Merge pull request #10 from jordojordo/main

Add sync mechanism with workflow and manifest

.github/workflows/sync-extensions.yml

@@ -0,0 +1,58 @@
+name: Sync and Release Extensions
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - manifest.json
+
+env:
+  ACTIONS_RUNNER_DEBUG: false
+  CI_COMMIT_MESSAGE: CI Build Artifacts
+
+jobs:
+  sync:
+    if: github.repository_owner == 'rancher'
+    name: Sync and Release Extensions
+    runs-on: ubuntu-latest
+    permissions: write-all
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+
+      - name: Configure Git
+        run: |
+          git config user.name github-actions
+          git config user.email github-actions@github.com
+      
+      - name: Setup Helm
+        uses: azure/setup-helm@v3
+        with:
+          version: v3.12.1
+
+      - name: Setup yq
+        uses: chrisdickinson/setup-yq@v1.0.1
+        with:
+          yq-version: v4.34.2
+
+      - name: Run sync script
+        shell: bash
+        id: sync_script
+        run: |
+          chmod +x ./scripts/sync
+          ./scripts/sync
+
+      - name: Commit build
+        run: |
+          git add ./{assets,charts,extensions,icons,index.yaml}
+          git commit -a -m "${{ env.CI_COMMIT_MESSAGE }}"
+          git push
+
+      - name: Run chart-releaser
+        uses: helm/chart-releaser-action@v1.5.0
+        with:
+          charts_dir: ./charts/*
+        env:
+          CR_TOKEN: '${{ secrets.GITHUB_TOKEN }}'
+          CR_SKIP_EXISTING: true

README.md

@@ -7,7 +7,6 @@ The "Tested & Certified -- Rancher Extension" certification is part of the SUSE
 ## Requirements
 
 * Charts must be Helm 3 compatible.
-    
 
 * Chart must be in a hosted [Helm](https://helm.sh/docs/topics/chart_repository/) or Git repository that we can reference.
 
@@ -17,94 +16,84 @@ The "Tested & Certified -- Rancher Extension" certification is part of the SUSE
 #### 1. Fork the [Partner Extensions](https://github.com/rancher/partner-charts/) repository, clone your fork, checkout the **main** branch and pull the latest changes. 
 Then create a new branch off of main
 
-#### 2. Create subdirectories in `packages` in the form of `<vendor>/<chart>`
-```bash
-cd partner-charts
-mkdir -p packages/suse/kubewarden-controller
-
+#### 2. Update the `manifest.json` with your Extension metadata.
+
+```json
+{
+  "extensions": {
+    "kubewarden": {
+      "repo": "kubewarden/ui",
+      "branch": "gh-pages",
+      "versions": [
+        "1.0.0",
+        "1.1.0"
+      ]
+    }
+  }
+}
 ```
-#### 3. Create your [upstream.yaml](#configuration-file) in `packages/<vendor>/<chart>`
-
-The tool reads a configuration yaml, `upstream.yaml`, to know where to fetch the upstream chart. This file is also able to define any alterations for valid variables in the Chart.yaml as described by [Helm](https://helm.sh/docs/topics/charts/#the-chartyaml-file).
 
-Some [example upstream.yaml](#examples) are provided below
+#### 3. Commit your changes
 ```bash
-cat <<EOF > packages/suse/kubewarden-controller/upstream.yaml
-HelmRepo: https://charts.kubewarden.io
-HelmChart: kubewarden-controller
-Vendor: SUSE
-DisplayName: Kubewarden Controller
-ChartMetadata:  
-  icon: https://www.kubewarden.io/images/icon-kubewarden.svg
-EOF
+git add manifest.json
+git commit -m "Submitting kubewarden/ui version 1.1.0"
 ```
-#### 4. Commit your packages directory
-```bash
-git add packages/suse/kubewarden-controller
-git commit -m "Submitting suse/kubewarden-controller"
-```
-#### 6. Test your configuration
-1. In Rancher UI > Extensions > Kebab menu (3 dot) > Manage repositories, add your github repository
-2. Go to Hamburger menu > Extensions and your extension should be listed under "Available." Install the extension
-#### 7. Push your commit
+
+#### 4. Push your commit
 ```bash
 git push origin <your_branch>
 ```
-#### 8. Open a pull request to **main** branch
+
+#### 5. Open a pull request on the **main** branch
+
+Once your pull request is approved and merged, an automated workflow will sync this repository with the build assets from the supplied repository within the `manifest.json` file. When fully synced, a new release will be created and added to the [releases](https://github.com/rancher/partner-extensions/releases) section. 
 
 ## Configuration File
 
-Options for `upstream.yaml`
-| Variable | Requires | Description |
-| ------------- | ------------- |------------- |
-| AutoInstall | | Allows setting a required additional chart to deploy prior to current chart, such as a dedicated CRDs chart
-| ChartMetadata | | Allows setting/overriding the value of any valid [Chart.yaml variable](https://helm.sh/docs/topics/charts/#the-chartyaml-file)
-| DisplayName | | Sets the name the chart will be listed under in the Rancher UI
-| Experimental | | Adds the 'experimental' annotation which adds a flag on the UI entry
-| Fetch | HelmChart, HelmRepo | Selects set of charts to pull from upstream.<br />- **latest** will pull only the latest chart version *default*<br />- **newer** will pull all newer versions than currently stored<br />- **all** will pull all versions
-| GitBranch | GitRepo | Defines which branch to pull from the upstream GitRepo
-| GitHubRelease | GitRepo | If true, will pull latest GitHub release from repo. Requires GitHub URL
-| GitRepo | | Defines the git repo to pull from
-| GitSubdirectory | GitRepo | Allows selection of a subdirectory of the upstream git repo to pull the chart from
-| HelmChart | HelmRepo | Defines which chart to pull from the upstream Helm repo
-| HelmRepo | HelmChart | Defines the upstream Helm repo to pull from
-| Hidden | | Adds the 'hidden' annotation which hides the chart from the Rancher UI
-| Namespace | | Addes the 'namespace' annotation which hard-codes a deployment namespace for the chart
-| PackageVersion | | Used to generate new patch version of chart
-| ReleaseName | | Sets the value of the release-name Rancher annotation. Defaults to the chart name
-| TrackVersions | HelmChart, HelmRepo | Allows selection of multiple *Major.Minor* versions to track from upstream independently.
-| Vendor | | Sets the vendor name providing the chart
+Required properties for `manifest.json`
+| Variable | Description |
+| ------------- |------------- |
+| [extension key] | This name is representative of the Extension **package** name. For example, the [clock](https://github.com/rancher/ui-plugin-examples/tree/main/pkg/clock) package within the [`ui-plugin-examples`](https://github.com/rancher/ui-plugin-examples/tree/main) repository, `clock` would be the extension key.
+| repo | Defines the upstream **Github** repository to pull the build assets from.
+| branch | Defines which branch to pull from the upstream `repo`
+| versions | An array of version strings which correspond to the Extension **package** version(s) to be synced with this repository. For example, the [clock](https://github.com/rancher/ui-plugin-examples/tree/main/charts/clock) extension package has two versions, `0.1.0` and `0.2.0` would be added.
 
-## Examples
-### Helm Repo
-#### Minimal Requirements
-```yaml
-HelmRepo: https://charts.kubewarden.io
-HelmChart: kubewarden-controller
-Vendor: SUSE
-DisplayName: Kubewarden Controller
-```
 
-### Git Repo
-```yaml
-GitRepo: https://github.com/kubewarden/helm-charts.git
-GitBranch: main
-GitSubdirectory: charts/kubewarden-controller
-Vendor: SUSE
-DisplayName: Kubewarden Controller
-ChartMetadata:
-  kubeVersion: '>=1.21-0'
-  icon: https://www.kubewarden.io/images/icon-kubewarden.svg
-```
+## Examples
 
-### GitHub Release
-```yaml
-GitRepo: https://github.com/kubewarden/helm-charts.git
-GitHubRelease: true
-GitSubdirectory: charts/kubewarden-controller
-Vendor: SUSE
-DisplayName: Kubewarden Controller
-ChartMetadata:
-  kubeVersion: '>=1.21-0'
-  icon: https://www.kubewarden.io/images/icon-kubewarden.svg
+```json
+{
+  "extensions": {
+    "kamaji": {
+      "repo": "clastix/rancher-extension-clastix",
+      "branch": "gh-pages",
+      "versions": [
+        "0.1.2"
+      ]
+    },
+    "elemental": {
+      "repo": "rancher/elemental-ui",
+      "branch": "main",
+      "versions": [
+        "1.2.0",
+        "1.1.0",
+        "1.0.0"
+      ]
+    },
+    "kubewarden": {
+      "repo": "kubewarden/ui",
+      "branch": "gh-pages",
+      "versions": [
+        "1.0.0",
+        "1.0.1",
+        "1.0.2",
+        "1.0.3",
+        "1.0.4",
+        "1.0.5",
+        "1.0.6",
+        "1.1.0"
+      ]
+    }
+  }
+}
 ```

manifest.json

@@ -0,0 +1,3 @@
+{
+  "extensions": {}
+}

scripts/sync

@@ -0,0 +1,187 @@
+#!/usr/bin/env bash
+
+SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
+BASE_DIR="$( cd $SCRIPT_DIR && cd .. & pwd)"
+
+CYAN="\033[96m"
+YELLOW="\033[93m"
+RESET="\033[0m"
+BOLD="\033[1m"
+NORMAL="\033[22m"
+CHECK="\xE2\x9C\x94"
+
+ORG=rancher
+BRANCH=main
+CLEAN=false
+
+usage() {
+  echo "Usage: $0 [<options>]"
+  echo " options:"
+  echo "  [-c | --clean]                   Clean the current extension assets including: ./{charts, assets, extensions, icons}"
+  echo "  [-o | --org] <name>              Specify the organization of the current repository (defaults to 'rancher')"
+  echo "  [-b | --branch] <name>           Specify the destination branch of the extension assets within the current repository (defaults to 'main')"
+  exit 1
+}
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    -h|--help)
+      usage
+      ;;
+    -o|--org)
+      if [[ -z $2 || $2 == -* ]]; then
+        echo "Error: Missing argument for $1 option"
+        usage
+      fi
+      ORG="${2}"
+      shift 2
+      ;;
+    -c|--clean)
+      CLEAN=true
+      shift
+      ;;
+    -b|--branch)
+      if [[ -z $2 || $2 == -* ]]; then
+        echo "Error: Missing argument for $1 option"
+        usage
+      fi
+      BRANCH="${2}"
+      shift 2
+      ;;
+    *)
+      echo "Error: Unknown option $1"
+      usage
+      ;;
+  esac
+done
+
+shift $((OPTIND-1))
+
+if [[ $CLEAN == true ]]; then
+  rm -rf ./charts
+  rm -rf ./assets
+  rm -rf ./extensions
+  rm -rf ./icons
+fi
+
+echo -e "${CYAN}${BOLD}Syncing Extensions${RESET}"
+
+EXTS=$(jq -r ".extensions | keys[]" manifest.json)
+
+TMP=${BASE_DIR}/tmp
+rm -rf ${TMP}
+mkdir -p ${TMP}
+
+HELM_INDEX=${BASE_DIR}/index.yaml
+CHART_TMP=${TMP}/_charts
+
+REPOSITORY=${ORG}/partner-extensions
+
+echo "GitHub Repository: ${REPOSITORY}"
+echo "GitHub Branch:     ${BRANCH}"
+
+for NAME in ${EXTS}
+do
+  echo -e "${CYAN} + Syncing: ${BOLD}${NAME}${RESET}"
+
+  # Make diretories for assets, charts, and extensions
+  mkdir -p ./assets/${NAME}
+  mkdir -p ./charts/${NAME}
+  mkdir -p ./extensions/${NAME}
+
+  # Get repository name, branch, and versions 
+  REPO=$(jq -r ".extensions.${NAME}.repo" manifest.json)
+  EXT_BRANCH=$(jq -r ".extensions.${NAME}.branch" manifest.json)
+  VERSIONS=$(jq -r ".extensions.${NAME}.versions[]" manifest.json)
+  VFORMAT=$(echo $VERSIONS | tr '\n' ' ')
+
+  echo -e "     Repository: ${REPO}"
+  echo -e "     Branch: ${EXT_BRANCH}"
+  echo -e "     Versions  : ${VFORMAT}"
+  echo ""
+
+  echo -e "  .. Cloning repository"
+  rm -rf ./tmp/${NAME}
+  pushd tmp > /dev/null
+  git clone https://github.com/${REPO}.git ${NAME}
+  cd ${NAME}
+  git checkout ${EXT_BRANCH}
+  pwd
+  popd > /dev/null
+
+  for VERSION in ${VERSIONS}
+  do
+    echo -e "${CYAN}     Syncing: ${BOLD}${NAME}@${VERSION}${RESET}"
+
+    if [ -d "./charts/${NAME}/${VERSION}" ]; then
+      echo "      Version already synced"
+    else
+      echo "     + Copying version ${VERSION}"
+      # cp ./tmp/${NAME}/extensions/${NAME}/${NAME}-${VERSION}.tgz ./assets/${NAME}
+      cp -R ./tmp/${NAME}/extensions/${NAME}/${VERSION} ./extensions/${NAME}/${VERSION}
+      cp -R ./tmp/${NAME}/charts/${NAME}/${VERSION} ./charts/${NAME}/${VERSION}
+
+      # Need to patch the charts
+      echo "     + Patching Helm chart"
+      CR_FILE=./charts/${NAME}/${VERSION}/templates/cr.yaml
+      ENDPOINT=https://raw.githubusercontent.com/${REPOSITORY}/${BRANCH}/extensions/${NAME}/${VERSION}
+      sed -i.bak -e 's@endpoint:.*@endpoint: '"$ENDPOINT"'@' ${CR_FILE}
+      rm -f ${CR_FILE}.bak
+
+      echo "     + Fetching and patching icon"
+
+      # Check the chart file for an icon
+      CHART_FILE=./charts/${NAME}/${VERSION}/Chart.yaml
+
+      ICON=$(sed -nr 's/icon: (.*)/\1/p' ${CHART_FILE})
+      
+      if [ -n "${ICON}" ]; then
+        # Downloading icon
+        ICON_FILE=$(basename $ICON)
+        echo "     + Fetching icon: ${ICON}"
+        ICON_REL=icons/${NAME}/${VERSION}-${ICON_FILE}
+        mkdir -p ${BASE_DIR}/icons/${NAME}
+        rm -f ${BASE_DIR}/${ICON_REL}
+        wget --quiet -O ${BASE_DIR}/${ICON_REL} ${ICON}
+
+        NEW_ICON=https://raw.githubusercontent.com/${REPOSITORY}/${BRANCH}/${ICON_REL}
+
+        sed -i.bak -e 's@icon:.*@icon: '"$NEW_ICON"'@' ${CHART_FILE}
+        rm -rf ${CHART_FILE}.bak
+
+        PKG_FILE=${BASE_DIR}/extensions/${NAME}/${VERSION}/plugin/package.json
+        sed -i.bak -e 's@\"icon\": \".*\"@\"icon\": \"'"$NEW_ICON"'\"@' ${PKG_FILE}
+        rm -rf ${PKG_FILE}.bak
+      fi
+
+      echo "     + Packaging Helm chart"
+      helm package ./charts/${NAME}/${VERSION} -d ./assets/${NAME}
+
+      echo "     + Updating Helm index"
+
+      # --------------------------------------------------------------------------------
+      # Update the helm index just for this chart 
+      # --------------------------------------------------------------------------------
+      if [ -f "${HELM_INDEX}" ]; then
+        UPDATE="--merge ${HELM_INDEX}"
+      fi
+
+      # Base URL referencing assets directly from GitHub
+      BASE_URL="assets/${NAME}"
+
+      rm -rf ${CHART_TMP}
+      mkdir -p ${CHART_TMP}
+      cp ${BASE_DIR}/assets/${NAME}/${NAME}-${VERSION}.tgz ${CHART_TMP}
+
+      helm repo index ${CHART_TMP} --url ${BASE_URL} ${UPDATE}
+
+      cp ${CHART_TMP}/index.yaml ${HELM_INDEX}
+    fi
+  done
+
+  echo ""
+done
+
+# Clean up
+rm -rf ${CHART_TMP}
+rm -rf ${TMP}