TRG 5.09 - Helm test

Status	Created	Post-History
Update	19-Jan-2024	add description for kubernetes version
Update	05-Dec-2023	Updated example, with customizable helm version
Update	10-Nov-2023	Bump kind version
Active	10-Nov-2023
Prerelease	7-Mar-2023	Moved out of draft
Draft	23-Feb-2023	Draft release

Why

Helm charts are defined as the sane default for cloud native applications in Eclipse Tractus-X. To make sure that the aligned quality verification of applications are as close as possible to the production setup, it is critical that the test setup is doing what it would do on a production setup. Therefore, Helm test will provide this mechanism together with kind and allows us to run e2e verifications directly on GitHub Actions.

GitHub Actions simplifies the open source infrastructure requirements as otherwise a test k8s cluster would be required, which might not be available in the long run.

This is the base for future e2e tests.

This also allows to move the responsibility of verifying the helm chart functionality from the system team back to the development team.

Description

Helm charts and their dependent images (the application to test) need to be validated by installing them and verifying that they are running successfully.

Helm test is the technical solution helm provides to verify that a helm chart works as expected. It basically installs the helm chart (which installs and runs the application) and then allows an additional container to run. This test container can do everything, especially calling an api, executing tests etc.

This needs to be running in a GitHub Action in the repository which contains the Helm Chart, which gets released.

Example GitHub Action Workflow:

Build all necessary container images / pull the container image under test
Create kind setup (kind creates a kubernetes cluster through container images, GitHub action supports this which allows you to create a kubernetes cluster inside the GitHub action run)
Make the necessary container images available through the kind setup (see GitHub action example, it uses the local kind registry)
Prepare and execute helm test
Fail on a failing helm test
The example also contains the upgradeability test as the last step which is described in more detail in TRG 5.11

Technical requirements:

A GitHub action exist which builds or uses the helm chart which gets released
The GitHub action can be triggered manually through GitHub WebUI manually running a workflow
Helm test verifies that the application is up and running
- Example verifications:
  - Frontend: Frontend returns valid page
  - API: API is reachable and returns valid response (access denied / test user can log in)
  - Kubernetes versions that are supported with the helm chart are tested

Example .github/workflows/helm-test.yaml

name: Lint and Test Charts

on:
    pull_request:
    workflow_dispatch:
      inputs:
        node_image:
          description: 'kindest/node image for k8s kind cluster'
          # k8s version from 3.1 release as default
          default: 'kindest/node:v1.27.3'
          required: false
          type: string
        upgrade_from:
          description: 'chart version to upgrade from'
          # chart version from 3.1 release as default
          default: 'x.x.x'
          required: false
          type: string
        helm_version:
          description: 'helm version to test (default = latest)'
          default: 'latest'
          required: false
          type: string

jobs:
    lint-test:
        runs-on: ubuntu-latest
        steps:
            - name: Checkout
              uses: actions/checkout@v3
              with:
                fetch-depth: 0

            - name: Kubernetes KinD Cluster
              uses: container-tools/kind-action@v1
              with:
                # upgrade version, default (v0.17.0) uses node image v1.21.1 and doesn't work with more recent node image versions
                version: v0.20.0
                # default value for event_name != workflow_dispatch
                node_image: ${{ github.event.inputs.node_image || 'kindest/node:v1.27.3' }}

            - name: Build image
              uses: docker/build-push-action@v3
              with:
                context: .
                push: true
                tags: kind-registry:5000/app-dashboard:testing

            - name: Set up Helm
              uses: azure/setup-helm@v3
              with:
                version: ${{ github.event.inputs.helm_version || 'latest' }}

            - uses: actions/setup-python@v4
              with:
                python-version: '3.9'
                check-latest: true
            - name: Set up chart-testing
              uses: helm/chart-testing-action@v2.3.1

            - name: Run chart-testing (list-changed)
              id: list-changed
              run: |
                changed=$(ct list-changed --target-branch ${{ github.event.repository.default_branch }})
                if [[ -n "$changed" ]]; then
                  echo "changed=true" >> $GITHUB_OUTPUT
                fi

            - name: Run chart-testing (lint)
              run: ct lint --validate-maintainers=false --target-branch ${{ github.event.repository.default_branch }}
            - name: Run chart-testing (install)
              run: ct install --charts charts/app-dashboard
              if: github.event_name != 'pull_request' || steps.list-changed.outputs.changed == 'true'

              # Upgrade the released chart version with the locally available chart
              # default value for event_name != workflow_dispatch
            - name: Run helm upgrade
              run: |
                helm repo add bitnami https://charts.bitnami.com/bitnami
                helm repo add tractusx-dev https://eclipse-tractusx.github.io/charts/dev
                helm install [NAME] tractusx-dev/[CHART] --version ${{ github.event.inputs.upgrade_from || 'x.x.x' }}
                helm dependency update charts/[CHART]
                helm upgrade [NAME] charts/[CHART]
              if: github.event_name != 'pull_request' || steps.list-changed.outputs.changed == 'true'

Why​

Description​

Why

Description