# ci-engine.yml — moottori ja consumerin job-tiedostot > **Tila:** DRAFT — työstettävä esimerkki, ei lopullinen suunnitelma. > **Liittyy:** ticket 0006, [analysis/ci-flow-values-vs-native-config.md](../../docs/analysis/ci-flow-values-vs-native-config.md) Esimerkkiprojekti: `temperature-store` (Java/Maven-mikropalvelu) Provider (`gitea-ci-library`) tarjoaa **vain yhden tiedoston**: `ci-engine.yml`. Consumer tuo omat job-tiedostonsa ja datansa. Moottori päättelee itse mitä ajaa. --- ## Tiedostot ja roolit ``` temperature-store/ ← CONSUMER ├── .gitea/workflows/ │ └── ci.yml ← [A] Kutsuja: molemmat jobit (feature + master) yhdessä ├── ci-flow-values.yaml ← [B] Projektin data — uniikki per projekti └── pom.xml gitea-ci-library/ ← PROVIDER (= MOOTTORI) ├── .gitea/workflows/ │ └── ci-engine.yml ← [C] AINOA workflow — kaikki pipelinetyypit ├── scripts/ │ ├── report-status.sh │ ├── dispatch-workflow.sh │ ├── push-reports.sh │ └── tag-commit.sh └── ci-flow-values.yaml ← Provider-testaus (dogfood — kirjasto testaa itsensä) ``` **Roolit:** | Tiedosto | Omistaja | Rooli | |----------|----------|-------| | `ci-engine.yml` | **Provider** | Moottori: build-logiikka, konffaus, komennot | | `ci.yml` | **Consumer** | Kutsuja: molemmat jobit yhdessä tiedostossa, `if:` valitsee | | `ci-flow-values.yaml` | **Consumer** | Data: ekosysteemi, docker-nimi, test-flow jne | | `scripts/*.sh` | **Provider** | Jaetut työkalut | --- ## [A] Consumerin `.gitea/workflows/ci.yml` — rooli: **kutsuja** ```yaml # temperature-store/.gitea/workflows/ci.yml # # Rooli: KUTSUJA. # - Valitsee branchin perusteella oikean jobin (feature / master) # - Välittää KONTIT moottorille (moottori ei tunne yhtään konttia) # - Välittää config-filen polun moottorille name: CI on: push: branches: ["**"] workflow_dispatch: jobs: feature: if: github.ref != 'refs/heads/master' uses: org/gitea-ci-library/.gitea/workflows/ci-engine.yml@v1 secrets: inherit with: config-file: ci-flow-values.yaml maven-image: maven:3.9-eclipse-temurin-21 # ← Consumer omistaa kontit master: if: github.ref == 'refs/heads/master' uses: org/gitea-ci-library/.gitea/workflows/ci-engine.yml@v1 secrets: inherit with: config-file: ci-flow-values.yaml maven-image: maven:3.9-eclipse-temurin-21 dind-image: docker:26-dind # ← Master tarvitsee DinD:n ``` **Mitä tässä tapahtuu:** 1. Push mihin tahansa branchiin → `ci.yml` triggeröityy 2. `if: github.ref != 'refs/heads/master'` → `feature`-job 3. `if: github.ref == 'refs/heads/master'` → `master`-job 4. Molemmat kutsuvat samaa moottoria (`ci-engine.yml@v1`) samoilla parametreilla 5. Moottori päättelee itse branchista mitä pipelinea ajaa **Miksi yksi tiedosto, kaksi jobia:** - Yksi paikka katsoa mitä CI tekee — `ci.yml` on koko projektin CI-määrittely - `if:`-ehto on riittävä erottelemaan featuren ja masterin - Kopioitavissa sellaisenaan — consumer ei muokkaa mitään --- ## [B] Consumerin `ci-flow-values.yaml` — rooli: **projektidata** ```yaml # temperature-store/ci-flow-values.yaml # # Rooli: DATA ("mikä tämä projekti on, mitä se tarvitsee") # Tämä tiedosto on jokaisessa projektissa ERI. # Kaikki non-secret-arvot tänne. Tokenit → Gitea org secrets. build: ecosystem: maven # maven | gradle | npm — moottori valitsee kontin ja komennot docker: registry: gitea imageName: temperature-store sonarqube: url: https://sonar.example.com projectKey: temperature-store deployment: projectFolder: microservices fileName: values-{.environment}.yaml property: container.version test-flow: - deploy: development wait: true - test: name: "integration fast" environment: integration repo: tests/integration workflow: test.yml ref: main tags: "@temperature and not @slow" ``` **Mitä `ci-engine.yml` lukee tästä (MVP, tiketti 0006):** - `build.ecosystem` — määrittää kontin ja komennot **Non-secret vs secret:** - Tänne: ekosysteemi, image-nimi, URL:t, test-flow — versioitavaa - Gitea org secrets: `GITEA_TOKEN`, `SONAR_TOKEN`, `DEPLOY_TOKEN` — salaisuudet --- ## [C] Providerin `ci-engine.yml` — rooli: **moottori** Providerin **ainoa** workflow. Moottori koostuu kolmesta kerroksesta: | Kerros | Mistä tulee | Sisältö | |--------|-------------|---------| | **Kontit** | `with:` parametrit consumerilta | `maven-image`, `dind-image`, … — moottori ei tunne yhtään konttia | | **Pipeline runko** | `ci-engine.yml` sisäinen | Feature-skeleton: start → test → coverage → end
Master-skeleton: start → build → deploy → [test-flow] → end | | **Test plan + data** | `ci-flow-values.yaml` | `build.ecosystem` → komennot
`test-flow[]` → test plan (vain master)
`docker.*`, `deployment.*` (vain master) | ```yaml # gitea-ci-library/.gitea/workflows/ci-engine.yml # # MOOTTORI — kolme kerrosta: # 1. Kontit: ${{ inputs.maven-image }} jne (consumerilta) # 2. Pipeline runko: job-graafi erikseen featurelle ja masterille (täällä) # 3. Data: ci-flow-values.yaml → ekosysteemi, test plan, docker (consumerilta) name: CI Engine on: workflow_call: inputs: config-file: required: true type: string # ← Polku ci-flow-values.yaml:aan maven-image: # ← KONTTI: consumerilta required: false type: string dind-image: # ← KONTTI: consumerilta (vain master) required: false type: string node-image: # ← KONTTI: consumerilta (vain npm) required: false type: string concurrency: group: ci-${{ github.ref }} cancel-in-progress: false jobs: # ═══════════════════════════════════════════════════════════════ # YHTEINEN: konfiguraation luku (data-kerros) # Raportoi: pending → config luettu → success # ═══════════════════════════════════════════════════════════════ start: runs-on: ubuntu-latest outputs: ecosystem: ${{ steps.read-config.outputs.ecosystem }} steps: - uses: actions/checkout@v4 - name: Report pipeline start run: | bash scripts/report-status.sh \ "pending" "Pipeline starting" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/start" - name: Read project config id: read-config run: | ECOSYSTEM=$(yq '.build.ecosystem' "${{ inputs.config-file }}") [ -z "$ECOSYSTEM" ] || [ "$ECOSYSTEM" = "null" ] && \ echo "ERROR: build.ecosystem not set" >&2 && exit 1 echo "ecosystem=$ECOSYSTEM" >> $GITHUB_OUTPUT - name: Report pipeline started if: success() run: | bash scripts/report-status.sh \ "success" "Pipeline started" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/start" - name: Report start failure if: failure() run: | bash scripts/report-status.sh \ "failure" "Pipeline start failed" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/start" # ═══════════════════════════════════════════════════════════════ # FEATURE-SKELETON: testit + coverage # Ajetaan kun github.ref != refs/heads/master # Jokainen steppi raportoi: alussa pending, lopussa success/failure # ═══════════════════════════════════════════════════════════════ unit-test: needs: start if: github.ref != 'refs/heads/master' runs-on: ubuntu-latest container: image: ${{ inputs.maven-image }} steps: - uses: actions/checkout@v4 - name: Report unit-test start run: | bash scripts/report-status.sh \ "pending" "Unit tests running" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/unit-test" - name: Run unit tests id: test env: ECOSYSTEM: ${{ needs.start.outputs.ecosystem }} run: | case "$ECOSYSTEM" in maven|gradle) mvn test ;; npm) npm test ;; esac - name: Report unit-test success if: success() run: | bash scripts/report-status.sh \ "success" "Unit tests passed" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/unit-test" - name: Report unit-test failure if: failure() run: | bash scripts/report-status.sh \ "failure" "Unit tests failed" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/unit-test" code-coverage: needs: [start, unit-test] if: github.ref != 'refs/heads/master' runs-on: ubuntu-latest container: image: ${{ inputs.maven-image }} steps: - uses: actions/checkout@v4 - name: Report coverage start run: | bash scripts/report-status.sh \ "pending" "Coverage running" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/code-coverage" - name: Run coverage id: coverage env: ECOSYSTEM: ${{ needs.start.outputs.ecosystem }} run: | case "$ECOSYSTEM" in maven) mvn jacoco:report ;; gradle) gradle jacocoTestReport ;; npm) npm run coverage ;; esac - name: Report coverage success if: success() run: | bash scripts/report-status.sh \ "success" "Coverage report generated" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/code-coverage" - name: Report coverage failure if: failure() run: | bash scripts/report-status.sh \ "failure" "Coverage failed" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/code-coverage" # ═══════════════════════════════════════════════════════════════ # MASTER-SKELETON: build + deploy + test plan (tiketti 0009) # github.ref == refs/heads/master # Kontti: ${{ inputs.maven-image }} + ${{ inputs.dind-image }} # Test plan: ci-flow-values.yaml → test-flow[] # ═══════════════════════════════════════════════════════════════ # build-container: # needs: start # if: github.ref == 'refs/heads/master' # container: # image: ${{ inputs.dind-image }} ← KONTTI consumerilta # ... # # deploy: # needs: build-container # if: github.ref == 'refs/heads/master' # ...lukee deployment.* configista... # # test-flow: # needs: deploy # if: github.ref == 'refs/heads/master' # ...lukee test-flow[] configista... ← TEST PLAN configista # ═══════════════════════════════════════════════════════════════ # LOPETUS: kaikille yhteinen # Raportoi: pending → aggregoi tulokset → success/failure # ═══════════════════════════════════════════════════════════════ end: needs: [unit-test, code-coverage] if: always() runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Report pipeline ending run: | bash scripts/report-status.sh \ "pending" "Pipeline finishing" \ "$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \ "ci/end" - name: Report final status run: | BUILD_URL="$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" if [ "${{ needs.unit-test.result }}" = "success" ] && \ [ "${{ needs.code-coverage.result }}" = "success" ]; then STATE="success"; DESC="All checks passed" else STATE="failure"; DESC="Some checks failed" fi bash scripts/report-status.sh "$STATE" "$DESC" "$BUILD_URL" "ci/end" ``` **Moottorin kolme kerrosta käytännössä:** 1. **Kontit** (consumerilta `with:` → moottori `inputs:` → job `container.image`) - Moottori ei sisällä yhtään kontti-imagea - Consumer päättää versiot: `maven:3.9-eclipse-temurin-21` 2. **Pipeline runko** (moottorin sisäinen job-graafi) - `if: github.ref != 'refs/heads/master'` → feature-skeleton - `if: github.ref == 'refs/heads/master'` → master-skeleton (myöhemmät tiketit) 3. **Data** (`ci-flow-values.yaml`) - `build.ecosystem` → mitä komentoja ajetaan - `test-flow[]` → test plan (vain master, tiketti 0009) **Mitä tässä EI ole (myöhemmät tiketit):** - `push-reports.sh` — raporttien julkaisu MinIO:hon (tiketti 0013) - `isContainerBuild()` / kontin buildaus (tiketti 0009) - SonarQube quality gate (tiketti 0009) - Test flow -ketjutus `test-flow[]`:sta (tiketti 0008) --- ## Yhteenveto ``` ┌──────────────────────────────────────────────────────────────────────┐ │ CONSUMER: temperature-store │ │ │ │ ci.yml ci-flow-values.yaml │ │ ┌──────────────────────────┐ ┌────────────────────────┐ │ │ │ jobs: │ │ build: │ │ │ │ feature (if !master) ───┼─┐ │ ecosystem: maven │ │ │ │ with: │ │ │ docker: │ │ │ │ config-file + kontit │ │ │ imageName: ... │ │ │ │ │ │ │ test-flow: [...] │ │ │ │ master (if master) ────┼─┤ └────────────────────────┘ │ │ │ with: │ │ │ │ │ config-file + kontit │ │ │ │ └──────────────────────────┘ │ │ └───────────────────────────────┼───────────────────────────────────────┘ │ with: kontit + config-file ▼ ┌──────────────────────────────────────────────────────────────────────┐ │ PROVIDER: gitea-ci-library (ci-engine.yml) │ │ │ │ Kerros 1: KONTIT (consumerilta) │ │ ┌──────────────────────────────────────────┐ │ │ │ inputs: maven-image, dind-image, ... │ │ │ └──────────────────────────────────────────┘ │ │ │ │ Kerros 2: PIPELINE RUNKO (moottorin sisäinen) │ │ ┌──────────────────────────────────────────┐ │ │ │ FEATURE-SKELETON MASTER-SKELETON │ │ │ │ start ──────────────────── start │ │ │ │ ↓ ↓ │ │ │ │ unit-test build-container│ │ │ │ ↓ ↓ │ │ │ │ code-coverage deploy │ │ │ │ ↓ ↓ │ │ │ │ end test-flow[] ───│── Kerros 3: TEST PLAN │ │ │ ↓ │ │ │ │ end │ │ │ └──────────────────────────────────────────┘ │ │ │ │ Kerros 3: DATA (ci-flow-values.yaml) │ │ ┌──────────────────────────────────────────┐ │ │ │ ecosystem → komennot │ │ │ │ test-flow[] → test plan (vain master) │ │ │ │ docker.*, deployment.* (vain master) │ │ │ └──────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────────┘ ``` ### Päätöstaulukko | Asia | Sijainti | Kerros | |------|----------|:--:| | Kontit (`maven-image`, `dind-image`) | Consumer (`ci.yml` `with:`) | 1 | | Pipeline runko (job-graafi) | Provider (`ci-engine.yml`) | 2 | | `build.ecosystem` → komennot | Consumer (`ci-flow-values.yaml`) | 3 | | `test-flow[]` → test plan | Consumer (`ci-flow-values.yaml`) | 3 | | `docker.*`, `deployment.*` | Consumer (`ci-flow-values.yaml`) | 3 | | Skriptit | Provider (`scripts/`) | — | | Tokenit, salasanat | Gitea org secrets | — |