450 lines
20 KiB
Markdown
450 lines
20 KiB
Markdown
# ci-engine.yml — moottori ja consumerin job-tiedostot
|
|
|
|
> **Tila:** DRAFT — työstettävä esimerkki, ei lopullinen suunnitelma.
|
|
> **Liittyy:** ticket 0006, [analysis/ci-flow-values-vs-native-config.md](../../docs/analysis/ci-flow-values-vs-native-config.md)
|
|
|
|
Esimerkkiprojekti: `temperature-store` (Java/Maven-mikropalvelu)
|
|
|
|
Provider (`gitea-ci-library`) tarjoaa **vain yhden tiedoston**: `ci-engine.yml`.
|
|
Consumer tuo omat job-tiedostonsa ja datansa. Moottori päättelee itse mitä ajaa.
|
|
|
|
---
|
|
|
|
## Tiedostot ja roolit
|
|
|
|
```
|
|
temperature-store/ ← CONSUMER
|
|
├── .gitea/workflows/
|
|
│ └── ci.yml ← [A] Kutsuja: molemmat jobit (feature + master) yhdessä
|
|
├── ci-flow-values.yaml ← [B] Projektin data — uniikki per projekti
|
|
└── pom.xml
|
|
|
|
gitea-ci-library/ ← PROVIDER (= MOOTTORI)
|
|
├── .gitea/workflows/
|
|
│ └── ci-engine.yml ← [C] AINOA workflow — kaikki pipelinetyypit
|
|
├── scripts/
|
|
│ ├── report-status.sh
|
|
│ ├── dispatch-workflow.sh
|
|
│ ├── push-reports.sh
|
|
│ └── tag-commit.sh
|
|
└── ci-flow-values.yaml ← Provider-testaus (dogfood — kirjasto testaa itsensä)
|
|
```
|
|
|
|
**Roolit:**
|
|
|
|
| Tiedosto | Omistaja | Rooli |
|
|
|----------|----------|-------|
|
|
| `ci-engine.yml` | **Provider** | Moottori: build-logiikka, konffaus, komennot |
|
|
| `ci.yml` | **Consumer** | Kutsuja: molemmat jobit yhdessä tiedostossa, `if:` valitsee |
|
|
| `ci-flow-values.yaml` | **Consumer** | Data: ekosysteemi, docker-nimi, test-flow jne |
|
|
| `scripts/*.sh` | **Provider** | Jaetut työkalut |
|
|
|
|
---
|
|
|
|
## [A] Consumerin `.gitea/workflows/ci.yml` — rooli: **kutsuja**
|
|
|
|
```yaml
|
|
# temperature-store/.gitea/workflows/ci.yml
|
|
#
|
|
# Rooli: KUTSUJA.
|
|
# - Valitsee branchin perusteella oikean jobin (feature / master)
|
|
# - Välittää KONTIT moottorille (moottori ei tunne yhtään konttia)
|
|
# - Välittää config-filen polun moottorille
|
|
|
|
name: CI
|
|
|
|
on:
|
|
push:
|
|
branches: ["**"]
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
feature:
|
|
if: github.ref != 'refs/heads/master'
|
|
uses: org/gitea-ci-library/.gitea/workflows/ci-engine.yml@v1
|
|
secrets: inherit
|
|
with:
|
|
config-file: ci-flow-values.yaml
|
|
maven-image: maven:3.9-eclipse-temurin-21 # ← Consumer omistaa kontit
|
|
|
|
master:
|
|
if: github.ref == 'refs/heads/master'
|
|
uses: org/gitea-ci-library/.gitea/workflows/ci-engine.yml@v1
|
|
secrets: inherit
|
|
with:
|
|
config-file: ci-flow-values.yaml
|
|
maven-image: maven:3.9-eclipse-temurin-21
|
|
dind-image: docker:26-dind # ← Master tarvitsee DinD:n
|
|
```
|
|
|
|
**Mitä tässä tapahtuu:**
|
|
1. Push mihin tahansa branchiin → `ci.yml` triggeröityy
|
|
2. `if: github.ref != 'refs/heads/master'` → `feature`-job
|
|
3. `if: github.ref == 'refs/heads/master'` → `master`-job
|
|
4. Molemmat kutsuvat samaa moottoria (`ci-engine.yml@v1`) samoilla parametreilla
|
|
5. Moottori päättelee itse branchista mitä pipelinea ajaa
|
|
|
|
**Miksi yksi tiedosto, kaksi jobia:**
|
|
- Yksi paikka katsoa mitä CI tekee — `ci.yml` on koko projektin CI-määrittely
|
|
- `if:`-ehto on riittävä erottelemaan featuren ja masterin
|
|
- Kopioitavissa sellaisenaan — consumer ei muokkaa mitään
|
|
|
|
---
|
|
|
|
## [B] Consumerin `ci-flow-values.yaml` — rooli: **projektidata**
|
|
|
|
```yaml
|
|
# temperature-store/ci-flow-values.yaml
|
|
#
|
|
# Rooli: DATA ("mikä tämä projekti on, mitä se tarvitsee")
|
|
# Tämä tiedosto on jokaisessa projektissa ERI.
|
|
# Kaikki non-secret-arvot tänne. Tokenit → Gitea org secrets.
|
|
|
|
build:
|
|
ecosystem: maven # maven | gradle | npm — moottori valitsee kontin ja komennot
|
|
|
|
docker:
|
|
registry: gitea
|
|
imageName: temperature-store
|
|
|
|
sonarqube:
|
|
url: https://sonar.example.com
|
|
projectKey: temperature-store
|
|
|
|
deployment:
|
|
projectFolder: microservices
|
|
fileName: values-{.environment}.yaml
|
|
property: container.version
|
|
|
|
test-flow:
|
|
- deploy: development
|
|
wait: true
|
|
- test:
|
|
name: "integration fast"
|
|
environment: integration
|
|
repo: tests/integration
|
|
workflow: test.yml
|
|
ref: main
|
|
tags: "@temperature and not @slow"
|
|
```
|
|
|
|
**Mitä `ci-engine.yml` lukee tästä (MVP, tiketti 0006):**
|
|
- `build.ecosystem` — määrittää kontin ja komennot
|
|
|
|
**Non-secret vs secret:**
|
|
- Tänne: ekosysteemi, image-nimi, URL:t, test-flow — versioitavaa
|
|
- Gitea org secrets: `GITEA_TOKEN`, `SONAR_TOKEN`, `DEPLOY_TOKEN` — salaisuudet
|
|
|
|
---
|
|
|
|
## [C] Providerin `ci-engine.yml` — rooli: **moottori**
|
|
|
|
Providerin **ainoa** workflow. Moottori koostuu kolmesta kerroksesta:
|
|
|
|
| Kerros | Mistä tulee | Sisältö |
|
|
|--------|-------------|---------|
|
|
| **Kontit** | `with:` parametrit consumerilta | `maven-image`, `dind-image`, … — moottori ei tunne yhtään konttia |
|
|
| **Pipeline runko** | `ci-engine.yml` sisäinen | Feature-skeleton: start → test → coverage → end<br>Master-skeleton: start → build → deploy → [test-flow] → end |
|
|
| **Test plan + data** | `ci-flow-values.yaml` | `build.ecosystem` → komennot<br>`test-flow[]` → test plan (vain master)<br>`docker.*`, `deployment.*` (vain master) |
|
|
|
|
```yaml
|
|
# gitea-ci-library/.gitea/workflows/ci-engine.yml
|
|
#
|
|
# MOOTTORI — kolme kerrosta:
|
|
# 1. Kontit: ${{ inputs.maven-image }} jne (consumerilta)
|
|
# 2. Pipeline runko: job-graafi erikseen featurelle ja masterille (täällä)
|
|
# 3. Data: ci-flow-values.yaml → ekosysteemi, test plan, docker (consumerilta)
|
|
|
|
name: CI Engine
|
|
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
config-file:
|
|
required: true
|
|
type: string # ← Polku ci-flow-values.yaml:aan
|
|
maven-image: # ← KONTTI: consumerilta
|
|
required: false
|
|
type: string
|
|
dind-image: # ← KONTTI: consumerilta (vain master)
|
|
required: false
|
|
type: string
|
|
node-image: # ← KONTTI: consumerilta (vain npm)
|
|
required: false
|
|
type: string
|
|
|
|
concurrency:
|
|
group: ci-${{ github.ref }}
|
|
cancel-in-progress: false
|
|
|
|
jobs:
|
|
# ═══════════════════════════════════════════════════════════════
|
|
# YHTEINEN: konfiguraation luku (data-kerros)
|
|
# Raportoi: pending → config luettu → success
|
|
# ═══════════════════════════════════════════════════════════════
|
|
start:
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
ecosystem: ${{ steps.read-config.outputs.ecosystem }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Report pipeline start
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"pending" "Pipeline starting" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/start"
|
|
|
|
- name: Read project config
|
|
id: read-config
|
|
run: |
|
|
ECOSYSTEM=$(yq '.build.ecosystem' "${{ inputs.config-file }}")
|
|
[ -z "$ECOSYSTEM" ] || [ "$ECOSYSTEM" = "null" ] && \
|
|
echo "ERROR: build.ecosystem not set" >&2 && exit 1
|
|
echo "ecosystem=$ECOSYSTEM" >> $GITHUB_OUTPUT
|
|
|
|
- name: Report pipeline started
|
|
if: success()
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"success" "Pipeline started" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/start"
|
|
|
|
- name: Report start failure
|
|
if: failure()
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"failure" "Pipeline start failed" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/start"
|
|
|
|
# ═══════════════════════════════════════════════════════════════
|
|
# FEATURE-SKELETON: testit + coverage
|
|
# Ajetaan kun github.ref != refs/heads/master
|
|
# Jokainen steppi raportoi: alussa pending, lopussa success/failure
|
|
# ═══════════════════════════════════════════════════════════════
|
|
unit-test:
|
|
needs: start
|
|
if: github.ref != 'refs/heads/master'
|
|
runs-on: ubuntu-latest
|
|
container:
|
|
image: ${{ inputs.maven-image }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Report unit-test start
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"pending" "Unit tests running" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/unit-test"
|
|
|
|
- name: Run unit tests
|
|
id: test
|
|
env:
|
|
ECOSYSTEM: ${{ needs.start.outputs.ecosystem }}
|
|
run: |
|
|
case "$ECOSYSTEM" in
|
|
maven|gradle) mvn test ;;
|
|
npm) npm test ;;
|
|
esac
|
|
|
|
- name: Report unit-test success
|
|
if: success()
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"success" "Unit tests passed" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/unit-test"
|
|
|
|
- name: Report unit-test failure
|
|
if: failure()
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"failure" "Unit tests failed" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/unit-test"
|
|
|
|
code-coverage:
|
|
needs: [start, unit-test]
|
|
if: github.ref != 'refs/heads/master'
|
|
runs-on: ubuntu-latest
|
|
container:
|
|
image: ${{ inputs.maven-image }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Report coverage start
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"pending" "Coverage running" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/code-coverage"
|
|
|
|
- name: Run coverage
|
|
id: coverage
|
|
env:
|
|
ECOSYSTEM: ${{ needs.start.outputs.ecosystem }}
|
|
run: |
|
|
case "$ECOSYSTEM" in
|
|
maven) mvn jacoco:report ;;
|
|
gradle) gradle jacocoTestReport ;;
|
|
npm) npm run coverage ;;
|
|
esac
|
|
|
|
- name: Report coverage success
|
|
if: success()
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"success" "Coverage report generated" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/code-coverage"
|
|
|
|
- name: Report coverage failure
|
|
if: failure()
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"failure" "Coverage failed" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/code-coverage"
|
|
|
|
# ═══════════════════════════════════════════════════════════════
|
|
# MASTER-SKELETON: build + deploy + test plan (tiketti 0009)
|
|
# github.ref == refs/heads/master
|
|
# Kontti: ${{ inputs.maven-image }} + ${{ inputs.dind-image }}
|
|
# Test plan: ci-flow-values.yaml → test-flow[]
|
|
# ═══════════════════════════════════════════════════════════════
|
|
# build-container:
|
|
# needs: start
|
|
# if: github.ref == 'refs/heads/master'
|
|
# container:
|
|
# image: ${{ inputs.dind-image }} ← KONTTI consumerilta
|
|
# ...
|
|
#
|
|
# deploy:
|
|
# needs: build-container
|
|
# if: github.ref == 'refs/heads/master'
|
|
# ...lukee deployment.* configista...
|
|
#
|
|
# test-flow:
|
|
# needs: deploy
|
|
# if: github.ref == 'refs/heads/master'
|
|
# ...lukee test-flow[] configista... ← TEST PLAN configista
|
|
|
|
# ═══════════════════════════════════════════════════════════════
|
|
# LOPETUS: kaikille yhteinen
|
|
# Raportoi: pending → aggregoi tulokset → success/failure
|
|
# ═══════════════════════════════════════════════════════════════
|
|
end:
|
|
needs: [unit-test, code-coverage]
|
|
if: always()
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Report pipeline ending
|
|
run: |
|
|
bash scripts/report-status.sh \
|
|
"pending" "Pipeline finishing" \
|
|
"$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER" \
|
|
"ci/end"
|
|
|
|
- name: Report final status
|
|
run: |
|
|
BUILD_URL="$GITEA_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_NUMBER"
|
|
if [ "${{ needs.unit-test.result }}" = "success" ] && \
|
|
[ "${{ needs.code-coverage.result }}" = "success" ]; then
|
|
STATE="success"; DESC="All checks passed"
|
|
else
|
|
STATE="failure"; DESC="Some checks failed"
|
|
fi
|
|
bash scripts/report-status.sh "$STATE" "$DESC" "$BUILD_URL" "ci/end"
|
|
```
|
|
|
|
**Moottorin kolme kerrosta käytännössä:**
|
|
|
|
1. **Kontit** (consumerilta `with:` → moottori `inputs:` → job `container.image`)
|
|
- Moottori ei sisällä yhtään kontti-imagea
|
|
- Consumer päättää versiot: `maven:3.9-eclipse-temurin-21`
|
|
|
|
2. **Pipeline runko** (moottorin sisäinen job-graafi)
|
|
- `if: github.ref != 'refs/heads/master'` → feature-skeleton
|
|
- `if: github.ref == 'refs/heads/master'` → master-skeleton (myöhemmät tiketit)
|
|
|
|
3. **Data** (`ci-flow-values.yaml`)
|
|
- `build.ecosystem` → mitä komentoja ajetaan
|
|
- `test-flow[]` → test plan (vain master, tiketti 0009)
|
|
|
|
**Mitä tässä EI ole (myöhemmät tiketit):**
|
|
- `push-reports.sh` — raporttien julkaisu MinIO:hon (tiketti 0013)
|
|
- `isContainerBuild()` / kontin buildaus (tiketti 0009)
|
|
- SonarQube quality gate (tiketti 0009)
|
|
- Test flow -ketjutus `test-flow[]`:sta (tiketti 0008)
|
|
|
|
---
|
|
|
|
## Yhteenveto
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────────────┐
|
|
│ CONSUMER: temperature-store │
|
|
│ │
|
|
│ ci.yml ci-flow-values.yaml │
|
|
│ ┌──────────────────────────┐ ┌────────────────────────┐ │
|
|
│ │ jobs: │ │ build: │ │
|
|
│ │ feature (if !master) ───┼─┐ │ ecosystem: maven │ │
|
|
│ │ with: │ │ │ docker: │ │
|
|
│ │ config-file + kontit │ │ │ imageName: ... │ │
|
|
│ │ │ │ │ test-flow: [...] │ │
|
|
│ │ master (if master) ────┼─┤ └────────────────────────┘ │
|
|
│ │ with: │ │ │
|
|
│ │ config-file + kontit │ │ │
|
|
│ └──────────────────────────┘ │ │
|
|
└───────────────────────────────┼───────────────────────────────────────┘
|
|
│ with: kontit + config-file
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────────┐
|
|
│ PROVIDER: gitea-ci-library (ci-engine.yml) │
|
|
│ │
|
|
│ Kerros 1: KONTIT (consumerilta) │
|
|
│ ┌──────────────────────────────────────────┐ │
|
|
│ │ inputs: maven-image, dind-image, ... │ │
|
|
│ └──────────────────────────────────────────┘ │
|
|
│ │
|
|
│ Kerros 2: PIPELINE RUNKO (moottorin sisäinen) │
|
|
│ ┌──────────────────────────────────────────┐ │
|
|
│ │ FEATURE-SKELETON MASTER-SKELETON │ │
|
|
│ │ start ──────────────────── start │ │
|
|
│ │ ↓ ↓ │ │
|
|
│ │ unit-test build-container│ │
|
|
│ │ ↓ ↓ │ │
|
|
│ │ code-coverage deploy │ │
|
|
│ │ ↓ ↓ │ │
|
|
│ │ end test-flow[] ───│── Kerros 3: TEST PLAN │
|
|
│ │ ↓ │ │
|
|
│ │ end │ │
|
|
│ └──────────────────────────────────────────┘ │
|
|
│ │
|
|
│ Kerros 3: DATA (ci-flow-values.yaml) │
|
|
│ ┌──────────────────────────────────────────┐ │
|
|
│ │ ecosystem → komennot │ │
|
|
│ │ test-flow[] → test plan (vain master) │ │
|
|
│ │ docker.*, deployment.* (vain master) │ │
|
|
│ └──────────────────────────────────────────┘ │
|
|
└──────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Päätöstaulukko
|
|
|
|
| Asia | Sijainti | Kerros |
|
|
|------|----------|:--:|
|
|
| Kontit (`maven-image`, `dind-image`) | Consumer (`ci.yml` `with:`) | 1 |
|
|
| Pipeline runko (job-graafi) | Provider (`ci-engine.yml`) | 2 |
|
|
| `build.ecosystem` → komennot | Consumer (`ci-flow-values.yaml`) | 3 |
|
|
| `test-flow[]` → test plan | Consumer (`ci-flow-values.yaml`) | 3 |
|
|
| `docker.*`, `deployment.*` | Consumer (`ci-flow-values.yaml`) | 3 |
|
|
| Skriptit | Provider (`scripts/`) | — |
|
|
| Tokenit, salasanat | Gitea org secrets | — |
|