Feature/git pages -> oma nginx perusteinen raporointipalvelu (#48)
CI Gitea Reports Main / Config load (push) Successful in 17s
CI Main / Config load (push) Successful in 20s
CI Gitea Reports Main / Latest version (push) Successful in 17s
CI Main / Latest versio (push) Successful in 19s
ci-helm-build-push Helm push 0.2.0
CI Gitea Reports Main / Build & Push Helm chart (push) Successful in 33s
gitops/gitea-ci-library/gitea-reports GitOps: gitea-reports 0.2.0
CI Gitea Reports Main / GitOps (push) Successful in 33s
CI Gitea Reports Main / Report Summary (push) Successful in 3s
CI Main / Bats tests (push) Failing after 1m16s
CI Main / Cucumber tests (push) Failing after 1m22s
CI Main / Build & Push Docker (push) Has been skipped
CI Main / GitOps (push) Has been skipped
CI Main / Move provider version tag (push) Has been skipped
CI Main / Report Summary (push) Successful in 3s
CI Gitea Reports Main / Config load (push) Successful in 17s
CI Main / Config load (push) Successful in 20s
CI Gitea Reports Main / Latest version (push) Successful in 17s
CI Main / Latest versio (push) Successful in 19s
ci-helm-build-push Helm push 0.2.0
CI Gitea Reports Main / Build & Push Helm chart (push) Successful in 33s
gitops/gitea-ci-library/gitea-reports GitOps: gitea-reports 0.2.0
CI Gitea Reports Main / GitOps (push) Successful in 33s
CI Gitea Reports Main / Report Summary (push) Successful in 3s
CI Main / Bats tests (push) Failing after 1m16s
CI Main / Cucumber tests (push) Failing after 1m22s
CI Main / Build & Push Docker (push) Has been skipped
CI Main / GitOps (push) Has been skipped
CI Main / Move provider version tag (push) Has been skipped
CI Main / Report Summary (push) Successful in 3s
Co-authored-by: moilanik <niko.moilanen@tietoevry.com> Reviewed-on: #48
This commit was merged in pull request #48.
This commit is contained in:
@@ -0,0 +1,551 @@
|
||||
# git-pages → Nginx + PV -arkkitehtuuri
|
||||
|
||||
> Päivitetty: 2026-06-28
|
||||
|
||||
## Ongelma
|
||||
|
||||
git-pages storage v2 pakottaa kaikki tiedostot yhteen `.index`-protobufiin per site.
|
||||
Raja on 1MB kovakoodattu. Yksi monorepo täytti tämän, eikä uusia raportteja voi
|
||||
julkaista ennen kuin ongelma on ratkaistu.
|
||||
|
||||
## Ratkaisu
|
||||
|
||||
Poista git-pages kokonaan. Korvaa Nginx:llä (static serving) + kevyellä upload-sidecarilla (tar.gz HTTP PUT → extract). Tiedostot suoraan PV:llä ilman `.index`-välikerrosta.
|
||||
|
||||
## Lukitut päätökset
|
||||
|
||||
1. **git-pages poistuu** — sovellus ei ole mukana luku- eikä kirjoitusketjussa
|
||||
2. **Nginx palvelee suoraan PV:ltä** — `root /data`, `autoindex on`, `try_files $uri $uri/ $uri/index.html =404`
|
||||
3. **Upload-sidecar** — busybox httpd + CGI, ottaa vastaan `PUT /path` + tar.gz body ja purkaa PV:lle
|
||||
4. **URL = FS-polku** — ei Host-header-kikkaa, ei slugia, ei rewritea
|
||||
5. **Traefik** — PATCH/PUT → upload-sidecar:8081 (BasicAuth), GET/HEAD → nginx:8080 (ForwardAuth myöh.)
|
||||
6. **CI-julkaisu** — `curl -X PUT --data-binary @report.tar.gz https://{host}/{owner}/{repo}/{branch_raw}/{sha8}/{suite}/`
|
||||
7. **Branch raakana URL:ssa** — `feature/x` URL:ssa = `feature/x/` FS:ssä. Ei slugitusta
|
||||
8. **Linkki on aina 1-suuntainen** — gitea → raportti. Raportista ei linkkiä giteaan
|
||||
9. **GITEA_API_URL poistuu** publish-skriptin pakollisista env-varista (retention käyttää omaansa)
|
||||
|
||||
## Ympäristömuuttujat
|
||||
|
||||
### GITHUB_REF_NAME
|
||||
|
||||
Gitea Actions runnerin automaattisesti asettama muuttuja, joka on branchin/tagin nimi ilman `refs/heads/`-etuliitettä.
|
||||
|
||||
| Tilanne | Arvo |
|
||||
|---------|------|
|
||||
| branch `main` | `main` |
|
||||
| branch `feature/branch` | `feature/branch` |
|
||||
| tag `v1.0.0` | `v1.0.0` |
|
||||
|
||||
Käytännössä `git rev-parse --abbrev-ref HEAD`. Branchin `/` on sallittu URL-polussa, joten `GITHUB_REF_NAME` voidaan käyttää sellaisenaan rakenteessa `${GITHUB_REF_NAME}/${SHA8}/${SUITE}`. Rinnakkaisissa buildeissa jokainen runner ajaa oman branchinsa kontekstissa → URL pysyy uniikkina.
|
||||
|
||||
```
|
||||
CI → curl -X PUT → Traefik (BasicAuth middleware)
|
||||
→ upload-sidecar:8081 (busybox httpd + CGI)
|
||||
→ tar -xzf - -C /data/{owner}/{repo}/{branch}/{sha8}/{suite}/
|
||||
→ PV /app/data/
|
||||
|
||||
Browser → GET → Traefik
|
||||
→ nginx:8080 (root /app/data)
|
||||
→ /data/{owner}/{repo}/{branch}/{sha8}/{suite}/index.html
|
||||
```
|
||||
|
||||
## Tiedostomuutokset
|
||||
|
||||
### Poistuu (4 tiedostoa)
|
||||
|
||||
| Tiedosto | Miksi |
|
||||
|----------|-------|
|
||||
| `templates/init-job.yaml` | Ei enää git-pages API:a alustettavaksi |
|
||||
| `templates/configmap.yaml` (nykyinen) | Korvataan nginx-conf + CGI |
|
||||
| `templates/deployment.yaml` (nykyinen) | Korvataan uudella |
|
||||
| `git-pages-image` (values.yaml) | Ei enää git-pages-konttia |
|
||||
|
||||
### Muuttuu (8 tiedostoa)
|
||||
|
||||
| # | Tiedosto | Muutos |
|
||||
|---|----------|--------|
|
||||
| 1 | `Chart.yaml` | description, poista appVersion |
|
||||
| 2 | `values.yaml` | Poista git-pages-keyt, lisää nginx/sidecar |
|
||||
| 3 | `dev-values.yaml` | Matchaa uusi values-rakenne |
|
||||
| 4 | `templates/configmap.yaml` | `default.conf` (nginx) + `upload.cgi` |
|
||||
| 5 | `templates/deployment.yaml` | 2 konttia: nginx:alpine (8080) + busybox httpd (8081) |
|
||||
| 6 | `templates/ingressroute.yaml` | Backend: PATCH/PUT → upload:8081, GET/HEAD → nginx:8080 |
|
||||
| 7 | `templates/service.yaml` | 2 porttia: http-read(8080), http-write(8081) |
|
||||
| 8 | `templates/NOTES.txt` | Uusi URL + esimerkit |
|
||||
|
||||
### Pysyy ennallaan (8 tiedostoa)
|
||||
|
||||
| Tiedosto | Huomio |
|
||||
|----------|--------|
|
||||
| `templates/certificate.yaml` | TLS hostille |
|
||||
| `templates/middleware.yaml` | BasicAuth + HTTPS redirect |
|
||||
| `templates/publish-auth-secret.yaml` | Sama BasicAuth-secret |
|
||||
| `templates/pvc.yaml` | Sama PVC |
|
||||
| `templates/retention-configmap.yaml` | Päivitä retention-scriptit |
|
||||
| `templates/retention-cronjob.yaml` | Pieni muutos (ei git-pages API) |
|
||||
| `templates/retention-rbac.yaml` | Ennallaan |
|
||||
| `_helpers.tpl` | Vain label-helpers |
|
||||
|
||||
### Chartin ulkopuolella
|
||||
|
||||
| Tiedosto | Muutos |
|
||||
|----------|--------|
|
||||
| `scripts/publish-git-pages.sh` | Uusi: `curl -X PUT` tar.gz upload-sidecariin |
|
||||
| `scripts/report-status.sh` | URL: `reports/` → `${GITHUB_REF_NAME}/` |
|
||||
| `scripts/ci-report.sh` | URL (rivi 101): `reports/` → `${GITHUB_REF_NAME}/` |
|
||||
| `git-pages/docs/architecture.md` | Uusi arkkitehtuuri |
|
||||
| `docs/design-rationale.md` | Päivitys |
|
||||
| `files/retention-*.sh` | Lue PV:tä suoraan, ei git-pages API:a |
|
||||
|
||||
## Upload CGI
|
||||
|
||||
Busybox httpd välittää CGI-skriptille tiedot ympäristömuuttujissa (`REQUEST_METHOD`, `PATH_INFO`, `CONTENT_LENGTH`). Skripti on tiedosto `/cgi/upload.cgi` — `.cgi`-pääte laukaisee CGI-execution.
|
||||
|
||||
```bash
|
||||
#!/bin/sh
|
||||
# REQUEST_METHOD, PATH_INFO, etc. asettaa busybox httpd automaattisesti
|
||||
|
||||
if [ "$REQUEST_METHOD" = "PUT" ]; then
|
||||
dest="/app/data${PATH_INFO%/}"
|
||||
mkdir -p "$dest"
|
||||
cat - | tar -xzf - -C "$dest"
|
||||
echo "Status: 201 Created"
|
||||
echo ""
|
||||
fi
|
||||
```
|
||||
|
||||
**PUT `https://{host}/{owner}/{repo}/{branch}/{sha8}/{suite}/`** → CGI purkaa:
|
||||
- `PATH_INFO` = `/{owner}/{repo}/{branch}/{sha8}/{suite}/`
|
||||
- `dest` = `/app/data/{owner}/{repo}/{branch}/{sha8}/{suite}`
|
||||
- Tarin tiedostot puretaan tähän
|
||||
|
||||
**Huomio:** Skripti käyttää tar.gz-pakkausta. Jos tar on raaka (ilman gzip:ia), vaihda `-xzf` → `-xf`.
|
||||
|
||||
### CGI-kutsu
|
||||
|
||||
Busybox httpd käynnistetään:
|
||||
|
||||
```bash
|
||||
busybox httpd -f -p 8081 -h /cgi
|
||||
```
|
||||
|
||||
`-h /cgi` on docroot. Skriptitiedoston pääte `.cgi` (esim. `/cgi/upload.cgi`) käynnistää CGI-execution automaattisesti.
|
||||
|
||||
## Nginx-konfiguraatio
|
||||
|
||||
```nginx
|
||||
client_max_body_size 100M; # nousevat raportit; oletus 1M on liian pieni
|
||||
|
||||
server {
|
||||
listen 8080;
|
||||
root /app/data;
|
||||
autoindex on;
|
||||
client_max_body_size 100M;
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ $uri/index.html =404;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Alpine-nginx lukee configit `/etc/nginx/http.d/default.conf`. ConfigMap mountataan tähän polkuun.
|
||||
|
||||
## Deployment (kontit)
|
||||
|
||||
```yaml
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:alpine
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
name: http-read
|
||||
volumeMounts:
|
||||
- name: nginx-conf
|
||||
mountPath: /etc/nginx/http.d
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
- name: upload
|
||||
image: alpine:latest
|
||||
command:
|
||||
- busybox
|
||||
- httpd
|
||||
- -f
|
||||
- -p
|
||||
- "8081"
|
||||
- -h
|
||||
- /cgi
|
||||
ports:
|
||||
- containerPort: 8081
|
||||
name: http-write
|
||||
volumeMounts:
|
||||
- name: cgi-scripts
|
||||
mountPath: /cgi
|
||||
readOnly: true
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
```
|
||||
|
||||
## values.yaml-rakenne
|
||||
|
||||
```yaml
|
||||
# git-pages → Nginx + upload-sidecar
|
||||
|
||||
nameOverride: ""
|
||||
fullnameOverride: ""
|
||||
|
||||
nginx:
|
||||
image: nginx
|
||||
tag: alpine
|
||||
port: 8080
|
||||
bodySize: 100M
|
||||
|
||||
upload:
|
||||
image: alpine
|
||||
tag: latest
|
||||
port: 8081
|
||||
|
||||
service:
|
||||
type: ClusterIP
|
||||
|
||||
persistence:
|
||||
enabled: true
|
||||
accessMode: ReadWriteOnce
|
||||
storageClass: ""
|
||||
size: 5Gi
|
||||
|
||||
ingress:
|
||||
enabled: true
|
||||
host: ci-reports.helm-dev.keskikuja.site
|
||||
entryPoints:
|
||||
websecure: websecure
|
||||
web: web
|
||||
|
||||
certificate:
|
||||
enabled: true
|
||||
issuerRef:
|
||||
name: letsencrypt-prod
|
||||
kind: ClusterIssuer
|
||||
|
||||
publishAuth:
|
||||
create: false
|
||||
htpasswdUsers: ""
|
||||
|
||||
retention:
|
||||
enabled: false
|
||||
mode: cronjob
|
||||
schedule: "0 3 * * *"
|
||||
image:
|
||||
repository: debian
|
||||
tag: bookworm-slim
|
||||
giteaApiUrl: ""
|
||||
rules:
|
||||
branches:
|
||||
default:
|
||||
minAgeDays: 7
|
||||
keepMin: 5
|
||||
```
|
||||
|
||||
## CI-julkaisu (publish-skripti)
|
||||
|
||||
### Muutokset nykyiseen
|
||||
|
||||
| Kohta | Nykyinen | Uusi |
|
||||
|-------|----------|------|
|
||||
| `GITEA_API_URL` | pakollinen env-var | **poistettu** (retention käyttää omaansa) |
|
||||
| Tar-prefiksi | `{owner}/{repo}/reports/{sha8}/{suite}` | ei prefiksiä — `{suite}` |
|
||||
| Paketti | `tar -cf` | `tar -czf` (gzip) |
|
||||
| Content-Type | `application/x-tar` | `application/tar+gz` |
|
||||
| Headerit | `Atomic: no`, `Create-Parents: yes` | poistettu |
|
||||
| Kohde | `PATCH {GIT_PAGES_URL}/` | `PUT {GIT_PAGES_URL}/{owner}/{repo}/{branch}/{sha8}/{suite}/` |
|
||||
| Branch | vain `.meta` | URL:ssa + `.meta` |
|
||||
|
||||
### Uusi skripti (ydin)
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env sh
|
||||
set -eu
|
||||
|
||||
SUITE_PATH="${1:-}"
|
||||
[ -n "$SUITE_PATH" ] || { echo "ERROR: suite_path required" >&2; exit 1; }
|
||||
[ -n "${GIT_PAGES_URL:-}" ] || { echo "ERROR: GIT_PAGES_URL is not set" >&2; exit 1; }
|
||||
[ -n "${REPORTS_PUBLISH_TOKEN:-}" ] || { echo "ERROR: REPORTS_PUBLISH_TOKEN is not set" >&2; exit 1; }
|
||||
[ -n "${GITHUB_REPOSITORY:-}" ] || { echo "ERROR: GITHUB_REPOSITORY is not set" >&2; exit 1; }
|
||||
[ -n "${GITHUB_SHA:-}" ] || { echo "ERROR: GITHUB_SHA is not set" >&2; exit 1; }
|
||||
[ -n "${GITHUB_REF_NAME:-}" ] || { echo "ERROR: GITHUB_REF_NAME is not set" >&2; exit 1; }
|
||||
|
||||
OWNER="${GITHUB_REPOSITORY%%/*}"
|
||||
REPO="${GITHUB_REPOSITORY##*/}"
|
||||
SHA8=$(echo "$GITHUB_SHA" | cut -c1-8)
|
||||
BRANCH="${GITHUB_REF_NAME}"
|
||||
SUITE="${SUITE_PATH%/}"
|
||||
PAGES_USER="${GIT_PAGES_PUBLISH_USER:-publish}"
|
||||
|
||||
[ -d "$SUITE" ] || { echo "ERROR: not a directory: $SUITE" >&2; exit 1; }
|
||||
|
||||
WORK=$(mktemp -d)
|
||||
TAR=$(mktemp)
|
||||
trap 'rm -rf "$WORK" "$TAR"' EXIT
|
||||
|
||||
# Kopioi raporttitiedostot
|
||||
mkdir -p "$WORK/$SUITE"
|
||||
cp -a "$SUITE/." "$WORK/$SUITE/"
|
||||
|
||||
# Generoi index.html (sama logiikka kuin nykyään)
|
||||
cd "$WORK/$SUITE"
|
||||
if [ ! -f "index.html" ]; then
|
||||
# identtinen item-listaus kuin nykyisessä skriptissä
|
||||
…
|
||||
fi
|
||||
|
||||
# .meta tiedosto retentionia varten
|
||||
cat > ".meta" <<EOF
|
||||
{"branch":"${BRANCH}","sha":"${GITHUB_SHA}","published_at":"$(date -u +%Y-%m-%dT%H:%M:%SZ)"}
|
||||
EOF
|
||||
|
||||
cd "$WORK"
|
||||
|
||||
# Pakkaa: pelkät tiedostot ilman $SUITE-etuliitettä
|
||||
# CGI purkaa destinationiin /data/{owner}/{repo}/{branch}/{sha8}/{suite}/
|
||||
# ilman --strip-components, joten tarin menee suoraan oikeaan paikkaan
|
||||
tar czf "$TAR" -C "$WORK/$SUITE" .
|
||||
|
||||
# PUT upload-sidecariin
|
||||
PUBLISH_URL="${GIT_PAGES_URL}/${OWNER}/${REPO}/${BRANCH}/${SHA8}/${SUITE}/"
|
||||
HTTP_CODE=$(curl -sS -X PUT "$PUBLISH_URL" \
|
||||
-u "${PAGES_USER}:${REPORTS_PUBLISH_TOKEN}" \
|
||||
-H "Content-Type: application/tar+gz" \
|
||||
--data-binary @"$TAR" \
|
||||
-o /tmp/git-pages-publish-response.txt \
|
||||
-w "%{http_code}")
|
||||
|
||||
case "$HTTP_CODE" in
|
||||
200|201|204) ;;
|
||||
*)
|
||||
echo "ERROR: publish HTTP ${HTTP_CODE}" >&2
|
||||
cat /tmp/git-pages-publish-response.txt >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "${PUBLISH_URL}"
|
||||
```
|
||||
|
||||
## Gitea commit -linkki (report-status.sh)
|
||||
|
||||
Gitean commit-näkymään tuleva raporttilinkki muodostetaan `scripts/report-status.sh`:ssa.
|
||||
|
||||
### Muutos
|
||||
|
||||
| Kohta | Nykyinen (rivi 21) | Uusi |
|
||||
|-------|-------------------|------|
|
||||
| URL | `${GIT_PAGES_URL}/${GITHUB_REPOSITORY}/reports/${SHA8}/${SUITE}` | `${GIT_PAGES_URL}/${GITHUB_REPOSITORY}/${GITHUB_REF_NAME}/${SHA8}/${SUITE}` |
|
||||
|
||||
`reports/` → `{GITHUB_REF_NAME}/`. Branchin raaka nimi URL-polussa on sallittu. Muu skripti (Gitea API -kutsu, token, json-body) pysyy identtisenä.
|
||||
|
||||
Tarkista: `GITHUB_REF_NAME` on oltava asetettu — Gitea Actions asettaa sen automaattisesti. Jos sitä tarvitaan tarkistuksena, lisätään `report-status.sh`:n env-var-tarkistuksiin.
|
||||
|
||||
## Gitea Actions step summary (report-summary.yml)
|
||||
|
||||
`.gitea/workflows/report-summary.yml` luo GITHUB_STEP_SUMMARY -taulukon raporttilinkeillä.
|
||||
|
||||
### Muutos
|
||||
|
||||
| Kohta | Nykyinen (rivi 28) | Uusi |
|
||||
|-------|-------------------|------|
|
||||
| BASE | `${GIT_PAGES_URL}/${GITHUB_REPOSITORY}/reports/${SHA8}` | `${GIT_PAGES_URL}/${GITHUB_REPOSITORY}/${GITHUB_REF_NAME}/${SHA8}` |
|
||||
|
||||
Sama muuri: `reports/` → `${GITHUB_REF_NAME}/`. Loppuosa identtinen.
|
||||
|
||||
## CI-report (ci-report.sh)
|
||||
|
||||
`scripts/ci-report.sh` rakentaa URL:n suoraan single-entry-tapauksessa (rivi 101).
|
||||
|
||||
### Muutos
|
||||
|
||||
| Kohta | Nykyinen (rivi 101) | Uusi |
|
||||
|-------|-------------------|------|
|
||||
| URL | `${GIT_PAGES_URL}/${GITHUB_REPOSITORY}/reports/${SHA8}/${SUITE}/${SINGLE_ENTRY}` | `${GIT_PAGES_URL}/${GITHUB_REPOSITORY}/${GITHUB_REF_NAME}/${SHA8}/${SUITE}/${SINGLE_ENTRY}` |
|
||||
|
||||
`reports/` → `${GITHUB_REF_NAME}/`. Lisäksi `GITHUB_REF_NAME` pitää lisätä env-var-tarkistuksiin, jos sitä ei ole.
|
||||
|
||||
## Retention
|
||||
|
||||
Retention siirtyy lukemaan PV:tä suoraan (find, ls, stat) git-pages API:n sijaan. Nykyinen malli luki `.git-pages/manifest.json` HTTP:lla — uusi lukee FS:ää.
|
||||
|
||||
### Data
|
||||
|
||||
```
|
||||
/app/data/{owner}/{repo}/{branch}/{sha8}/
|
||||
.meta ← {"branch":"...","sha":"...","published_at":"..."}
|
||||
index.html
|
||||
style.css
|
||||
...
|
||||
```
|
||||
|
||||
### Logiikka commit-tasolla
|
||||
|
||||
Retention on commit-kohtainen. Yksi commit = yksi kansio `/{sha8}/`, jonka alla voi olla useita suiteja (`cucumber/`, `bats/`, jne.).
|
||||
|
||||
```
|
||||
/app/data/{owner}/{repo}/{branch}/
|
||||
abc12345/ ← yksi commit
|
||||
cucumber/
|
||||
index.html
|
||||
.meta
|
||||
bats/
|
||||
index.html
|
||||
.meta
|
||||
def67890/ ← toinen commit
|
||||
cucumber/
|
||||
index.html
|
||||
.meta
|
||||
```
|
||||
|
||||
Toiminta aktiiviselle branchille (löytyy Giteasta):
|
||||
|
||||
1. Listaa commit-kansiot (`abc12345/`, `def67890/`, ...) uusin ensin (viimeksi muokattu)
|
||||
2. Ohita `keepMin` kpl — nämä säilytetään aina
|
||||
3. Jos raportti on alle `minAgeDays` päivää vanha → skip (liian tuore poistettavaksi)
|
||||
4. Muut poistetaan (`rm -rf {sha8}/`), jolloin kaikki commitin suite-tulokset poistuvat kerralla
|
||||
|
||||
Poistuneelle branchille (404 Giteasta): poista koko `{branch}/`-kansio.
|
||||
|
||||
### Parametrit
|
||||
|
||||
```yaml
|
||||
retention:
|
||||
rules:
|
||||
branches:
|
||||
default:
|
||||
keepMin: 5
|
||||
minAgeDays: 7
|
||||
main:
|
||||
keepMin: 20
|
||||
minAgeDays: 14
|
||||
```
|
||||
|
||||
**Parametrin nimi:** `maxAgeDays` → `minAgeDays` (arvo sama, uusi nimi kertoo mitä se todella tekee: minimi-ikä ennen poistoa).
|
||||
|
||||
### Skripti (ydin)
|
||||
|
||||
```bash
|
||||
DATA_ROOT="/app/data"
|
||||
|
||||
for owner_dir in "$DATA_ROOT"/*/; do
|
||||
owner=$(basename "$owner_dir")
|
||||
for repo_dir in "$owner_dir"*/; do
|
||||
repo=$(basename "$repo_dir")
|
||||
for branch_dir in "$repo_dir"*/; do
|
||||
branch=$(basename "$branch_dir")
|
||||
branch_urlenc=$(echo "$branch" | sed 's/\//%2F/g')
|
||||
|
||||
# Tarkista onko branch yhä olemassa Giteassa
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||
-H "Authorization: token $GITEA_TOKEN" \
|
||||
"$GITEA_API_URL/api/v1/repos/$owner/$repo/branches/$branch_urlenc")
|
||||
|
||||
if [ "$HTTP_CODE" = "404" ]; then
|
||||
# Branch poistettu — tuhoa koko branch
|
||||
rm -rf "$branch_dir"
|
||||
continue
|
||||
fi
|
||||
|
||||
# Aktiivinen branch — hae säännöt
|
||||
rules=$(jq -r ".branches.\"$branch\" // .branches.default" "$RETENTION_CONFIG")
|
||||
keep_min=$(echo "$rules" | jq -r '.keepMin // 5')
|
||||
min_age=$(echo "$rules" | jq -r '.minAgeDays // 7')
|
||||
|
||||
# Listaa commit-kansiot uusin ensin
|
||||
commits=$(ls -1t "$branch_dir")
|
||||
count=0
|
||||
for sha8 in $commits; do
|
||||
[ -d "$branch_dir/$sha8" ] || continue
|
||||
count=$((count + 1))
|
||||
|
||||
# Ohita keepMin kpl
|
||||
[ "$count" -le "$keep_min" ] && continue
|
||||
|
||||
# Tarkista ikä
|
||||
age_seconds=$(( $(date +%s) - $(stat -c %Y "$branch_dir/$sha8") ))
|
||||
age_days=$(( age_seconds / 86400 ))
|
||||
[ "$age_days" -lt "$min_age" ] && continue
|
||||
|
||||
# Poista koko commit
|
||||
rm -rf "$branch_dir/$sha8"
|
||||
done
|
||||
done
|
||||
done
|
||||
done
|
||||
```
|
||||
|
||||
## Deploy-järjestys (vaiheistus)
|
||||
|
||||
### Vaiheet
|
||||
|
||||
| # | Vaihe | Mitä | Riippuvuus |
|
||||
|---|-------|------|-----------|
|
||||
| 1 | **Poista vanha PVC** | `kubectl delete pvc git-pages-data -n gitea-reports` (tai helm uninstall + uusi asennus) | Vanha git-pages-data .index-muodossa — tuhotaan |
|
||||
| 2 | **Deployaa uusi chart** | `helm upgrade --install gitea-reports ./gitea-reports -n gitea-reports -f dev-values.yaml` | Uusi PVC luodaan tyhjänä. Vanhoja raportteja ei ole. |
|
||||
| 3 | **Varmenna Nginx** | `curl localhost:8080/` → 404/autoindex | Nginx palvelee |
|
||||
| 4 | **Varmenna upload** | `curl -X PUT -T test.tar.gz localhost:8081/test/` → 201 | CGI toimii |
|
||||
| 5 | **Päivitä provider-skriptit** | `scripts/publish-git-pages.sh`, `scripts/report-status.sh`, `scripts/ci-report.sh` | Uusi URL-muoto |
|
||||
| 6 | **Päivitä provider-workflow** | `.gitea/workflows/report-summary.yml` | BASE-muuttuja |
|
||||
| 7 | **Varmenna CI-julkaisu** | `bash publish-git-pages.sh <test-suite>` → 201 + URL | Koko ketju |
|
||||
|
||||
### Consumer-workflowt EI päivitetä
|
||||
|
||||
`.gitea/workflows/example-*` ja `.ci/scripts/*` (consumerien kopioimat skriptit) eivät muutu.
|
||||
|
||||
### report-summary.yml
|
||||
|
||||
On provider-workflow (ei `example-`-prefiksiä). Saa päivittyä.
|
||||
|
||||
## Rollback-suunnitelma
|
||||
|
||||
### Jos chart epäonnistuu
|
||||
|
||||
```bash
|
||||
helm rollback git-pages <revision>
|
||||
git revert HEAD # tai git reset --hard ennen script-muutoksia
|
||||
```
|
||||
|
||||
- Vanha PVC on poistettu → rollback palauttaa vanhan git-pages-deploymentin mutta ilman dataa
|
||||
- Vanhaa `.index`-dataa ei palaudu — se tuhottiin vaiheessa 1
|
||||
- Jos uutta dataa on ehtinyt kertyä, se on uudella PVC:llä → vanha deployment ei löydä sitä
|
||||
|
||||
### Suojaus
|
||||
|
||||
Pidä vanha PVC hengessä (nimeä uudelleen, älä poista) kunnes rollback-ikkuna on umpeutunut:
|
||||
|
||||
```bash
|
||||
# Ennen vaihetta 1: uudelleennimeä vanha PVC
|
||||
kubectl delete pvc git-pages-data -n gitea-reports --wait=false
|
||||
kubectl patch pvc git-pages-data -n gitea-reports -p '{"metadata":{"finalizers":[]}}' --type merge # tarvittaessa
|
||||
```
|
||||
|
||||
Jos rollback tarvitaan 24h sisällä, palauta vanha PVC backupista (esim. snapshot).
|
||||
|
||||
## Varmennus (check-stepit)
|
||||
|
||||
Jokaiselle komponentille check ennen seuraavaan vaiheeseen siirtymistä:
|
||||
|
||||
| Vaihe | Check |
|
||||
|-------|-------|
|
||||
| **helm template** | `helm template gitea-reports ./gitea-reports -n gitea-reports -f dev-values.yaml` → deploymentissa 2 konttia (nginx + upload) |
|
||||
| **Nginx** | `kubectl exec <pod> -- wget -qO- http://localhost:8080/` → 404 tai autoindex (ei connection refused) |
|
||||
| **Upload CGI** | `echo "hello" \| tar czf /tmp/test.tar.gz -C /tmp . && curl -sS -X PUT -H "Content-Type: application/tar+gz" --data-binary @/tmp/test.tar.gz http://localhost:8081/test/hello/` → status 201. Sitten `kubectl exec <pod> -- ls /app/data/test/hello/` → tiedostot näkyvissä |
|
||||
| **Retention** | `kubectl exec <pod> -- bash /scripts/retention-run.sh` → exit 0, ei virheitä |
|
||||
| **Provider-skripti (end-to-end)** | 1. `kubectl port-forward pod/<pod> 8080:8080 8081:8081` 2. `bash scripts/publish-git-pages.sh <test-suite>` → output URL (`https://{host}/{owner}/{repo}/{branch}/{sha8}/{suite}/`) 3. `curl http://localhost:8080/{owner}/{repo}/{branch}/{sha8}/{suite}/` → 200, HTML näkyy 4. `curl -I http://localhost:8080/{owner}/{repo}/{branch}/{sha8}/{suite}/index.html` → 200 |
|
||||
|
||||
## Data migration
|
||||
|
||||
**Ei migraatiota.** Vanha PVC sisältää git-pages storage v2 -dataa (`.index` protobuf). Tämä data tuhotaan tarkoituksella:
|
||||
|
||||
1. CI on rikki 1MB rajoitteen takia — uusia raportteja ei voi julkaista ennen kuin git-pages on poistettu
|
||||
2. Vanhat raportit ovat vanhentuneita (branchit on jo saatettu poistaa, commitit vanhoja)
|
||||
3. Uudet buildit tuottavat raportit uuteen järjestelmään ilman migraatiota
|
||||
|
||||
**Toimenpide:**
|
||||
- Poista vanha PVC ennen uuden chartin deployausta
|
||||
- Uusi chart luo uuden tyhjän PVC:n
|
||||
- CI ajaa uudet buildit → raportit kirjoitetaan Nginx+PV-malliin
|
||||
Reference in New Issue
Block a user