Feature/git pages -> oma nginx perusteinen raporointipalvelu #48
@@ -67,7 +67,7 @@ Browser → GET → Traefik
|
|||||||
| 1 | `Chart.yaml` | description, poista appVersion |
|
| 1 | `Chart.yaml` | description, poista appVersion |
|
||||||
| 2 | `values.yaml` | Poista git-pages-keyt, lisää nginx/sidecar |
|
| 2 | `values.yaml` | Poista git-pages-keyt, lisää nginx/sidecar |
|
||||||
| 3 | `dev-values.yaml` | Matchaa uusi values-rakenne |
|
| 3 | `dev-values.yaml` | Matchaa uusi values-rakenne |
|
||||||
| 4 | `templates/configmap.yaml` | `nginx.conf` + `upload.sh` (CGI) |
|
| 4 | `templates/configmap.yaml` | `default.conf` (nginx) + `upload.cgi` |
|
||||||
| 5 | `templates/deployment.yaml` | 2 konttia: nginx:alpine (8080) + busybox httpd (8081) |
|
| 5 | `templates/deployment.yaml` | 2 konttia: nginx:alpine (8080) + busybox httpd (8081) |
|
||||||
| 6 | `templates/ingressroute.yaml` | Backend: PATCH/PUT → upload:8081, GET/HEAD → nginx:8080 |
|
| 6 | `templates/ingressroute.yaml` | Backend: PATCH/PUT → upload:8081, GET/HEAD → nginx:8080 |
|
||||||
| 7 | `templates/service.yaml` | 2 porttia: http-read(8080), http-write(8081) |
|
| 7 | `templates/service.yaml` | 2 porttia: http-read(8080), http-write(8081) |
|
||||||
@@ -96,13 +96,50 @@ Browser → GET → Traefik
|
|||||||
| `docs/design-rationale.md` | Päivitys |
|
| `docs/design-rationale.md` | Päivitys |
|
||||||
| `files/retention-*.sh` | Lue PV:tä suoraan, ei git-pages API:a |
|
| `files/retention-*.sh` | Lue PV:tä suoraan, ei git-pages API:a |
|
||||||
|
|
||||||
## Nginx-konfiguraatio (ydin)
|
## Upload CGI
|
||||||
|
|
||||||
|
Busybox httpd välittää CGI-skriptille tiedot ympäristömuuttujissa (`REQUEST_METHOD`, `PATH_INFO`, `CONTENT_LENGTH`). Skripti on tiedosto `/cgi/upload.cgi` — `.cgi`-pääte laukaisee CGI-execution.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
#!/bin/sh
|
||||||
|
# REQUEST_METHOD, PATH_INFO, etc. asettaa busybox httpd automaattisesti
|
||||||
|
|
||||||
|
if [ "$REQUEST_METHOD" = "PUT" ]; then
|
||||||
|
dest="/app/data${PATH_INFO%/}"
|
||||||
|
mkdir -p "$dest"
|
||||||
|
cat - | tar -xzf - -C "$dest"
|
||||||
|
echo "Status: 201 Created"
|
||||||
|
echo ""
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
|
||||||
|
**PUT `https://{host}/{owner}/{repo}/{branch}/{sha8}/{suite}/`** → CGI purkaa:
|
||||||
|
- `PATH_INFO` = `/{owner}/{repo}/{branch}/{sha8}/{suite}/`
|
||||||
|
- `dest` = `/app/data/{owner}/{repo}/{branch}/{sha8}/{suite}`
|
||||||
|
- Tarin tiedostot puretaan tähän
|
||||||
|
|
||||||
|
**Huomio:** Skripti käyttää tar.gz-pakkausta. Jos tar on raaka (ilman gzip:ia), vaihda `-xzf` → `-xf`.
|
||||||
|
|
||||||
|
### CGI-kutsu
|
||||||
|
|
||||||
|
Busybox httpd käynnistetään:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
busybox httpd -f -p 8081 -h /cgi
|
||||||
|
```
|
||||||
|
|
||||||
|
`-h /cgi` on docroot. Skriptitiedoston pääte `.cgi` (esim. `/cgi/upload.cgi`) käynnistää CGI-execution automaattisesti.
|
||||||
|
|
||||||
|
## Nginx-konfiguraatio
|
||||||
|
|
||||||
```nginx
|
```nginx
|
||||||
|
client_max_body_size 100M; # nousevat raportit; oletus 1M on liian pieni
|
||||||
|
|
||||||
server {
|
server {
|
||||||
listen 8080;
|
listen 8080;
|
||||||
root /app/data;
|
root /app/data;
|
||||||
autoindex on;
|
autoindex on;
|
||||||
|
client_max_body_size 100M;
|
||||||
|
|
||||||
location / {
|
location / {
|
||||||
try_files $uri $uri/ $uri/index.html =404;
|
try_files $uri $uri/ $uri/index.html =404;
|
||||||
@@ -110,21 +147,7 @@ server {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
## Upload CGI (ydin)
|
Alpine-nginx lukee configit `/etc/nginx/http.d/default.conf`. ConfigMap mountataan tähän polkuun.
|
||||||
|
|
||||||
```bash
|
|
||||||
#!/bin/sh
|
|
||||||
read -r method uri _
|
|
||||||
case "$method" in
|
|
||||||
PUT)
|
|
||||||
dest="/app/data${uri%/*}" # strip filename
|
|
||||||
mkdir -p "$dest"
|
|
||||||
cat - | tar -xzf - -C "$dest"
|
|
||||||
echo "Status: 201 Created"
|
|
||||||
echo ""
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
```
|
|
||||||
|
|
||||||
## Deployment (kontit)
|
## Deployment (kontit)
|
||||||
|
|
||||||
@@ -137,12 +160,19 @@ containers:
|
|||||||
name: http-read
|
name: http-read
|
||||||
volumeMounts:
|
volumeMounts:
|
||||||
- name: nginx-conf
|
- name: nginx-conf
|
||||||
mountPath: /etc/nginx/conf.d
|
mountPath: /etc/nginx/http.d
|
||||||
- name: data
|
- name: data
|
||||||
mountPath: /app/data
|
mountPath: /app/data
|
||||||
- name: upload
|
- name: upload
|
||||||
image: alpine:latest
|
image: alpine:latest
|
||||||
command: ["busybox", "httpd", "-f", "-p", "8081", "-h", "/cgi"]
|
command:
|
||||||
|
- busybox
|
||||||
|
- httpd
|
||||||
|
- -f
|
||||||
|
- -p
|
||||||
|
- "8081"
|
||||||
|
- -h
|
||||||
|
- /cgi
|
||||||
ports:
|
ports:
|
||||||
- containerPort: 8081
|
- containerPort: 8081
|
||||||
name: http-write
|
name: http-write
|
||||||
@@ -154,6 +184,66 @@ containers:
|
|||||||
mountPath: /app/data
|
mountPath: /app/data
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## values.yaml-rakenne
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# git-pages → Nginx + upload-sidecar
|
||||||
|
|
||||||
|
nameOverride: ""
|
||||||
|
fullnameOverride: ""
|
||||||
|
|
||||||
|
nginx:
|
||||||
|
image: nginx
|
||||||
|
tag: alpine
|
||||||
|
port: 8080
|
||||||
|
bodySize: 100M
|
||||||
|
|
||||||
|
upload:
|
||||||
|
image: alpine
|
||||||
|
tag: latest
|
||||||
|
port: 8081
|
||||||
|
|
||||||
|
service:
|
||||||
|
type: ClusterIP
|
||||||
|
|
||||||
|
persistence:
|
||||||
|
enabled: true
|
||||||
|
accessMode: ReadWriteOnce
|
||||||
|
storageClass: ""
|
||||||
|
size: 5Gi
|
||||||
|
|
||||||
|
ingress:
|
||||||
|
enabled: true
|
||||||
|
host: ci-reports.helm-dev.keskikuja.site
|
||||||
|
entryPoints:
|
||||||
|
websecure: websecure
|
||||||
|
web: web
|
||||||
|
|
||||||
|
certificate:
|
||||||
|
enabled: true
|
||||||
|
issuerRef:
|
||||||
|
name: letsencrypt-prod
|
||||||
|
kind: ClusterIssuer
|
||||||
|
|
||||||
|
publishAuth:
|
||||||
|
create: false
|
||||||
|
htpasswdUsers: ""
|
||||||
|
|
||||||
|
retention:
|
||||||
|
enabled: false
|
||||||
|
mode: cronjob
|
||||||
|
schedule: "0 3 * * *"
|
||||||
|
image:
|
||||||
|
repository: debian
|
||||||
|
tag: bookworm-slim
|
||||||
|
giteaApiUrl: ""
|
||||||
|
rules:
|
||||||
|
branches:
|
||||||
|
default:
|
||||||
|
maxAgeDays: 90
|
||||||
|
keepMin: 5
|
||||||
|
```
|
||||||
|
|
||||||
## CI-julkaisu (publish-skripti)
|
## CI-julkaisu (publish-skripti)
|
||||||
|
|
||||||
### Muutokset nykyiseen
|
### Muutokset nykyiseen
|
||||||
@@ -200,19 +290,23 @@ mkdir -p "$WORK/$SUITE"
|
|||||||
cp -a "$SUITE/." "$WORK/$SUITE/"
|
cp -a "$SUITE/." "$WORK/$SUITE/"
|
||||||
|
|
||||||
# Generoi index.html (sama logiikka kuin nykyään)
|
# Generoi index.html (sama logiikka kuin nykyään)
|
||||||
cd "$WORK"
|
cd "$WORK/$SUITE"
|
||||||
if [ ! -f "$SUITE/index.html" ]; then
|
if [ ! -f "index.html" ]; then
|
||||||
# identtinen item-listaus kuin nykyisessä skriptissä
|
# identtinen item-listaus kuin nykyisessä skriptissä
|
||||||
…
|
…
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# .meta tiedosto retentionia varten
|
# .meta tiedosto retentionia varten
|
||||||
cat > "$SUITE/.meta" <<EOF
|
cat > ".meta" <<EOF
|
||||||
{"branch":"${BRANCH}","sha":"${GITHUB_SHA}","published_at":"$(date -u +%Y-%m-%dT%H:%M:%SZ)"}
|
{"branch":"${BRANCH}","sha":"${GITHUB_SHA}","published_at":"$(date -u +%Y-%m-%dT%H:%M:%SZ)"}
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
# Pakkaa
|
cd "$WORK"
|
||||||
tar czf "$TAR" -C "$WORK" "$SUITE"
|
|
||||||
|
# Pakkaa: pelkät tiedostot ilman $SUITE-etuliitettä
|
||||||
|
# CGI purkaa destinationiin /data/{owner}/{repo}/{branch}/{sha8}/{suite}/
|
||||||
|
# ilman --strip-components, joten tarin menee suoraan oikeaan paikkaan
|
||||||
|
tar czf "$TAR" -C "$WORK/$SUITE" .
|
||||||
|
|
||||||
# PUT upload-sidecariin
|
# PUT upload-sidecariin
|
||||||
PUBLISH_URL="${GIT_PAGES_URL}/${OWNER}/${REPO}/${BRANCH}/${SHA8}/${SUITE}/"
|
PUBLISH_URL="${GIT_PAGES_URL}/${OWNER}/${REPO}/${BRANCH}/${SHA8}/${SUITE}/"
|
||||||
@@ -263,12 +357,54 @@ Sama muuri: `reports/` → `${GITHUB_REF_NAME}/`. Loppuosa identtinen.
|
|||||||
|
|
||||||
## Retention
|
## Retention
|
||||||
|
|
||||||
Retention siirtyy lukemaan PV:tä suoraan (find, ls, stat) git-pages API:n sijaan:
|
Retention siirtyy lukemaan PV:tä suoraan (find, ls, stat) git-pages API:n sijaan. Nykyinen malli luki `.git-pages/manifest.json` HTTP:lla — uusi lukee FS:ää.
|
||||||
|
|
||||||
1. Listaa kaikki kansiot `/app/data/{owner}/{repo}/{branch}/{sha8}/`
|
### Data
|
||||||
2. Lue `.meta` jokaisesta
|
|
||||||
3. Tarkista branchin olemassaolo Gitea API:lla
|
```
|
||||||
4. Poista vanhat/kuolleet branchit kansiorakenteesta (`rm -rf`)
|
/app/data/{owner}/{repo}/{branch}/{sha8}/
|
||||||
|
.meta ← {"branch":"...","sha":"...","published_at":"..."}
|
||||||
|
index.html
|
||||||
|
style.css
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
### Logiikka
|
||||||
|
|
||||||
|
```bash
|
||||||
|
DATA_ROOT="/app/data"
|
||||||
|
|
||||||
|
for owner_dir in "$DATA_ROOT"/*/; do
|
||||||
|
owner=$(basename "$owner_dir")
|
||||||
|
for repo_dir in "$owner_dir"*/; do
|
||||||
|
repo=$(basename "$repo_dir")
|
||||||
|
for branch_dir in "$repo_dir"*/; do
|
||||||
|
branch=$(basename "$branch_dir")
|
||||||
|
|
||||||
|
# Tarkista onko branch yhä olemassa Giteassa
|
||||||
|
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||||
|
-H "Authorization: token $GITEA_TOKEN" \
|
||||||
|
"$GITEA_API_URL/api/v1/repos/$owner/$repo/branches/$(echo "$branch" | sed 's/\//%2F/g')")
|
||||||
|
|
||||||
|
if [ "$HTTP_CODE" = "404" ]; then
|
||||||
|
# Branch kuollut — poista kaikki
|
||||||
|
rm -rf "$branch_dir"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Aktiivinen branch — retention-säännöt
|
||||||
|
for sha8_dir in "$branch_dir"*/; do
|
||||||
|
[ -d "$sha8_dir" ] || continue
|
||||||
|
age=$(( ( $(date +%s) - $(stat -c %Y "$sha8_dir") ) / 86400 ))
|
||||||
|
# maxAgeDays + keepMin (sama logiikka kuin nykyään)
|
||||||
|
…
|
||||||
|
done
|
||||||
|
done
|
||||||
|
done
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
**Tarkennettava:** Branch-nimen erikoismerkit (`feature/x`) pitää URL-enkoodata Gitea API -kutsuun (`%2F`), koska branch voi sisältää `/`.
|
||||||
|
|
||||||
## Avoimet asiat (ei tässä MVP:ssä)
|
## Avoimet asiat (ei tässä MVP:ssä)
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user