# dotnews-paas

Minimal self-hosted deployment platform (a very basic Coolify/Vercel), Docker-free:

```
Internet → OPNsense (NAT 80/443 → VM)
  → Caddy :443        auto-TLS, redirects, serves static assets directly
      ├─ cached apps  → Varnish :6081 (page cache, purge-on-publish) → app port
      ├─ other apps   → app port directly
      └─ deploy panel → PM2 app (deploy.younex.de)
  PM2 · one Node app per port (3001+) · deploys via panel webhooks, Jenkins or GitLab CI
```

Everything is conventions + one bash CLI (`paas`) + a small web panel. The CLI
is the only thing that writes server state; the panel and all CI paths call it.

Two app runtimes, chosen per project at creation (`RUNTIME=` in the registry):

| | `node` (default) | `laravel` (servers bootstrapped with `WITH_LARAVEL=1`) |
|---|---|---|
| Build | `npm ci && npm run build` → `.output` | `composer install --no-dev … && npm run build` → whole checkout |
| Runtime | PM2 process on `127.0.0.1:PORT` | PHP-FPM pool `paas-<app>` (unix socket), Caddy `php_fastcgi` |
| Edge cache | optional Varnish | not available (VCL strips cookies → sessions/CSRF) |
| Health-gate | release booted on `PORT+1000` | release served via a `staging` symlink on `PORT+1000` through the real pool |
| Extras | — | migrations on deploy, scheduler timer, PM2 queue worker, `paas db` MariaDB |

## Install (fresh Debian 12 VM)

```bash
# 1. from the workstation: create the VM on Proxmox
PROXMOX_HOST=root@<proxmox> TEMPLATE_VMID=9000 NEW_VMID=210 \
  VM_IP_CIDR=192.168.12.80/24 VM_GW=192.168.12.1 \
  SSH_PUBKEY_FILE=~/.ssh/id_ed25519.pub infra/provision-vm.sh

# 2. copy the repo over and bootstrap
rsync -a ./ debian@192.168.12.80:dotnews-paas/
ssh debian@192.168.12.80 "cd dotnews-paas && sudo PANEL_DOMAIN=deploy.younex.de PANEL_ADMIN_PASSWORD='...' ./bootstrap.sh"
#    Laravel-capable server (adds PHP-FPM 8.3/8.4, composer, MariaDB, timers):
#    sudo WITH_LARAVEL=1 PANEL_DOMAIN=deploy-php.younex.de PANEL_ADMIN_PASSWORD='...' ./bootstrap.sh

# 3. OPNsense: NAT 80/443 from a free virtual IP to the VM (infra/opnsense-notes.md)
# 4. DNS: point the panel + app domains at that virtual IP
```

## New app in 3 steps

1. **Create** — in the panel (`https://deploy.younex.de` → New project) or CLI:
   `paas create myapp --domains www.example.com --cache`
2. **DNS** — point the domain at the virtual IP. Caddy fetches certificates
   automatically.
3. **Connect the repo** — paste the project's webhook URL (shown on the project
   page) into GitHub/GitLab/Gitea webhook settings. Every push to the
   configured branch builds and deploys. Alternatively use `ci/deploy.yml`
   (GitLab CI) or `ci/Jenkinsfile.template` (Jenkins) — both just call
   `paas deploy` over SSH, and all deployments show up in the panel dashboard.
   The platform itself (this repo) deploys to production the same way: a push
   to `main` runs the Jenkins job `paas-platform`, see `docs/ci-platform-deploy.md`.

## Daily operations

| Action | Panel | CLI |
|---|---|---|
| Deploy | "Deploy now" button / git push | `paas deploy <app> <output-dir>` |
| Rollback | "Rollback" button | `paas rollback <app>` |
| Logs | project → Logs | `paas logs <app>` |
| Purge cache | — | `paas ban <app>` |
| Warm cache (pre-render WARM_PATHS) | — | `paas warm <app>` |
| Domains | — | `paas domains <app> --add/--remove <d>` |
| Status | dashboard | `paas list` |
| Restart / apply env | "Restart app" | `paas restart <app>` |
| Database | project → Database card | `paas db create\|show\|drop\|backup\|shell <app>` |
| PHP version | project → Runtime | `paas php <app> --set 8.4` |
| Laravel switches | project → Laravel card | `paas set <app> --migrate\|--queue\|--scheduler on\|off` |
| Repair configs | — | `paas regen` |

## How deploys work (safety model)

`paas deploy` copies the build into a new release dir, boots it on a **test
port** and only if it answers its health check does the `current` symlink flip
and PM2 restart — a failing build or broken release can never take the running
version down. After activation the previous release is kept (last 5) for
one-command rollback. For cached apps the deploy ends with a Varnish purge.

## Laravel apps (`--runtime laravel`)

Only on servers bootstrapped with `WITH_LARAVEL=1` (`RUNTIMES=node,laravel` in
`/srv/paas/paas.conf` is what the panel and CLI gate on). Everything else —
domains, TLS, geo/domain rules, basic-auth, traffic metrics, Checkmk, ZIP
deploys, webhooks — works exactly as for Node apps.

- **Serving**: Caddy `php_fastcgi` → per-app PHP-FPM pool (`/srv/paas/php/<ver>/paas-<app>.conf`,
  user `deploy`, socket `/run/php/paas-<app>.sock`), `public/` served by Caddy's
  `file_server`, `/build/*` immutable. The loopback `http://127.0.0.1:PORT` listener
  is what health probes/monitoring hit (same contract as PM2 apps).
- **Releases**: `paas deploy <app> <checkout>` copies the *built* checkout
  (needs `vendor/` + `public/`), links `storage/` and `.env` from `shared/`,
  runs `config|route|view|event:cache` (fatal = release can't boot), health-gates
  it through the real pool (`staging` symlink, port `PORT+1000`), runs
  `migrate --force` (toggle `MIGRATE`), flips `current`. No PHP-FPM reload per
  deploy: Caddy's `resolve_root_symlink` makes opcache key on the real release
  path. Rollback = symlink flip (migrations are not reversed).
- **Env**: `shared/.env` is the Laravel `.env` (symlinked into every release).
  `paas create` seeds `APP_KEY`, `APP_ENV=production`, `APP_DEBUG=false`,
  `APP_URL`, `LOG_CHANNEL=daily`. Because of `config:cache`, env edits need
  `paas restart <app>` (rebuilds the cache, reloads fpm) or a deploy.
- **Health**: default `HEALTH_PATH=/up` (Laravel 11+). Older apps: `--health /`.
  The gate runs *before* migrations — keep it independent of new schema.
- **Scheduler**: `paas set <app> --scheduler on` → the `paas-laravel-scheduler`
  timer runs `schedule:run` every minute (log `storage/logs/scheduler.log`).
- **Queue**: `paas set <app> --queue on` → PM2 process `<app>-queue` running
  `QUEUE_CMD` (`--queue-cmd` to change); deploys send `queue:restart`.
- **PHP versions**: `paas php <app> --set 8.4` moves the pool (socket path stays).
  Versions installed: `PHP_VERSIONS` at bootstrap (`/srv/paas/php/bin/<ver>/php` shims).
- **Logs**: `paas logs <app>` tails `storage/logs/{laravel*,php-fpm,scheduler}.log`.

## Databases (`paas db`, MariaDB — any runtime)

`WITH_LARAVEL=1` installs MariaDB bound to `127.0.0.1` and a `paas_admin` user
whose credentials live in `/srv/paas/db-admin.cnf` (0600 `deploy`; no sudo
involved). Per app: `paas db create <app>` makes database + user
`<app_with_underscores>` (utf8mb4), writes `DB_CONNECTION/DB_HOST/DB_PORT/
DB_DATABASE/DB_USERNAME/DB_PASSWORD` (Node apps also `DATABASE_URL`) into
`shared/.env` — existing differing keys are kept unless `--force-env` — and is
idempotent (re-run = show). `show`, `shell`, `backup` (`mariadb-dump` →
`/srv/paas/backups/db/<app>-<ts>.sql.gz`, last 7 kept; nightly timer runs
`backup --all`), `drop --yes` (backs up first). `paas remove --purge` drops the
database after a final dump. From a workstation:
`ssh -L 3306:127.0.0.1:3306 deploy@<server>` then connect to `localhost:3306`.

## Caching (opt-in per app, `--cache`)

Varnish sits between Caddy and the app. Policy (templates/base.vcl):

- TTLs come from the app: `Cache-Control: s-maxage=…, stale-while-revalidate=…`
  (no header → 120 s micro-cache). Browsers always get `no-cache` — the edge
  absorbs revalidation.
- Only GET/HEAD are cached; POST always passes; other methods → 405.
- 5xx never enters the cache — stale content is served instead (up to 72 h,
  168 h if the backend is down entirely).
- Tracking params (`utm_*`, `gclid`, `fbclid`, …) are stripped from the cache
  key; remaining query params are sorted.
- Purge: `paas ban <app>`, or from an app on the box:
  `curl -X BAN -H "X-Ban-Host: www.example.com" http://127.0.0.1:6081/`
  (localhost only). Wire a CMS publish-webhook to an app route that calls this
  and long TTLs stay instantly fresh.
- Deploy/rollback bans send `X-Ban-Path: ^/(?!api/img)` — everything for the
  host is purged EXCEPT `/api/img` objects (content-hashed immutable images;
  re-encoding them after every deploy was pure waste). For apps without an
  `/api/img` route the regex is equivalent to a full host ban.
- Per-app never-cache routes: `--pass-paths "<regex>"` at create time.
- Warming: deploys/rollbacks end with `paas warm` (fetches `WARM_PATHS`, default `/`,
  through Varnish so the cache is HIT-ready). With `WARM_SITEMAP=/news-sitemap.xml`
  (`--warm-sitemap`, set automatically by the panel's multi-label preset) the warm
  additionally fetches that sitemap and pre-renders up to `WARM_SITEMAP_LIMIT`
  (default 50) of its `<loc>` URLs on the primary domain — so recent articles are
  HIT-ready again right after the deploy ban. CMS publish-webhooks can warm a richer
  list app-side (see the dotnews `_hooks/cms` receiver: homepage, changed entry,
  latest articles).

## Layout on the server

```
/usr/local/bin/paas             the CLI
/srv/paas/apps.d/<app>.conf     app registry (edit + `paas regen`)
/srv/paas/caddy/*.caddy         generated Caddy site blocks
/srv/paas/vcl/paas-*.vcl        generated Varnish backends/routing
/srv/paas/panel/                the web panel (PM2: paas-panel)
/srv/paas/builds/<app>/         panel/CI build workspaces
/srv/paas/paas.db               sqlite: projects + deployment history
/srv/apps/<app>/releases/       one dir per deploy
/srv/apps/<app>/current         symlink to the live release
/srv/apps/<app>/shared/.env     app environment (Laravel: the app's .env, symlinked per release)
/srv/apps/<app>/static/         public/ assets, served by Caddy (accumulating; node apps)
/srv/apps/<app>/shared/storage/ Laravel storage/ (shared across releases)
/srv/paas/php/<ver>/*.conf      generated PHP-FPM pools (laravel servers)
/srv/paas/db-admin.cnf          MariaDB admin creds for 'paas db' (laravel servers)
/srv/paas/backups/db/           nightly per-app database dumps
```

## Further features

- **System domains**: every `paas create` generates `<app>-<rand>.<SYSTEM_DOMAIN_BASE>`
  (wildcard DNS → this server; always available, forced noindex). Base in
  `/srv/paas/paas.conf`.
- **Redirect domains**: `paas domains <app> --add-redirect example.com` → 308 to
  the primary domain (or via the panel's Domains card).
- **Geo-blocking**: per-path country rules (block / allow-only) via the panel's
  rule editor (searchable flag picker, with a Clone button to reuse a rule's
  countries for a different path) or `paas geo <app> --set-json -`. Enforced
  in Caddy before the cache; needs the xcaddy build (maxmind module — see
  bootstrap output) + DB-IP mmdb (auto-refreshed monthly). Internal/unknown IPs
  always pass — so do Google's crawlers, by default: `paas-google-crawlers-refresh`
  (systemd timer, daily) fetches Google's official AdsBot-Google/
  AdsBot-Google-Mobile/Googlebot IP ranges (`special-crawlers.json` +
  `common-crawlers.json`) into `/srv/paas/geoip/google-crawlers.cidr`, which
  `emit_geo_directives` bakes into an app's geo matchers as a `not
  remote_ip` exemption — geo-blocking a market never breaks Google's
  indexing/ad-verification crawls of it. Per-app toggle, `GEO_EXEMPT_GOOGLE`
  (default `on`): `paas set <app> --geo-exempt-google off` or the panel's
  Geo-blocking card switch makes Google's crawlers subject to the app's geo
  rules like any other visitor.
- **Domain rules**: restrict one of a project's custom domains to a path
  allowlist (everything else 404s on that host, before Varnish/PM2 ever see
  it) and/or mark it `noindex` (`X-Robots-Tag`, independent of the app-wide
  flag) — via the panel's Domain rules editor (Clone button to reuse a rule's
  paths/noindex on another domain) or `paas domain-rules <app> --set-json -`.
  Same enforcement mechanism as geo-blocking. Useful for a secondary
  marketing/campaign domain (e.g. an `invest.` subdomain used only for a
  handful of landing pages) sharing an app instance with the primary domain
  without duplicating the whole site onto it or into search results — the
  app's own canonical/sitemap logic (when host-agnostic, as in the dotnews
  Nuxt apps) already keeps such a domain out of duplicate-content trouble;
  this feature only needs to scope which paths and headers apply.
- **Traffic metrics**: per-app daily requests/bandwidth/HIT-rate/status from
  Caddy access logs (zero hot-path impact); 30-day chart + immutable monthly
  ledger in the panel.
- **Checkmk**: agent-over-HTTPS datasource (`GET /cmk-agent`, Bearer token) +
  dynamic local checks — every app automatically gets health/cert/deploy/pm2/5xx
  services; deploy failures alert through the existing Checkmk notification
  rules. Monitoring-side setup is a one-time host + 2 rules (see plan history).
- **Matrix projects**: one repo → N apps. Set the project's Variants JSON
  (`[{"app": "...", "prep_cmd": "..."}]`); each push then preps/builds/deploys
  every variant with isolated failures. This is the dotnews production pattern.

## Future (design-ready, not built)

Per-URL cache tags (xkey), multi-server, preview deploys, panel 2FA.
Migrating the four production dotnews sites = 4× `paas create` (+ redirect
domains) + one matrix project with the four tenant prep commands.