# Server deployment notes (logship-production, 192.168.12.72)

Configuration that lives on the production server but must stay reproducible.
Findings/date: 2026-07-25 log review.

## 1. pm2 boot ordering (fixes 502 storm after reboot)

The frontend depends on the iDempiere REST API (`https://127.0.0.1:8443`,
proxied by nginx on port 80). At boot, pm2 starts within seconds while
iDempiere needs ~2 minutes → ~2 min of `502 Bad Gateway` on every reboot.

Fix: systemd drop-in for `pm2-root.service` that waits for port 8443.

**⚠️ The production deploy pipeline rewrites this file on every deploy**
(`reference/ci/deploy-production.sh`, manually pasted into the Jenkins job
`erp-frontend-pipeline-production`). The drop-in content therefore lives in
that script's heredoc — that is the source of truth. When changing the
drop-in: update the script here, update the pasted copy in the Jenkins job,
and keep [pm2-root.service.d/override.conf](pm2-root.service.d/override.conf)
in sync as the standalone reference copy.

Manual one-off install (e.g. before the next deploy has run):

```bash
cp pm2-root.service.d/override.conf /etc/systemd/system/pm2-root.service.d/override.conf
systemctl daemon-reload   # takes effect on next boot, no restart needed
```

## 2. Persistent pm2 logs

pm2 does not rotate logs by itself, and root's crontab wiped them daily
(`0 4 * * * /usr/bin/rm -rf /root/.pm2/logs/*.log`) — after any incident the
frontend log history was gone.

```bash
pm2 install pm2-logrotate
pm2 set pm2-logrotate:max_size 20M
pm2 set pm2-logrotate:retain 14
pm2 set pm2-logrotate:compress true
# then REMOVE the crontab line above (crontab -e), it defeats rotation
```

## 3. Known crontab leftovers on the server

- `0 3 * * 0 /usr/sbin/qm reboot 105` — Proxmox command inside the VM; `qm`
  does not exist there, fails silently every Sunday. Belongs on the Proxmox
  host (prox2) if a weekly reboot of VM 105 is actually wanted; remove here.

## 4. Open item for the app itself

The production `.env` sets `NODE_TLS_REJECT_UNAUTHORIZED=0` (loaded via
`ecosystem.config.cjs`), which disables TLS verification for the whole node
process. Preferable: trust the internal CA / self-signed cert explicitly
(e.g. `NODE_EXTRA_CA_CERTS=/path/to/ca.pem`) and drop the flag.