- Add backup-b2-failed oneshot for OnFailure notification
- Add onFailure handler to both backup-b2 and backup-b2-check
- Add network-online.target dependency to backup-b2-check
- Add TimeoutStartSec (2h for backup, 1h for check)
Found via ops-review lenses.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tested restore of:
- PostgreSQL dumps (forgejo: 112 tables, mautrix_slack: 32 tables)
- Forgejo repositories
- User home directories
Also updated known gaps status (sops key, PostgreSQL pin fixed).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Prevents automatic upgrade to PostgreSQL 16 when upgrading NixOS.
This allows a safer two-step approach: upgrade NixOS first, then
pg_upgrade later.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Secrets now encrypted to three recipients:
- vultr_vps: server SSH host key (primary)
- admin: workstation key (local editing)
- recovery: offline key at ~/.config/sops/age/recovery.key
If server dies and admin key unavailable, recovery key can
still decrypt secrets to bootstrap restore.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Closes r177. Critical DR gap - user home directories and ACME
certificates were not being backed up.
Excludes common caches that can be rebuilt:
- .cache, .npm/_cacache, .bun/install/cache
- node_modules, .nix-profile, .nix-defexpr
- Trash
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Documents restore procedures for full server loss, partial restore,
and user data recovery scenarios. Includes verification checklists,
time estimates, and break-glass quick reference.
Also documents known gaps (home dirs, ACME, RocksDB consistency)
that need fixing before the runbook is production-ready.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Allows dev users to use nix develop, nix build, etc.
Previously blocked by daemon access restrictions.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Generate server-side SSH keypair for git access from server
- Upload both laptop key and server key to Forgejo
- Add mandatory key revocation in dev-remove.sh
- Fix: use forgejo@ instead of git@ for SSH URLs
- Keys named username-laptop and username-devserver
- Key comment includes DO-NOT-REUSE warning
Closes ops-jrz1-rfx
Write credentials to ~/.forgejo-credentials (JSON, mode 600) when
creating new Forgejo users. Onboarding message points to file
instead of showing password in terminal output.
Addresses ops-jrz1-ofw.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace openssl rand with /dev/urandom (openssl not in NixOS path)
- Update forgejo-api-token with admin scope for user provisioning
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add programs.ssh.knownHosts for git.clarun.xyz (prevents SSH prompts)
- Expose forgejo-api-token via sops-nix for provisioning
- dev-add.sh: Create Forgejo account + upload SSH key via API
- dev-add.sh: Set up .gitconfig with user.name/email
- dev-remove.sh: Print warning to manually suspend Forgejo account
Addresses ops-jrz1-qts.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- forgejo-admin-password: dan user password
- forgejo-api-token: API token for automation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes statix W20 warning. No functional change.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- VM test boots a VM and verifies PostgreSQL, conduwuit, dnsmasq, nginx
- Shellcheck runs on all shell scripts (errors and warnings)
- Fix unused variables in sanitize-files.sh
- Use initialHashedPassword for root in VM config
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove beads from VPS deployment (kept locally for dev workflow)
- Add slack-bot-token and slack-app-token secrets with devs group access
- Remove dead acme-email secret reference
- Increase egress limits from 30/min to 150/min (burst 60→300)
- Change egress blocking from REJECT to DROP for better app behavior
- Add egress-status script for user self-diagnosis
- Update dev-slack-direct.md with new /run/secrets access patterns
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
VM imports configuration.nix which already has the permission.
Clarified comments explaining why both flake.nix and configuration.nix
need the permission (different pkgs sources).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Prevents unexpected breakage from upstream changes.
To update: nix flake update beads opencode
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Safe to re-run: updates SSH key and config if user exists,
creates new user if not. Matches NixOS declarative model.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Change [ -f /etc/slack-dev.env ] to [ -r ... ] so users not in
devs group don't get permission denied errors on login.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Preview mode shows what would be removed without making changes.
Skips confirmation prompt and outputs cyan-colored dry-run messages.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- mosh package in systemPackages
- UDP ports 60000-60010 for mosh sessions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ssh-hardening.nix had fatal bugs:
- UsePAM=false breaks NixOS SSH auth
- Protocol=2 deprecated, crashes modern sshd
- AllowUsers defaulted to ["admin"], locks out all users
Partial fixes applied but module still unsafe to enable.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Documents who, w, finger, write, wall, ytalk and .plan files.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- bsd-finger, ytalk, fortune in systemPackages
- Fortune displays on interactive shell login via programs.bash.interactiveShellInit
- Avoids breaking nix copy/rsync/scp (loginShellInit was wrong approach)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
