dan/ops-jrz1
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
ops-jrz1/.beads/issues.jsonl
Dan 043d7ab73a bd daemon sync: 2026-01-22 15:03:27
2026-01-22 15:03:27 -08:00

180 lines
140 KiB
JSON
Raw Blame History

{"id":"ops-jrz1-00e","title":"Upgrade NixOS from 24.05 to 24.11","description":"Upgrade NixOS from 24.05 to 24.11\n\n## Current State (2026-01-09)\n- Server: 24.05.20241230 (Uakari)\n- PostgreSQL 15.10 (NOT pinned - risk!)\n- Python 3.11.10\n- Unstable packages: conduwuit, mautrix-slack, mautrix-gmessages, mautrix-whatsapp\n\n## Orch Consensus (gemini, gpt)\nBoth recommend upgrading to 24.11 now:\n- Mixing unstable with old 24.05 risks glibc incompatibility\n- 24.11 is mature, waiting for 25.05 compounds upgrade complexity\n- Main risks: PostgreSQL version, Python 3.12, module option changes\n\n## Implementation Phases\nSee child issues for detailed tasks.\n\n## Rollback Plan\n- Keep previous NixOS generation\n- VNC console access for bootloader selection\n- Backups are the real safety net (rollback doesn't restore DB state)","status":"closed","priority":3,"issue_type":"epic","created_at":"2025-12-04T21:03:22.760228514-08:00","updated_at":"2026-01-10T18:34:57.295254773-08:00","closed_at":"2026-01-10T18:34:57.295254773-08:00","close_reason":"NixOS 24.11 upgrade complete - all services verified","comments":[{"id":1,"issue_id":"ops-jrz1-00e","author":"dan","text":"Analysis Findings:\n1. Version Mismatch: Local flake.nix is pinned to 'nixos-24.05', but the dev environment reports '25.11' (Unstable), indicating state divergence.\n2. Upstream Bugs: Blocking issues in mautrix-slack (ops-jrz1-blh) and maubot (sync failure) are present in the current unstable revision (2025-12-02).\n3. Recommendation: Upgrade platform to NixOS 24.11 (Stable) to align environment, ensure stability, and pull fresh upstream fixes.","created_at":"2025-12-08T23:54:57Z"}]}
{"id":"ops-jrz1-03o","title":"Upgrade mautrix-slack to v25.11","description":"Upgrade is just flake update + deploy. Current deployed: v0.2.3+dev.unknown (Oct 13). Flake lock: v25.10 (Oct 22). Latest nixpkgs-unstable: v25.11. Run: nix flake update nixpkgs-unstable \u0026\u0026 deploy. May fix edit panic (ops-jrz1-qxr).","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T18:24:18.332067067-08:00","updated_at":"2025-12-05T19:07:09.156981447-08:00","closed_at":"2025-12-05T19:07:09.156981447-08:00"}
{"id":"ops-jrz1-09o","title":"Review NixOS 24.11 release notes for breaking changes","description":"Review release notes and changelogs for affected services.\n\n## Check\n- NixOS 24.11 release notes\n- conduwuit/matrix-continuwuity changelog\n- mautrix-slack changelog\n- Forgejo changelog\n- PostgreSQL 15 → any changes in 24.11\n- Python 3.11 → 3.12 impact on maubot/bridges\n\n## Document\nAny required config changes or migration steps.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-09T17:06:49.040107986-08:00","created_by":"dan","updated_at":"2026-01-10T15:58:14.228615942-08:00","closed_at":"2026-01-10T15:58:14.228615942-08:00","close_reason":"Documented in docs/nixos-24.11-upgrade-notes.md. Key: pin PostgreSQL 15, review Forgejo 8/9 notes, continuwuity fork is fine.","dependencies":[{"issue_id":"ops-jrz1-09o","depends_on_id":"ops-jrz1-00e","type":"parent-child","created_at":"2026-01-09T17:07:09.688966515-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-0nt","title":"Enable fail2ban module (imported but not active)","description":"modules/security/fail2ban.nix is imported in hosts/ops-jrz1.nix but security.fail2ban-enhanced.enable is never set to true. SSH brute force protection is not active. Fix: Add security.fail2ban-enhanced.enable = true to hosts config.","status":"closed","priority":1,"issue_type":"bug","created_at":"2026-01-05T15:44:24.970052917-08:00","created_by":"dan","updated_at":"2026-01-05T22:58:18.196427827-08:00","closed_at":"2026-01-05T22:58:18.196427827-08:00","close_reason":"Wontfix: key-only SSH is sufficient. fail2ban adds complexity without real security value. If log noise becomes an issue, CrowdSec is the modern alternative."}
{"id":"ops-jrz1-0tk","title":"Set up backup strategy for /home dev directories","description":"User home directories contain work that should be backed up. Options: (1) Daily tarball to object storage, (2) Restic/borg to remote, (3) Syncthing to another host, (4) Just document that users should push to git. Consider: frequency, retention, restore process, cost.","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-03T14:17:56.208262155-08:00","created_by":"dan","updated_at":"2026-01-03T14:17:56.208262155-08:00"}
{"id":"ops-jrz1-13e","title":"Emes agent workflow rollout on ops-jrz1","description":"Document and stage server-first adoption of emes (tissue/jwz/idle) and multi-agent CLI workflow. Add server docs + tool install guidance, outline adapter/event model, and identify required Nix changes for system-wide install.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-07T09:43:31.294962856-08:00","created_by":"dan","updated_at":"2026-01-07T09:47:56.535884086-08:00","closed_at":"2026-01-07T09:47:56.535884086-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-1bk","title":"Add CPU watchdog timer","description":"Systemd timer that detects sustained CPU abuse and kills offending user.\n\n## Script: /usr/local/bin/cpu-watchdog\n```bash\n#\\!/usr/bin/env bash\n# Detect sustained CPU abuse, kill after 5 consecutive violations\nTHRESHOLD=180 # 180% CPU (almost 2 cores)\nCOUNTFILE=\"/var/lib/cpu-watchdog\"\nmkdir -p \"$COUNTFILE\"\n\nfor user in $(ls /home); do\n id \"$user\" \u0026\u003e/dev/null || continue\n pct=$(ps -u \"$user\" -o %cpu= 2\u003e/dev/null | awk '{s+=$1}END{print int(s)}')\n pct=${pct:-0}\n \n if [ \"$pct\" -gt \"$THRESHOLD\" ]; then\n count=$(cat \"$COUNTFILE/$user\" 2\u003e/dev/null || echo 0)\n count=$((count + 1))\n echo \"$count\" \u003e \"$COUNTFILE/$user\"\n logger -t cpu-watchdog \"User $user at ${pct}% CPU (strike $count/5)\"\n \n if [ \"$count\" -ge 5 ]; then\n /usr/local/bin/killswitch \"$user\" \"sustained CPU abuse (${pct}%)\"\n rm -f \"$COUNTFILE/$user\"\n fi\n else\n rm -f \"$COUNTFILE/$user\"\n fi\ndone\n```\n\n## Systemd timer\n```nix\nsystemd.services.cpu-watchdog = {\n script = ''/usr/local/bin/cpu-watchdog'';\n serviceConfig.Type = \"oneshot\";\n};\nsystemd.timers.cpu-watchdog = {\n wantedBy = [ \"timers.target\" ];\n timerConfig = {\n OnBootSec = \"1min\";\n OnUnitActiveSec = \"1min\";\n };\n};\n```\n\n## Behavior\n- Runs every minute\n- 5 consecutive minutes at \u003e180% CPU = kill\n- Resets counter if CPU drops below threshold","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T20:20:53.246401154-08:00","created_by":"dan","updated_at":"2026-01-02T21:02:35.469465906-08:00","closed_at":"2026-01-02T21:02:35.469465906-08:00","close_reason":"Closed","dependencies":[{"issue_id":"ops-jrz1-1bk","depends_on_id":"ops-jrz1-396","type":"blocks","created_at":"2026-01-02T20:21:14.270063028-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-1kv","title":"Set up terminfo for xterm-ghostty and other modern terminals","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T11:34:42.133192121-08:00","created_by":"dan","updated_at":"2026-01-03T18:02:28.819727599-08:00","closed_at":"2026-01-03T18:02:28.819727599-08:00","close_reason":"Added pkgs-unstable.ghostty.terminfo and kitty.terminfo to configuration.nix, deployed successfully"}
{"id":"ops-jrz1-1t1","title":"Migrate existing users to dual-key setup","description":"Add server-side keys for existing users (frank).\n\n## Existing users\n- dan: Already has Forgejo account (manually created)\n- frank: Provisioned recently, has login key in Forgejo\n\n## Migration steps\n1. Generate server-side keypair for each user\n2. Upload to Forgejo with \"-devserver\" title\n3. Verify git push works\n\n## Script or manual?\nCould create migrate-git-keys.sh or just run commands manually for 2 users.\n\n## Commands\n```bash\n# For each user:\nsudo -u frank ssh-keygen -t ed25519 -f /home/frank/.ssh/id_ed25519 -N '' -C 'frank@jrz1-devserver'\n# Upload via API...\n```","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:48:01.793174851-08:00","created_by":"dan","updated_at":"2026-01-09T19:36:04.345813628-08:00","closed_at":"2026-01-09T19:36:04.345813628-08:00","close_reason":"Implemented in commit d9c1848","dependencies":[{"issue_id":"ops-jrz1-1t1","depends_on_id":"ops-jrz1-rfx","type":"parent-child","created_at":"2026-01-09T17:48:12.67845924-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-1t1","depends_on_id":"ops-jrz1-5ag","type":"blocks","created_at":"2026-01-09T17:49:06.866614276-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-2bh","title":"Notification system for Claude agent input requests","description":"Notify phone when Claude Code needs input. Research notes:\n\nDetection options:\n- Claude Code hooks (check if CC has \"waiting for input\" hook)\n- Terminal idle detection (tmux/script watches for prompt pattern)\n- Process state (detect claude waiting on stdin)\n\nNotification services to evaluate:\n- ntfy.sh (free, self-hostable, good mobile apps)\n- Matrix (already running - could use dedicated room)\n- Pushover ($5 one-time, dead simple)\n- Custom webhook\n\nReference: granda.org article mentions \"Poke webhooks\"","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-05T17:25:32.788319213-08:00","created_by":"dan","updated_at":"2026-01-05T17:57:45.888639591-08:00"}
{"id":"ops-jrz1-2bu","title":"Direct Slack bot path for learners","description":"Alternative path: learners write Python bots using slack-bolt, connect directly to Slack via Socket Mode. No Matrix, no bridge.\n\n## Architecture\n```\nlearner code → slack-bolt → Socket Mode WebSocket → Slack API\n```\n\n## Status\n\n**Done:**\n- [x] /etc/slack-learner.env with shared tokens (xoxb-, xapp-)\n- [x] learners group for access control (dantest is member)\n- [x] learner-add.sh adds users to group, sources env in .bashrc\n- [x] Design doc: docs/learner-slack-direct.md\n\n**Not Done:**\n- [ ] Starter template (~/slack-bot-template/)\n- [ ] Process management (systemd user services or supervisor)\n- [ ] #learner-sandbox channel in Slack\n- [ ] End-to-end test with real learner\n\n## Tradeoffs vs Maubot/Matrix (ops-jrz1-2pm)\n- Faster feedback (direct to Slack)\n- Excellent slack-bolt docs\n- But: shared bot identity, manual process management\n\n## Ready to Use NOW\nWorks today with terminal editors (vim/nano):\n```bash\nssh alice@ops-jrz1\npip install slack-bolt\npython bot.py # responds in Slack\n```\n\nVS Code Remote-SSH needs nix-ld deployed first.","status":"closed","priority":2,"issue_type":"epic","created_at":"2025-12-29T18:56:10.239324326-05:00","created_by":"dan","updated_at":"2026-01-09T16:15:30.912654052-08:00","closed_at":"2026-01-09T16:15:30.912654052-08:00","close_reason":"Working, learners actively using"}
{"id":"ops-jrz1-2pm","title":"Remote dev environment for learners","description":"Set up dev environments for learners to build maubot plugins (Matrix bots that can bridge to Slack).\n\n## Approach\nVS Code Remote-SSH + shared maubot + per-user Unix accounts\n\n## Architecture\n```\nlearner code → maubot → Matrix → mautrix-slack bridge → Slack\n```\n\n## Status\n\n**Done:**\n- [x] learner-add.sh / learner-remove.sh scripts\n- [x] Hello-world plugin template (templates/plugin-skeleton/)\n- [x] Test user `dantest` created with ~/plugins/hello-dantest/\n- [x] Maubot running and healthy\n\n**Not Done:**\n- [ ] nix-ld for VS Code Remote-SSH (config added, not deployed)\n- [ ] Test full VS Code Remote-SSH flow\n- [ ] Test Claude Code extension over Remote-SSH\n- [ ] #learners-sandbox Matrix room\n- [ ] Onboarding doc polish\n\n## Tradeoffs vs Direct Slack (ops-jrz1-2bu)\n- Slower feedback (bridge hop)\n- Sparse maubot docs\n- But: managed process lifecycle, per-bot identity\n\n## Docs\n- docs/learner-onboarding.md\n- docs/learner-admin.md","status":"closed","priority":2,"issue_type":"epic","created_at":"2025-12-28T10:13:21.90764918-05:00","created_by":"dan","updated_at":"2026-01-09T16:15:30.898845361-08:00","closed_at":"2026-01-09T16:15:30.898845361-08:00","close_reason":"Working, learners actively using"}
{"id":"ops-jrz1-2r9","title":"Remove unused 'config' parameter from configuration.nix","description":"deadnix reports unused lambda pattern 'config' at line 1. Either remove it or prefix with underscore.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-03T17:35:57.074498327-08:00","created_by":"dan","updated_at":"2026-01-05T18:40:08.564606182-08:00","closed_at":"2026-01-05T18:40:08.564606182-08:00","close_reason":"Fixed by deadnix cleanup in commit 2dd5684"}
{"id":"ops-jrz1-2y4","title":"Test musiclink deployment end-to-end","description":"Verification steps:\n\n1. systemctl status matterbridge musiclink\n2. journalctl -u matterbridge -u musiclink (check for errors)\n3. Post a Spotify link in configured Slack channel\n4. Verify bot responds with YouTube/Apple/Deezer links\n5. Test reverse direction (YouTube → Spotify)\n\nDocument any issues for future reference.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-08T12:59:10.160713165-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.955964801-08:00","closed_at":"2026-01-08T16:18:17.955964801-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves.","dependencies":[{"issue_id":"ops-jrz1-2y4","depends_on_id":"ops-jrz1-hkb","type":"blocks","created_at":"2026-01-08T15:56:08.357534903-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-2y4","depends_on_id":"ops-jrz1-8j9","type":"blocks","created_at":"2026-01-08T15:56:08.404257595-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-30e","title":"Add gastown (gt) to system packages via flake input","description":"Add gastown CLI (gt) as a system-wide package.\n\n## What is gastown?\nGastown (gt) is the orchestration layer to beads' memory layer. Together they form a workflow for AI-supervised coding.\n\n## Status\n**BLOCKED**: No releases available yet. Repo uses GoReleaser but no tags/releases published.\n- Requires Go 1.24 to build from source\n- Once releases exist, add like beads: flake input + systemPackages\n\n## Source\ngithub:steveyegge/gastown\n\n## Workaround\nUsers can build locally: `go install github.com/steveyegge/gastown@latest`","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T16:37:47.093900357-08:00","created_by":"dan","updated_at":"2026-01-02T19:08:28.742017652-08:00","closed_at":"2026-01-02T19:08:28.742017652-08:00","close_reason":"Won't fix - not adding gastown to this server"}
{"id":"ops-jrz1-396","title":"Add killswitch script","description":"Script to immediately terminate all processes for a user.\n\n## Script: /usr/local/bin/killswitch\n```bash\n#!/usr/bin/env bash\n# Usage: killswitch \u003cusername\u003e [reason]\nset -euo pipefail\nUSER=\"$1\"\nREASON=\"${2:-manual kill}\"\n\nif ! id \"$USER\" \u0026\u003e/dev/null; then\n echo \"User not found: $USER\" \u003e\u00262\n exit 1\nfi\n\nlogger -t killswitch \"Killing all processes for $USER: $REASON\"\npkill -u \"$USER\" || true\nloginctl terminate-user \"$USER\" 2\u003e/dev/null || true\necho \"Killed $USER: $REASON\"\n```\n\n## Usage\n```bash\n# Manual kill\nkillswitch dan \"investigating suspicious activity\"\n\n# From watchdog\nkillswitch dan \"sustained CPU abuse (250%)\"\n```\n\n## Notes\n- Logs to syslog with 'killswitch' tag\n- Terminates user session and all processes\n- Safe to run if user has no processes","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T20:19:35.306146948-08:00","created_by":"dan","updated_at":"2026-01-02T20:25:38.950670564-08:00","closed_at":"2026-01-02T20:25:38.950670564-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-3au","title":"Research: Learner deployment pipeline","description":"How does learner code get to \"prod\" (running services)?\n\n## Current context\n- Maubot: Upload .mbp via web UI\n- Slack bots: Manual `python bot.py` or systemd user service\n\n## Questions\n1. Can learners run persistent services? (`systemctl --user`)\n2. Should they have access to maubot admin UI?\n3. Git-based deploy? Push to trigger reload?\n4. Who can restart what?\n\n## Options\n- **Manual only** - Learner runs in foreground/tmux\n- **User systemd** - `systemctl --user enable mybot`\n- **Supervised** - Central supervisor manages learner procs\n- **GitOps** - Push to deploy (complex)\n\n## Security considerations\n- What if learner bot crashes in loop?\n- Resource limits on user services?\n- Can learner affect other learners' services?","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-02T12:27:34.107447487-08:00","created_by":"dan","updated_at":"2026-01-02T12:27:34.107447487-08:00"}
{"id":"ops-jrz1-3b1","title":"Research: Agentic coder sandboxing","description":"When Claude Code runs on server, what can it do? Should we limit it?\n\n## Current state\n- No sandbox\n- Claude has full user privileges\n- Can run any command user can run\n\n## Risks\n- `rm -rf ~` (accidental or hallucinated)\n- Network exfiltration\n- Resource exhaustion (fork bomb, disk fill)\n- Credential theft from env/files\n\n## Options\n1. **Trust the agent** - User's problem if Claude breaks things\n2. **Command allowlist** - Only approved commands\n3. **Container sandbox** - Run agent in container\n4. **Snapshot/rollback** - Easy recovery if things break\n5. **Audit logging** - At least know what happened\n\n## Questions\n- What do other agentic coding setups do?\n- Is this overkill for a learning environment?\n- Does Claude Code have built-in safety?","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T12:27:33.705283658-08:00","created_by":"dan","updated_at":"2026-01-10T09:33:43.750905801-08:00","closed_at":"2026-01-10T09:33:43.750905801-08:00","close_reason":"Codex CLI uses seccomp+NoNewPrivs sandbox blocking nix daemon. Fix: codex -s danger-full-access or sandbox_mode='danger-full-access' in ~/.codex/config.toml. Documented in server-AGENTS.md."}
{"id":"ops-jrz1-3ca","title":"Persist opencode state/cache across restarts","description":"opencode may store index/cache in ~/.cache or other dirs not covered by current bind mounts. AI context could be lost on container restart. Verify and add mounts.","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-05T15:32:30.90315778-08:00","updated_at":"2025-12-28T00:05:44.753074955-05:00","closed_at":"2025-12-28T00:05:44.753074955-05:00","close_reason":"Parent epic cancelled - browser-based dev approach abandoned","dependencies":[{"issue_id":"ops-jrz1-3ca","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.247361009-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-3fd","title":"Deploy and test single-user instance (Phase 1)","description":"Deploy one container for testing. Validate: WebSocket, extensions, terminal, opencode, memory usage. Access via SSH tunnel initially.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T17:16:54.783260036-08:00","updated_at":"2026-01-05T18:40:33.004809178-08:00","closed_at":"2026-01-05T18:40:33.004809178-08:00","close_reason":"Dependencies cancelled - browser-based dev approach abandoned","dependencies":[{"issue_id":"ops-jrz1-3fd","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:17:36.400677984-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-3fd","depends_on_id":"ops-jrz1-5oe","type":"blocks","created_at":"2025-12-05T17:17:38.708397909-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-3fd","depends_on_id":"ops-jrz1-av0","type":"blocks","created_at":"2025-12-05T17:17:38.721665448-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-3fd","depends_on_id":"ops-jrz1-9gd","type":"blocks","created_at":"2025-12-05T17:17:38.737824478-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-3jo","title":"Test: VS Code extension execution location","description":"When using VS Code Remote-SSH, does Claude Code extension run locally or on server?\n\n## Why it matters\n- If local: needs to proxy commands to server, may have latency/sync issues\n- If remote: has direct access, simpler model, but needs install on server\n\n## Test plan\n1. Connect VS Code Remote-SSH to jrz1 as dantest\n2. Install Claude Code extension\n3. Observe where extension runs (check processes on both sides)\n4. Run a simple Claude command, see where execution happens\n5. Check if extension needs API key on server or uses local\n\n## Expected outcome\nDocument actual behavior for onboarding docs.","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-02T12:27:34.517565101-08:00","created_by":"dan","updated_at":"2026-01-02T12:27:34.517565101-08:00"}
{"id":"ops-jrz1-3lgf","title":"chmod runs even if key already existed","description":"[IDEMPOTENCY] dev-add.sh:105-106\n\n```bash\nchmod 600 \"$server_key\"\nchmod 644 \"$server_key.pub\"\n```\n\nIssue: chmod on key files runs even if key already existed - harmless but unnecessary\n\nSuggest: Move inside the `if [[ ! -f \"$server_key\" ]]` block, or accept as defense-in-depth\n\nSource: ops-review","status":"closed","priority":4,"issue_type":"chore","created_at":"2026-01-09T19:50:50.566743222-08:00","created_by":"dan","updated_at":"2026-01-09T20:15:05.201821452-08:00","closed_at":"2026-01-09T20:15:05.201821452-08:00","close_reason":"Keeping as-is - defense-in-depth ensures correct permissions every run"}
{"id":"ops-jrz1-3so","title":"Browser-based dev environment with opencode","description":"Epic: Provide VS Code in browser via code-server with opencode AI integration.\n\nKey decisions:\n- code-server in Podman containers (rootless)\n- opencode CLI + VS Code extension pre-installed\n- Subdomain routing (dan.code.clarun.xyz)\n- Custom container image\n- Target users: non-programmers, testers, learners\n\nDesign doc: specs/004-browser-dev-environment/design.md\n\nMigrated from ops-jrz1-ndl","status":"closed","priority":1,"issue_type":"epic","created_at":"2025-12-05T17:04:36.709352529-08:00","updated_at":"2025-12-28T00:01:08.825440506-05:00","closed_at":"2025-12-28T00:01:08.825440506-05:00","close_reason":"Won't fix - non-MS browser-based VSCode solutions have reliability/compatibility issues"}
{"id":"ops-jrz1-3ue","title":"Generate server-side SSH keypair in dev-add.sh","description":"Add server-side keypair generation to dev-add.sh.\n\n## Changes\nIn `create_user()` function, after setting up SSH authorized_keys:\n\n```bash\n# Generate server-side keypair for git access\nlog_info \"Generating server-side SSH key for git access...\"\nssh-keygen -t ed25519 \\\n -f \"/home/$username/.ssh/id_ed25519\" \\\n -N '' \\\n -C \"$username@jrz1-server-DO-NOT-REUSE\"\n\n# Strict permissions (guardrail from threat model analysis)\nchmod 700 \"/home/$username/.ssh\"\nchmod 600 \"/home/$username/.ssh/id_ed25519\"\nchmod 644 \"/home/$username/.ssh/id_ed25519.pub\"\nchown -R \"$username:users\" \"/home/$username/.ssh\"\n```\n\n## Guardrails (from orch consensus)\n- chmod 700 on .ssh directory (prevent cross-user access)\n- chmod 600 on private key\n- Key comment includes DO-NOT-REUSE warning\n- Idempotent: check if key exists before generating\n\n## Verification\n- Key created at ~/.ssh/id_ed25519\n- Permissions: drwx------ on .ssh, -rw------- on key\n- Comment visible in .pub file","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:47:49.658989402-08:00","created_by":"dan","updated_at":"2026-01-09T19:36:04.328129622-08:00","closed_at":"2026-01-09T19:36:04.328129622-08:00","close_reason":"Implemented in commit d9c1848","dependencies":[{"issue_id":"ops-jrz1-3ue","depends_on_id":"ops-jrz1-rfx","type":"parent-child","created_at":"2026-01-09T17:48:12.617565638-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-3ux","title":"Document activation script behavior for user slice limits","description":"configuration.nix:171-178 uses activation script to write to /run/systemd/system/. This survives reboot but behavior on re-activation should be documented. Consider if systemd-tmpfiles would be more appropriate.","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-03T17:35:59.432265237-08:00","created_by":"dan","updated_at":"2026-01-03T17:35:59.432265237-08:00"}
{"id":"ops-jrz1-3wd","title":"Update flake to nixos-24.11 and build","description":"The actual upgrade execution.\n\n## Steps\n1. Update flake.nix: `nixpkgs.url = \"github:NixOS/nixpkgs/nixos-24.11\";`\n2. Update sops-nix pin if needed (check compatibility)\n3. Run `nix flake update`\n4. Build locally: `nixos-rebuild build --flake .#ops-jrz1`\n5. Diff closures: `nvd diff /run/current-system result` (run on server)\n6. Review diff for unexpected changes\n\n## Do NOT deploy yet\nThis task is build-only. Deployment is separate task.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:06:49.152287142-08:00","created_by":"dan","updated_at":"2026-01-10T18:16:12.032056496-08:00","closed_at":"2026-01-10T18:16:12.032056496-08:00","close_reason":"Build successful, closure diffed, ready for deployment","dependencies":[{"issue_id":"ops-jrz1-3wd","depends_on_id":"ops-jrz1-00e","type":"parent-child","created_at":"2026-01-09T17:07:09.709811064-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-3wd","depends_on_id":"ops-jrz1-7qg","type":"blocks","created_at":"2026-01-09T17:07:54.508131117-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-3wd","depends_on_id":"ops-jrz1-09o","type":"blocks","created_at":"2026-01-09T17:07:54.54061966-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-3x4","title":"Add maubot SDK and deploy script to container image","description":"Container image needs:\n- Python 3.11 + maubot SDK\n- deploy.sh script (zip → .mbp → curl to maubot API)\n- maubot API reachable from container (host network or port forward)\n\nPart of learner onboarding for bot development.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-06T12:18:06.841708662-08:00","updated_at":"2025-12-28T00:05:44.748474842-05:00","closed_at":"2025-12-28T00:05:44.748474842-05:00","close_reason":"Parent epic cancelled - browser-based dev approach abandoned","dependencies":[{"issue_id":"ops-jrz1-3x4","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-06T12:18:16.085519885-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-3x4","depends_on_id":"ops-jrz1-d58","type":"blocks","created_at":"2025-12-06T12:18:16.110944935-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-3zo","title":"Post-upgrade verification checklist","description":"Verify all services after NixOS 24.11 upgrade.\n\n## Service Health\n- [ ] PostgreSQL: `systemctl status postgresql`\n- [ ] Conduwuit: `systemctl status matrix-continuwuity`\n- [ ] Forgejo: `systemctl status forgejo` + web UI\n- [ ] mautrix-slack: `systemctl status mautrix-slack` + bridge test\n- [ ] maubot: `systemctl status maubot` + web UI\n- [ ] nginx: `systemctl status nginx`\n\n## Functional Tests\n- [ ] Matrix: can send/receive messages\n- [ ] Forgejo: can clone repo, web UI works\n- [ ] Bridge: Slack ↔ Matrix message flow\n- [ ] Dev users: SSH access works\n- [ ] git.clarun.xyz: clone via SSH works\n\n## Monitoring\n- Watch logs for 24-48 hours: `journalctl -f`\n- Check for errors: `journalctl --since '1 hour ago' | grep -E 'ERR|error|failed'`","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:06:49.33480326-08:00","created_by":"dan","updated_at":"2026-01-10T18:34:47.906747684-08:00","closed_at":"2026-01-10T18:34:47.906747684-08:00","close_reason":"All services verified, PostgreSQL collation fixed, no errors","dependencies":[{"issue_id":"ops-jrz1-3zo","depends_on_id":"ops-jrz1-00e","type":"parent-child","created_at":"2026-01-09T17:07:09.754831381-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-3zo","depends_on_id":"ops-jrz1-a9d","type":"blocks","created_at":"2026-01-09T17:07:54.617559299-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-44x","title":"Consolidate repeated Nix keys in configuration.nix","description":"statix W20: networking.*, environment.*, systemd.* keys are scattered. Consolidate into single attribute sets for cleaner code.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-07T10:45:11.74130885-08:00","created_by":"dan","updated_at":"2026-01-08T12:54:10.768691041-08:00","closed_at":"2026-01-08T12:54:10.768691041-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-45v","title":"Matrix/Slack identity mismatch: dan vs vlad","description":"Matrix user @dan:clarun.xyz is linked to Slack user 'vlad'. Messages appear as vlad in Slack but dan in Element. Cosmetic confusion. Options: rename Matrix display name, or re-login bridge with different Slack account.","status":"open","priority":4,"issue_type":"task","created_at":"2025-12-05T19:38:19.899555475-08:00","updated_at":"2026-01-08T12:16:37.173602537-08:00"}
{"id":"ops-jrz1-46y","title":"Write onboarding documentation","description":"Critical for non-programmers. Cover: login, opencode usage, Git setup (PAT workflow), resource limits, security hygiene. Keep concise.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T15:32:41.586544583-08:00","updated_at":"2025-12-28T00:08:06.743739166-05:00","closed_at":"2025-12-28T00:08:06.743739166-05:00","close_reason":"Browser-based dev environment cancelled","dependencies":[{"issue_id":"ops-jrz1-46y","depends_on_id":"ops-jrz1-7j4","type":"blocks","created_at":"2025-12-05T15:33:25.328712413-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-46y","depends_on_id":"ops-jrz1-wj2","type":"blocks","created_at":"2025-12-05T15:33:25.351559821-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-46y","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.401868669-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-4jm","title":"Smoke test Matrix server (conduwuit)","description":"Verify Matrix homeserver is healthy: check /_matrix/client/versions endpoint, test registration, verify federation status (disabled). Quick health check after deployments.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T18:09:47.220765063-08:00","updated_at":"2025-12-05T18:19:33.059734881-08:00","closed_at":"2025-12-05T18:19:33.059734881-08:00"}
{"id":"ops-jrz1-4kf","title":"No timeout on curl commands - could hang","description":"[RESILIENCE] dev-add.sh:214,254 dev-remove.sh:56,79\n\n```bash\ncurl -s ... 2\u003e/dev/null\n```\n\nIssue: No timeout on curl commands - could hang indefinitely if Forgejo unresponsive\n\nSuggest: Add `--connect-timeout 5 --max-time 30` to curl calls\n\nSource: ops-review","status":"closed","priority":3,"issue_type":"bug","created_at":"2026-01-09T19:50:50.144564222-08:00","created_by":"dan","updated_at":"2026-01-09T20:10:13.583587673-08:00","closed_at":"2026-01-09T20:10:13.583587673-08:00","close_reason":"Added --connect-timeout 5 --max-time 30 to all 4 curl calls in dev-add.sh and dev-remove.sh"}
{"id":"ops-jrz1-4m5","title":"VM test: Fix skipTypeCheck/skipLint properly","description":"Currently using skipTypeCheck=true and skipLint=true due to dynamic node names. Consensus says this is a maintainability smell. Would need to restructure test so node names are static. Medium-high effort.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-08T00:58:32.815754848-08:00","created_by":"dan","updated_at":"2026-01-08T11:03:29.333525231-08:00","closed_at":"2026-01-08T11:03:29.333525231-08:00","close_reason":"skipTypeCheck and skipLint are both necessary for NixOS VM tests because node variables are dynamically injected. The Python variable name comes from hostname (hyphens→underscores), not the node key. Added comments explaining why each skip flag is needed. This is standard practice, not a code smell for this use case."}
{"id":"ops-jrz1-4oj","title":"Phone-based Claude Code workflow (mosh + notifications)","description":"Enable Claude Code sessions from phone. Target: Phone (Termius) via Tailscale to VPS running Claude Code agents. Components: mosh (config ready), Tailscale, notification webhooks for agent input, git worktrees for parallel work. Reference: granda.org/en/2026/01/02/claude-code-on-the-go/","status":"open","priority":2,"issue_type":"epic","created_at":"2026-01-05T17:25:16.929379168-08:00","created_by":"dan","updated_at":"2026-01-05T17:25:21.606713513-08:00","dependencies":[{"issue_id":"ops-jrz1-4oj","depends_on_id":"ops-jrz1-92t","type":"blocks","created_at":"2026-01-05T17:25:54.124020166-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-4oj","depends_on_id":"ops-jrz1-5wf","type":"blocks","created_at":"2026-01-05T17:25:54.160078437-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-4oj","depends_on_id":"ops-jrz1-2bh","type":"blocks","created_at":"2026-01-05T17:25:54.195131809-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-4oj","depends_on_id":"ops-jrz1-unn","type":"blocks","created_at":"2026-01-05T17:25:54.233360492-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-4wz","title":"VM test: Silence root password warning","description":"Use initialHashedPassword = '' instead of password = 'test' to silence the NixOS warning about multiple password options. Quick win, cleaner output.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-08T00:58:32.045234794-08:00","created_by":"dan","updated_at":"2026-01-08T09:11:12.857067129-08:00","closed_at":"2026-01-08T09:11:12.857067129-08:00","close_reason":"Changed to initialHashedPassword. Warning persists due to module interaction - cosmetic, tests pass."}
{"id":"ops-jrz1-5ag","title":"Upload both keys to Forgejo with distinct titles","description":"Update provision_forgejo() to upload both keys with clear naming.\n\n## Current behavior\nUploads login key with title \"dev-server-key\"\n\n## New behavior\nUpload TWO keys:\n1. Login key → title \"$username-laptop\"\n2. Server key → title \"$username-devserver\"\n\n## Code changes\n```bash\n# Upload login key (for direct laptop→git access)\nupload_forgejo_key \"$username\" \"$ssh_key\" \"$username-laptop\"\n\n# Upload server-side key (for server→git access) \nlocal server_pubkey\nserver_pubkey=$(cat \"/home/$username/.ssh/id_ed25519.pub\")\nupload_forgejo_key \"$username\" \"$server_pubkey\" \"$username-devserver\"\n```\n\n## Implementation notes\n- Extract key upload to helper function (avoid duplication)\n- Handle 422 (key already exists) gracefully\n- Both keys scoped to Forgejo only","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:47:59.406469574-08:00","created_by":"dan","updated_at":"2026-01-09T19:36:04.340046727-08:00","closed_at":"2026-01-09T19:36:04.340046727-08:00","close_reason":"Implemented in commit d9c1848","dependencies":[{"issue_id":"ops-jrz1-5ag","depends_on_id":"ops-jrz1-rfx","type":"parent-child","created_at":"2026-01-09T17:48:12.638501829-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-5ag","depends_on_id":"ops-jrz1-3ue","type":"blocks","created_at":"2026-01-09T17:49:06.835599069-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-5ef","title":"Add local-scripts to systemPackages","description":"Add the local-scripts derivation to environment.systemPackages so scripts are available system-wide.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-03T08:39:53.979819917-08:00","created_by":"dan","updated_at":"2026-01-03T08:45:47.753131986-08:00","closed_at":"2026-01-03T08:45:47.753131986-08:00","close_reason":"Merged into ops-jrz1-o9c (admin-scripts package includes systemPackages addition)","dependencies":[{"issue_id":"ops-jrz1-5ef","depends_on_id":"ops-jrz1-vw4","type":"blocks","created_at":"2026-01-03T08:40:02.788186493-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-5fk","title":"Smoke test Maubot service","description":"Verify Maubot is healthy: check management UI accessible via SSH tunnel, verify bot instances running, test plugin functionality. Quick health check after deployments.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T18:09:47.33773092-08:00","updated_at":"2025-12-05T18:19:33.061388913-08:00","closed_at":"2025-12-05T18:19:33.061388913-08:00"}
{"id":"ops-jrz1-5ki","title":"Set up programmatic QA test user for bridge testing","status":"open","priority":3,"issue_type":"task","created_at":"2025-12-05T20:17:04.312571398-08:00","updated_at":"2025-12-05T20:17:04.312571398-08:00"}
{"id":"ops-jrz1-5l5","title":"Communicate dev-add changes to users (Forgejo integration)","description":"dev-add.sh now provisions Forgejo accounts. Need to communicate this to:\n- Existing dev users (may need retroactive Forgejo account setup)\n- Any documentation/onboarding materials\n- Consider: How do existing users get their Forgejo accounts created?","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-08T17:44:34.644232142-08:00","created_by":"dan","updated_at":"2026-01-09T10:11:03.087410216-08:00","closed_at":"2026-01-09T10:11:03.087410216-08:00","close_reason":"Retroactively provisioned frank with Forgejo access. dan already had access. Both existing dev users now have Forgejo accounts."}
{"id":"ops-jrz1-5oe","title":"Create NixOS module for code-server containers","description":"Module to manage per-user Podman containers, nginx routing, secrets. Use virtualisation.oci-containers. Generate systemd units.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T17:16:54.656121092-08:00","updated_at":"2025-12-28T00:05:44.743524099-05:00","closed_at":"2025-12-28T00:05:44.743524099-05:00","close_reason":"Parent epic cancelled - browser-based dev approach abandoned","dependencies":[{"issue_id":"ops-jrz1-5oe","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:17:36.386278268-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-5oe","depends_on_id":"ops-jrz1-d58","type":"blocks","created_at":"2025-12-05T17:17:38.694752468-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-5wf","title":"Evaluate Tailscale for private VPS access","description":"Research Tailscale setup on NixOS. Consider: replaces public SSH, integrates with phone, MagicDNS for easy naming.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-05T17:25:32.639656649-08:00","created_by":"dan","updated_at":"2026-01-05T17:29:39.308362443-08:00","closed_at":"2026-01-05T17:29:39.308362443-08:00","close_reason":"Not needed - public SSH with key-only auth is sufficient"}
{"id":"ops-jrz1-62b","title":"dev-add: check devs group exists before creating user","description":"dev-add failed silently when devs group was missing. User was created but SSH key wasn't set up. Script should validate prerequisites first.","status":"closed","priority":2,"issue_type":"bug","created_at":"2026-01-03T11:50:57.134573631-08:00","created_by":"dan","updated_at":"2026-01-03T11:53:40.714806901-08:00","closed_at":"2026-01-03T11:53:40.714806901-08:00","close_reason":"Added devs group check before user creation in dev-add.sh"}
{"id":"ops-jrz1-6dd","title":"Manage Slack tokens via sops-nix instead of /etc/slack-dev.env","description":"/etc/slack-dev.env with Slack tokens is managed manually outside NixOS. Not declarative, could be lost on rebuild. Add to secrets.yaml and deploy via sops-nix for consistency.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-05T15:44:41.749258935-08:00","created_by":"dan","updated_at":"2026-01-07T10:45:26.269107512-08:00","closed_at":"2026-01-07T10:45:26.269107512-08:00","close_reason":"Implemented: Slack tokens now in sops-nix with group=devs, mode=0440. Deployed and verified."}
{"id":"ops-jrz1-6es","title":"Update egress-watchdog source to use 'killswitch' not /usr/local/bin path","description":"scripts/egress-watchdog:44 has hardcoded /usr/local/bin/killswitch. The Nix build uses replaceStrings to fix this, but source should reflect reality. Change to just 'killswitch'.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T17:35:58.211053165-08:00","created_by":"dan","updated_at":"2026-01-05T09:12:47.279119754-08:00","closed_at":"2026-01-05T09:12:47.279119754-08:00","close_reason":"Updated scripts to use killswitch directly, removed replaceStrings from config"}
{"id":"ops-jrz1-6hu4","title":"Finalize MusicLink integration","description":"1. Verify link conversion logic.\\n2. Revoke admin privileges from @musiclink.\\n3. Resolve git access issue (ops-jrz1-zr0q).","notes":"Matrix-native MusicLink config deployed; service running on ops-jrz1 with new matrix settings. Still pending: revoke @musiclink admin privileges and resolve git access issue (ops-jrz1-zr0q).","status":"in_progress","priority":2,"issue_type":"task","owner":"dleink@gmail.com","created_at":"2026-01-20T15:00:38.86497614-08:00","created_by":"Dan","updated_at":"2026-01-22T09:13:17.982083947-08:00"}
{"id":"ops-jrz1-6ip","title":"Remove unused Nix lambda patterns (deadnix findings)","description":"deadnix found 4 unused declarations: configuration.nix:1 (config), flake.nix:27 (pkgs), hosts/ops-jrz1.nix:1 (config, pkgs-unstable), modules/dev-services.nix:2 (pkgs). Fix: Remove or prefix with underscore.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-05T15:44:25.617107094-08:00","created_by":"dan","updated_at":"2026-01-05T18:23:54.976849305-08:00","closed_at":"2026-01-05T18:23:54.976849305-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-6o6","title":"Shellcheck: Consider enabling warnings","description":"Currently using -S error (errors only). Consensus split: warnings catch shell logic bugs but may be noisy. Evaluate if switching to include warnings surfaces real issues or just noise.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-08T00:58:32.656003167-08:00","created_by":"dan","updated_at":"2026-01-08T10:35:39.717266176-08:00","closed_at":"2026-01-08T10:35:39.717266176-08:00","close_reason":"Removed unused vars from sanitize-files.sh, enabled warnings in shellcheck"}
{"id":"ops-jrz1-6of","title":"AI cost/rate limiting per user","description":"One user could drain API credits with runaway script. Need rate limiting per user, either via proxy middleware or opencode config. Track usage.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T15:32:30.772304538-08:00","updated_at":"2025-12-05T17:42:42.773613559-08:00","closed_at":"2025-12-05T17:42:42.773613559-08:00","dependencies":[{"issue_id":"ops-jrz1-6of","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.206816868-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-6of","depends_on_id":"ops-jrz1-wj2","type":"blocks","created_at":"2025-12-05T17:17:38.658742196-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-6t9","title":"Evaluate llm CLI: per-repo vs system-wide install","description":"Simon Willison's llm CLI tool. Options: (1) System-wide via nixpkgs, (2) Per-user via uv/pip, (3) Per-project .envrc. Consider: multiple users, plugin ecosystem, update frequency.","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-03T09:35:13.705897177-08:00","created_by":"dan","updated_at":"2026-01-03T09:35:13.705897177-08:00"}
{"id":"ops-jrz1-758","title":"VM test: Add config rendering verification","description":"Verify generated config files contain expected values (ports, DB DSN, server_name). Catches config generation bugs. Medium effort.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-08T00:58:33.180370223-08:00","created_by":"dan","updated_at":"2026-01-08T00:58:33.180370223-08:00"}
{"id":"ops-jrz1-7j4","title":"Git credential strategy for non-programmers","description":"Non-programmers can't manage SSH keys. Pre-configure git-credential-store or provide simple PAT workflow with docs. Store in persistent home with 600 perms.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T15:32:19.673999683-08:00","updated_at":"2025-12-05T17:38:54.788694408-08:00","closed_at":"2025-12-05T17:38:54.788694408-08:00","dependencies":[{"issue_id":"ops-jrz1-7j4","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.139749437-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-7qg","title":"Pin PostgreSQL to version 15 before upgrade","description":"PostgreSQL is currently NOT explicitly pinned in the NixOS config. If 24.11 defaults to a different version, the service will fail to start.\n\n## Action\nAdd to dev-services.nix or hosts/ops-jrz1.nix:\n```nix\nservices.postgresql.package = pkgs.postgresql_15;\n```\n\n## Verification\nDeploy this change BEFORE the main upgrade to confirm no breakage.\n\n## Why Critical\n- PostgreSQL data directory is version-specific\n- NixOS does NOT auto-migrate databases\n- #1 cause of upgrade failures per orch consensus","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:06:19.512966961-08:00","created_by":"dan","updated_at":"2026-01-10T16:07:17.560819957-08:00","closed_at":"2026-01-10T16:07:17.560819957-08:00","close_reason":"Pinned to postgresql_15 in dev-services.nix","dependencies":[{"issue_id":"ops-jrz1-7qg","depends_on_id":"ops-jrz1-00e","type":"parent-child","created_at":"2026-01-09T17:07:04.385150479-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-7vf","title":"Enable ssh-hardening module (imported but not active)","description":"ssh-hardening module has critical bugs that break SSH when enabled:\n\n1. UsePAM=false - NixOS SSH auth requires PAM even for key-based auth\n2. Protocol=2 - deprecated in OpenSSH 7.6+, causes sshd to crash\n3. AllowUsers default [\"admin\"] - locks out root and all dev users\n\nPartial fixes applied (removed UsePAM/Protocol, made AllowUsers conditional) but module needs full review before enabling. See commit bcfdf96.\n\nTODO:\n- [ ] Test module in VM before production\n- [ ] Verify all settings against modern OpenSSH\n- [ ] Consider removing AllowUsers entirely or rethinking default\n- [ ] Add integration test","status":"closed","priority":1,"issue_type":"bug","created_at":"2026-01-05T15:44:25.134403571-08:00","created_by":"dan","updated_at":"2026-01-05T23:09:25.390696268-08:00","closed_at":"2026-01-05T23:09:25.390696268-08:00","close_reason":"Wontfix: server already has solid SSH config (key-only, modern ciphers, no password auth). Module adds marginal hardening with deployment risk. Not worth the complexity."}
{"id":"ops-jrz1-85v","title":"npm registry blocked by Cloudflare (IP reputation)","description":"Cannot install/update/audit npm packages from VPS. Cloudflare blocking registry.npmjs.org and google.com. Likely IP reputation filtering on Vultr VPS range. Workarounds: use alternative registry, proxy through allowed endpoint, or request IP reputation review.","status":"closed","priority":2,"issue_type":"bug","created_at":"2026-01-07T10:45:54.736616259-08:00","created_by":"dan","updated_at":"2026-01-07T10:46:23.987248502-08:00","closed_at":"2026-01-07T10:46:23.987248502-08:00","close_reason":"Working now - was likely transient Cloudflare block or user hit egress rate limit (30/min new connections). Reopen if recurs."}
{"id":"ops-jrz1-86g","title":"Add per-user resource limits (not just slice-wide)","description":"Currently user.slice has TasksMax=500, MemoryMax=80%, but individual user-XXXX.slice has infinity. One user can starve others. Add per-user limits via systemd drop-ins or user-XXXX.slice config.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-03T08:40:25.937465595-08:00","created_by":"dan","updated_at":"2026-01-03T10:00:36.908904643-08:00","closed_at":"2026-01-03T10:00:36.908904643-08:00","close_reason":"Added per-user limits via activation script drop-in: MemoryMax=50%, TasksMax=200, CPUQuota=200%"}
{"id":"ops-jrz1-88o","title":"Implement backup strategy for VPS","description":"No backups configured. Critical data: Matrix DB (622M), PostgreSQL (161M), Forgejo (2.5M), maubot (320K). No recovery path if disk fails. Need automated backups with off-site storage.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-12-04T22:55:25.546850172-08:00","updated_at":"2025-12-05T00:56:27.720623612-08:00","closed_at":"2025-12-05T00:56:27.720623612-08:00"}
{"id":"ops-jrz1-8eb","title":"Obtain Spotify API credentials for musiclink","description":"Get Spotify API credentials:\n\n1. Go to https://developer.spotify.com/dashboard\n2. Create new app or use existing\n3. Get Client ID and Client Secret\n4. Add to sops secrets or musiclink config\n\nNote: Needed for Spotify link detection/conversion","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-08T12:59:08.801148544-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.95023091-08:00","closed_at":"2026-01-08T16:18:17.95023091-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves."}
{"id":"ops-jrz1-8j9","title":"Create musiclink config.toml with API credentials","description":"Create /var/lib/musiclink/config.toml with:\n\n[matterbridge]\n- url = ws://127.0.0.1:4242/api/websocket\n- token = (same as matterbridge API token)\n- gateway = \"main\"\n\n[services]\n- enabled = [\"spotify\", \"youtube\", \"apple\", \"deezer\"]\n\n[services.spotify]\n- client_id, client_secret from Spotify Developer Dashboard\n\n[services.youtube]\n- api_key from Google Cloud Console (optional, can skip initially)\n\nApple Music and Deezer don't need credentials.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-08T12:59:08.362889466-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.947482218-08:00","closed_at":"2026-01-08T16:18:17.947482218-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves.","dependencies":[{"issue_id":"ops-jrz1-8j9","depends_on_id":"ops-jrz1-qgm","type":"blocks","created_at":"2026-01-08T15:56:08.129810568-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-8j9","depends_on_id":"ops-jrz1-8eb","type":"blocks","created_at":"2026-01-08T15:56:08.167928618-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-8m7","title":"Add cgroups limits for user slices","description":"Add soft resource limits to prevent one user/agent from crashing server.\n\n## Config\n```nix\nsystemd.slices.\"user\".sliceConfig = {\n MemoryMax = \"80%\";\n TasksMax = 500;\n CPUWeight = 100; # Fair sharing, no hard quota\n};\n```\n\n## Behavior\n- Memory: Users collectively can't exceed 80% RAM\n- Tasks: Max 500 processes per user (prevents fork bombs)\n- CPU: Fair sharing when contended, bursts allowed\n\n## Testing\n- Verify with `systemctl show user-1001.slice`\n- Test fork bomb doesn't crash server","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T20:16:22.600133044-08:00","created_by":"dan","updated_at":"2026-01-02T21:02:35.455928291-08:00","closed_at":"2026-01-02T21:02:35.455928291-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-8mc","title":"configuration.nix: Document UID range 1000:65534 rationale","description":"UID range 1000:65534 excludes root but includes nobody (65534). Add comment explaining rationale. configuration.nix:70","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-03T08:17:35.893969961-08:00","created_by":"dan","updated_at":"2026-01-03T09:32:23.604873295-08:00","closed_at":"2026-01-03T09:32:23.604873295-08:00","close_reason":"Added comment explaining UID range 1000:65534"}
{"id":"ops-jrz1-8mm","title":"Consolidate olm insecure package permission to one location","description":"olm-3.2.16 is permitted in 3 places: configuration.nix:177-179, flake.nix:47-49, flake.nix:70-72. Redundant. Consolidate to one location.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-05T15:44:41.269512798-08:00","created_by":"dan","updated_at":"2026-01-05T22:54:52.747856599-08:00","closed_at":"2026-01-05T22:54:52.747856599-08:00","close_reason":"Removed 1 redundant location. Full consolidation not possible: maubot needs it in nixpkgs, mautrix-slack needs it in pkgs-unstable."}
{"id":"ops-jrz1-92t","title":"Deploy mosh for mobile shell access","description":"Add mosh package and UDP ports 60000-60010. Config already in configuration.nix, just needs deploy.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-05T17:25:32.52879135-08:00","created_by":"dan","updated_at":"2026-01-05T17:30:57.233018578-08:00","closed_at":"2026-01-05T17:30:57.233018578-08:00","close_reason":"Deployed mosh package and UDP ports 60000-60010"}
{"id":"ops-jrz1-93q9","title":"Add offline sops recovery key","description":"Currently sops decryption key is derived from SSH host key. If host key is lost, all secrets are unrecoverable. Add a second age recipient key stored offline in password manager.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-10T14:01:51.151422159-08:00","created_by":"dan","updated_at":"2026-01-10T15:40:40.908254691-08:00","closed_at":"2026-01-10T15:40:40.908254691-08:00","close_reason":"Added recovery age key at ~/.config/sops/age/recovery.key. Secrets now encrypted to 3 recipients. Tested decryption with recovery key."}
{"id":"ops-jrz1-98q","title":"Update dev-remove.sh to revoke Forgejo keys","description":"Make Forgejo key revocation MANDATORY in dev-remove.sh.\n\n## Why mandatory (from threat model analysis)\n- Zombie keys = persistence after offboarding\n- 'Warnings don't work' - if not automated, won't happen\n- Keys allow access even after Unix account deleted\n\n## Implementation\n```bash\nrevoke_forgejo_keys() {\n local username=\"$1\"\n local token_file=\"/run/secrets/forgejo-api-token\"\n \n # List user's keys\n # DELETE /api/v1/admin/users/{username}/keys/{id}\n # for each key\n}\n```\n\n## Fallback\nIf API call fails, still print warning but make it clear this is a FAILURE state, not normal.\n\n## Alternative considered\nCould delete Forgejo user entirely, but that may delete repos/PRs (too destructive).","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:47:59.501623842-08:00","created_by":"dan","updated_at":"2026-01-09T19:36:04.34309976-08:00","closed_at":"2026-01-09T19:36:04.34309976-08:00","close_reason":"Implemented in commit d9c1848","dependencies":[{"issue_id":"ops-jrz1-98q","depends_on_id":"ops-jrz1-rfx","type":"parent-child","created_at":"2026-01-09T17:48:12.658838096-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-9gd","title":"Upgrade VPS RAM for dev environments","description":"Current: 2GB. Need 4-8GB for multiple code-server containers. Coordinate with Vultr, plan maintenance window.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T17:16:54.267689439-08:00","updated_at":"2025-12-28T00:08:06.748175273-05:00","closed_at":"2025-12-28T00:08:06.748175273-05:00","close_reason":"Browser-based dev environment cancelled","dependencies":[{"issue_id":"ops-jrz1-9gd","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:17:36.331146543-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-9hq","title":"Add earlier root check to dev-add.sh","description":"dev-add.sh has EUID check at line 138, but could fail earlier with clearer message before doing validation work.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-03T17:35:58.633740163-08:00","created_by":"dan","updated_at":"2026-01-03T17:35:58.633740163-08:00"}
{"id":"ops-jrz1-9pe","title":"Research: System packages for learner accounts","description":"How do dev users get access to toolchains (Go, Node, Rust, etc.)?\n\n## Findings\n\n**Users CAN self-install packages:**\n```bash\nnix profile install nixpkgs#go\nnix profile install nixpkgs#nodejs\nnix profile install nixpkgs#rustc\n```\n\nPackages go to `~/.nix-profile/bin`, already in PATH. Works today.\n\n**Devshells work too:**\n```bash\n# In project with flake.nix\nnix develop\n```\n\n## Options\n\n| Option | Pros | Cons |\n|--------|------|------|\n| **Self-service only** | Minimal config, user learns nix | Cold start friction |\n| **Global defaults** | Zero friction for common tools | Bloats system, version conflicts |\n| **Starter script** | One command setup, customizable | Another thing to maintain |\n| **direnv + devshells** | Per-project envs, reproducible | Needs direnv installed globally |\n\n## Current State\n- `nix profile install` works for users ✅\n- `nix develop` works ✅\n- direnv NOT installed globally\n- Only python3, uv in system packages\n\n## Recommendation\n1. Add `direnv` to global packages (enables per-project devshells)\n2. Document `nix profile install` for quick one-offs\n3. Provide example flake.nix templates for Go, Node, Rust projects\n4. Keep system packages minimal (python3, uv, direnv, git, vim)\n\n## Test Results\n```\n$ nix profile install nixpkgs#go\n$ go version\ngo version go1.22.8 linux/amd64\n```","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T12:27:32.894163417-08:00","created_by":"dan","updated_at":"2026-01-02T12:32:32.502649201-08:00","closed_at":"2026-01-02T12:32:32.502649201-08:00","close_reason":"Users can self-install via nix profile. Added direnv globally for devshells."}
{"id":"ops-jrz1-9sy","title":"Consolidate repeated sops.* keys in hosts/ops-jrz1.nix","description":"statix W20: sops.defaultSopsFile, sops.age.*, sops.secrets.* scattered. Consolidate into single sops = { ... } block.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-07T10:45:11.853476629-08:00","created_by":"dan","updated_at":"2026-01-08T12:44:06.433830146-08:00","closed_at":"2026-01-08T12:44:06.433830146-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-9x8","title":"Claude CLI update mechanism","description":"Claude Code CLI is manually installed to /usr/local/bin/claude.\n\n## Current state\n- Installed via: curl -fsSL https://claude.ai/install.sh | bash\n- Copied to /usr/local/bin/claude\n- No automatic updates\n\n## Options\n1. Periodic manual update (run install script again)\n2. Systemd timer to check for updates\n3. Package via nix (would need custom derivation)\n\n## Acceptance criteria\nDocument the update process at minimum.","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-02T16:46:03.908575951-08:00","created_by":"dan","updated_at":"2026-01-02T16:46:03.908575951-08:00"}
{"id":"ops-jrz1-a3w","title":"egress-watchdog: Document GNU grep -P dependency","description":"grep -oP requires GNU grep with PCRE. NixOS always has it, but add comment for portability awareness. scripts/egress-watchdog:25","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-03T08:17:35.642729617-08:00","created_by":"dan","updated_at":"2026-01-03T09:32:23.646375858-08:00","closed_at":"2026-01-03T09:32:23.646375858-08:00","close_reason":"Added comment noting grep -oP requires GNU grep with PCRE"}
{"id":"ops-jrz1-a9d","title":"Deploy NixOS 24.11 with boot and reboot","description":"Deploy the upgrade using boot (not switch) for cleaner service restarts.\n\n## Steps\n1. Ensure backup is verified (ops-jrz1-xxx)\n2. Deploy: `nixos-rebuild boot --flake .#ops-jrz1 --target-host root@ops-jrz1`\n3. Note the new generation number\n4. Reboot: `ssh root@ops-jrz1 reboot`\n5. Wait for server to come back (~2 min)\n6. Verify SSH access\n\n## If boot fails\n- VNC console to access bootloader\n- Select previous generation\n- Boot and diagnose","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:06:49.241721331-08:00","created_by":"dan","updated_at":"2026-01-10T18:26:57.764571244-08:00","closed_at":"2026-01-10T18:26:57.764571244-08:00","close_reason":"Deployed gen 72, all services active, NixOS 24.11 running","dependencies":[{"issue_id":"ops-jrz1-a9d","depends_on_id":"ops-jrz1-00e","type":"parent-child","created_at":"2026-01-09T17:07:09.732906874-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-a9d","depends_on_id":"ops-jrz1-3wd","type":"blocks","created_at":"2026-01-09T17:07:54.572445895-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-a9d","depends_on_id":"ops-jrz1-asi","type":"blocks","created_at":"2026-01-09T17:07:54.594901975-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-ab6","title":"Enable fail2ban for SSH brute force protection","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-05T11:13:47.305821-08:00","created_by":"dan","updated_at":"2026-01-05T11:13:54.497754296-08:00","closed_at":"2026-01-05T11:13:54.497754296-08:00","close_reason":"Duplicate of ops-jrz1-gci"}
{"id":"ops-jrz1-asi","title":"Take verified backup before NixOS upgrade","description":"Full backup of all stateful data before upgrade.\n\n## What to backup\n- PostgreSQL (all databases): pg_dumpall\n- Forgejo repos: /var/lib/forgejo\n- Matrix state: /var/lib/matrix-continuwuity\n- Maubot: /var/lib/maubot\n- Secrets: /run/secrets (or source sops file)\n- Current NixOS generation: note the generation number\n\n## Verification\n- Test backup restorability (at minimum: gzip -t on archives)\n- Store off-server if possible\n\n## Note\nRollback restores binaries/config, NOT database state. This backup is the real safety net.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:06:48.908603718-08:00","created_by":"dan","updated_at":"2026-01-10T16:18:26.598623173-08:00","closed_at":"2026-01-10T16:18:26.598623173-08:00","close_reason":"Restore drill passed: PostgreSQL dumps, Forgejo repos, home dirs all verified","dependencies":[{"issue_id":"ops-jrz1-asi","depends_on_id":"ops-jrz1-00e","type":"parent-child","created_at":"2026-01-09T17:07:09.665384279-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-asi","depends_on_id":"ops-jrz1-7qg","type":"blocks","created_at":"2026-01-09T17:07:54.478050281-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-av0","title":"Configure wildcard DNS and ACME cert","description":"Set up *.code.clarun.xyz DNS record and wildcard SSL cert via ACME. Depends on subdomain routing decision (kg0).","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T17:16:54.387356964-08:00","updated_at":"2026-01-08T16:28:58.851235463-08:00","closed_at":"2026-01-08T16:28:58.851235463-08:00","close_reason":"Parent epic 3so (browser-based dev) closed as won't-fix. No current use case for wildcard DNS.","dependencies":[{"issue_id":"ops-jrz1-av0","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:17:36.34918436-08:00","created_by":"daemon","metadata":"{}"},{"issue_id":"ops-jrz1-av0","depends_on_id":"ops-jrz1-kg0","type":"blocks","created_at":"2025-12-05T17:17:38.676800677-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-ayl","title":"Rename sna-instagram-bot to something memorable","notes":"Deferred - active development using this bot on server","status":"open","priority":3,"issue_type":"task","created_at":"2025-12-08T16:54:31.223265094-08:00","updated_at":"2026-01-09T16:01:06.938166124-08:00"}
{"id":"ops-jrz1-b09","title":"Research: Forgejo backup strategy","description":"Consider backup strategy for the Forgejo instance.\n\n## Questions\n- What data needs backing up? (repos, issues, users, config)\n- Where to back up to? (off-site, object storage, etc.)\n- Frequency?\n- Restore procedure?\n\n## Context\nForgejo runs on ops-jrz1 at :3000, stores git repos and metadata.","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-02T19:01:55.854723242-08:00","created_by":"dan","updated_at":"2026-01-02T19:01:55.854723242-08:00"}
{"id":"ops-jrz1-b8v","title":"Enable nix-ld for VS Code Remote SSH","description":"VS Code Remote-SSH requires nix-ld to run the VS Code Server on NixOS.\n\n## Status\n- [x] Config added to hosts/ops-jrz1.nix\n- [x] nix flake check passed\n- [ ] Deploy to server\n\n## Config Added\n```nix\nprograms.nix-ld.enable = true;\nprograms.nix-ld.libraries = with pkgs; [\n stdenv.cc.cc.lib\n zlib\n openssl\n];\n```\n\n## Blocks\n- VS Code Remote-SSH (won't work without this)\n- Testing Claude Code extension over Remote-SSH\n- JetBrains Gateway (same issue)\n\n## Deploy Command\n```bash\nnixos-rebuild switch --flake .#ops-jrz1 --target-host root@ops-jrz1 --build-host localhost\n```","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-30T22:24:15.465531482-05:00","created_by":"dan","updated_at":"2026-01-02T10:22:07.3354851-08:00","closed_at":"2026-01-02T10:22:07.3354851-08:00","close_reason":"Deployed - nix-ld active at /lib64/ld-linux-x86-64.so.2"}
{"id":"ops-jrz1-bb5","title":"Test dual-key git workflow end-to-end","description":"Verify the complete git workflow works.\n\n## Test cases\n\n### From server (primary use case)\n1. SSH into server as test user\n2. `git clone git@git.clarun.xyz:dan/ops-jrz1.git`\n3. Make a change, commit\n4. `git push` → should succeed\n\n### From laptop (if externally accessible)\n1. Add git.clarun.xyz to laptop's known_hosts\n2. `git clone git@git.clarun.xyz:dan/ops-jrz1.git`\n3. Push → should work with laptop key\n\n### Error cases\n- User without Forgejo account → clear error message\n- User without repo access → clear error message\n- Wrong key → \"Permission denied (publickey)\"\n\n## Cleanup\nRemove test commits/branches after verification","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-09T17:48:01.887022617-08:00","created_by":"dan","updated_at":"2026-01-09T19:36:04.348735003-08:00","closed_at":"2026-01-09T19:36:04.348735003-08:00","close_reason":"Implemented in commit d9c1848","dependencies":[{"issue_id":"ops-jrz1-bb5","depends_on_id":"ops-jrz1-rfx","type":"parent-child","created_at":"2026-01-09T17:48:12.706061398-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-bb5","depends_on_id":"ops-jrz1-1t1","type":"blocks","created_at":"2026-01-09T17:49:06.901096245-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-bbn","title":"Research: Resource limits and quotas","description":"Should we limit CPU/memory/disk per learner?\n\n## Current state\n- No limits configured\n- Single VPS shared by all users\n- 2GB RAM, 1 vCPU (Vultr $6 tier?)\n\n## Options\n1. **No limits** - Trust learners, monitor manually\n2. **Systemd slices** - cgroups for user sessions\n3. **Disk quotas** - Limit ~/\n4. **ulimits** - Process limits\n\n## Questions\n- What resources does a typical dev session use?\n- What about `go build` or `npm install`?\n- Is this premature optimization?","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-02T12:27:34.884865507-08:00","created_by":"dan","updated_at":"2026-01-03T10:11:09.79120117-08:00","closed_at":"2026-01-03T10:11:09.79120117-08:00","close_reason":"Implemented: systemd slices (user.slice MemoryMax=80%, TasksMax=500) + per-user limits (MemoryMax=50%, TasksMax=200, CPUQuota=200%). Disk quotas tracked separately in ops-jrz1-oxx."}
{"id":"ops-jrz1-bhk","title":"Add disk quotas for user workspaces","description":"User could fill host disk via /var/lib/vscode/\u003cuser\u003e/. Add per-directory quotas or monitoring/alerting on disk usage.","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-05T15:32:41.199417226-08:00","updated_at":"2025-12-28T00:05:44.7635372-05:00","closed_at":"2025-12-28T00:05:44.7635372-05:00","close_reason":"Parent epic cancelled - browser-based dev approach abandoned","dependencies":[{"issue_id":"ops-jrz1-bhk","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.309592029-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-blh","title":"mautrix-slack edit panic persists in v25.11","description":"mautrix-slack panic on rapid message edits (race condition)\n\n**Root cause**: Edit event arrives before original message is stored in DB. ConvertEdit accesses nil metadata.\n\n**Location**: handleslack.go:575 - has TODO comment: 'this can panic?'\n\n**Reproduction**: Edit a Slack message within ~1 second of sending\n\n**Upstream status**: \n- v25.11 is latest (we're on it)\n- Known to devs (TODO in code)\n- No open issue filed yet\n\n**Stack trace**:\ngo.mau.fi/mautrix-slack/pkg/connector.(*SlackMessage).ConvertEdit\n handleslack.go:575\nmaunium.net/go/mautrix/bridgev2.(*Portal).handleRemoteEdit\n portal.go:2838","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-05T19:40:33.255395189-08:00","updated_at":"2025-12-28T00:06:14.637057055-05:00","closed_at":"2025-12-28T00:06:14.637057055-05:00","close_reason":"Duplicate of ops-jrz1-f15 which has fix ready","comments":[{"id":2,"issue_id":"ops-jrz1-blh","author":"dan","text":"Confirmed panic exists in nixpkgs-unstable from 2025-12-02. Fix will be addressed via platform upgrade (see ops-jrz1-00e).","created_at":"2025-12-08T23:54:57Z"}]}
{"id":"ops-jrz1-bom","title":"Consolidate repeated attribute keys in configuration.nix","description":"statix W20: networking and environment keys are repeated across the file. Consolidate into single blocks for readability.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-03T17:35:57.496164613-08:00","created_by":"dan","updated_at":"2026-01-08T16:36:50.918598935-08:00","closed_at":"2026-01-08T16:36:50.918598935-08:00","close_reason":"Duplicate of 44x - already completed this session"}
{"id":"ops-jrz1-c0i","title":"Investigate intermittent DNS/connectivity issues","description":"SSH connections timing out intermittently. May be DNS resolution or network instability. Investigate: (1) DNS resolver config, (2) Network interface stability, (3) Firewall rules interaction, (4) VPS provider network issues.","status":"closed","priority":2,"issue_type":"bug","created_at":"2026-01-04T16:43:45.330599724-08:00","created_by":"dan","updated_at":"2026-01-04T16:56:24.821706332-08:00","closed_at":"2026-01-04T16:56:24.821706332-08:00","close_reason":"DNS healthy: 3-7ms resolution, 0% packet loss, interface stable. Timeout was transient."}
{"id":"ops-jrz1-cboo","title":"Setup and integrate pi-mono toolkit","description":"Goal: Setup the pi-mono toolset (specifically pi-coding-agent and pi-mom) on ops-jrz1.\\n\\nFound Information:\\n- Source: /tmp/pi-mono (TS monorepo)\\n- Core Tools: pi-coding-agent (CLI), pi-mom (Slack bridge), pi-ai (LLM API).\\n- Workflow: standard npm install/build/check. Requires build before check.\\n- Constraints: Strict dev rules in AGENTS.md (no inline imports, short style, mandatory check after edits).\\n- Relevance: pi-mom can delegate Slack messages to the coding agent, aligning with our current bridge infrastructure.","status":"open","priority":2,"issue_type":"task","owner":"dleink@gmail.com","created_at":"2026-01-20T15:37:55.231274188-08:00","created_by":"Dan","updated_at":"2026-01-20T15:37:55.231274188-08:00"}
{"id":"ops-jrz1-cmv","title":"Add egress rate limiting (iptables)","description":"Hard limit outbound connections per user to prevent mass exfil/scanning.\n\n## Config\n```nix\nnetworking.firewall.extraCommands = ''\n # Rate limit new outbound connections for regular users (uid 1000+)\n iptables -A OUTPUT -m state --state NEW -m owner --uid-owner 1000:65534 \\\n -m limit --limit 30/min --limit-burst 60 -j ACCEPT\n iptables -A OUTPUT -m state --state NEW -m owner --uid-owner 1000:65534 \\\n -j LOG --log-prefix \"EGRESS-LIMIT: \"\n iptables -A OUTPUT -m state --state NEW -m owner --uid-owner 1000:65534 \\\n -j REJECT\n'';\n```\n\n## Behavior\n- 30 new connections/min sustained, burst of 60\n- Over limit: logged and rejected\n- Doesn't affect established connections\n\n## Testing\n- `for i in {1..100}; do curl -s ifconfig.me \u0026 done`\n- Should see EGRESS-LIMIT in journal after ~60","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T20:16:32.276607792-08:00","created_by":"dan","updated_at":"2026-01-02T21:12:35.5888406-08:00","closed_at":"2026-01-02T21:12:35.5888406-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-cpm","title":"Add admin alerting when killswitch fires","description":"Killswitch only logs to journald. Consider external notification (email, Matrix message, webhook) when users are terminated.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-03T08:40:26.416998752-08:00","created_by":"dan","updated_at":"2026-01-03T08:40:26.416998752-08:00"}
{"id":"ops-jrz1-cs2","title":"Docs deployment strategy: AGENTS.md for bots, README.md for humans","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-07T10:15:27.94438992-08:00","created_by":"dan","updated_at":"2026-01-07T10:15:27.94438992-08:00"}
{"id":"ops-jrz1-cxf","title":"VM test: Add nginx reverse proxy test","description":"Test nginx with forceSSL=false or self-signed cert. Consensus strongly recommends this - nginx is a high failure point and testing direct :8008 misses header forwarding issues (X-Forwarded-For, etc).","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-08T00:58:32.506341558-08:00","created_by":"dan","updated_at":"2026-01-08T07:10:30.057772105-08:00","closed_at":"2026-01-08T07:10:30.057772105-08:00","close_reason":"Implemented plain HTTP nginx test with recommendedProxySettings"}
{"id":"ops-jrz1-d38","title":"Add tmux to system packages","description":"Add tmux for session persistence. Users can run bots in tmux, disconnect, reconnect.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T15:13:58.514256583-08:00","created_by":"dan","updated_at":"2026-01-02T17:25:59.102158299-08:00","closed_at":"2026-01-02T17:25:59.102158299-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-d58","title":"Build custom code-server container image","description":"Dockerfile with: code-server, opencode CLI, opencode VS Code extension (Open VSX), Python, Node, Git. Push to registry or build locally.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T17:16:54.507577308-08:00","updated_at":"2025-12-28T00:05:44.736614157-05:00","closed_at":"2025-12-28T00:05:44.736614157-05:00","close_reason":"Parent epic cancelled - browser-based dev approach abandoned","dependencies":[{"issue_id":"ops-jrz1-d58","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:17:36.369590207-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-d8o","title":"Add --dry-run flag to dev-remove.sh","description":"scripts/dev-remove.sh has no way to preview what would be deleted before committing. Add --dry-run flag that shows actions without executing them.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-05T15:44:33.81954695-08:00","created_by":"dan","updated_at":"2026-01-05T19:40:15.319784807-08:00","closed_at":"2026-01-05T19:40:15.319784807-08:00","close_reason":"Implemented --dry-run flag for dev-remove.sh"}
{"id":"ops-jrz1-dg9","title":"Document pattern for adding dev tools to system","description":"Create documentation for the standard pattern of adding dev tools.\n\n## Pattern\n1. Add flake input (if not in nixpkgs)\n2. Add to environment.systemPackages\n3. Run nixos-rebuild switch\n4. Config stays per-user/per-repo\n\n## Document should cover\n- How to add a tool from nixpkgs\n- How to add a tool from external flake\n- How to package a tool not yet packaged\n- How to update a tool (flake lock update)\n\n## Location\ndocs/adding-dev-tools.md or similar","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-02T16:36:04.613581812-08:00","created_by":"dan","updated_at":"2026-01-03T10:47:43.998225051-08:00","closed_at":"2026-01-03T10:47:43.998225051-08:00","close_reason":"Created docs/adding-dev-tools.md covering system-wide, per-user, per-project, and external flake methods"}
{"id":"ops-jrz1-dhj","title":"Port forwarding strategy for user apps","description":"When user runs app on localhost:3000, how do they view it? code-server has /proxy/\u003cport\u003e but URL is confusing for learners. Need clear UX or docs.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T15:32:30.649292743-08:00","updated_at":"2025-12-05T17:41:01.486505687-08:00","closed_at":"2025-12-05T17:41:01.486505687-08:00","dependencies":[{"issue_id":"ops-jrz1-dhj","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.175857247-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-dt9","title":"Increase container RAM limits (2GB too tight)","description":"2GB hard limit will OOM with code-server + opencode + LSP + user app. Gemini/GPT recommend 3-4GB per container or add swap. Need to size server appropriately.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T15:32:19.400171408-08:00","updated_at":"2025-12-05T17:38:54.770433169-08:00","closed_at":"2025-12-05T17:38:54.770433169-08:00","dependencies":[{"issue_id":"ops-jrz1-dt9","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.066130377-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-dux","title":"Container isolation: maubot API access only","description":"Security design for learner containers:\n\n**Container CAN access**:\n- maubot API (:29316) for plugin deploy\n- Matrix rooms via bot (through maubot)\n- Slack via bridge (through Matrix)\n\n**Container CANNOT access**:\n- Host filesystem\n- Other containers\n- PostgreSQL directly\n- Matrix homeserver directly\n- sops secrets\n\nImplementation: Podman network config, no --privileged, limited port exposure.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-06T12:18:16.212646624-08:00","updated_at":"2025-12-28T10:12:31.107338781-05:00","closed_at":"2025-12-28T10:12:31.107338781-05:00","close_reason":"Depends on cancelled browser-dev epic (ops-jrz1-3so)","dependencies":[{"issue_id":"ops-jrz1-dux","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-06T12:18:21.627621772-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-dw7","title":"Review opencode settings and configuration","description":"Investigate opencode CLI configuration options. What settings are available? Where does config live? Any server-specific tuning needed?","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-03T10:46:29.904950557-08:00","created_by":"dan","updated_at":"2026-01-03T10:46:29.904950557-08:00"}
{"id":"ops-jrz1-e0s","title":"Deploy MusicLink bot with matterbridge","description":"Deploy the MusicLink bot created by a dev. The bot detects music links (Spotify, YouTube, Apple Music, etc.) in Slack and responds with equivalent links on other streaming services.\n\n## Architecture\n- matterbridge: Bridges Slack to local WebSocket API (nixpkgs module)\n- musiclink: Go bot connecting to matterbridge API, does link detection/conversion\n\n## Location\n- Source: /home/dan/proj/musiclink on VPS\n- Docs: /home/dan/proj/musiclink/docs/platform-setup.md\n\n## Dependencies\n- Slack bot token (reuse existing or create new app)\n- Spotify API credentials\n- YouTube API key (optional)\n- matterbridge ↔ musiclink shared token\n\n## Services\n- services.matterbridge (existing nixpkgs module)\n- systemd.services.musiclink (custom, with hardening)","status":"closed","priority":2,"issue_type":"feature","created_at":"2026-01-08T12:58:24.210407462-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.932458481-08:00","closed_at":"2026-01-08T16:18:17.932458481-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves."}
{"id":"ops-jrz1-ecw","title":"Support multiple agentic coders: opencode, gemini, codex","description":"Ensure dev users can use their choice of agentic coding tool, not just Claude.\n\n## Tools Supported\n\n| Tool | Status | Install Method |\n|------|--------|----------------|\n| Claude Code | ✅ /usr/local/bin/claude | Manual install |\n| opencode | ✅ System package | Flake input (github:sst/opencode) |\n| gemini-cli | ✅ User install | `npm install -g @google/gemini-cli` |\n| codex | ✅ User install | `npm install -g @openai/codex` |\n\n## System Packages Added\n- `opencode` - AI coding agent (v1.0.224)\n- `nodejs_22` - For npm-based tool installation\n\n## User Install Instructions\nFor gemini-cli or codex:\n```bash\nnpm install -g @google/gemini-cli\nnpm install -g @openai/codex\n```\n\n## Auth Storage\nSee ops-jrz1-xz7 for multi-user auth research.\n\n## Remaining\n- gastown (gt) blocked - no release binaries yet (ops-jrz1-30e)","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T16:25:11.578992492-08:00","created_by":"dan","updated_at":"2026-01-02T17:41:40.793211973-08:00","closed_at":"2026-01-02T17:41:40.793211973-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-ezf","title":"Maubot plugin dev workflow for learners","description":"Design frictionless dev workflow for Python/Go learners building maubot plugins.\n\n**Requirements**:\n- No SSH tunnel setup for learners\n- Fast feedback loop (edit → see bot respond)\n- Circuit breakers (allowed_rooms, rate limits)\n- Test channel: #vlads-pad (Slack) ↔ Matrix\n\n**Options being considered**:\n1. Git-push deploy: push to repo → CI builds .mbp → deploys to maubot\n2. Code-server containers: browser IDE on VPS, deploy script talks to maubot locally\n3. Hybrid: code-server + git workflow\n\n**Related**: ops-jrz1-3so (browser-dev-environment epic)","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-12-06T01:36:26.529372206-08:00","updated_at":"2025-12-28T10:12:31.096280407-05:00","closed_at":"2025-12-28T10:12:31.096280407-05:00","close_reason":"Depends on cancelled browser-dev epic (ops-jrz1-3so)","dependencies":[{"issue_id":"ops-jrz1-ezf","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-06T12:18:06.743837766-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-f15","title":"mautrix-slack ConvertEdit panic on message edits","description":"mautrix-slack v25.11 panics with nil pointer dereference at handleslack.go:575 when processing Slack message_changed events. \n\nFIX READY at /tmp/mautrix-slack-fix (branch: fix-convert-edit-nil-panic)\n\nThree changes needed in pkg/connector/handleslack.go:\n1. Line 181: Add 'evt.SubMessage != nil \u0026\u0026' before evt.SubMessage.SubType\n2. Lines 573-578: Add nil checks for SubMessage and SubMessage.Edited at start of ConvertEdit()\n3. Line 609: Add 's.Data.SubMessage != nil \u0026\u0026' before accessing Timestamp\n\nTo submit PR: fork mautrix/slack, push branch, open PR against main.","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-27T10:03:28.857940916-05:00","created_by":"dan","updated_at":"2025-12-28T10:02:58.660206297-05:00","closed_at":"2025-12-28T10:02:58.660206297-05:00","close_reason":"Won't fix for now - fix ready in /tmp/mautrix-slack-fix if needed later"}
{"id":"ops-jrz1-f6i","title":"Update docs for declarative script workflow","description":"Update AGENTS.md and any onboarding docs to remove manual scp deployment steps. Scripts now deploy automatically with nixos-rebuild.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T08:39:54.759066801-08:00","created_by":"dan","updated_at":"2026-01-03T09:21:09.793836001-08:00","closed_at":"2026-01-03T09:21:09.793836001-08:00","close_reason":"Updated AGENTS.md with declarative script deployment info","dependencies":[{"issue_id":"ops-jrz1-f6i","depends_on_id":"ops-jrz1-sdz","type":"blocks","created_at":"2026-01-03T08:40:02.875606451-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-fs2","title":"Enable psacct for lightweight command audit logging","description":"Enable process accounting (services.acct.enable = true) on the dev server. Provides 'who ran what, when' visibility without capturing arguments. Query with lastcomm/sa. Lightweight, kernel-level (can't be bypassed), minimal overhead. Consensus from flash-or + gpt recommended this over auditd (too verbose) and shell history (bypassable).","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-05T11:16:24.469058505-08:00","created_by":"dan","updated_at":"2026-01-05T11:16:24.469058505-08:00"}
{"id":"ops-jrz1-gci","title":"Enable fail2ban for SSH brute force protection","description":"SSH brute force attempts generate log noise but don't pose security risk (key-only auth). fail2ban would help but is low priority. Deferred pending RFC on SSH log management strategy.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-04T21:03:22.651495544-08:00","updated_at":"2026-01-05T18:40:08.35294139-08:00","closed_at":"2026-01-05T18:40:08.35294139-08:00","close_reason":"Duplicate of ops-jrz1-0nt","dependencies":[{"issue_id":"ops-jrz1-gci","depends_on_id":"ops-jrz1-nir","type":"blocks","created_at":"2025-12-04T22:56:14.777377818-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-ghd","title":"Research: User isolation and sandboxing options","description":"Can users see each other's code? What isolation do we need?\n\n## Current state\n- Standard Unix permissions\n- No containers or VMs\n- Users in same `users` group\n\n## Options to evaluate\n1. **Unix perms only** - `chmod 700 ~` (current)\n2. **Separate groups** - Per-user primary group\n3. **Containers** - systemd-nspawn, podman\n4. **VMs** - microVMs per user (heavy)\n5. **Firejail/bubblewrap** - Process sandboxing\n\n## Questions\n- What's the threat model? (curious learners vs malicious actors)\n- How much isolation is worth the complexity?\n- Does NixOS offer anything special here?","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T12:27:33.291275677-08:00","created_by":"dan","updated_at":"2026-01-09T16:31:11.739552144-08:00","closed_at":"2026-01-09T16:31:11.739552144-08:00","close_reason":"Unix perms sufficient for learning environment, documented in server-AGENTS.md"}
{"id":"ops-jrz1-glk","title":"VS Code extension policy (security)","description":"Extensions can run arbitrary code. Decide: allow arbitrary installs, or curate/restrict? For non-programmers, pre-install safe set and optionally disable marketplace.","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-05T15:32:41.463030936-08:00","updated_at":"2025-12-28T00:08:06.752037252-05:00","closed_at":"2025-12-28T00:08:06.752037252-05:00","close_reason":"Browser-based dev environment cancelled","dependencies":[{"issue_id":"ops-jrz1-glk","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.372120465-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-gwk","title":"Declarative script deployment via NixOS","description":"Scripts in scripts/ are manually deployed via scp to /usr/local/bin/. Convert to NixOS declarative deployment using writeShellApplication.\n\nTwo packages (per orch consensus):\n1. watchdog-scripts - killswitch, cpu-watchdog, egress-watchdog\n → Referenced directly by systemd, NOT in PATH\n2. admin-scripts - learner-add.sh, learner-remove.sh\n → Added to systemPackages for interactive use\n\nSubtasks:\n- ops-jrz1-vw4: Create watchdog-scripts package\n- ops-jrz1-o9c: Create admin-scripts package (parallel)\n- ops-jrz1-ujw: Update systemd services to use store paths\n- ops-jrz1-sdz: Remove manual /usr/local/bin scripts\n- ops-jrz1-f6i: Update docs","status":"closed","priority":2,"issue_type":"epic","created_at":"2026-01-03T08:39:41.242421474-08:00","created_by":"dan","updated_at":"2026-01-03T09:21:09.83027789-08:00","closed_at":"2026-01-03T09:21:09.83027789-08:00","close_reason":"Epic complete: All scripts now deployed declaratively via writeShellApplication"}
{"id":"ops-jrz1-heb2","title":"Arithmetic increment without || true","description":"[BLAST-RADIUS] dev-remove.sh:84,87\n\n```bash\n((revoked++))\n((failed++))\n```\n\nIssue: Arithmetic with `set -e` can exit on zero - mitigated by `|| true` elsewhere but these lines don't have it\n\nSuggest: Use `((revoked++)) || true` for consistency, or `revoked=$((revoked + 1))`\n\nSource: ops-review","status":"closed","priority":4,"issue_type":"chore","created_at":"2026-01-09T19:50:52.81062048-08:00","created_by":"dan","updated_at":"2026-01-09T20:16:24.047884065-08:00","closed_at":"2026-01-09T20:16:24.047884065-08:00","close_reason":"Added || true to both arithmetic increments for consistency and set -e safety"}
{"id":"ops-jrz1-hjs","title":"Evaluate wall/talk for dev-to-dev messaging","description":"Look at classic Unix messaging utilities (wall, talk, write, mesg) for dev-to-dev communication. Goal: make the server feel like a university CS department Unix box - collaborative, retro, educational vibe. Quick terminal pings, 'hey about to reboot', pair debugging invites. Part of the shared-machine culture.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-05T11:14:14.813033486-08:00","created_by":"dan","updated_at":"2026-01-05T15:32:22.430901047-08:00","closed_at":"2026-01-05T15:32:22.430901047-08:00","close_reason":"Added bsd-finger, ytalk, fortune to systemPackages. Fortune on login via programs.bash.interactiveShellInit. Classic Unix social tools now available."}
{"id":"ops-jrz1-hkb","title":"Create matterbridge.toml config with Slack + API bridge","description":"Create /var/lib/matterbridge/matterbridge.toml with:\n\n1. [slack.workspace] - Connect to Slack using bot token\n2. [api.musiclink] - Local WebSocket API on 127.0.0.1:4242\n3. [[gateway]] - Route messages between Slack and API\n\nTemplate in /home/dan/proj/musiclink/docs/platform-setup.md\n\nSecrets needed:\n- Slack bot token (xoxb-...)\n- API token (generate with openssl rand -hex 32)","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-08T12:59:07.733852326-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.942095796-08:00","closed_at":"2026-01-08T16:18:17.942095796-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves.","dependencies":[{"issue_id":"ops-jrz1-hkb","depends_on_id":"ops-jrz1-kpw","type":"blocks","created_at":"2026-01-08T15:56:08.043060147-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-hkb","depends_on_id":"ops-jrz1-mul","type":"blocks","created_at":"2026-01-08T15:56:08.311144656-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-i1g","title":"Pin beads and opencode flake inputs to commit hashes","description":"flake.nix:13-21 - beads and opencode inputs track HEAD, not pinned to specific commits. Builds may break unexpectedly. Fix: Pin to commit hashes like sops-nix is.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-05T15:44:25.36700409-08:00","created_by":"dan","updated_at":"2026-01-05T20:56:59.260867509-08:00","closed_at":"2026-01-05T20:56:59.260867509-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-i1z","title":"Add timestamp to killswitch output","description":"scripts/killswitch echo output has no timestamp. Consider adding or document reliance on systemd journal.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-03T17:35:59.019670457-08:00","created_by":"dan","updated_at":"2026-01-03T17:35:59.019670457-08:00"}
{"id":"ops-jrz1-i8i","title":"Enable mautrix-slack relay mode for bot bridging","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-06T19:09:42.087506995-08:00","updated_at":"2025-12-06T19:09:47.612545472-08:00","closed_at":"2025-12-06T19:09:47.612545472-08:00"}
{"id":"ops-jrz1-iok","title":"Instagram bot missing base-config.yaml","description":"Plugin was missing base-config.yaml required by maubot Config class. Fixed in commit 4b9481d.","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-06T13:02:10.103730128-08:00","updated_at":"2025-12-06T13:02:15.055396318-08:00","closed_at":"2025-12-06T13:02:15.055396318-08:00"}
{"id":"ops-jrz1-j13","title":"VM test: Add negative tests","description":"Test that services fail appropriately when dependencies are down (e.g., continuwuity fails when DB unavailable). Useful to prevent silent misconfig. Higher effort.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-08T00:58:33.000296348-08:00","created_by":"dan","updated_at":"2026-01-08T00:58:33.000296348-08:00"}
{"id":"ops-jrz1-jboq","title":"Mirror ops-jrz1 flake to GitHub","description":"If self-hosted Forgejo is dead, NixOS config is inaccessible. Set up GitHub mirror with auto-push on successful deploy.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-10T14:01:51.357452269-08:00","created_by":"dan","updated_at":"2026-01-10T15:04:22.770582838-08:00"}
{"id":"ops-jrz1-jho","title":"Create or reuse Slack app for matterbridge","description":"Options:\nA) Reuse existing mautrix-slack bot token (may have sufficient scopes)\nB) Create new Slack app for MusicLink\n\nIf creating new:\n1. https://api.slack.com/apps → Create New App\n2. OAuth scopes needed:\n - channels:history (read messages)\n - channels:read (see channel list)\n - chat:write (send messages)\n - users:read (get user info)\n3. Install to workspace\n4. Copy Bot User OAuth Token (xoxb-...)\n5. Invite bot to target channel\n\nDecide: Reuse existing or create dedicated app?","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-08T12:59:09.240785874-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.952072013-08:00","closed_at":"2026-01-08T16:18:17.952072013-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves."}
{"id":"ops-jrz1-jit","title":"Logging and monitoring for dev environments","description":"No observability plan. Need: container CPU/mem metrics, nginx logs, disk usage monitoring, alert on repeated 401s or resource exhaustion.","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-05T15:32:41.318448038-08:00","updated_at":"2025-12-28T00:08:06.750234292-05:00","closed_at":"2025-12-28T00:08:06.750234292-05:00","close_reason":"Browser-based dev environment cancelled","dependencies":[{"issue_id":"ops-jrz1-jit","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.343610481-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-jvt","title":"Add beads (bd) to system packages via flake input","description":"Add beads CLI as a system-wide package using the flake input pattern.\n\n## Implementation\n```nix\n# flake.nix inputs\nbeads = {\n url = \"github:steveyegge/beads\";\n inputs.nixpkgs.follows = \"nixpkgs-unstable\";\n};\n\n# configuration.nix or module\nenvironment.systemPackages = [\n inputs.beads.packages.${system}.default\n];\n```\n\n## Why this approach\nPer orch consensus:\n- Flake inputs + systemPackages is NixOS best practice\n- Reproducible, version-pinned via flake.lock\n- All users get same version\n- Config stays per-repo (.beads/)\n\n## Acceptance criteria\n- `bd --version` works for all users\n- No manual install needed\n- Pinned in flake.lock","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T16:36:04.106750429-08:00","created_by":"dan","updated_at":"2026-01-02T17:25:59.114217153-08:00","closed_at":"2026-01-02T17:25:59.114217153-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-jwl","title":"Update cpu-watchdog source to use 'killswitch' not /usr/local/bin path","description":"scripts/cpu-watchdog:36 has hardcoded /usr/local/bin/killswitch. The Nix build uses replaceStrings to fix this, but source should reflect reality. Change to just 'killswitch'.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T17:35:57.848239876-08:00","created_by":"dan","updated_at":"2026-01-05T09:12:47.271207156-08:00","closed_at":"2026-01-05T09:12:47.271207156-08:00","close_reason":"Updated scripts to use killswitch directly, removed replaceStrings from config"}
{"id":"ops-jrz1-k1z","title":"killswitch: Rename USER variable to avoid shadowing","description":"USER=\"$1\" shadows shell builtin $USER. Rename to TARGET_USER for clarity. scripts/killswitch:15","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T08:17:35.258067292-08:00","created_by":"dan","updated_at":"2026-01-03T09:31:54.318499842-08:00","closed_at":"2026-01-03T09:31:54.318499842-08:00","close_reason":"Renamed USER to TARGET_USER to avoid shadowing shell builtin"}
{"id":"ops-jrz1-k2a","title":"Security posture and threat analysis for dev server","description":"Perform security analysis for the shared dev server environment.\n\n## Scope\n- Multi-user SSH access\n- Agentic coders (Claude, etc.) running with user privileges\n- Shared Slack tokens\n- Network egress\n- Nix store access\n\n## Questions to answer\n- What's the threat model? (curious devs vs malicious actors)\n- What can a compromised user account access?\n- What can an agentic coder do that a human couldn't?\n- Are there secrets at risk?\n- What isolation is needed?\n\n## Deliverables\n- Threat model document\n- Risk assessment\n- Recommendations for mitigations","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T16:34:32.681248615-08:00","created_by":"dan","updated_at":"2026-01-02T19:14:53.774623716-08:00","closed_at":"2026-01-02T19:14:53.774623716-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-k3e0","title":"Take a look at AGENTS.md","status":"open","priority":3,"issue_type":"task","owner":"dleink@gmail.com","created_at":"2026-01-20T11:34:23.2711918-08:00","created_by":"Dan","updated_at":"2026-01-20T11:34:23.2711918-08:00"}
{"id":"ops-jrz1-kfk","title":"VM test: Scope DNS to specific hostnames","description":"Change dnsmasq from wildcard 'address=/#/127.0.0.1' to specific hostnames like 'address=/matrix.example.org/127.0.0.1'. More explicit, less likely to mask DNS misconfig bugs.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-08T00:58:32.367556436-08:00","created_by":"dan","updated_at":"2026-01-08T10:26:56.433679799-08:00","closed_at":"2026-01-08T10:26:56.433679799-08:00","close_reason":"Changed to explicit hostnames: matrix.example.org, example.org"}
{"id":"ops-jrz1-kg0","title":"Switch to subdomain routing (dan.code.clarun.xyz)","description":"Path-based routing (/code/dan/) is fragile. Extensions assume root path, cookies scope incorrectly, PWA breaks. Switch to wildcard subdomains for cleaner isolation.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-12-05T15:32:19.283887085-08:00","updated_at":"2025-12-05T17:23:11.983564455-08:00","closed_at":"2025-12-05T17:23:11.983564455-08:00","dependencies":[{"issue_id":"ops-jrz1-kg0","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.043217984-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-kgx","title":"Improve egress rate limiting: higher limits, better feedback, DROP instead of REJECT","description":"Current 30/min burst 60 is too aggressive for npm install. Changes: (1) Raise to 150/min burst 300, (2) Add logging users can check, (3) Switch REJECT to DROP so apps back off naturally instead of hard fail.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-07T11:04:51.82736813-08:00","created_by":"dan","updated_at":"2026-01-07T11:10:35.322969917-08:00","closed_at":"2026-01-07T11:10:35.322969917-08:00","close_reason":"Implemented: 150/min burst 300, DROP instead of REJECT, added egress-status command for users"}
{"id":"ops-jrz1-kia","title":"Container reset mechanism (keep workspace)","description":"If user breaks their environment, need simple way to wipe container and restore default image while preserving /workspace. Script or admin command.","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-05T15:32:31.045592689-08:00","updated_at":"2025-12-28T00:05:44.757842852-05:00","closed_at":"2025-12-28T00:05:44.757842852-05:00","close_reason":"Parent epic cancelled - browser-based dev approach abandoned","dependencies":[{"issue_id":"ops-jrz1-kia","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.275530016-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-kpw","title":"Add matterbridge NixOS module to configuration","description":"Enable services.matterbridge in NixOS config.\n\n- Add to configuration.nix or create modules/matterbridge.nix\n- Configure configPath for toml file\n- Ensure matterbridge package is available\n\nReference: https://github.com/42wim/matterbridge","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-08T12:58:42.122007711-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.940156572-08:00","closed_at":"2026-01-08T16:18:17.940156572-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves."}
{"id":"ops-jrz1-kvb","title":"ssh-keygen stderr suppressed - hides failures","description":"[OBSERVABILITY] dev-add.sh:98\n\n```bash\n\u003e/dev/null 2\u003e\u00261\n```\n\nIssue: ssh-keygen stderr redirected to /dev/null - hides why key generation might fail\n\nSuggest: Log stderr or check exit code explicitly before suppressing\n\nSource: ops-review","status":"closed","priority":3,"issue_type":"bug","created_at":"2026-01-09T19:50:49.890772307-08:00","created_by":"dan","updated_at":"2026-01-09T19:58:03.976630058-08:00","closed_at":"2026-01-09T19:58:03.976630058-08:00","close_reason":"Fixed: removed 2\u003e\u00261, stderr now visible"}
{"id":"ops-jrz1-l5s","title":"Clean up slack-oauth-token null value from secrets.yaml","description":"Old placeholder key set to null, should be fully removed","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-07T10:22:29.567949459-08:00","created_by":"dan","updated_at":"2026-01-08T17:06:51.959818842-08:00","closed_at":"2026-01-08T17:06:51.959818842-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-lae","title":"egress-watchdog: Fix subshell gotcha in while-read pipeline","description":"while-read in pipeline runs in subshell - variables don't persist outside loop. Use process substitution: while read ...; done \u003c \u003c(echo \"$hits\" | grep ...). scripts/egress-watchdog:25","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-03T08:17:35.401495377-08:00","created_by":"dan","updated_at":"2026-01-03T09:30:50.535018144-08:00","closed_at":"2026-01-03T09:30:50.535018144-08:00","close_reason":"Fixed: using process substitution instead of pipeline subshell"}
{"id":"ops-jrz1-ld4","title":"VM test: Add fail2ban smoke test","description":"Add basic check that fail2ban.service starts. Consensus says it's a cheap smoke test even without full integration. May need to check if it requires sshd or other deps.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-08T00:58:32.21084618-08:00","created_by":"dan","updated_at":"2026-01-08T08:49:02.035160545-08:00","closed_at":"2026-01-08T08:49:02.035160545-08:00","close_reason":"Won't do - fail2ban requires sshd and other deps, not worth test complexity for smoke check"}
{"id":"ops-jrz1-lxai","title":"curl stderr suppressed - network errors invisible","description":"[OBSERVABILITY] dev-add.sh:222,264 dev-remove.sh:58,81\n\n```bash\n2\u003e/dev/null) || true\n```\n\nIssue: curl stderr suppressed - network errors invisible (DNS, connection refused, etc.)\n\nSuggest: Log curl errors to aid debugging; consider `2\u003e\u00261` with conditional logging\n\nSource: ops-review","status":"closed","priority":3,"issue_type":"bug","created_at":"2026-01-09T19:50:50.351933034-08:00","created_by":"dan","updated_at":"2026-01-09T20:10:52.1630535-08:00","closed_at":"2026-01-09T20:10:52.1630535-08:00","close_reason":"Keeping 2\u003e/dev/null - HTTP status codes provide sufficient error context for this use case"}
{"id":"ops-jrz1-meh","title":"cpu-watchdog: Add flock for atomic strike counter updates","description":"Read-modify-write of strike counter not atomic. Systemd timer serializes runs so low risk now, but add flock if parallelism added later. scripts/cpu-watchdog:29-31","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T08:17:35.759126212-08:00","created_by":"dan","updated_at":"2026-01-03T10:08:38.50903714-08:00","closed_at":"2026-01-03T10:08:38.50903714-08:00","close_reason":"Wontfix: systemd timer serializes runs, race condition is theoretical only"}
{"id":"ops-jrz1-mh2","title":"Research: Forgejo integration for shared projects","description":"How does beads/bd integrate with our Forgejo git server (git.clarun.xyz)?\n\n## Questions\n- Can bd sync to Forgejo repos?\n- How do dev users on the server collaborate on shared projects?\n- Is there a git workflow that makes sense (forks? shared repo? branches?)\n- Does bd need any special config for Forgejo vs GitHub?\n\n## Context\n- Forgejo running at git.clarun.xyz\n- Dev users have SSH access to server\n- May want shared project tracking via beads","notes":"Related work: qts completed - dev users now get Forgejo accounts automatically. This issue focuses on bd/beads collaboration workflow which is still open.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T16:24:01.771168961-08:00","created_by":"dan","updated_at":"2026-01-22T15:03:26.971447477-08:00","closed_at":"2026-01-22T15:03:26.971447477-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-mul","title":"Add musiclink secrets to sops-nix","description":"Add to secrets/secrets.yaml:\n\n- matterbridge-api-token: (generated, shared between services)\n- matterbridge-slack-token: (xoxb-... for Slack connection)\n- spotify-client-id: (from Spotify dashboard)\n- spotify-client-secret: (from Spotify dashboard)\n- youtube-api-key: (optional, from Google Cloud)\n\nUpdate sops config in hosts/ops-jrz1.nix to expose these secrets.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-08T12:59:09.71440955-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.954141689-08:00","closed_at":"2026-01-08T16:18:17.954141689-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves.","dependencies":[{"issue_id":"ops-jrz1-mul","depends_on_id":"ops-jrz1-jho","type":"blocks","created_at":"2026-01-08T15:56:08.223426556-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-mul","depends_on_id":"ops-jrz1-8eb","type":"blocks","created_at":"2026-01-08T15:56:08.266067255-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-n4g","title":"Benchmark VPS performance and evaluate upgrade options","description":"Current specs: 1 vCPU, 2GB RAM, swap in use (363MB). npm installs slow, general sluggishness reported. Investigate: (1) What's using memory? (2) Would 2 vCPU/4GB help significantly? (3) Any NixOS/systemd tuning possible? (4) Cost/benefit of upgrade vs optimization.","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-03T12:29:00.208834799-08:00","created_by":"dan","updated_at":"2026-01-03T12:29:00.208834799-08:00"}
{"id":"ops-jrz1-ndl","title":"Browser-based dev environment (code-server)","description":"Explore setting up browser-based development:\n\nOptions:\n- code-server / openvscode-server - VS Code in browser\n- ttyd / wetty - terminal in browser \n- PWA install to home screen for native app feel\n\nCould combine with Tailscale for secure access without exposing ports.\n\nRef: ops-dev thin client brainstorm session","notes":"Design doc created: specs/004-browser-dev-environment/design.md - covers architecture, tech choices, resource planning, security model, rollout phases","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-12-04T15:08:02.406274744-08:00","updated_at":"2025-12-05T17:05:52.872944892-08:00","closed_at":"2025-12-05T17:05:52.872944892-08:00"}
{"id":"ops-jrz1-nir","title":"RFC: SSH log noise reduction strategy","description":"Research showed 99.8% of SSH logs are scanner noise (9000 failed attempts/day). Options: (1) Change SSH port - simple, ~99% reduction (2) journald filter - surgical but complex (3) LogLevel ERROR - loses successful login audit trail (4) fail2ban - bans IPs, partial reduction. Orch consensus: Gemini opposed LogLevel ERROR due to losing audit trail, GPT supported. Need RFC to decide approach. See posture review from Dec 2025 session.","status":"open","priority":4,"issue_type":"task","created_at":"2025-12-04T22:55:13.990334935-08:00","updated_at":"2026-01-15T21:51:04.294876686-08:00"}
{"id":"ops-jrz1-nvx","title":"Slack bot architecture: Matrix-first approach","description":"**Decision**: Use Matrix as primary platform for Slack bot development.\n\n**Architecture**: Bots run as maubot plugins (or Matrix bots), communicate to Slack via mautrix-slack bridge.\n\n**Rationale**:\n- Existing infrastructure (maubot deployed, bridge working)\n- Single platform to manage\n- Bots work with Matrix users too\n- Avoid Socket Mode contention (only one xapp- connection allowed)\n\n**Trade-offs accepted**:\n- Bridge dependency (edit panic bug exists)\n- Extra latency through bridge hop\n- Limited to bridged channels\n\n**Alternative considered (Option B - direct Slack API)**:\n- Could use xoxb- token for outbound-only (REST)\n- Would need new Slack app for full Socket Mode independence\n- Deferred for now\n\n**Credentials available**:\n- slack-oauth-token (xoxb-) - shareable for REST calls if needed\n- slack-app-token (xapp-) - reserved for bridge Socket Mode\n\n**Status**: DECIDED - staying with Matrix-first","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-05T23:12:22.011872713-08:00","updated_at":"2025-12-05T23:12:28.329467732-08:00","closed_at":"2025-12-05T23:12:28.329467732-08:00"}
{"id":"ops-jrz1-nwv","title":"Package graphite CLI (gt) for NixOS","description":"Graphite CLI (gt) is not in nixpkgs. Need to package it.\n\n## Research needed\n- How is gt distributed? (npm, binary, go?)\n- Is there an existing nix package or flake?\n- If not, create minimal derivation\n\n## Options\n1. Find existing flake/overlay\n2. Use buildNpmPackage if it's npm-based\n3. Fetch pre-built binary\n\n## Once packaged\nAdd to system packages via flake input pattern (same as beads).","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-02T16:36:04.374192123-08:00","created_by":"dan","updated_at":"2026-01-02T16:37:46.981193033-08:00","closed_at":"2026-01-02T16:37:46.981193033-08:00","close_reason":"Wrong tool - gt is gastown, not graphite"}
{"id":"ops-jrz1-o2h","title":"Consider making --archive default in dev-remove.sh","description":"scripts/dev-remove.sh defaults to permanent delete; archive is opt-in. Easy to accidentally lose data. Consider making --archive default with --force for delete.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-05T15:44:41.504268553-08:00","created_by":"dan","updated_at":"2026-01-05T15:44:41.504268553-08:00"}
{"id":"ops-jrz1-o9c","title":"Create admin-scripts package for systemPackages","description":"Package learner-add.sh, learner-remove.sh using writeShellApplication. Add to environment.systemPackages so they're available in PATH for interactive admin use.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-03T08:45:35.623169977-08:00","created_by":"dan","updated_at":"2026-01-03T09:20:08.655105165-08:00","closed_at":"2026-01-03T09:20:08.655105165-08:00","close_reason":"Implemented admin-scripts (learner-add, learner-remove) using writeShellApplication, added to systemPackages"}
{"id":"ops-jrz1-ofw","title":"Implement password delivery for Forgejo provisioning","description":"Choose and implement password delivery mechanism for auto-provisioned Forgejo users.\n\n## Options\n1. Print to terminal (simple, visible in scrollback)\n2. Write to ~/.forgejo-credentials (secure file, mode 600)\n3. Hybrid: file + pointer in onboarding message\n\n## Decision needed\n- Which approach fits our security/usability tradeoff?\n- Should file auto-delete after first use?\n\n## Depends on\n- s6p research findings","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-08T18:09:41.352537918-08:00","created_by":"dan","updated_at":"2026-01-09T10:03:21.397286363-08:00","closed_at":"2026-01-09T10:03:21.397286363-08:00","close_reason":"Implemented secure password delivery: JSON file at ~/.forgejo-credentials (mode 600), pointer in terminal output.","dependencies":[{"issue_id":"ops-jrz1-ofw","depends_on_id":"ops-jrz1-s6p","type":"blocks","created_at":"2026-01-08T18:09:41.405676758-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-oxx","title":"Add disk quota or watchdog for /home","description":"No disk limits for users. Could fill /home. Options: ext4 quotas, btrfs subvolume limits, or simple watchdog.","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-03T08:40:26.188569342-08:00","created_by":"dan","updated_at":"2026-01-03T08:40:26.188569342-08:00"}
{"id":"ops-jrz1-p2d","title":"Add egress connection logging","description":"Log all new outbound connections for forensics.\n\n## Config\n```nix\nnetworking.firewall.extraCommands = ''\n # Log all new outbound from regular users\n iptables -A OUTPUT -m state --state NEW -m owner --uid-owner 1000:65534 \\\n -j LOG --log-prefix \"EGRESS: \" --log-level info\n'';\n```\n\n## Usage\n```bash\n# View egress logs\njournalctl -k | grep EGRESS\n\n# Watch live\njournalctl -kf | grep EGRESS\n```\n\n## Notes\n- Logs before rate limit rules (if both implemented)\n- Includes source UID, dest IP, dest port","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T20:17:39.566590459-08:00","created_by":"dan","updated_at":"2026-01-02T21:12:35.575052381-08:00","closed_at":"2026-01-02T21:12:35.575052381-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-qgm","title":"Create musiclink NixOS service with buildGoModule","description":"Add systemd service for musiclink bot:\n\n1. Build Go binary with pkgs.buildGoModule from /home/dan/proj/musiclink\n2. Create systemd.services.musiclink with:\n - DynamicUser for isolation\n - StateDirectory for config\n - Hardening (ProtectSystem, ProtectHome, NoNewPrivileges)\n - After/Requires matterbridge.service\n3. Restart policy on failure\n\nAlternative: fetchFromGitHub if we push to git.clarun.xyz","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-08T12:59:08.021057917-08:00","created_by":"dan","updated_at":"2026-01-08T16:18:17.943916003-08:00","closed_at":"2026-01-08T16:18:17.943916003-08:00","close_reason":"Handed off to musiclink team. They can use Odesli API (free, no creds) or get Spotify creds themselves.","dependencies":[{"issue_id":"ops-jrz1-qgm","depends_on_id":"ops-jrz1-kpw","type":"blocks","created_at":"2026-01-08T15:56:08.088021978-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-qj4","title":"Evaluate bun as faster npm alternative for AI tool installs","description":"npm install -g @google/gemini-cli takes ~1 min (580 packages). Bun is much faster. Consider: (1) Add bun to system packages, (2) Update dev-add onboarding to suggest bun install -g, (3) Or pre-install popular tools system-wide.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T12:26:38.457885819-08:00","created_by":"dan","updated_at":"2026-01-04T13:49:50.343836853-08:00","closed_at":"2026-01-04T13:49:50.343836853-08:00","close_reason":"Added bun to systemPackages, updated dev-add.sh to use bun in PATH and onboarding"}
{"id":"ops-jrz1-qts","title":"RFC: Seamless dev access to ops-jrz1 repo","description":"Devs/agents on jrz1 should be able to access git.clarun.xyz (Forgejo on same server) seamlessly. Goal: minimal friction, leverage existing SSH access.\n\n## Orch Consensus (2026-01-08)\n\nRan serial consensus with flash-or, gemini, gpt, qwen. **Unanimous recommendation: Option B (SSH keys synced to Forgejo)**.\n\n### Why Option B Wins\n\n| Criterion | Advantage |\n|-----------|-----------|\n| Attribution | Each commit/push tied to specific user (critical for agents vs humans) |\n| Security | No PATs in plaintext, keys already used for VPS access |\n| Friction | SSH \"just works\" once key is in Forgejo |\n| Hooks/CI | Forgejo's server-side hooks, protections, webhooks all work |\n\n### Options Rejected\n\n- **D (shared account)**: Loses attribution, security nightmare as project grows\n- **E (filesystem access)**: Bypasses hooks, causes permission conflicts between `git` user and Unix users\n- **C (PAM/LDAP)**: Overkill for single VPS\n- **A (HTTP+PAT)**: Token lifecycle/rotation is extra work, agents leak tokens\n\n### Implementation Plan\n\nExtend `dev-add.sh` to:\n\n1. **Create Forgejo user** via admin API (using token from sops)\n2. **Upload SSH key** to Forgejo profile\n3. **Set git identity** (`user.name`, `user.email`)\n4. **Pre-seed known_hosts** (critical for agents - prevents interactive prompt hang)\n\n```bash\n# Key additions to dev-add.sh:\n\n# 1. Create Forgejo user\ncurl -X POST \"http://localhost:3000/api/v1/admin/users\" \\\n -H \"Authorization: token $FORGEJO_TOKEN\" \\\n -d '{\"username\": \"'$USER'\", \"email\": \"'$USER'@clarun.xyz\", \"password\": \"...\", \"must_change_password\": false}'\n\n# 2. Upload SSH key\ncurl -X POST \"http://localhost:3000/api/v1/admin/users/$USER/keys\" \\\n -H \"Authorization: token $FORGEJO_TOKEN\" \\\n -d '{\"title\": \"vps-local-key\", \"key\": \"'$PUB_KEY'\"}'\n\n# 3. Git identity\nsu - \"$USER\" -c \"git config --global user.name '$USER'\"\nsu - \"$USER\" -c \"git config --global user.email '$USER@clarun.xyz'\"\n\n# 4. Pre-seed known_hosts (prevents agent hang on first git operation)\nsu - \"$USER\" -c \"ssh-keyscan localhost \u003e\u003e ~/.ssh/known_hosts 2\u003e/dev/null\"\nsu - \"$USER\" -c \"ssh-keyscan git.clarun.xyz \u003e\u003e ~/.ssh/known_hosts 2\u003e/dev/null\"\n```\n\n### Optional Add-on\n\nOption F (anonymous reads) can supplement B for public repos - useful for \"clone without setup\" but doesn't solve writes.\n\n### Implementation Considerations\n\n- Make idempotent (don't fail if user/key already exists)\n- Handle `dev-remove.sh` cleanup (delete Forgejo user too)\n- Forgejo API token available in sops: `forgejo-api-token`\n- Consider retroactive sync script for existing users\n","notes":"PAM check: No external auth sources configured. Proceeding with API path.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-05T18:42:23.691289101-08:00","created_by":"dan","updated_at":"2026-01-08T18:00:57.755160223-08:00","closed_at":"2026-01-08T18:00:57.755160223-08:00","close_reason":"Implemented Forgejo integration for dev user provisioning. dev-add.sh now creates Forgejo accounts and uploads SSH keys automatically."}
{"id":"ops-jrz1-qxr","title":"mautrix-slack message edit panic (upstream bug)","description":"Bridge upgraded to v25.11. Need to verify if edit panic is fixed by testing a Slack message edit. Watch logs: journalctl -u mautrix-slack -f | grep -E 'ERR|panic|edit'","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-05T18:22:38.18203834-08:00","updated_at":"2025-12-05T19:36:00.556011621-08:00","closed_at":"2025-12-05T19:36:00.556011621-08:00","dependencies":[{"issue_id":"ops-jrz1-qxr","depends_on_id":"ops-jrz1-03o","type":"blocks","created_at":"2025-12-05T18:24:23.259399275-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-r177","title":"Add /home and /var/lib/acme to B2 backups","description":"Critical DR gap: user home directories and ACME certs not backed up. Without /home, user work is lost. Without /var/lib/acme, Let's Encrypt rate limits may block HTTPS for 7 days on restore.","status":"closed","priority":1,"issue_type":"task","created_at":"2026-01-10T14:01:50.750615523-08:00","created_by":"dan","updated_at":"2026-01-10T14:36:45.169229564-08:00","closed_at":"2026-01-10T14:36:45.169229564-08:00","close_reason":"Added /home and /var/lib/acme to backup paths. Tested: snapshot 76633d30 includes both."}
{"id":"ops-jrz1-rfx","title":"Generate server-side SSH keypair for git access","description":"Users can't push to git.clarun.xyz from the server because they have no private key for outgoing SSH.\n\n## Problem\n- dev-add.sh uploads user's login pubkey to Forgejo\n- But that key's private half is on user's laptop, not the server\n- User has no private key on server for `git@git.clarun.xyz`\n\n## Solution: Dual-Key Approach\n\n**Orch Consensus (gemini, gpt, qwen): Unanimous Support**\n\nEach user gets TWO SSH keys in Forgejo:\n\n| Key | Location | Purpose | Forgejo Title |\n|-----|----------|---------|---------------|\n| Login key | User's laptop | SSH login + direct git from laptop | `username-laptop` |\n| Server key | Server ~/.ssh/id_ed25519 | Git from within server | `username-devserver` |\n\nThis follows the \"One Key Per Device\" industry standard:\n- Laptop and server are separate devices\n- If laptop stolen → revoke laptop key, server access intact\n- If server compromised → attacker only gets server key\n- Better than SSH agent forwarding (hijackable on shared server)\n\n## Architecture\n```\nLaptop [private key] ──SSH──▶ Server [authorized_keys]\n [private key] ──git──▶ Forgejo (alice-laptop)\n \nServer [~/.ssh/id_ed25519] ──git──▶ Forgejo (alice-devserver)\n```\n\n## Implementation\n\n### 1. dev-add.sh changes\n```bash\n# Generate server-side keypair\nssh-keygen -t ed25519 -f /home/$username/.ssh/id_ed25519 \\\n -N '' -C \"$username@jrz1-devserver\"\nchown $username:users /home/$username/.ssh/id_ed25519*\nchmod 600 /home/$username/.ssh/id_ed25519\n\n# Upload BOTH keys to Forgejo:\n# 1. Login key with title \"$username-laptop\"\n# 2. Server key with title \"$username-devserver\"\n```\n\n### 2. dev-remove.sh changes\n- Revoke both keys from Forgejo via API, OR\n- Enhance warning to mention both keys need manual removal\n\n### 3. Migration for existing users\n- Generate server-side keypair\n- Upload to Forgejo with \"-devserver\" title\n\n## Key Naming Convention\n```\nssh-keygen -C \"username@jrz1-devserver\" # in key comment\nTitle in Forgejo: \"username-devserver\" # in API call\n```\n\n## Lifecycle\n- Disable Forgejo user → both keys stop working immediately\n- Delete Unix account → server key deleted with home dir\n- Offboarding is clean and automatable\n\n## Testing\n1. Create test user via dev-add.sh\n2. SSH in as test user\n3. `git clone git@git.clarun.xyz:dan/ops-jrz1.git`\n4. Make change, commit, push\n5. Verify push succeeds\n6. (Optional) Test push from laptop if externally accessible","status":"closed","priority":2,"issue_type":"epic","created_at":"2026-01-09T17:16:28.590318796-08:00","created_by":"dan","updated_at":"2026-01-09T19:36:09.412119418-08:00","closed_at":"2026-01-09T19:36:09.412119418-08:00","close_reason":"Dual-key git access implemented and tested"}
{"id":"ops-jrz1-rkp","title":"Add egress abuse watchdog","description":"Monitor for users hitting egress rate limits, kill if sustained.\n\n## Script: /usr/local/bin/egress-watchdog\n```bash\n#\\!/usr/bin/env bash\n# Kill users who keep hitting egress limits\nTHRESHOLD=10 # EGRESS-LIMIT hits per minute\nCOUNTFILE=\"/var/lib/egress-watchdog\"\nmkdir -p \"$COUNTFILE\"\n\n# Count recent limit hits per UID\njournalctl -k --since \"1 minute ago\" 2\u003e/dev/null | grep \"EGRESS-LIMIT\" | \\\n grep -oP 'UID=\\K[0-9]+' | sort | uniq -c | while read count uid; do\n \n user=$(getent passwd \"$uid\" | cut -d: -f1)\n [ -z \"$user\" ] \u0026\u0026 continue\n \n if [ \"$count\" -gt \"$THRESHOLD\" ]; then\n strikes=$(cat \"$COUNTFILE/$user\" 2\u003e/dev/null || echo 0)\n strikes=$((strikes + 1))\n echo \"$strikes\" \u003e \"$COUNTFILE/$user\"\n logger -t egress-watchdog \"User $user hit egress limit $count times (strike $strikes/3)\"\n \n if [ \"$strikes\" -ge 3 ]; then\n /usr/local/bin/killswitch \"$user\" \"egress abuse ($count hits)\"\n rm -f \"$COUNTFILE/$user\"\n fi\n else\n rm -f \"$COUNTFILE/$user\"\n fi\ndone\n```\n\n## Behavior\n- Runs every minute (same timer as CPU watchdog, or separate)\n- 3 consecutive minutes of \u003e10 blocked connections = kill\n- Works with egress rate limiting (ops-jrz1-cmv)\n\n## Dependencies\n- Requires ops-jrz1-cmv (egress rate limiting)\n- Requires ops-jrz1-396 (killswitch script)","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-02T20:21:09.516724064-08:00","created_by":"dan","updated_at":"2026-01-03T06:02:02.132992356-08:00","closed_at":"2026-01-03T06:02:02.132992356-08:00","close_reason":"Egress watchdog deployed and tested. Script monitors EGRESS-LIMIT kernel log entries, tracks strikes per user, kills after 3 strikes.","dependencies":[{"issue_id":"ops-jrz1-rkp","depends_on_id":"ops-jrz1-396","type":"blocks","created_at":"2026-01-02T20:21:14.314011866-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-rkp","depends_on_id":"ops-jrz1-cmv","type":"blocks","created_at":"2026-01-02T20:21:14.352411765-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-s6p","title":"Research: Password delivery for auto-provisioned Forgejo users","description":"When dev-add.sh creates Forgejo accounts, users get random passwords with must_change_password=true but no way to learn the password.\n\n## Problem\n- SSH git access works (key uploaded)\n- Web UI access broken (can't log in to change password)\n\n## Options to Research\n- Print to terminal\n- Secure file in home directory\n- Email delivery\n- Matrix message\n- QR code\n- Skip password entirely (SSH-only)\n- OAuth/SSO integration\n\n## Context\n- Most users are agents (don't need web UI)\n- Human devs may want PR/issue access\n- Low frequency onboarding","notes":"Research findings:\n- No Forgejo API for password reset tokens/magic links\n- CLI can set passwords directly\n- Practical options: print to terminal, write to secure file in home dir\n- True magic links would require custom implementation","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-08T18:06:56.473648691-08:00","created_by":"dan","updated_at":"2026-01-09T10:03:21.382978266-08:00","closed_at":"2026-01-09T10:03:21.382978266-08:00","close_reason":"Implemented secure password delivery: JSON file at ~/.forgejo-credentials (mode 600), pointer in terminal output."}
{"id":"ops-jrz1-s8x","title":"Add health check endpoint for Matrix homeserver monitoring","description":"modules/dev-services.nix Matrix service has no health check for monitoring. Add nginx location for /_matrix/client/versions health probe.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-05T15:44:34.407481321-08:00","created_by":"dan","updated_at":"2026-01-05T15:44:34.407481321-08:00"}
{"id":"ops-jrz1-sdz","title":"Remove /usr/local/bin scripts from server","description":"After declarative deployment works, clean up manually deployed scripts from /usr/local/bin on the server.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T08:39:54.483032394-08:00","created_by":"dan","updated_at":"2026-01-03T09:20:34.591216526-08:00","closed_at":"2026-01-03T09:20:34.591216526-08:00","close_reason":"Removed all manual scripts from /usr/local/bin/","dependencies":[{"issue_id":"ops-jrz1-sdz","depends_on_id":"ops-jrz1-ujw","type":"blocks","created_at":"2026-01-03T08:40:02.851476398-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-sdz","depends_on_id":"ops-jrz1-o9c","type":"blocks","created_at":"2026-01-03T08:45:48.023849189-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-svr","title":"Make dev-add.sh idempotent or add --update flag","description":"scripts/dev-add.sh:42-45 exits with error if user exists. Can't safely re-run to update SSH key or fix partial failures. Add --update flag or make idempotent by default.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-05T15:44:33.963322361-08:00","created_by":"dan","updated_at":"2026-01-05T20:45:31.653910595-08:00","closed_at":"2026-01-05T20:45:31.653910595-08:00","close_reason":"Made idempotent - safe to re-run, updates existing users"}
{"id":"ops-jrz1-t73","title":"Rename learner to dev in scripts and docs","description":"Rename terminology from \"learner\" to \"dev\" or \"user\" across:\n\n- scripts/learner-add.sh → dev-add.sh\n- scripts/learner-remove.sh → dev-remove.sh\n- /etc/slack-learner.env → /etc/slack-dev.env\n- learners group → devs group\n- docs/learner-*.md\n- tests/test-learner-env.sh\n\nLow priority cleanup.","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-02T12:32:40.340984626-08:00","created_by":"dan","updated_at":"2026-01-03T10:37:34.321661169-08:00","closed_at":"2026-01-03T10:37:34.321661169-08:00","close_reason":"Renamed learner to dev across scripts, docs, tests, and configuration"}
{"id":"ops-jrz1-u0w","title":"Security review of running server","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-04T21:03:22.420507724-08:00","updated_at":"2025-12-04T21:04:31.989886731-08:00","closed_at":"2025-12-04T21:04:31.989886731-08:00"}
{"id":"ops-jrz1-ujw","title":"Update systemd services to use nix store paths","description":"Change ExecStart from /usr/local/bin/cpu-watchdog to use the derivation path. Either reference package directly or use pkgs.writeShellApplication.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-03T08:39:54.227335183-08:00","created_by":"dan","updated_at":"2026-01-03T09:20:08.685831615-08:00","closed_at":"2026-01-03T09:20:08.685831615-08:00","close_reason":"Systemd services now reference Nix store paths via ${pkg}/bin/script","dependencies":[{"issue_id":"ops-jrz1-ujw","depends_on_id":"ops-jrz1-5ef","type":"blocks","created_at":"2026-01-03T08:40:02.815677839-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-unn","title":"Git worktree workflow for parallel Claude sessions","description":"Tooling for managing multiple git worktrees, each with own tmux window and Claude session. Enables parallel feature development.","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-05T17:25:32.913162022-08:00","created_by":"dan","updated_at":"2026-01-05T17:25:32.913162022-08:00"}
{"id":"ops-jrz1-vbt","title":"Improve test coverage: NixOS VM tests, script tests, integration tests","description":"Current testing is minimal - just build checks and post-deploy smoke tests.\n\n## Current state\n- `nix flake check` - verifies configs build\n- `scripts/smoke-test.sh` - post-deploy service/port/HTTP checks\n\n## Gaps\n\n### NixOS VM tests (nixosTest)\n- No actual VM tests despite ops-jrz1-vm.nix existing\n- Could test: service startup, port bindings, basic Matrix API\n- AGENTS.md says \"Add VM tests\" but none exist\n\n### Script tests\n- dev-add.sh / dev-remove.sh have no tests\n- killswitch / watchdogs untested\n- Edge cases: duplicate user, missing args, permissions\n\n### Integration tests\n- No message flow tests (Matrix ↔ Slack bridge)\n- No maubot plugin deployment test\n- No user onboarding end-to-end test\n\n### Security tests\n- fail2ban trigger verification\n- User isolation (can user A see user B files?)\n- Egress rate limit enforcement\n\n## Options\n1. Start with nixosTest for core services (low effort, high value)\n2. Add bats/shunit2 tests for shell scripts\n3. Integration tests need real credentials (harder)\n\n## Reference\n- NixOS testing: https://nixos.org/manual/nixos/stable/#sec-nixos-tests\n- smoke-test.sh pattern could extend to VM","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-05T19:14:36.134101488-08:00","created_by":"dan","updated_at":"2026-01-08T11:04:11.186009521-08:00","closed_at":"2026-01-08T11:04:11.186009521-08:00","close_reason":"Implemented Phase 1+2: VM integration test (PostgreSQL, conduwuit, nginx, dnsmasq) and shellcheck linting. Filed additional improvements (negative tests, config verification) as separate P4 issues."}
{"id":"ops-jrz1-vix","title":"Evaluate home-manager for per-user config","description":"Evaluate whether home-manager adds value for our setup.\n\n## What home-manager could manage\n- Shell config (.bashrc, .zshrc)\n- Git config (.gitconfig)\n- Tool configs (~/.config/*)\n- direnv integration\n\n## Questions\n- Do we need declarative per-user dotfiles?\n- Is the complexity worth it for a small team?\n- Can we start without it and add later?\n\n## Recommendation from consensus\n\"Optional but recommended\" - good for pushing default configs to all devs.\nStart without it, add if pain point emerges.","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-02T16:36:04.849881753-08:00","created_by":"dan","updated_at":"2026-01-02T16:36:04.849881753-08:00"}
{"id":"ops-jrz1-vw4","title":"Create watchdog-scripts package (writeShellApplication)","description":"Package killswitch, cpu-watchdog, egress-watchdog using writeShellApplication. Include runtimeInputs for procps, gawk, systemd, etc. These scripts are NOT added to PATH - only referenced directly by systemd services.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-03T08:39:53.745385806-08:00","created_by":"dan","updated_at":"2026-01-03T09:20:08.62999227-08:00","closed_at":"2026-01-03T09:20:08.62999227-08:00","close_reason":"Implemented watchdog-scripts (killswitch, cpu-watchdog, egress-watchdog) using writeShellApplication with proper runtimeInputs"}
{"id":"ops-jrz1-w1ll","title":"Document and test disaster recovery restore procedure","description":"Goal: Be able to restore user data from B2 backups in case of disaster, misconfiguration, or user error.\n\n## Scope\n- Document step-by-step restore procedure for each backup type:\n - PostgreSQL databases (forgejo, mautrix_slack)\n - Forgejo repos and state\n - Matrix homeserver (RocksDB)\n - Maubot plugins/state\n- Test restore to a fresh system or /tmp\n- Document partial restore (single database, single user's repos)\n- Quarterly restore drill schedule\n\n## Acceptance\n- Written runbook in docs/\n- At least one successful test restore\n- Restore time estimate documented","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-10T13:53:16.948489834-08:00","created_by":"dan","updated_at":"2026-01-10T13:53:16.948489834-08:00","dependencies":[{"issue_id":"ops-jrz1-w1ll","depends_on_id":"ops-jrz1-r177","type":"blocks","created_at":"2026-01-10T14:02:01.261838782-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-w1ll","depends_on_id":"ops-jrz1-y8le","type":"blocks","created_at":"2026-01-10T14:02:01.297605462-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-w1mb","title":"Bug: VS Code Claude extension loses auth on restart","description":"## Symptom\nClaude Code VS Code extension loses API key authentication after VS Code restart.\n\n## Root Cause Found\nhome-manager symlinks ~/.config/Code/User/settings.json to read-only Nix store.\nVS Code extensions cannot persist settings/auth state.\n\n## Workaround\nLogin via CLI first: claude /login\nExtension reads shared credentials from ~/.claude/.credentials.json\n\n## Fix\nFiled dotfiles-j06 to address home-manager VS Code config.\n\n## References\n- GitHub anthropics/claude-code#12204","status":"closed","priority":3,"issue_type":"bug","owner":"dleink@gmail.com","created_at":"2026-01-15T21:55:03.276834742-08:00","created_by":"Dan","updated_at":"2026-01-16T11:58:58.256452631-08:00","closed_at":"2026-01-16T11:58:58.256452631-08:00","close_reason":"Root cause identified: home-manager read-only symlink. Filed dotfiles-j06 for fix."}
{"id":"ops-jrz1-w68","title":"Remote dev environment security and setup research","description":"Research and test security/setup questions for learner remote dev environments.\n\n## Context\nTwo dev paths identified:\n1. **Server-first**: SSH in, run agentic coders on server\n2. **Local VS Code**: Remote-SSH extension, code on server\n\nBoth have open questions around sandboxing, system packages, deployment access, and security boundaries.\n\n## Scope\n- System package management for learners\n- User isolation and sandboxing options\n- Agentic coder security (what can Claude do?)\n- Deployment pipeline security\n- VS Code extension behavior testing\n- Resource limits and quotas\n\n## Deliverables\n- Answers to open questions (documented)\n- Security recommendations\n- Implementation plan for chosen approach","status":"open","priority":2,"issue_type":"epic","created_at":"2026-01-02T12:26:48.104374079-08:00","created_by":"dan","updated_at":"2026-01-02T12:26:48.104374079-08:00","dependencies":[{"issue_id":"ops-jrz1-w68","depends_on_id":"ops-jrz1-9pe","type":"blocks","created_at":"2026-01-02T12:27:59.277687811-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-w68","depends_on_id":"ops-jrz1-ghd","type":"blocks","created_at":"2026-01-02T12:27:59.32841874-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-w68","depends_on_id":"ops-jrz1-3b1","type":"blocks","created_at":"2026-01-02T12:27:59.375858081-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-w68","depends_on_id":"ops-jrz1-3au","type":"blocks","created_at":"2026-01-02T12:27:59.428509997-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-w68","depends_on_id":"ops-jrz1-3jo","type":"blocks","created_at":"2026-01-02T12:27:59.473581774-08:00","created_by":"dan"},{"issue_id":"ops-jrz1-w68","depends_on_id":"ops-jrz1-bbn","type":"blocks","created_at":"2026-01-02T12:27:59.523975339-08:00","created_by":"dan"}]}
{"id":"ops-jrz1-wj2","title":"Design API key provisioning strategy","description":"opencode needs API keys (OpenAI, Anthropic). Options: 1) Shared key with proxy + rate limiting, 2) Per-user keys in sops-nix. Need to prevent key exposure and enable usage tracking.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-12-05T15:32:19.526073243-08:00","updated_at":"2025-12-05T17:25:10.534718515-08:00","closed_at":"2025-12-05T17:25:10.534718515-08:00","dependencies":[{"issue_id":"ops-jrz1-wj2","depends_on_id":"ops-jrz1-3so","type":"parent-child","created_at":"2025-12-05T17:05:47.103332379-08:00","created_by":"daemon","metadata":"{}"}]}
{"id":"ops-jrz1-xoad","title":"Create release cycle and changelog process","description":"Need a way to communicate changes to users - changelog, release notes, or similar.\n\nIdeas:\n- MOTD on SSH login showing recent changes\n- /etc/motd.d/ with dynamic changelog\n- bd-powered changelog generation\n\nNext session priority.","status":"in_progress","priority":2,"issue_type":"task","created_at":"2026-01-10T13:49:46.492349303-08:00","created_by":"dan","updated_at":"2026-01-16T15:29:40.249800763-08:00"}
{"id":"ops-jrz1-xz1","title":"Fix maubot admin UI exposed to internet (port 29316)","description":"Maubot admin UI on port 29316 is publicly accessible (returns 401 but API surface exposed). Firewall explicitly allows this port. Risk: brute force on admin password, direct exploit of any maubot vulnerabilities. Fix: bind to 127.0.0.1 only, remove from firewall, access via SSH tunnel.","status":"closed","priority":1,"issue_type":"bug","created_at":"2025-12-04T21:03:22.531676543-08:00","updated_at":"2025-12-04T22:35:24.162735368-08:00","closed_at":"2025-12-04T22:35:24.162735368-08:00"}
{"id":"ops-jrz1-xz7","title":"Research: Multi-user auth storage for agentic coders","description":"Investigate where auth credentials are stored for each agentic coder when multiple users authenticate:\n\n## Questions\n- Claude Code: Where is OAuth token stored? ~/.claude? Conflicts between users?\n- opencode: Auth storage location?\n- gemini-cli: Auth storage?\n- codex: Auth storage?\n\n## Goal\nUnderstand if there are isolation issues when multiple users auth on same server.","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-02T17:30:15.028994987-08:00","created_by":"dan","updated_at":"2026-01-02T17:30:15.028994987-08:00"}
{"id":"ops-jrz1-y8le","title":"Stop Matrix before backup for RocksDB consistency","description":"RocksDB is backed up while running, risking corrupt snapshots. Add systemd pre-hook to stop matrix-continuwuity during backup window.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-10T14:01:50.945222296-08:00","created_by":"dan","updated_at":"2026-01-10T20:15:25.90394816-08:00","closed_at":"2026-01-10T20:15:25.90394816-08:00","close_reason":"Accepting risk: RocksDB has crash consistency, 3 AM backup window has minimal activity, and we have multiple daily snapshots. Can re-evaluate if restore drill shows corruption."}
{"id":"ops-jrz1-yhu","title":"configuration.nix: Consider custom iptables chain for egress rules","description":"Same iptables match pattern repeated 8 times. Could create custom chain for cleaner rule management. Optional - readability tradeoff. configuration.nix:68-79","status":"closed","priority":3,"issue_type":"task","created_at":"2026-01-03T08:17:35.532609792-08:00","created_by":"dan","updated_at":"2026-01-03T10:07:28.725278889-08:00","closed_at":"2026-01-03T10:07:28.725278889-08:00","close_reason":"Wontfix: current inline rules work fine, custom chain is marginal improvement"}
{"id":"ops-jrz1-zgs8","title":"Set up B2 automated backups with restic","status":"closed","priority":1,"issue_type":"task","created_at":"2026-01-10T13:32:34.753782465-08:00","created_by":"dan","updated_at":"2026-01-10T13:52:46.518193686-08:00","closed_at":"2026-01-10T13:52:46.518193686-08:00","close_reason":"B2 backup operational: bucket ops-jrz1-backup, daily 3 AM, weekly integrity check, tested successfully"}
{"id":"ops-jrz1-zk9","title":"Enhance egress watchdog to alert on suspicious traffic patterns","description":"Current egress watchdog counts rate-limit hits and kills users after 3 strikes. Consider adding alerts for: (1) Known malicious IPs/domains, (2) Crypto mining pool connections, (3) Unusual port patterns (IRC, etc), (4) High volume to single destination. Could integrate with fail2ban or custom alerting.","status":"open","priority":3,"issue_type":"task","created_at":"2026-01-04T13:49:38.854307436-08:00","created_by":"dan","updated_at":"2026-01-04T13:49:50.048618935-08:00"}
{"id":"ops-jrz1-zm2","title":"Remove empty Slack token placeholders from secrets.yaml","description":"secrets/secrets.yaml:3-4 has empty strings for slack-oauth-token and slack-app-token. Confusing; suggests they should be populated. Remove if unused or document why empty.","status":"closed","priority":4,"issue_type":"task","created_at":"2026-01-05T15:44:34.18597284-08:00","created_by":"dan","updated_at":"2026-01-07T10:45:26.397155955-08:00","closed_at":"2026-01-07T10:45:26.397155955-08:00","close_reason":"Superseded by ops-jrz1-l5s (cleanup null value). Old placeholders replaced with real tokens."}
{"id":"ops-jrz1-zr0q","title":"Fix git access to git.clarun.xyz for musiclink input","description":"Nix flake check failed to fetch musiclink via HTTPS/SSH. Currently using a local file path as a workaround.","status":"closed","priority":3,"issue_type":"task","owner":"dleink@gmail.com","created_at":"2026-01-20T12:42:19.113702543-08:00","created_by":"Dan","updated_at":"2026-01-20T20:34:41.735832471-08:00","closed_at":"2026-01-20T20:34:41.735832471-08:00","close_reason":"Closed"}
{"id":"ops-jrz1-zvh","title":"Fix maubot health check (failing every 5 min)","description":"Health check at /_matrix/maubot/v1/version returns 401 (auth required). Check script doesn't provide auth token. Spamming error logs every 5 minutes.","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-04T22:55:25.755541054-08:00","updated_at":"2025-12-05T02:00:19.284410671-08:00","closed_at":"2025-12-05T02:00:19.284410671-08:00"}
Reference in a new issue View git blame Copy permalink