ops-jrz1/specs/003-maubot-integration/tasks.md

# Implementation Tasks: Maubot Integration

**Feature**: 003-maubot-integration
**Branch**: `003-maubot-integration`
**Target**: ops-jrz1 VPS (45.77.205.49)
**Estimated Duration**: 2-3 hours deployment + 7 days validation

## Task Summary

- **Total Tasks**: 47 (updated for incremental deployment strategy)
- **Setup Phase**: 4 tasks
- **Foundational Phase**: 6 tasks
- **User Story 1 (P1)**: 20 tasks - Instagram content sharing (MVP)
  - Infrastructure: 3 tasks (T011-T013)
  - Phase 1 deployment: 4 tasks (T013a-d)
  - Phase 2 deployment: 4 tasks (T013e-h)
  - Phase 3 deployment: 6 tasks (T014-T017c)
  - Phase 4 bot config: 6 tasks (T018-T023)
  - Testing: 4 tasks (T024-T027)
- **User Story 2 (P2)**: 6 tasks - Management interface
- **User Story 3 (P2)**: 5 tasks - Service reliability
- **User Story 4 (P3)**: 3 tasks - Additional bot deployment
- **Polish Phase**: 3 tasks

**MVP Scope**: User Story 1 (20 tasks) - validates core value proposition with incremental deployment

---

## Phase 1: Setup (Project Initialization)

**Goal**: Prepare development environment and extract source modules from ops-base

- [X] T001 Create feature branch 003-maubot-integration from main
- [X] T002 Copy maubot.nix module from /home/dan/proj/ops-base/vm-configs/modules/maubot.nix to modules/maubot.nix
- [X] T003 Copy Instagram bot plugin from /home/dan/proj/sna/sna-instagram-bot.mbp to local working directory
- [X] T004 Generate maubot secrets (admin password 32 chars, secret key 48 bytes) using openssl rand -base64

**Checkpoint**: Source files ready for adaptation

---

## Phase 2: Foundational (Blocking Prerequisites)

**Goal**: Adapt maubot module for ops-jrz1 and configure secrets

**Independent Test**: Deploy adapted module and verify service starts without errors

### Module Adaptation

- [X] T005 Update module namespace from services.matrix-vm.maubot to services.maubot in modules/maubot.nix
- [X] T006 Update homeserver URL from http://127.0.0.1:6167 to http://127.0.0.1:8008 in modules/maubot.nix
- [X] T007 Remove registration_secrets section from config generation in modules/maubot.nix (lines ~140-150, conduwuit doesn't support shared secret)
- [X] T008 Change config path from /run/maubot/config.yaml to /var/lib/maubot/config/config.yaml in modules/maubot.nix
- [X] T009 Add LoadCredential removal for registration-secret (keep admin-password and secret-key only) in modules/maubot.nix systemd service section
- [X] T010 [P] Add maubot secrets to secrets/secrets.yaml (maubot-admin-password, maubot-secret-key) using sops secrets/secrets.yaml

**Checkpoint**: Module adapted for conduwuit, secrets encrypted

---

## Phase 3: User Story 1 - Instagram Content Sharing to Matrix (Priority: P1)

**Goal**: Deploy maubot service with Instagram bot and validate content fetching

**Independent Test**: Post Instagram URL in enabled Matrix room and verify bot responds with image/video/caption within 5 seconds

**Why MVP**: Core value proposition - brings Instagram content into team communication, validates integration works

### Infrastructure Deployment

- [X] T011 [US1] Add sops secret declarations to hosts/ops-jrz1.nix (sops.secrets.maubot-admin-password, sops.secrets.maubot-secret-key)
- [X] T012 [US1] Create dev-platform wrapper options in modules/dev-services.nix (services.dev-platform.maubot with enable and port options)
- [X] T013 [US1] Add dev-platform config block in modules/dev-services.nix (maps to services.maubot with homeserverUrl, serverName, port, secret paths)

### Service Deployment - Phase 1: Module Files

- [ ] T013a [US1] Deploy Phase 1 to VPS (modules added, service disabled) using nixos-rebuild switch --flake .#ops-jrz1 --target-host root@45.77.205.49 --build-host localhost
- [ ] T013b [US1] Verify Phase 1: Check nixos-rebuild output reports "no services changed" or only unrelated service restarts
- [ ] T013c [US1] Verify existing services healthy: ssh root@45.77.205.49 'systemctl status matrix-continuwuity mautrix-slack forgejo postgresql nginx'
- [ ] T013d [US1] Git commit Phase 1 with message "Add maubot module files (service disabled)"

### Service Deployment - Phase 2: Secrets

- [ ] T013e [US1] Deploy Phase 2 to VPS (secrets added in Phase 0 and Phase 1, service still disabled) using nixos-rebuild switch
- [ ] T013f [US1] Verify Phase 2: Check secrets decrypted via ssh root@45.77.205.49 'ls -la /run/secrets/maubot-*' (expect 0400 permissions)
- [ ] T013g [US1] Verify existing services healthy (same command as T013c)
- [ ] T013h [US1] Git commit Phase 2 with message "Add maubot secrets (service not enabled)"

### Service Deployment - Phase 3: Enable Service

- [ ] T014 [US1] Enable maubot service in hosts/ops-jrz1.nix (services.dev-platform.maubot.enable = true, port = 29316)
- [ ] T015 [US1] Deploy Phase 3 to VPS (enable maubot service) using nixos-rebuild switch --flake .#ops-jrz1 --target-host root@45.77.205.49 --build-host localhost
- [ ] T016 [US1] Verify service status via ssh root@45.77.205.49 'systemctl status maubot.service' (expect active running)
- [ ] T017 [US1] Check logs for errors via ssh root@45.77.205.49 'journalctl -u maubot.service -n 50'
- [ ] T017a [US1] Verify existing services still healthy after maubot deployment (same command as T013c)
- [ ] T017b [US1] Test Slack bridge functionality (post message in Slack, verify appears in Matrix within 5 seconds)
- [ ] T017c [US1] Git commit Phase 3 with message "Enable maubot service (no bots deployed yet)"

### Bot Configuration - Phase 4: Manual Deployment

- [ ] T018 [US1] Create SSH tunnel to management interface: ssh -L 29316:localhost:29316 root@45.77.205.49
- [ ] T019 [US1] Login to maubot web UI at http://localhost:29316/_matrix/maubot (username: admin, password from sops secrets)
- [ ] T020 [US1] Create bot Matrix user @instagram-bot:clarun.xyz via conduwuit registration token (Clients tab → Add client → Register)
- [ ] T021 [US1] Upload Instagram plugin sna-instagram-bot.mbp via web UI (Plugins tab → Upload)
- [ ] T022 [US1] Create bot instance instagram-bot-1 (type: sna.instagram, primary_user: @instagram-bot:clarun.xyz, config: {"enabled": true, "max_file_size": 50000000, "room_subscriptions": []})
- [ ] T023 [US1] Invite bot to test Matrix room via /invite @instagram-bot:clarun.xyz

### Testing & Validation

- [ ] T024 [US1] Add test room ID to bot config room_subscriptions in maubot web UI
- [ ] T025 [US1] Restart bot instance (Stop → Start in web UI)
- [ ] T026 [US1] Post public Instagram URL in test room and verify bot responds within 5 seconds with image/video/caption (SC-001)
- [ ] T027 [US1] Post Instagram URL in non-subscribed room and verify bot ignores it (FR-006 enforcement)

**Acceptance Criteria**:
- ✅ Bot responds to Instagram URLs in subscribed rooms only
- ✅ Content fetched within 5 seconds (SC-001)
- ✅ Images, videos, and captions displayed correctly
- ✅ Bot ignores URLs in non-subscribed rooms

**MVP Checkpoint**: Core functionality working - Instagram content visible in Matrix

---

## Phase 4: User Story 2 - Bot Management Interface (Priority: P2)

**Goal**: Validate management interface functionality for bot lifecycle operations

**Independent Test**: Access management UI, create/stop/restart bot instance, view logs and status

**Why this priority**: Essential for operations but bot works without admin features initially

### Management Interface Validation

- [ ] T028 [US2] Access management dashboard via SSH tunnel and verify all bot instances listed with status (instances tab)
- [ ] T029 [US2] Test plugin upload via web UI (upload test .mbp file, verify appears in plugins list)
- [ ] T030 [US2] Test bot instance creation via web UI (create test instance, verify appears online in Matrix within 30 seconds)
- [ ] T031 [US2] Test bot configuration edit (edit room_subscriptions via config JSON, restart instance, verify bot responds only in new rooms)
- [ ] T032 [US2] Test bot stop/start via web UI (click Stop button, verify bot goes offline, click Start, verify reconnects)
- [ ] T033 [US2] View bot logs in UI and verify error messages display with timestamps and severity levels

**Acceptance Criteria**:
- ✅ Dashboard displays all bot instances with status
- ✅ Plugin upload succeeds and validates
- ✅ Bot lifecycle operations (create/stop/start) work via UI
- ✅ Configuration changes take effect after restart
- ✅ Logs visible with proper formatting

---

## Phase 5: User Story 3 - Bot Framework Service Reliability (Priority: P2)

**Goal**: Validate auto-start, auto-recovery, and failure handling

**Independent Test**: Reboot server and verify maubot service and all bot instances resume automatically

**Why this priority**: Critical for production reliability but can be validated after basic functionality proven

### Reliability Testing

- [ ] T034 [US3] Test server reboot recovery (ssh root@45.77.205.49 'reboot', wait 2 minutes, verify service auto-starts via systemctl status maubot)
- [ ] T035 [US3] Test Matrix homeserver restart handling (restart matrix-continuwuity service, verify bot reconnects automatically without manual intervention)
- [ ] T036 [US3] Verify health check timers active (ssh root@45.77.205.49 'systemctl list-timers | grep maubot', expect maubot-health.timer and maubot-health-restart.timer)
- [ ] T037 [US3] Test manual health check (curl http://localhost:29316/_matrix/maubot/v1/version, verify JSON response with version field)
- [ ] T038 [US3] Monitor 7-day uptime for SC-003 validation (99% uptime target, check periodically: uptime -p, journalctl -u maubot | grep -i error)

**Acceptance Criteria**:
- ✅ Service auto-starts on server boot within 2 minutes
- ✅ Bot instances reconnect after Matrix homeserver restart
- ✅ Health timers operational
- ✅ 99% uptime achieved over 7-day period

---

## Phase 6: User Story 4 - Additional Bot Deployment (Priority: P3)

**Goal**: Demonstrate platform extensibility by deploying a second bot type

**Independent Test**: Deploy echo bot or reaction bot from maubot plugin repository and verify independent operation

**Why this priority**: Future-proofs investment, not required for initial Instagram bot value

### Extensibility Validation

- [ ] T039 [US4] Download additional maubot plugin from community repository (e.g., echo bot, reaction bot)
- [ ] T040 [US4] Upload second plugin via management UI and verify validation succeeds
- [ ] T041 [US4] Create second bot instance using new plugin and verify appears in dashboard with type, status, and resource usage
- [ ] T042 [US4] Test SC-002 multi-instance validation (run 3 concurrent bot instances, verify no performance degradation)

**Acceptance Criteria**:
- ✅ Multiple plugin types supported
- ✅ Dashboard shows all bots with clear differentiation
- ✅ 3+ concurrent instances run without degradation (SC-002)

---

## Phase 7: Polish & Cross-Cutting Concerns

**Goal**: Complete documentation and prepare for merge

### Documentation

- [ ] T043 Update CLAUDE.md with maubot management commands (service status, logs, SSH tunnel, room subscription workflow)
- [ ] T044 Create deployment worklog in docs/worklogs/2025-10-26-maubot-deployment.org documenting session
- [ ] T045 Commit changes and tag release v0.3.0 (message: "Add maubot bot framework with Instagram bot - Implements 003-maubot-integration")

**Final Checkpoint**: All documentation complete, ready for 7-day validation period

---

## Dependencies & Execution Order

### User Story Dependencies

```
Phase 1 (Setup)
  ↓
Phase 2 (Foundational) ← BLOCKING for all user stories
  ↓
├─→ User Story 1 (P1) ← MVP, no dependencies
├─→ User Story 2 (P2) ← depends on US1 (needs running bot to manage)
├─→ User Story 3 (P2) ← depends on US1 (needs service deployed to test reliability)
└─→ User Story 4 (P3) ← depends on US2 (needs management UI working)
  ↓
Phase 7 (Polish) ← depends on all user stories complete
```

### Critical Path

1. Setup (T001-T004)
2. Foundational (T005-T010) - **MUST complete before user stories**
3. User Story 1 (T011-T027) - **MVP - Deploy first, validate before continuing**
4. Validate MVP success before proceeding to US2/US3/US4
5. User Stories 2, 3, 4 can proceed in parallel after US1 validates
6. Polish (T043-T045) after all user stories complete

---

## Parallel Execution Opportunities

### Phase 2 (Foundational)

**Parallel**:
- T010 can run in parallel with T005-T009 (secrets vs module editing, different files)

### Phase 3 (User Story 1)

**Parallel**:
- T011, T012, T013 can run in parallel (different files: hosts/ops-jrz1.nix, modules/dev-services.nix)
- After T015 deploys: T016, T017 can run in parallel (both read-only checks)

**Sequential**:
- T014 depends on T011, T012, T013 (needs config in place)
- T015 depends on T014 (deployment needs config)
- T018-T027 must run sequentially (UI workflow dependencies)

### Phase 4-6 (User Stories 2, 3, 4)

**Parallel after US1**:
- US2 tasks (T028-T033) can run in parallel with US3 tasks (T034-T038) if US1 validates
- US4 tasks (T039-T042) should wait for US2 to confirm management UI working

---

## Implementation Strategy

### MVP-First Approach

**Week 1**: Focus exclusively on User Story 1 (T001-T027)
- Goal: Working Instagram bot responding to URLs in designated rooms
- Success: Can demo "post Instagram URL → see content in Matrix"
- Decision point: If MVP fails, stop and reassess before continuing

**Week 2**: Expand to User Stories 2 & 3 (T028-T038) in parallel
- Goal: Operational management and reliability validated
- Success: Admins can manage bots via UI, service survives restarts

**Week 3**: Add extensibility (User Story 4) if needed (T039-T042)
- Goal: Prove multi-bot capability
- Success: 3 concurrent bot instances running

**Week 4+**: 7-day validation period
- Monitor uptime (SC-003: 99% target)
- Monitor Instagram fetch success rate (SC-006: 95% target)
- Collect user feedback

### Incremental Delivery

Each user story delivers independently testable value:
- **US1**: Instagram content in Matrix (core value)
- **US2**: Self-service bot management (operational efficiency)
- **US3**: Production reliability (reduces maintenance burden)
- **US4**: Platform extensibility (future-proofing)

Can stop after any user story and still have working system.

---

## Testing Strategy

**Manual QA** (no automated tests per plan.md):
- Each user story has "Independent Test" criteria
- Acceptance scenarios from spec.md validated manually
- Success criteria (SC-001 through SC-008) checked via quickstart.md checklist

**Validation Period**:
- 7 days operational before merging to main (per constitution Principle III)
- Monitor metrics: uptime, response time, fetch success rate
- Document issues in worklog

---

## Risk Mitigation

**High-risk tasks**:
- T007: Removing registration_secrets (conduwuit incompatibility) - carefully test bot registration after change
- T015: Initial deployment (first time on ops-jrz1) - have rollback ready via nixos-rebuild switch --rollback
- T020: Bot user registration (new auth pattern) - document exact steps in worklog for repeatability

**Rollback points**:
- After T010: Can rollback before deployment if module adaptation fails
- After T015: NixOS generation rollback if service won't start
- After T027: Can remove bot and redeploy if issues found

---

## Success Metrics

**Per User Story**:
- US1: Bot responds to Instagram URLs within 5 seconds (SC-001)
- US2: Management UI loads within 2 seconds (SC-007)
- US3: 99% uptime over 7 days (SC-003), auto-recovery within 2 minutes (SC-004)
- US4: 3 concurrent instances without degradation (SC-002)

**Overall**:
- [ ] All 8 success criteria validated (SC-001 through SC-008)
- [ ] Constitution check passes (all 4 principles compliant)
- [ ] 7-day stability period completed without critical issues
- [ ] Documentation complete (spec, plan, quickstart, worklog, CLAUDE.md updated)

---

**Estimated Timeline**:
- **MVP (US1)**: 2-3 hours deployment + testing
- **Full Feature (US1-4)**: 1 week implementation + 1 week validation
- **Production Ready**: 2 weeks total (including 7-day stability period)

**Next Command**: `/speckit.implement` to begin execution (start with T001)