# Feature Specification: Matrix Bot Framework (Maubot) Integration **Feature Branch**: `003-maubot-integration` **Created**: 2025-10-26 **Status**: Draft **Input**: User description: "Begin maubot feature spec. instagram bot is one of our goals." ## Clarifications ### Session 2025-10-26 - Q: Instagram bot activation behavior - should it respond to all Instagram URLs, only when mentioned, or in designated rooms? → A: Bot responds to Instagram URLs only in designated bot-enabled rooms - Q: Bot error notification method - how should errors be communicated to administrators? → A: Error notification behavior based on severity levels (DEBUG/INFO logs only, WARN logs + dashboard visibility, ERROR/CRITICAL logs + dashboard + Matrix admin room notifications) - Q: Room enablement mechanism - how do administrators enable bot in specific rooms? → A: Edit bot configuration file with room IDs, restart bot instance - Q: Admin notification room configuration - should each bot have dedicated admin room, shared room, or reuse homeserver admin room? → A: Reuse Matrix homeserver admin room for bot ERROR/CRITICAL notifications - Q: Management interface authentication - single shared account, multi-user, or Matrix homeserver auth? → A: Single shared admin account (username/password configured in sops-nix secrets) ## User Scenarios & Testing *(mandatory)* ### User Story 1 - Instagram Content Sharing to Matrix (Priority: P1) A team member shares an Instagram post URL in a Matrix room, and the bot automatically fetches and displays the content (image, caption, metadata) directly in the chat, allowing team members to view and discuss Instagram content without leaving Matrix. **Why this priority**: This is the core value proposition - bringing Instagram content into team communication. Demonstrates immediate utility of the bot framework and validates the integration works correctly. **Independent Test**: Can be fully tested by posting an Instagram URL in a Matrix room and verifying the bot responds with content preview, delivering immediate value as an Instagram content viewer. **Acceptance Scenarios**: 1. **Given** Instagram bot is enabled in a specific Matrix room, **When** user posts "https://instagram.com/p/ABC123/" in that room, **Then** bot responds within 5 seconds with image, caption, and post metadata (likes, comments count) 2. **Given** Instagram bot is NOT enabled in a Matrix room, **When** user posts Instagram URL in that room, **Then** bot ignores the URL and does not respond 3. **Given** bot receives Instagram URL in enabled room, **When** content is a video, **Then** bot provides video thumbnail, caption, and download link 4. **Given** bot receives Instagram URL in enabled room, **When** content is a carousel (multiple images), **Then** bot displays all images in sequence with navigation 5. **Given** bot receives Instagram profile URL in enabled room, **When** URL is "https://instagram.com/username", **Then** bot displays profile info (bio, follower count, recent posts preview) 6. **Given** bot encounters rate limiting in enabled room, **When** too many requests in short period, **Then** bot queues request and notifies user of delay --- ### User Story 2 - Bot Management Interface (Priority: P2) Platform administrators can configure, start, stop, and monitor bots through a web-based management interface without editing configuration files or restarting services. **Why this priority**: Essential for operational management and enables non-developer administrators to manage bots. Required for long-term maintainability but bot can work without it initially. **Independent Test**: Can be tested by accessing management interface, creating a test bot instance, and verifying it appears in Matrix - demonstrates full bot lifecycle management. **Acceptance Scenarios**: 1. **Given** administrator accesses Maubot management UI, **When** they log in with shared admin credentials, **Then** dashboard displays all bot instances, their status, and health metrics 2. **Given** administrator wants to deploy Instagram bot, **When** they upload maubot plugin file (.mbp), **Then** plugin appears in available plugins list 3. **Given** plugin is uploaded, **When** administrator creates new bot instance with Matrix user credentials and room subscription list, **Then** bot appears online in Matrix within 30 seconds and only responds in configured rooms 4. **Given** administrator wants to change enabled rooms, **When** they edit bot configuration file with new room IDs and restart bot instance, **Then** bot begins responding only in newly configured rooms 5. **Given** bot is running, **When** administrator clicks "Stop" button, **Then** bot goes offline and stops responding to commands 6. **Given** bot encounters error, **When** viewing bot logs in UI, **Then** error messages are displayed with timestamps, severity level, and context 7. **Given** bot experiences CRITICAL error, **When** error occurs, **Then** notification is sent to Matrix homeserver admin room with error details and affected bot instance --- ### User Story 3 - Bot Framework Service Reliability (Priority: P2) The Maubot service starts automatically on server boot, maintains bot instances across restarts, and recovers from failures without manual intervention. **Why this priority**: Critical for production use but can be validated after basic functionality works. Prevents the bot framework from being a maintenance burden. **Independent Test**: Can be tested by rebooting the server and verifying Maubot service auto-starts and all bot instances resume operation automatically. **Acceptance Scenarios**: 1. **Given** server reboots, **When** system comes back online, **Then** Maubot service starts automatically within 2 minutes and all bot instances reconnect to Matrix 2. **Given** Matrix homeserver restarts, **When** homeserver is available again, **Then** bot instances re-establish connections and resume operation without manual intervention 3. **Given** bot instance crashes, **When** Maubot detects failure, **Then** service attempts automatic restart with exponential backoff 4. **Given** bot encounters persistent error (ERROR/CRITICAL severity), **When** restart attempts fail, **Then** service logs detailed diagnostics, updates dashboard status, and sends notification to Matrix homeserver admin room 5. **Given** database connection lost, **When** connectivity is restored, **Then** Maubot reconnects automatically and restores bot state --- ### User Story 4 - Additional Bot Deployment (Priority: P3) Platform administrators can deploy additional custom bots beyond Instagram bot by uploading plugin files and configuring bot instances, enabling extensible bot functionality for future team needs. **Why this priority**: Demonstrates platform extensibility and future-proofs the investment, but not required for initial value delivery. Can be added after Instagram bot proves value. **Independent Test**: Can be tested by deploying a simple echo bot or reaction bot from maubot plugin repository and verifying it works independently. **Acceptance Scenarios**: 1. **Given** administrator has custom maubot plugin (.mbp file), **When** they upload via management interface, **Then** plugin is validated and added to available plugins 2. **Given** plugin requires configuration, **When** creating bot instance, **Then** administrator can provide plugin-specific settings through UI 3. **Given** multiple bot instances exist, **When** administrator views dashboard, **Then** all bots are clearly listed with their types, status, and resource usage 4. **Given** bot requires database storage, **When** bot instance is created, **Then** Maubot automatically provisions isolated database for that bot 5. **Given** plugin has dependencies, **When** uploading plugin, **Then** Maubot validates dependencies and reports missing requirements --- ## Requirements *(mandatory)* ### Functional Requirements - **FR-001**: System MUST extract and deploy maubot module from ops-base repository to ops-jrz1 infrastructure - **FR-002**: System MUST integrate Maubot with existing conduwuit Matrix homeserver on clarun.xyz - **FR-003**: System MUST provide web-based management interface on dedicated port (default: 29316) accessible to platform administrators via single shared admin account credentials stored in sops-nix secrets - **FR-004**: Maubot service MUST support automatic startup on system boot and auto-recovery from failures - **FR-005**: System MUST support Instagram bot plugin deployment with content fetching capabilities - **FR-006**: Instagram bot MUST fetch and display images, videos, captions, and metadata from Instagram URLs posted only in designated bot-enabled Matrix rooms (bot ignores URLs in rooms where it is not explicitly enabled) - **FR-007**: Instagram bot MUST handle rate limiting gracefully with user-friendly error messages - **FR-008**: System MUST support multiple bot instances running concurrently with isolated configurations (architecture supports 3+ instances per SC-002, production deploys 1 instance initially per quickstart.md) - **FR-009**: System MUST persist bot configurations and state to survive service restarts - **FR-010**: Administrators MUST be able to configure bot room subscriptions by editing bot configuration file with Matrix room IDs and restarting the bot instance - **FR-011**: System MUST provide health monitoring for bot instances with status indicators (health check API endpoint and dashboard status display via management interface) - **FR-012**: System MUST integrate with existing sops-nix secrets management for bot credentials - **FR-013**: System MUST support uploading and deploying additional maubot plugins (.mbp files) - functionality inherited from ops-base maubot.nix module, validated in T029 - **FR-014**: System MUST provide logging capabilities for bot activity and errors accessible via management interface with severity-based propagation (DEBUG/INFO to logs only, WARN to logs and dashboard, ERROR/CRITICAL to logs, dashboard, and Matrix homeserver admin room) - **FR-015**: Bot instances MUST authenticate with Matrix homeserver using registration tokens (conduwuit compatibility requirement, shared secret not supported) - **FR-016**: System MUST support per-bot database storage with automatic provisioning ### Key Entities - **Maubot Service**: Plugin-based Matrix bot framework that manages multiple bot instances, provides management interface, and handles Matrix homeserver integration - **Bot Instance**: Individual bot deployment with specific configuration, Matrix user account, and plugin assignment (e.g., "instagram-bot-1") - **Plugin**: Packaged bot functionality (.mbp file) containing code, metadata, and dependencies (e.g., Instagram content fetcher, echo bot, reaction bot) - **Bot Configuration**: Settings specific to bot instance including Matrix credentials, plugin settings, room subscriptions (list of enabled room IDs), and command prefixes - **Management Interface**: Web UI for administrators to create, configure, monitor, and control bot instances, displaying logs with severity levels and real-time status updates - **Admin Notification**: ERROR and CRITICAL level bot notifications sent to existing Matrix homeserver admin room (shared with other platform notifications) - **Bot Database**: Per-instance isolated SQLite database for plugin state and data persistence ## Success Criteria *(mandatory)* ### Measurable Outcomes - **SC-001**: Instagram bot responds to Instagram URLs with content preview within 5 seconds under normal conditions - **SC-002**: System supports at least 3 concurrent bot instances without performance degradation - **SC-003**: Maubot service maintains 99% uptime over 7-day testing period - **SC-004**: Bot instances automatically recover within 2 minutes after service restart - **SC-005**: Administrators can deploy a new bot instance from scratch in under 10 minutes - **SC-006**: Instagram bot successfully fetches content for 95% of public Instagram post URLs - **SC-007**: Management interface loads and displays bot status within 2 seconds - **SC-008**: System handles server reboot without data loss or manual intervention required **Validation Note**: SC-001, SC-002, SC-003, SC-004, SC-008 have explicit task validation (T026, T042, T038, T034, T034). SC-005, SC-006, SC-007 are measured during the 7-day operational validation period (T038) and documented in deployment worklog (T044). ## Scope *(mandatory)* ### In Scope - Extract and adapt maubot.nix module from ops-base to ops-jrz1 - Configure Maubot to integrate with conduwuit Matrix homeserver - Deploy Instagram bot plugin as primary use case - Set up management web interface with authentication - Implement health monitoring and auto-recovery mechanisms - Configure sops-nix secrets for bot credentials - Document bot deployment and management procedures including room subscription configuration workflow - Support for uploading additional maubot plugins ### Out of Scope - Custom Instagram bot development (use existing maubot Instagram plugin from community) - Migration of other bots from ops-base besides Instagram bot - Advanced analytics or metrics dashboard for bot performance - Multi-homeserver support (only clarun.xyz) - Custom plugin development beyond Instagram bot deployment - Mobile app for bot management (web interface only) - Automatic Instagram authentication (manual token provisioning acceptable) - Real-time Instagram feed monitoring or notifications ## Constraints *(mandatory)* ### Technical Constraints - Must work with conduwuit Matrix homeserver (ops-base used continuwuity, may require compatibility testing) - Limited to Python 3.11 for maubot runtime (nixpkgs availability) - Instagram bot functionality depends on Instagram API/scraping availability and rate limits - Must adapt from ops-base VM-based deployment pattern to ops-jrz1 VPS single-host pattern - Dependent on deprecated olm-3.2.16 library for Matrix encryption (known CVEs, acceptable risk documented in ops-base) ### Operational Constraints - Deployment must not disrupt existing services (Matrix homeserver, Slack bridge, Forgejo) - Management interface must be secured (single admin account authentication, localhost-only access) - Management interface credentials must be stored in sops-nix encrypted secrets - Bot Matrix accounts require registration tokens from homeserver - Instagram tokens may require periodic renewal based on Instagram API policies ### Resource Constraints - Maubot service limited to 512M memory (as per ops-base configuration) - Additional database space required for bot state (estimated <100MB initially) - Management interface port 29316 must not conflict with existing services ## Dependencies *(mandatory)* ### External Dependencies - ops-base repository access to extract maubot.nix module and documentation - Instagram bot plugin from maubot community or ops-base implementation - Instagram authentication tokens (if required by current Instagram API policies) - Matrix homeserver registration token for bot user creation ### Internal Dependencies - conduwuit Matrix homeserver must be operational on clarun.xyz - sops-nix secrets management must be configured for bot credentials - SQLite for bot state storage (decision per plan.md research: lightweight isolation better than shared PostgreSQL) - Existing NixOS infrastructure and deployment patterns ### Blocking Issues - Need to verify conduwuit compatibility with maubot (ops-base used continuwuity) - Need to assess current Instagram API access requirements and scraping feasibility - Need to extract and adapt ops-base module configuration options from `services.matrix-vm.maubot` to `services.dev-platform.maubot` ## Assumptions *(mandatory)* - Instagram content fetching remains technically feasible (no major Instagram API changes blocking access) - Maubot works with conduwuit Matrix homeserver with minimal or no modifications - ops-base maubot module can be adapted to VPS deployment with reasonable effort - Instagram bot plugin from ops-base is functional and can be reused or community plugin exists - Team accepts olm-3.2.16 security risk with documented mitigation plan (migration to vodozemac when available) - Bot traffic will remain under Instagram rate limits for small team usage (<100 requests/hour) - Single VPS deployment sufficient (no distributed bot architecture needed) - Single shared admin account sufficient for initial deployment (no multi-user management required) ## Non-Goals *(optional)* - Automated Instagram post monitoring or scheduled fetching - Direct posting to Instagram from Matrix (read-only integration) - Instagram DM integration or two-way messaging - Advanced content moderation or filtering - Custom Instagram analytics or engagement tracking - Multi-tenant bot hosting for external teams - Commercial Instagram API integration (acceptable to use community scraping approaches) - Real-time Instagram notifications or webhooks ## Known Limitations *(optional)* The following edge cases are known limitations not addressed in MVP scope: - **Deleted/private Instagram posts**: Bot does not handle posts that become private or deleted after initial fetch (content remains in Matrix chat history) - **Instagram rate limiting**: System may experience delays during high-traffic periods (429 responses). FR-007 requires graceful handling with user notifications. - **Matrix account credential expiry**: Bot user account credentials are managed via registration tokens and do not expire automatically. Manual re-authentication required if revoked. - **Instagram story URLs**: 24-hour expiry stories not supported (yt-dlp limitation for ephemeral content) - **Command collision**: Multiple bot instances in same room may respond to overlapping triggers. Recommendation: enable only one bot per room or use distinct command prefixes. - **Age-restricted/geo-blocked content**: Instagram content with access restrictions may fail to fetch depending on VPS location and yt-dlp capabilities - **Management interface connection loss**: If Maubot loses connection to Matrix homeserver, bot instances stop responding until connection restored (monitored via health checks in FR-011) - **Database corruption**: No automated backup/recovery. Recommendation: implement manual backup procedure for /var/lib/maubot/ during operational period. ## Risks *(optional)* ### Technical Risks - **Risk**: Instagram API/scraping methods may break with Instagram updates - **Mitigation**: Document bot as best-effort, plan for periodic maintenance, monitor Instagram bot community for updates - **Risk**: Conduwuit compatibility issues with maubot not discovered until integration - **Mitigation**: Test maubot registration and basic functionality early in implementation phase - **Risk**: olm-3.2.16 vulnerabilities may be exploited - **Mitigation**: Follow ops-base mitigation strategy - monitor for vodozemac migration, limit bot network exposure, document accepted risk ### Operational Risks - **Risk**: Instagram rate limiting may impact bot responsiveness during high usage - **Mitigation**: Implement request queuing, user notifications for delays, consider rate limit monitoring - **Risk**: Bot management interface security breach could compromise Matrix homeserver - **Mitigation**: Require strong authentication, limit network exposure, regular security audits, use sops-nix for credential storage - **Risk**: Bot instance failure may go unnoticed without monitoring - **Mitigation**: Implement health checks, automated restarts, log monitoring, administrator alerts for persistent failures ## Clarified Requirements *(resolved 2025-10-26)* ### Instagram Authentication Approach **Decision**: Use community scraping methods (instaloader, yt-dlp) for Instagram content fetching. **Rationale**: Easier to set up immediately without requiring Facebook developer account approval. Acceptable for internal team use with understanding that scraping methods may require periodic updates if Instagram changes their interface. ### Management Interface Network Exposure **Decision**: Restrict management interface to localhost only, requiring SSH tunnel for remote administration. **Rationale**: Maximizes security by eliminating network attack surface. Administrators already have SSH access for deployment, so tunnel setup is acceptable operational overhead for the security benefit. ### Bot Instance Quantity Planning **Decision**: Support single Instagram bot instance initially (1 instance). **Rationale**: Minimal resource requirements, proves concept quickly, demonstrates value before scaling. Architecture can support additional instances later if needed without major rework. **Note**: SC-002 requires validating 3-instance capability during testing to ensure architecture can scale when needed, but production deployment starts with single instance.