From 1e1965dc17001ccde73e5f5d745b3fcf4a27edcc Mon Sep 17 00:00:00 2001 From: dan Date: Sat, 24 Jan 2026 09:40:38 -0800 Subject: [PATCH] docs: add worklogs and work documents from Jan 21 sessions - 2026-01-21-add-review-skills.md - 2026-01-21-ralph-wiggum-lan-web-ui-prototype.md - 2026-01-21-worker-rpc-web-skills-audit.md - skill-compatibility.md - lan-web-ui-prototype work doc --- docs/skill-compatibility.md | 41 +++++ docs/work/2026-01-21-lan-web-ui-prototype.md | 47 +++++ docs/worklogs/2026-01-21-add-review-skills.md | 161 ++++++++++++++++ ...01-21-ralph-wiggum-lan-web-ui-prototype.md | 172 ++++++++++++++++++ .../2026-01-21-worker-rpc-web-skills-audit.md | 60 ++++++ 5 files changed, 481 insertions(+) create mode 100644 docs/skill-compatibility.md create mode 100644 docs/work/2026-01-21-lan-web-ui-prototype.md create mode 100644 docs/worklogs/2026-01-21-add-review-skills.md create mode 100644 docs/worklogs/2026-01-21-ralph-wiggum-lan-web-ui-prototype.md create mode 100644 docs/worklogs/2026-01-21-worker-rpc-web-skills-audit.md diff --git a/docs/skill-compatibility.md b/docs/skill-compatibility.md new file mode 100644 index 0000000..313a9a0 --- /dev/null +++ b/docs/skill-compatibility.md @@ -0,0 +1,41 @@ +# Skill Compatibility Audit (Top 10) + +Status legend: +- ✅ Works as-is +- ⚠️ Works with prerequisites / setup +- ❌ Not supported + +Agents: +- **Claude Code** +- **Gemini CLI** +- **Codex** +- **OpenCode** +- **Pi** + +## Summary Table + +| Skill | Key Dependencies | Claude | Gemini | Codex | OpenCode | Pi | Notes | +| --- | --- | --- | --- | --- | --- | --- | --- | +| **hq** | `worker`, `bd`, `git` | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Requires worker/bd installed; worker launch uses agent-specific CLI. | +| **review-gate** | `review-gate`, `jq` | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Claude supports Stop hook; others need manual orchestration. | +| **orch** | `orch` + API keys | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Requires orch CLI + GEMINI/OPENAI/OPENROUTER keys. | +| **code-review** | `bd`, lenses in `~/.config/lenses/code/` | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Works if bd + lenses are installed; optional orch. | +| **ops-review** | `bd`, `jq`, shellcheck/statix/deadnix/hadolint/yamllint | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Heavy toolchain; works with shell access + deps. | +| **spec-review** | `orch` + API keys | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Requires orch CLI; spec-kit repo structure. | +| **web-search** | `claude` CLI | ✅ | ❌ | ❌ | ❌ | ⚠️ | Claude-only unless claude CLI installed. | +| **web-research** | `claude` CLI (+ optional `llm`, `kagi`) | ✅ | ❌ | ❌ | ❌ | ⚠️ | Claude-only unless claude CLI installed. | +| **brave-search** | Node 18+, `npm install`, `BRAVE_API_KEY` | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Portable but requires Node + Brave API key. | +| **playwright-visit** | Playwright + Chromium (Nix devshell) | ⚠️ | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Works with shell + devshell setup; used for QA. | + +## Key Gaps + +1. **Claude-dependent web search**: `web-search` and `web-research` require Claude CLI and will fail for Gemini/Codex/OpenCode by default. +2. **Toolchain-heavy reviews**: `ops-review` and `code-review` depend on local tool installs and lens files. +3. **Worker orchestration relies on bd**: `hq` and `review-gate` depend on `bd` and `worker` being present. + +## Immediate Opportunities + +- Encourage **brave-search** as the default fallback when Claude search is unavailable. +- Add dependency checks and clearer failure messaging in `web-search` / `web-research` scripts. +- Document minimal install steps for review skills (lenses + lint toolchain). + diff --git a/docs/work/2026-01-21-lan-web-ui-prototype.md b/docs/work/2026-01-21-lan-web-ui-prototype.md new file mode 100644 index 0000000..487f631 --- /dev/null +++ b/docs/work/2026-01-21-lan-web-ui-prototype.md @@ -0,0 +1,47 @@ +# Work: LAN-only pi-web-ui prototype + +## Intent (Why) +- Explore a repo from a phone/browser while another terminal is in use. +- Provide a simple, local-only web UI for pi without cloud hosting. + +## Approach (How) +- Create a minimal web app using `@mariozechner/pi-web-ui`. +- Run a lightweight local server bound to LAN IP. +- Keep it unauthenticated and local-only for now. + +## Work Checklist +- [x] W001: Decide project location + folder layout for the prototype + - Verification: `prototypes/pi-web-ui-lan/` created in repo +- [x] W002: Document user workflow + test plan + - Verification: workflow + test steps captured in this doc +- [ ] W003: Scaffold a minimal web UI app (pi-web-ui ChatPanel) + - Verification: app builds and renders chat UI in browser +- [ ] W004: Provide a lightweight LAN server command + - Verification: open UI from phone on LAN +- [x] W005: Document minimal run steps + - Verification: README shows exact commands + +## Workflow (Draft) + +1. **Start local web UI** from the repo root. +2. **Find LAN IP + port** printed by the server. +3. **Open URL on phone** on the same Wi‑Fi network. +4. **Chat in web UI** to explore the repo while terminal is free. + +## Test Plan (Draft) + +1. Start the dev server on the laptop. +2. Confirm the UI loads in a desktop browser. +3. Open the LAN URL on the phone and confirm it loads. +4. Send a prompt and verify the chat UI streams a response. + +## Verification Evidence +- (2026-01-21) W001: Created `prototypes/pi-web-ui-lan/` with Vite layout. +- (2026-01-21) W002: Workflow + test plan documented here. +- (2026-01-21) W005: Added `prototypes/pi-web-ui-lan/README.md` with run steps. +- (2026-01-21) Dependencies installed via nix shell + pnpm (requires approving esbuild build scripts). + +## Notes +- Keep scope local-only; no auth, no external deployment. +- W003/W004 require running the dev server and validating LAN access. +- pnpm reported ignored build scripts for esbuild; may need `pnpm approve-builds` before running Vite. diff --git a/docs/worklogs/2026-01-21-add-review-skills.md b/docs/worklogs/2026-01-21-add-review-skills.md new file mode 100644 index 0000000..29b84c8 --- /dev/null +++ b/docs/worklogs/2026-01-21-add-review-skills.md @@ -0,0 +1,161 @@ +--- +title: "Worker RPC, Web Skills Expansion, and Compatibility Audit" +date: 2026-01-21 +keywords: [hq, worker-cli, brave-search, browser-tools, skills-audit] +commits: 0 +compression_status: uncompressed +--- + +# Session Summary + +**Date:** 2026-01-21 (Day 1 of project) +**Focus Area:** Worker RPC orchestration, portable web skills, and cross-agent skill compatibility + +# Accomplishments + +- [x] Implemented `skills/hq/scripts/worker-rpc.py` and a simulation harness to validate RPC event handling. +- [x] Added portable `brave-search` skill with auto-loading of `/run/secrets/api_keys/brave`. +- [x] Ported `browser-tools` skill (CDP-based Chrome automation) distinct from Playwright. +- [x] Created `handoff` skill to generate portable Markdown summaries across agents. +- [x] Produced `docs/skill-compatibility.md` (top 10 skills audit). +- [x] Drafted `specs/worker-message-bus.md` design spec for messaging + tmux observability. +- [x] Updated flake skill registry and API key exports (BRAVE/KAGI). + +# Key Decisions + +## Decision 1: Focus on skills (not pi extensions) + +- **Context:** pi extensions are powerful but Pi-only; we need skills that work across Claude, Codex, Gemini, and OpenCode. +- **Options considered:** + 1. Port pi extensions directly (TS) + 2. Use portable skills + CLI tools +- **Rationale:** Cross-agent portability wins; skills are the shared layer. +- **Impact:** Future work (loop mode, oracle, etc.) will be implemented as skills or CLI utilities, not TS. + +## Decision 2: Brave search as web fallback + +- **Context:** Claude web tools may be unavailable or out of credits. +- **Options considered:** + 1. Keep Claude-only web skills + 2. Add Brave API-based search + 3. Add Kagi only +- **Rationale:** Brave offers raw search results with minimal dependencies. +- **Impact:** `brave-search` added; `web-research` now documents fallback strategy. + +# Problems & Solutions + +| Problem | Solution | Learning | +|---------|----------|----------| +| `brave-search` failed when `BRAVE_API_KEY` missing | Added fallback to `/run/secrets/api_keys/brave` in script | Prefer sops-backed defaults for skills | +| `brave-search` not deployable via flake | Added skill to `skills.nix` and updated `flake.nix` API key exports | Flake registry must be kept in sync with skills/ | +| Browser automation needs visible Chrome (not Playwright) | Ported `browser-tools` from pi-skills | CDP + profile copy covers authenticated workflows | + +# Technical Details + +## Code Changes + +- Total files modified: 21 +- Key files changed: + - `skills/hq/SKILL.md` - RPC event mapping + WIP limit + - `skills/brave-search/scripts/search.js` - API key fallback + - `skills/browser-tools/scripts/browser-*.js` - CDP automation suite + - `skills.nix` / `flake.nix` - skill registry + API key exports + - `docs/skill-compatibility.md` - audit matrix +- New files created: + - `skills/brave-search/` (SKILL, README, scripts, package.json) + - `skills/browser-tools/` (SKILL, README, scripts, package.json) + - `skills/handoff/` (SKILL, README, scripts) + - `specs/worker-message-bus.md` +- Files deleted: + - None + +## Commands Used + +```bash +# Install brave-search deps and test +cd skills/brave-search +npm install +node scripts/search.js "nix flakes" -n 1 + +# Skill deployment test +nix build ~/proj/skills#brave-search + +# Worklog metrics + filename +skills/worklog/scripts/extract-metrics.sh +skills/worklog/scripts/suggest-filename.sh +``` + +## Architecture Notes + +- The worker CLI already contains a schema-ready SQLite message bus; only CLI commands are missing. +- `browser-tools` complements `playwright-visit` by enabling authenticated and interactive sessions. +- Skill install is mediated by `.skills` manifests and `use-skills.sh` with Nix builds. + +# Process and Workflow + +## What Worked Well + +- Reusing pi-skills implementations for Brave and browser CDP tools reduced risk. +- Rooting in skills rather than TS extensions kept portability high. +- Early audit (`docs/skill-compatibility.md`) made gaps visible. + +## What Was Challenging + +- Synchronizing flake registry with new skills. +- Navigating multiple agent ecosystems (pi vs Claude vs OpenCode). +- Deciding how to represent runtime orchestration across tools. + +# Learning and Insights + +## Technical Insights + +- Pi skills are separate from pi-mono; extensions are Pi-only. +- Brave search provides low-latency, API-driven results without browser overhead. +- The worker bus schema is already adequate for structured inter-agent comms. + +## Process Insights + +- Keeping skills portable (bash + node + python) is the best compatibility strategy. +- Central registry (`skills.nix`) is the single point of failure for deployability. + +## Architectural Insights + +- Messaging and observability should be CLI-driven, not skill-driven. +- Review workflows (code-review, ops-review, spec-review) are portable but require a heavy toolchain. + +# Context for Future Work + +## Open Questions + +- Should `browser-tools` be added to `.skills` by default or remain opt-in? +- How to structure log retention and pruning for worker message bus? +- Do we need a standardized JSON schema for worker messages in v1? + +## Next Steps + +- Implement `worker msg` CLI commands per `specs/worker-message-bus.md`. +- Add `worker watch` tmux integration. +- Expand skill compatibility audit beyond the top 10. +- Update dotfiles system packages (Chromium + Node) via `dotfiles-scvr`. + +## Related Work + +- `docs/worklogs/2026-01-21-worker-rpc-web-skills-audit.md` +- `docs/worklogs/2026-01-13-hq-deployment-codex-skills-integration.md` +- `docs/worklogs/2026-01-11-worker-cli-cleanup-refactors.org` +- `specs/worker-message-bus.md` +- Beads: `skills-29bp`, `skills-imei`, `skills-s5xl`, `skills-wxbs`, `skills-telx`, `dotfiles-scvr` + +# Raw Notes + +- Brave search default should not rely on Claude credits. +- Pi extensions stay in dotfiles; skills remain in this repo. +- `browser-tools` uses CDP with Chrome `:9222`, suitable for authenticated testing. + +# Session Metrics + +- Commits made: 0 +- Files touched: 21 +- Lines added/removed: +682/-20 +- Tests added: 0 +- Tests passing: 0/0 diff --git a/docs/worklogs/2026-01-21-ralph-wiggum-lan-web-ui-prototype.md b/docs/worklogs/2026-01-21-ralph-wiggum-lan-web-ui-prototype.md new file mode 100644 index 0000000..1b4395c --- /dev/null +++ b/docs/worklogs/2026-01-21-ralph-wiggum-lan-web-ui-prototype.md @@ -0,0 +1,172 @@ +--- +title: "Ralph Wiggum Integration + LAN pi-web-ui Prototype" +date: 2026-01-21 +keywords: [pi, ralph-wiggum, skills, extensions, web-ui, lan] +commits: 0 +compression_status: uncompressed +--- + +# Session Summary + +**Date:** 2026-01-21 (Day 21 of project) +**Focus Area:** Ralph Wiggum loop integration for pi + LAN-only pi-web-ui prototype + +# Accomplishments + +- [x] Located and reviewed pi-specific Ralph Wiggum implementations (awesome-pi-agent, tmustier/pi-extensions). +- [x] Forked Ralph Wiggum extension into the repo with Intent/Approach/Work dialect enforcement. +- [x] Added project-local pi skill (`ralph-work-loop`) to start loops on existing Work docs. +- [x] Ran a multi-lens code review on the extension and filed beads issues. +- [x] Implemented fixes for archive safety, path confirmation, and init deduplication; marked SRP refactor as wontfix. +- [x] Copied extension + skill to global pi directories for immediate use. +- [x] Scaffolding for a LAN-only pi-web-ui prototype in `prototypes/pi-web-ui-lan`. +- [x] Documented workflow + test plan in `docs/work/2026-01-21-lan-web-ui-prototype.md`. +- [x] Resolved blank-screen errors by pinning to published npm versions and a valid model. +- [x] Fixed Lit dev-mode class-field-shadowing error by forcing production condition resolution in Vite. + +# Key Decisions + +## Decision 1: Fork and adapt Ralph Wiggum instead of rewriting + +- **Context:** Needed pi-compatible Ralph loop with Intent/Approach/Work dialect. +- **Options considered:** + 1. Reimplement from scratch + 2. Fork tmustier/pi-extensions and customize templates/prompts +- **Rationale:** Forking provided stable loop mechanics and UI while allowing prompt/template customization. +- **Impact:** Faster delivery with minimal behavioral drift. + +## Decision 2: Keep extension single-file (wontfix SRP) + +- **Context:** Code review flagged 802-line monolith. +- **Options considered:** + 1. Split into modules + 2. Keep single file for now +- **Rationale:** Low maintenance risk and fewer moving parts in early stage. +- **Impact:** SRP refactor deferred; functionality fixes prioritized. + +## Decision 3: LAN prototype uses published npm packages + +- **Context:** pi packages on npm were at 0.49.3, not 1.37.x. +- **Options considered:** + 1. Use npm 0.49.3 + 2. Point to local pi-mono checkout +- **Rationale:** LAN prototype should be quick to run; npm is simplest. +- **Impact:** Updated prototype deps to 0.49.3 and selected valid model IDs. + +# Problems & Solutions + +| Problem | Solution | Learning | +|---------|----------|----------| +| Blank web UI at startup | Model ID mismatch caused boot failure | `getModel()` throws; use valid IDs from published package | +| Lit class-field-shadowing error in dev | Forced Vite to resolve production conditions | Disable Lit dev bundle in Vite to avoid shadowing error | +| pnpm install failed for pi 1.37.x | Switched to published 0.49.3 packages | Published npm versions lag internal tags | +| `pnpm approve-builds` required | Manual approval for esbuild | Nix shell + pnpm still requires approval for build scripts | + +# Technical Details + +## Code Changes + +- Total files modified: 24 (uncommitted) +- Key files changed: + - `.pi/extensions/ralph-wiggum/index.ts` - Intent/Approach/Work fork, safety fixes, dedupe + - `.pi/skills/ralph-work-loop/SKILL.md` - Skill to start Ralph loops on Work docs + - `.pi/skills/ralph-work-loop/scripts/find-latest-work.sh` - Helper for latest Work doc + - `prototypes/pi-web-ui-lan/src/main.ts` - Minimal web UI, model selection + - `prototypes/pi-web-ui-lan/vite.config.ts` - Force production condition resolution + - `docs/work/2026-01-21-lan-web-ui-prototype.md` - Work doc + workflow/test plan +- New files created: + - `prototypes/pi-web-ui-lan/*` - LAN prototype scaffold + - `.pi/extensions/ralph-wiggum/README.md` + - `.pi/skills/ralph-work-loop/README.md` + - `docs/work/2026-01-21-lan-web-ui-prototype.md` +- Files deleted: + - None + +## Commands Used + +```bash +# Locate Ralph implementation +./skills/brave-search/scripts/search.js "ralph wiggum" --content + +# Copy extension/skill to global pi +cp -r .pi/extensions/ralph-wiggum ~/.pi/agent/extensions/ +cp -r .pi/skills/ralph-work-loop ~/.pi/agent/skills/ + +# Nix shell + pnpm +nix shell nixpkgs#nodejs_20 nixpkgs#pnpm +pnpm install +pnpm approve-builds +pnpm run dev:lan +``` + +## Architecture Notes + +- Ralph Wiggum loop is packaged as a **pi extension** plus a **pi skill** wrapper for Work docs. +- LAN prototype currently runs in **direct mode** (browser calls provider APIs), no repo access. +- Next step for repo access: a lightweight RPC backend bridging to pi’s agent session. + +# Process and Workflow + +## What Worked Well + +- Reusing tmustier’s extension avoided rework and validated loop mechanics. +- Pi skill wrapping made natural-language trigger feasible for Work docs. +- Vite condition override resolved Lit dev error quickly. + +## What Was Challenging + +- pnpm approval flow is interactive under nix shell. +- Published npm versions lagged internal pi tags. +- LAN test still pending due to API key entry and phone access. + +# Learning and Insights + +## Technical Insights + +- Lit dev bundle throws class-field-shadowing errors under certain builds; forcing production conditions avoids it. +- Published npm packages are much older than pi-mono tags; choose models accordingly. + +## Process Insights + +- Shipping a loop in pi is best done as extension + skill rather than only one or the other. +- LAN-only prototype should stay minimal until the RPC bridge is designed. + +## Architectural Insights + +- Intent/Approach/Work dialect can be enforced in loop prompts without changing loop mechanics. +- A future RPC bridge would allow filesystem exploration without exposing API keys to the browser. + +# Context for Future Work + +## Open Questions + +- Should LAN prototype add a lightweight RPC bridge to enable repo access? +- Where should the nixian deploy live in dotfiles for pi extensions/skills? +- Should we seed API keys from env for LAN-only use (risk of exposure)? + +## Next Steps + +- Run LAN prototype end-to-end from phone (W003/W004 in work doc). +- Decide if API key seeding from env is acceptable for local use. +- If so, add optional `VITE_GOOGLE_API_KEY` seeding in `src/main.ts`. + +## Related Work + +- `docs/work/2026-01-21-lan-web-ui-prototype.md` +- `docs/worklogs/2026-01-18-intent-approach-work-framework-design.md` +- `docs/worklogs/2026-01-21-add-review-skills.md` + +# Raw Notes + +- Ralph Wiggum extension fork located under `.pi/extensions/ralph-wiggum/` in this repo. +- Skill wrapper under `.pi/skills/ralph-work-loop/`. +- LAN prototype shows API key prompt for Google when using `gemini-3-flash-preview`. +- The UI now renders after Vite resolve conditions update. + +# Session Metrics + +- Commits made: 0 +- Files touched: 24 +- Lines added/removed: +682/-20 (from extract-metrics) +- Tests added: 0 +- Tests passing: N/A diff --git a/docs/worklogs/2026-01-21-worker-rpc-web-skills-audit.md b/docs/worklogs/2026-01-21-worker-rpc-web-skills-audit.md new file mode 100644 index 0000000..8eeb9ad --- /dev/null +++ b/docs/worklogs/2026-01-21-worker-rpc-web-skills-audit.md @@ -0,0 +1,60 @@ +# Worker RPC, Web Skills, and Multi-Agent Audit + +**Date:** 2026-01-21 +**Author:** dan +**Tags:** hq, worker, pi, brave-search, browser-tools, research + +## Intent +- Upgrade **HQ worker orchestration** to support headless control via RPC. +- Audit and expand **web research capabilities** to be agent-neutral (less Claude-dependent). +- Investigate **pi-mono** patterns (extensions, messaging) for architectural alignment. +- Prepare the ground for **Beads messaging** replacement and **tmux observability**. + +## Approach +1. **Worker RPC**: Created `worker-rpc.py` to wrap the `pi` agent process, translating JSONL RPC events into semantic signals (`worker start`, `worker done`) for HQ monitoring. +2. **Web Skills**: Created `brave-search` (portable API-based search) and `browser-tools` (interactive CDP automation) to reduce dependency on Claude's built-in web tool. +3. **Audit**: Conducted a top-10 skill compatibility audit across 5 agents (Claude, Gemini, Codex, OpenCode, Pi). +4. **Research**: Cloned and inspected `pi-mono`, `pi-skills`, and `awesome-pi-agent` to identify reusable patterns (state-in-session, extensions vs skills). +5. **Planning**: Defined epics for "Runtime Control" and "Messaging/Observability" (skills-29bp). + +## Work + +### 1. HQ RPC Control Plane +- Implemented `skills/hq/scripts/worker-rpc.py`. + - Wraps `pi --mode rpc`. + - Detects `worker done` bash commands via RPC events. + - Streams logs to stdout/stderr. +- Added simulation test suite `skills/hq/test/run-simulation.sh`. +- Updated `HQ/SKILL.md` with RPC event mapping and WIP limits (hard cap: 3). + +### 2. Portable Web Skills +- **brave-search**: + - `scripts/search.js`: API query, supports `-n`, `--freshness`, `--content`. + - `scripts/content.js`: Readability/Turndown markdown extraction. + - Auto-loads key from `/run/secrets/api_keys/brave`. +- **browser-tools**: + - Ported from `pi-skills`. + - Provides `browser-start.js` (Chrome :9222), `browser-nav.js`, `browser-pick.js` (interactive selector). +- **handoff**: + - Created `skills/handoff/` for structured session summaries (git status, diffs, open questions). + +### 3. Skill System Updates +- Updated `skills.nix` to include `brave-search`, `browser-tools`, `handoff`. +- Updated `flake.nix` to expose `BRAVE_API_KEY` and `KAGI_API_KEY` to direnv environments. +- Updated `docs/skill-compatibility.md` with the multi-agent audit results. + +### 4. Beads Tracking +- Created Epic **skills-29bp**: "Worker runtime + observability research branches". +- Created **skills-imei**: "Worker CLI: Implement Message Bus and Tmux Observability". +- Created **skills-q8i0**: "Background launch mechanism". +- Created **dotfiles-scvr**: System setup for new skills (Chromium/Node). + +## Refined Views +- **Pi Extensions vs Skills**: Confirmed that TypeScript extensions are Pi-only. We will adopt their *patterns* (state-in-session, loop files) but implement them as portable CLI tools (like `worker` msg bus). +- **Web Search**: Moving away from `claude -p` dependency toward `brave-search` allows Gemini/Codex to perform research reliably. +- **Messaging**: The `worker` CLI's SQLite DB is already schema-ready for a message bus; we just need to expose `send/follow` commands. + +## Next Steps +1. Implement `worker msg` commands (send, poll, follow) in Nim. +2. Implement `worker watch` (tmux integration). +3. Standardize logging for `tail -f` observability.