docs: correct alice framing - adversarial agent review for automation

alice is for reviewing AGENT work in unattended/autonomous contexts,
not code review. Key use cases:
- Autonomous runs on ops-jrz1
- CI/CD pipelines with agents
- High-stakes changes without human oversight

Added hybrid approach recommendation: use alice concepts (Stop hook,
adversarial methodology) with our infrastructure (beads, orch).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
dan 2026-01-09 16:45:49 -08:00
parent 144bafd2b5
commit 4773abe56f

View file

@ -158,60 +158,133 @@ From emes architecture:
- `codex` - OpenAI CLI - `codex` - OpenAI CLI
- `gemini` - Google CLI - `gemini` - Google CLI
## Applicability to Our Skills ## Use Case: Adversarial Agent Review
### code-review Skill alice is NOT about code review - it's about **adversarial review of agent work** in automation contexts.
**Current state:** Interactive - runs lenses, presents findings, asks before filing issues. ### The Problem alice Solves
**Potential enhancement:** Add quality gate that blocks exit until findings are addressed. When agents work autonomously (unattended, CI/CD, batch jobs):
- Agent may claim "done" when work is incomplete
- Agent may miss requirements or make incorrect assumptions
- Agent may take shortcuts that don't serve user intent
- No human in the loop to catch mistakes
**Challenges:** ### alice's Solution
1. We don't have jwz - would need state management
2. Our review IS the quality gate (not a separate reviewer)
3. Different use case: code-review reviews code, alice reviews agent work
**Options:** An independent adversarial reviewer (Opus) that:
1. **Works for the user, not the agent** - grounds truth in user's actual words
2. **Assumes errors exist** - actively looks for problems
3. **Steel-mans then attacks** - gives benefit of doubt, then systematically challenges
4. **Seeks external opinions** - uses Codex/Gemini for second opinions
5. **Mechanically blocks exit** - can't be talked out of it via prompt
### When to Use alice Pattern
| Scenario | Why alice helps |
|----------|-----------------|
| **Autonomous/unattended runs** | No human watching - need automated QA |
| **CI/CD with agents** | Quality gate before merge |
| **Complex multi-step features** | Verify each deliverable meets requirements |
| **Refactoring** | Ensure nothing broke |
| **ops-jrz1 deployment** | Remote server, less oversight |
### When NOT to Use
- Interactive sessions with human oversight
- Simple, low-risk changes
- Exploratory/research work (no deliverable to review)
## Applicability to Our Workflow
### Potential Use Cases
1. **Autonomous runs on ops-jrz1**
- Agent implements feature on VPS
- alice reviews before agent exits
- Issues filed to tissue if problems found
2. **Batch processing**
- Agent processes multiple tasks
- alice spot-checks work quality
3. **High-stakes changes**
- Security-sensitive code
- Infrastructure changes
- Production deployments
### Integration Options
| Approach | Pros | Cons | | Approach | Pros | Cons |
|----------|------|------| |----------|------|------|
| **A: Adopt jwz** | Full emes compatibility | Another dependency, Zig tool | | **A: Adopt alice directly** | Battle-tested, full features | Requires jwz, tissue, Zig deps |
| **B: Use beads** | Already have it | Not designed for transient session state | | **B: Build our own** | Tailored to our needs, use beads | Dev effort, reinventing wheel |
| **C: Simple file state** | Minimal, portable | DIY circuit breakers | | **C: Hybrid** | Use alice concepts, our infra | Best of both, some integration work |
| **D: Hook-only (stateless)** | Simplest | No persistence across tool calls | | **D: orch-as-reviewer** | Already have orch for multi-model | Different purpose, not adversarial |
### Recommendation ### Hybrid Approach (Recommended)
For code-review, the alice pattern is overkill. Our skill already does the review - we don't need a second reviewer to review the review. Use alice's **concepts** with our **infrastructure**:
**More useful pattern:** Use `Stop` hook to remind agent to run code-review before exiting if significant code changes were made. This is a "did you remember to review?" gate, not a "did review pass?" gate. 1. **Stop hook** - Block exit until review passes
2. **beads for state** - Track review status per session
3. **orch for second opinions** - We already have multi-model consensus
4. **Adversarial prompt** - Adapt alice's methodology
Example: Example hooks.json:
```json ```json
{ {
"hooks": { "hooks": {
"Stop": [{ "Stop": [{
"hooks": [{ "hooks": [{
"type": "command", "type": "command",
"command": "check-review-reminder.sh", "command": "review-gate check",
"timeout": 5 "timeout": 30
}] }]
}] }]
} }
} }
``` ```
The script checks if: `review-gate` would:
1. Significant code changes exist (git diff) 1. Check if review mode is enabled (beads flag or env var)
2. code-review was invoked this session 2. If enabled, check for approval in beads
3. If changes but no review → return non-zero (block with reminder) 3. If unapproved, block and instruct agent to spawn reviewer
4. Circuit breaker after N failures
### Reviewer Agent Design
Inspired by alice but using our tools:
```markdown
# Adversarial Reviewer
You review agent work for the USER, not the agent.
## Methodology
1. Read the user's original request (not agent's summary)
2. Examine all changes made (git diff, file reads)
3. Assume errors exist - find them
4. Use orch for second opinions on non-trivial work
5. Post decision to beads
## Decision
- APPROVED: Work meets user's actual request
- ISSUES: Problems found (file beads issues)
## Tools Available
- Read, Grep, Glob (read-only)
- orch (second opinions)
- bd (issue tracking)
```
## Open Questions ## Open Questions
1. Should we adopt jwz for cross-skill state coordination? 1. Do we need jwz or can beads handle session state?
2. Is the "review reminder" pattern valuable enough to implement? 2. Should the reviewer be a separate skill or plugin?
3. Could ops-review benefit from similar gating? 3. How do we handle the "review the reviewer" problem?
4. How do hooks interact with our dual-publish strategy? 4. What's the circuit breaker threshold (3 like alice)?
5. Should this be opt-in (`#review`) or always-on for certain contexts?
## References ## References