docs(ui-query): add full command documentation
Document all commands: list-windows, get-text, find-element, query-state, benchmark. Add common flags section. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
parent
ca98eec7a6
commit
6c09bee2c4
|
|
@ -129,7 +129,7 @@
|
||||||
{"id":"skills-bo8","title":"Gemini skills access: ReadFile path restrictions block .claude/skills/","description":"Gemini agent couldn't read skill files from .claude/skills/orch/SKILL.md due to path restrictions. ReadFile tool restricts paths to workspace directories, so .claude/skills/ (symlinked from home-manager) is blocked. Agent had to fall back to shell cat command. Breaks skills portability across agents. Potential fixes: copy skills into repo, configure allowed paths, use MCP, or document workaround.","status":"closed","priority":3,"issue_type":"bug","created_at":"2026-01-09T10:58:04.037329419-08:00","created_by":"dan","updated_at":"2026-01-09T19:35:28.068433744-08:00","closed_at":"2026-01-09T19:35:28.068433744-08:00","close_reason":"Fix found: Gemini includeDirectories setting"}
|
{"id":"skills-bo8","title":"Gemini skills access: ReadFile path restrictions block .claude/skills/","description":"Gemini agent couldn't read skill files from .claude/skills/orch/SKILL.md due to path restrictions. ReadFile tool restricts paths to workspace directories, so .claude/skills/ (symlinked from home-manager) is blocked. Agent had to fall back to shell cat command. Breaks skills portability across agents. Potential fixes: copy skills into repo, configure allowed paths, use MCP, or document workaround.","status":"closed","priority":3,"issue_type":"bug","created_at":"2026-01-09T10:58:04.037329419-08:00","created_by":"dan","updated_at":"2026-01-09T19:35:28.068433744-08:00","closed_at":"2026-01-09T19:35:28.068433744-08:00","close_reason":"Fix found: Gemini includeDirectories setting"}
|
||||||
{"id":"skills-buh","title":"Document SQLite compile flags in config.nims","description":"[EVOLVE] LOW - SQLite compile flags (SQLITE_THREADSAFE, SQLITE_ENABLE_JSON1, SQLITE_OMIT_LOAD_EXTENSION) are hardcoded. Add comments explaining purpose.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-10T18:50:54.19875394-08:00","created_by":"dan","updated_at":"2026-01-10T18:50:54.19875394-08:00"}
|
{"id":"skills-buh","title":"Document SQLite compile flags in config.nims","description":"[EVOLVE] LOW - SQLite compile flags (SQLITE_THREADSAFE, SQLITE_ENABLE_JSON1, SQLITE_OMIT_LOAD_EXTENSION) are hardcoded. Add comments explaining purpose.","status":"open","priority":4,"issue_type":"task","created_at":"2026-01-10T18:50:54.19875394-08:00","created_by":"dan","updated_at":"2026-01-10T18:50:54.19875394-08:00"}
|
||||||
{"id":"skills-bvz","title":"spec-review: Add Definition of Ready checklists for each phase","description":"'Ready for /speckit.plan' and similar are underspecified.\n\nAdd concrete checklists:\n- Spec ready for planning: problem statement, goals, constraints, acceptance criteria, etc.\n- Plan ready for tasks: milestones, risks, dependencies, test strategy, etc.\n- Tasks ready for bd: each task has acceptance criteria, dependencies explicit, etc.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-15T00:23:24.877531852-08:00","updated_at":"2025-12-15T14:05:26.880419097-08:00","closed_at":"2025-12-15T14:05:26.880419097-08:00"}
|
{"id":"skills-bvz","title":"spec-review: Add Definition of Ready checklists for each phase","description":"'Ready for /speckit.plan' and similar are underspecified.\n\nAdd concrete checklists:\n- Spec ready for planning: problem statement, goals, constraints, acceptance criteria, etc.\n- Plan ready for tasks: milestones, risks, dependencies, test strategy, etc.\n- Tasks ready for bd: each task has acceptance criteria, dependencies explicit, etc.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-15T00:23:24.877531852-08:00","updated_at":"2025-12-15T14:05:26.880419097-08:00","closed_at":"2025-12-15T14:05:26.880419097-08:00"}
|
||||||
{"id":"skills-bww","title":"Benchmark AT-SPI overhead and coverage","description":"## Goal\nMeasure AT-SPI's runtime overhead and coverage across apps.\n\n## Prerequisites\n- Enable `services.gnome.at-spi2-core.enable = true` in NixOS\n- Set `QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1` for Qt apps\n- Rebuild and re-login\n\n## Overhead benchmarks\n1. **Startup time**: App launch with/without AT-SPI\n2. **Memory**: RSS delta with AT-SPI enabled\n3. **CPU**: Idle CPU with AT-SPI bus running\n4. **UI latency**: Input-to-paint latency (if measurable)\n\n## Coverage audit\nFor each app, document:\n- Does it expose accessibility tree?\n- How complete is the tree? (all elements vs partial)\n- Are coordinates accurate?\n- Are element types/roles correct?\n\n### Apps to test\n- [ ] Firefox\n- [ ] Ghostty terminal\n- [ ] Nautilus/file manager\n- [ ] VS Code / Electron app\n- [ ] A Qt app (if any installed)\n\n## Query benchmarks\n- Time to enumerate all elements in a window\n- Time to find element by role/name\n- Memory overhead of pyatspi queries\n\n## Depends on\n- skills-pdg (Enable AT-SPI for UI tree access)","status":"in_progress","priority":2,"issue_type":"task","created_at":"2025-12-17T14:13:21.599259773-08:00","updated_at":"2026-01-15T19:07:42.733336455-08:00","dependencies":[{"issue_id":"skills-bww","depends_on_id":"skills-pdg","type":"blocks","created_at":"2025-12-17T14:13:41.633210539-08:00","created_by":"daemon","metadata":"{}"}]}
|
{"id":"skills-bww","title":"Benchmark AT-SPI overhead and coverage","description":"## Goal\nMeasure AT-SPI's runtime overhead and coverage across apps.\n\n## Prerequisites\n- Enable `services.gnome.at-spi2-core.enable = true` in NixOS\n- Set `QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1` for Qt apps\n- Rebuild and re-login\n\n## Overhead benchmarks\n1. **Startup time**: App launch with/without AT-SPI\n2. **Memory**: RSS delta with AT-SPI enabled\n3. **CPU**: Idle CPU with AT-SPI bus running\n4. **UI latency**: Input-to-paint latency (if measurable)\n\n## Coverage audit\nFor each app, document:\n- Does it expose accessibility tree?\n- How complete is the tree? (all elements vs partial)\n- Are coordinates accurate?\n- Are element types/roles correct?\n\n### Apps to test\n- [ ] Firefox\n- [ ] Ghostty terminal\n- [ ] Nautilus/file manager\n- [ ] VS Code / Electron app\n- [ ] A Qt app (if any installed)\n\n## Query benchmarks\n- Time to enumerate all elements in a window\n- Time to find element by role/name\n- Memory overhead of pyatspi queries\n\n## Depends on\n- skills-pdg (Enable AT-SPI for UI tree access)","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-17T14:13:21.599259773-08:00","updated_at":"2026-01-15T19:13:34.088025944-08:00","closed_at":"2026-01-15T19:13:34.088025944-08:00","close_reason":"Benchmark complete. Results: 4.6s enumeration for 792 elements, 1.7s find-by-role avg. Coverage partial - Firefox needs explicit config, Ghostty works but registers as Unnamed. See skills/ui-query/docs/benchmark-results.md","dependencies":[{"issue_id":"skills-bww","depends_on_id":"skills-pdg","type":"blocks","created_at":"2025-12-17T14:13:41.633210539-08:00","created_by":"daemon","metadata":"{}"}]}
|
||||||
{"id":"skills-byq","title":"Integrate: review-gate with worker primitives","description":"Connect existing review-gate CLI with new worker system.\n\n## Current state\nreview-gate CLI exists with:\n- check/enable/approve/reject\n- Circuit breaker (3 strikes)\n- Stop hook integration (for Claude)\n\n## Integration needed\n- worker spawn enables review-gate automatically\n- worker status shows review state\n- worker approve/reject wraps review-gate\n- Evidence artifacts feed into review-gate\n\n## File coordination\n.worker-state/X.json includes:\n - review_session_id (links to .review-state/)\n - needs_review: true/false\n - review_status: pending/approved/rejected","notes":"MVP Tier 1: Wire review-gate to worker state machine","status":"closed","priority":1,"issue_type":"task","created_at":"2026-01-10T12:15:04.625083755-08:00","created_by":"dan","updated_at":"2026-01-10T23:24:21.172713875-08:00","closed_at":"2026-01-10T23:24:21.172713875-08:00","close_reason":"Integrated review-gate with worker: spawn enables review, status/show display review state, approve/reject update review-gate, cancel/merge clean up review state","dependencies":[{"issue_id":"skills-byq","depends_on_id":"skills-s6y","type":"blocks","created_at":"2026-01-10T12:15:10.376067847-08:00","created_by":"dan"}]}
|
{"id":"skills-byq","title":"Integrate: review-gate with worker primitives","description":"Connect existing review-gate CLI with new worker system.\n\n## Current state\nreview-gate CLI exists with:\n- check/enable/approve/reject\n- Circuit breaker (3 strikes)\n- Stop hook integration (for Claude)\n\n## Integration needed\n- worker spawn enables review-gate automatically\n- worker status shows review state\n- worker approve/reject wraps review-gate\n- Evidence artifacts feed into review-gate\n\n## File coordination\n.worker-state/X.json includes:\n - review_session_id (links to .review-state/)\n - needs_review: true/false\n - review_status: pending/approved/rejected","notes":"MVP Tier 1: Wire review-gate to worker state machine","status":"closed","priority":1,"issue_type":"task","created_at":"2026-01-10T12:15:04.625083755-08:00","created_by":"dan","updated_at":"2026-01-10T23:24:21.172713875-08:00","closed_at":"2026-01-10T23:24:21.172713875-08:00","close_reason":"Integrated review-gate with worker: spawn enables review, status/show display review state, approve/reject update review-gate, cancel/merge clean up review state","dependencies":[{"issue_id":"skills-byq","depends_on_id":"skills-s6y","type":"blocks","created_at":"2026-01-10T12:15:10.376067847-08:00","created_by":"dan"}]}
|
||||||
{"id":"skills-cc0","title":"spec-review: Add anti-hallucination constraints to prompts","description":"Models may paraphrase and present as quotes, or invent requirements/risks not in the doc.\n\nAdd:\n- 'Quotes must be verbatim'\n- 'Do not assume technologies/constraints not stated'\n- 'If missing info, list as open questions rather than speculating'","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-15T00:23:26.045478292-08:00","updated_at":"2025-12-15T14:07:19.556888057-08:00","closed_at":"2025-12-15T14:07:19.556888057-08:00"}
|
{"id":"skills-cc0","title":"spec-review: Add anti-hallucination constraints to prompts","description":"Models may paraphrase and present as quotes, or invent requirements/risks not in the doc.\n\nAdd:\n- 'Quotes must be verbatim'\n- 'Do not assume technologies/constraints not stated'\n- 'If missing info, list as open questions rather than speculating'","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-15T00:23:26.045478292-08:00","updated_at":"2025-12-15T14:07:19.556888057-08:00","closed_at":"2025-12-15T14:07:19.556888057-08:00"}
|
||||||
{"id":"skills-cg7c","title":"Design worker system prompt template","description":"Create the system prompt/context that spawned workers receive.\n\nContents:\n- Role definition (you are a worker agent)\n- Task context (from bd issue or description)\n- Available tools (worker start/done/heartbeat, bd comments)\n- Completion criteria\n- How to signal blockers/questions\n- How to hand off for review\n\nOutput: skills/hq/templates/worker-system.md","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-11T21:06:34.943983399-08:00","created_by":"dan","updated_at":"2026-01-12T10:41:56.919305275-08:00","closed_at":"2026-01-12T10:41:56.919305275-08:00","close_reason":"Completed - skills/hq/templates/worker-system.md created with role definition, available commands, communication protocol, and completion criteria"}
|
{"id":"skills-cg7c","title":"Design worker system prompt template","description":"Create the system prompt/context that spawned workers receive.\n\nContents:\n- Role definition (you are a worker agent)\n- Task context (from bd issue or description)\n- Available tools (worker start/done/heartbeat, bd comments)\n- Completion criteria\n- How to signal blockers/questions\n- How to hand off for review\n\nOutput: skills/hq/templates/worker-system.md","status":"closed","priority":2,"issue_type":"task","created_at":"2026-01-11T21:06:34.943983399-08:00","created_by":"dan","updated_at":"2026-01-12T10:41:56.919305275-08:00","closed_at":"2026-01-12T10:41:56.919305275-08:00","close_reason":"Completed - skills/hq/templates/worker-system.md created with role definition, available commands, communication protocol, and completion criteria"}
|
||||||
|
|
|
||||||
|
|
@ -28,11 +28,57 @@ scripts/ui-query list-windows --json # JSON output
|
||||||
scripts/ui-query list-windows -v # Include element counts (slower)
|
scripts/ui-query list-windows -v # Include element counts (slower)
|
||||||
```
|
```
|
||||||
|
|
||||||
Output includes:
|
### get-text
|
||||||
- Application name
|
|
||||||
- Window name and role
|
Extract text content from windows:
|
||||||
- Window geometry (position, size)
|
|
||||||
- States (active, focused, visible, etc.)
|
```bash
|
||||||
|
scripts/ui-query get-text 'Firefox' # Window containing 'Firefox'
|
||||||
|
scripts/ui-query get-text --app waybar # All windows from app
|
||||||
|
scripts/ui-query get-text --all # Everything (verbose)
|
||||||
|
scripts/ui-query get-text 'Editor' --json
|
||||||
|
```
|
||||||
|
|
||||||
|
### find-element
|
||||||
|
|
||||||
|
Find UI elements by role or name:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scripts/ui-query find-element --role button # All buttons
|
||||||
|
scripts/ui-query find-element --name 'Save' # Elements named 'Save'
|
||||||
|
scripts/ui-query find-element -r button -n OK # OK buttons
|
||||||
|
scripts/ui-query find-element -w Firefox -r link # Links in Firefox
|
||||||
|
scripts/ui-query find-element --role text --limit 5 # First 5 text elements
|
||||||
|
```
|
||||||
|
|
||||||
|
### query-state
|
||||||
|
|
||||||
|
Get detailed state for specific elements:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scripts/ui-query query-state --focused # Current focused element
|
||||||
|
scripts/ui-query query-state --role checkbox # All checkboxes with states
|
||||||
|
scripts/ui-query query-state --name 'Save' -a # 'Save' elements, all states
|
||||||
|
```
|
||||||
|
|
||||||
|
Shows: states, actions, values, text content, geometry, caret position.
|
||||||
|
|
||||||
|
### benchmark
|
||||||
|
|
||||||
|
Measure AT-SPI coverage and performance:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scripts/ui-query benchmark # Full audit
|
||||||
|
scripts/ui-query benchmark --app waybar # Single app
|
||||||
|
scripts/ui-query benchmark --json # JSON output
|
||||||
|
```
|
||||||
|
|
||||||
|
See `docs/benchmark-results.md` for baseline measurements.
|
||||||
|
|
||||||
|
## All Commands Support
|
||||||
|
|
||||||
|
- `--json` - JSON output
|
||||||
|
- `--debug` - Show AT-SPI errors (normally suppressed)
|
||||||
|
|
||||||
## Supported Applications
|
## Supported Applications
|
||||||
|
|
||||||
|
|
@ -40,8 +86,9 @@ Output includes:
|
||||||
|------|---------|
|
|------|---------|
|
||||||
| GTK apps | Good - automatic AT-SPI support |
|
| GTK apps | Good - automatic AT-SPI support |
|
||||||
| Qt apps | Good - with QT_LINUX_ACCESSIBILITY_ALWAYS_ON |
|
| Qt apps | Good - with QT_LINUX_ACCESSIBILITY_ALWAYS_ON |
|
||||||
|
| Firefox | Needs explicit enablement in about:config |
|
||||||
| Electron | Varies - often poor support |
|
| Electron | Varies - often poor support |
|
||||||
| Terminals | Basic - window frames only |
|
| Terminals | Basic - window frames, no buffer text |
|
||||||
|
|
||||||
## Debugging
|
## Debugging
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue