docs: initial Hermes Mobile planning scaffold

2026-07-09 03:45:48 +00:00
commit 2cd4730324
14 changed files with 2139 additions and 0 deletions
@@ -0,0 +1,119 @@
+# Testing Strategy
+
+## Philosophy
+
+Hermes Mobile must be tested as an app plus a local service plus a Hermes integration. Generic unit tests are not enough; the important thing is whether a phone can send work to Hermes, see activity, close the app, and get notified when done.
+
+## Test Layers
+
+### 1. Unit Tests
+
+Frontend:
+- event reducer/idempotency
+- upload tray state
+- voice recorder state machine
+- API client error handling
+- cron job card states
+
+Companion:
+- config loading
+- auth pairing tokens
+- notification provider interface
+- upload path normalization
+- event bus fanout
+- task state transitions
+
+### 2. Integration Tests
+
+Companion server with fake Hermes adapter:
+- create task
+- stream events
+- upload file + attach to task
+- completion notification queued
+- cron list/run mocks
+
+Hermes adapter contract tests:
+- CLI adapter health
+- API adapter health
+- Python worker JSONL protocol
+
+### 3. End-to-End Tests
+
+Use Playwright against local dev server.
+
+Scenarios:
+- first-time pairing
+- send text prompt
+- upload PNG and send prompt
+- record/upload fake audio blob
+- watch tool timeline update
+- cron run button
+- notification permission flow mocked
+
+### 4. Real Hermes Smoke Tests
+
+On a machine with Hermes installed:
+
+```bash
+hermes chat -q "Reply with exactly: hermes-mobile-ok"
+```
+
+Companion health test should do the equivalent and verify response.
+
+More advanced smoke:
+- Ask Hermes to run a harmless terminal command: `pwd`.
+- Verify a tool-call event appears.
+- Upload a small text file and ask Hermes to summarize it.
+
+## Manual Android QA
+
+Test on Android Chrome installed PWA:
+- Install app to home screen.
+- Launch standalone mode.
+- Keyboard does not break composer layout.
+- Back button closes sheets/modals first.
+- Voice recording works.
+- File picker works for image/zip/pdf.
+- Push notification arrives after app is backgrounded.
+- App reconnects to running task after being killed/reopened.
+
+## Compatibility Targets
+
+- Android Chrome latest
+- Android Firefox optional
+- Desktop Chrome for dev
+- iOS Safari later, not MVP priority
+
+## Observability for Testing
+
+Companion should expose:
+
+```http
+GET /api/health
+GET /api/debug/events/recent
+GET /api/debug/config/redacted
+POST /api/hermes/test
+POST /api/notifications/test
+```
+
+Only expose debug endpoints in dev mode or authenticated admin mode.
+
+## CI Plan
+
+Later:
+- pnpm lint
+- pnpm typecheck
+- pnpm test
+- pnpm e2e with fake adapter
+- build release artifact
+
+## Dogfooding Plan
+
+The app should be used to prompt Hermes about its own repo.
+
+Example dogfood tasks:
+- “Run the test suite for Hermes Mobile and summarize failures.”
+- “Inspect the companion logs and fix the upload bug.”
+- “Create a cron job that reminds me if companion is down.”
+
+This is important because Hermes is both the target agent and the tool used to build the app.