docs: initial Hermes Mobile planning scaffold
This commit is contained in:
Hermes Agent
@@ -0,0 +1,250 @@
|
||||
# Product Plan
|
||||
|
||||
## Working Name
|
||||
|
||||
Hermes Mobile
|
||||
|
||||
## One-line Pitch
|
||||
|
||||
A private Android-first app for talking to Hermes Agent with voice, files, live tool calls, approvals, cron controls, and completion notifications.
|
||||
|
||||
## Target User
|
||||
|
||||
Primary: Zeb / homelab power user who already runs Hermes Agent on a Linux machine and wants a better phone interface than Telegram.
|
||||
|
||||
Secondary later:
|
||||
- Developers running Hermes locally/remotely
|
||||
- Homelab admins
|
||||
- People who want an agent command center rather than a chatbot
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Not a generic OpenAI chat frontend.
|
||||
- Not a multi-provider model playground.
|
||||
- Not a public SaaS.
|
||||
- Not a replacement for Hermes Agent internals.
|
||||
- Not a Matrix/Discord/Telegram clone.
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. Agent-first, not chat-first
|
||||
- The UI should show what Hermes is doing, not just the final response.
|
||||
- Tool calls are first-class objects.
|
||||
|
||||
2. Phone-native ergonomics
|
||||
- Thumb-reachable controls.
|
||||
- Large tap targets.
|
||||
- Voice and upload actions always nearby.
|
||||
- Background task status survives app close/reopen.
|
||||
|
||||
3. Private by default
|
||||
- Self-hosted companion server.
|
||||
- Local uploads.
|
||||
- Optional LAN/Tailscale-only mode.
|
||||
- No third-party notification requirement unless chosen.
|
||||
|
||||
4. Modular integrations
|
||||
- Hermes CLI bridge now.
|
||||
- Hermes API bridge where useful.
|
||||
- Direct Python/event bridge later.
|
||||
- Notification providers swappable.
|
||||
|
||||
5. Install should feel boring
|
||||
- One command.
|
||||
- Clear service status.
|
||||
- Easy update/uninstall.
|
||||
|
||||
## Core Screens
|
||||
|
||||
### 1. Home / Ask
|
||||
|
||||
Purpose: send a prompt fast.
|
||||
|
||||
Features:
|
||||
- Text box with multiline support
|
||||
- Hold-to-record voice note
|
||||
- Attach button
|
||||
- Model/session selector compact chip
|
||||
- “New task” vs “continue session” toggle
|
||||
- Streaming answer
|
||||
- Activity timeline under/alongside answer
|
||||
- Completion toast/push
|
||||
|
||||
### 2. Activity
|
||||
|
||||
Purpose: see active and recent agent runs.
|
||||
|
||||
Features:
|
||||
- Running tasks
|
||||
- Tool-call cards
|
||||
- Logs/output previews
|
||||
- Generated files
|
||||
- Error states
|
||||
- Stop/cancel if supported
|
||||
|
||||
### 3. Sessions
|
||||
|
||||
Purpose: browse/resume history.
|
||||
|
||||
Features:
|
||||
- Recent conversations
|
||||
- Search
|
||||
- Rename/pin/archive
|
||||
- Session metadata: platform, started, last active, tool count, attachments
|
||||
|
||||
### 4. Cron
|
||||
|
||||
Purpose: manage scheduled Hermes tasks from phone.
|
||||
|
||||
Features:
|
||||
- Job list grouped by active/paused/completed
|
||||
- Run now
|
||||
- Pause/resume/remove
|
||||
- Edit prompt/schedule later
|
||||
- Last output viewer
|
||||
- Delivery target setting: in-app inbox, push, Matrix/Telegram/etc.
|
||||
|
||||
### 5. Files
|
||||
|
||||
Purpose: manage uploaded/generated artifacts.
|
||||
|
||||
Features:
|
||||
- Upload inbox
|
||||
- Generated media/files from responses
|
||||
- Download/share
|
||||
- Link file into a new prompt
|
||||
- Retention controls later
|
||||
|
||||
### 6. Approvals
|
||||
|
||||
Purpose: approve dangerous actions safely.
|
||||
|
||||
Features:
|
||||
- Pending approvals queue
|
||||
- Command/action preview
|
||||
- Risk label
|
||||
- Approve once / deny
|
||||
- Expiry countdown
|
||||
- Audit log
|
||||
|
||||
### 7. Settings
|
||||
|
||||
Purpose: connect to companion server and configure app.
|
||||
|
||||
Features:
|
||||
- Server URL
|
||||
- Pairing code / passkey login
|
||||
- Notification test
|
||||
- Theme
|
||||
- Hermes health
|
||||
- Storage/retention
|
||||
- STT/TTS preferences
|
||||
|
||||
## Feature Details
|
||||
|
||||
### Text Prompting
|
||||
|
||||
Modes:
|
||||
- Quick ask: starts new task/session.
|
||||
- Continue: appends to selected session.
|
||||
- Background task: app can be closed, push when finished.
|
||||
|
||||
### Voice Notes
|
||||
|
||||
MVP flow:
|
||||
1. Browser MediaRecorder captures audio.
|
||||
2. Upload audio file to companion.
|
||||
3. Companion either:
|
||||
- passes audio path to Hermes gateway/STT path, or
|
||||
- transcribes with configured local Whisper/faster-whisper, then sends text prompt.
|
||||
4. UI shows transcript for confirmation if desired.
|
||||
|
||||
Later:
|
||||
- Voice-to-voice replies using Hermes TTS.
|
||||
- Streaming partial transcript.
|
||||
|
||||
### File Uploads
|
||||
|
||||
Supported MVP:
|
||||
- png/jpg/webp/gif
|
||||
- zip/tar/gz
|
||||
- pdf
|
||||
- txt/log/md/json/csv
|
||||
- arbitrary binary as attachment
|
||||
|
||||
Prompt format to Hermes:
|
||||
- Store file locally in companion upload store.
|
||||
- Send prompt with absolute paths and metadata.
|
||||
- Example: `User uploaded files: /var/lib/hermes-mobile/uploads/.../archive.zip`
|
||||
|
||||
### Tool Calls
|
||||
|
||||
Event model:
|
||||
- `tool_call.started`
|
||||
- `tool_call.delta`
|
||||
- `tool_call.finished`
|
||||
- `tool_call.failed`
|
||||
|
||||
Tool cards show:
|
||||
- Icon/name
|
||||
- Arguments summary
|
||||
- Status
|
||||
- Duration
|
||||
- Expandable result
|
||||
- Copy/open file actions where relevant
|
||||
|
||||
### Notifications
|
||||
|
||||
MVP notification events:
|
||||
- task finished
|
||||
- task failed
|
||||
- approval required
|
||||
- cron job completed/failed
|
||||
|
||||
Providers:
|
||||
- Web Push for installed PWA
|
||||
- ntfy fallback
|
||||
- Gotify fallback
|
||||
- Home Assistant persistent notification optional
|
||||
|
||||
### Cron Triggers
|
||||
|
||||
The app should not reimplement Hermes cron. It should wrap Hermes cronjob functions/CLI:
|
||||
- list jobs
|
||||
- run job
|
||||
- pause/resume/remove
|
||||
- create/edit later
|
||||
|
||||
### “Prompting itself” / Hermes test harness
|
||||
|
||||
Because Hermes Mobile runs next to Hermes Agent, the companion can test connectivity by sending a harmless prompt through:
|
||||
- Hermes CLI: `hermes chat -q "ping"`
|
||||
- API server: `/v1/chat/completions`
|
||||
- Direct Python bridge later
|
||||
|
||||
The Settings health page should include a `Test Hermes` button.
|
||||
|
||||
## MVP Acceptance Criteria
|
||||
|
||||
- Install companion server on same Linux machine as Hermes.
|
||||
- Open mobile PWA on Android, pair to companion.
|
||||
- Send text prompt to Hermes and receive answer.
|
||||
- Upload a PNG or ZIP and include it in prompt context.
|
||||
- Record a voice note and turn it into a prompt.
|
||||
- See at least coarse tool activity while Hermes runs.
|
||||
- Receive push/ntfy notification when task completes.
|
||||
- View cron jobs and trigger one manually.
|
||||
|
||||
## Risks
|
||||
|
||||
1. Hermes API server may not expose enough event detail.
|
||||
- Mitigation: start with CLI/direct Python bridge and add event callbacks.
|
||||
|
||||
2. PWA push notification quirks.
|
||||
- Mitigation: support ntfy/Gotify fallback.
|
||||
|
||||
3. Mobile browser audio quirks.
|
||||
- Mitigation: test Android Chrome first; Capacitor later if needed.
|
||||
|
||||
4. Tool-call streaming requires Hermes integration changes.
|
||||
- Mitigation: companion adapter interface; use available callbacks in AIAgent where possible.
|
||||
Reference in New Issue
Block a user