hermes-mobile/docs/PRODUCT_PLAN.md

# Product Plan

## Working Name

Hermes Mobile

## One-line Pitch

A private Android-first app for talking to Hermes Agent with voice, files, live tool calls, approvals, cron controls, and completion notifications.

## Target User

Primary: Zeb / homelab power user who already runs Hermes Agent on a Linux machine and wants a better phone interface than Telegram.

Secondary later:
- Developers running Hermes locally/remotely
- Homelab admins
- People who want an agent command center rather than a chatbot

## Non-goals

- Not a generic OpenAI chat frontend.
- Not a multi-provider model playground.
- Not a public SaaS.
- Not a replacement for Hermes Agent internals.
- Not a Matrix/Discord/Telegram clone.

## Design Principles

1. Agent-first, not chat-first
   - The UI should show what Hermes is doing, not just the final response.
   - Tool calls are first-class objects.

2. Phone-native ergonomics
   - Native Android app first, PWA fallback second.
   - Thumb-reachable controls.
   - Large tap targets.
   - Voice and upload actions always nearby.
   - Background task status survives app close/reopen.
   - Haptics, native push, status bar/splash, and Android back behavior should be part of the product, not afterthoughts.

3. Private by default
   - Self-hosted companion server.
   - Local uploads.
   - Optional LAN/Tailscale-only mode.
   - No third-party notification requirement unless chosen.

4. Modular integrations
   - Hermes CLI bridge now.
   - Hermes API bridge where useful.
   - Direct Python/event bridge later.
   - Notification providers swappable.

5. Install should feel boring
   - One command.
   - Clear service status.
   - Easy update/uninstall.

## Core Screens

### 1. Home / Ask

Purpose: send a prompt fast.

Features:
- Text box with multiline support
- Hold-to-record voice note
- Attach button
- Model/session selector compact chip
- “New task” vs “continue session” toggle
- Streaming answer
- Activity timeline under/alongside answer
- Completion toast/push

### 2. Activity

Purpose: see active and recent agent runs.

Features:
- Running tasks
- Tool-call cards
- Logs/output previews
- Generated files
- Error states
- Stop/cancel if supported

### 3. Sessions

Purpose: browse/resume history.

Features:
- Recent conversations
- Search
- Rename/pin/archive
- Session metadata: platform, started, last active, tool count, attachments

### 4. Cron

Purpose: manage scheduled Hermes tasks from phone.

Features:
- Job list grouped by active/paused/completed
- Run now
- Pause/resume/remove
- Edit prompt/schedule later
- Last output viewer
- Delivery target setting: in-app inbox, push, Matrix/Telegram/etc.

### 5. Files

Purpose: manage uploaded/generated artifacts.

Features:
- Upload inbox
- Generated media/files from responses
- Download/share
- Link file into a new prompt
- Retention controls later

### 6. Approvals

Purpose: approve dangerous actions safely.

Features:
- Pending approvals queue
- Command/action preview
- Risk label
- Approve once / deny
- Expiry countdown
- Audit log

### 7. Settings

Purpose: connect to companion server and configure app.

Features:
- Server URL
- Pairing code / passkey login
- Notification test
- Theme
- Hermes health
- Storage/retention
- STT/TTS preferences

## Feature Details

### Text Prompting

Modes:
- Quick ask: starts new task/session.
- Continue: appends to selected session.
- Background task: app can be closed, push when finished.

### Voice Notes

MVP flow:
1. Browser MediaRecorder captures audio.
2. Upload audio file to companion.
3. Companion either:
   - passes audio path to Hermes gateway/STT path, or
   - transcribes with configured local Whisper/faster-whisper, then sends text prompt.
4. UI shows transcript for confirmation if desired.

Later:
- Voice-to-voice replies using Hermes TTS.
- Streaming partial transcript.

### File Uploads

Supported MVP:
- png/jpg/webp/gif
- zip/tar/gz
- pdf
- txt/log/md/json/csv
- arbitrary binary as attachment

Prompt format to Hermes:
- Store file locally in companion upload store.
- Send prompt with absolute paths and metadata.
- Example: `User uploaded files: /var/lib/hermes-mobile/uploads/.../archive.zip`

### Tool Calls

Event model:
- `tool_call.started`
- `tool_call.delta`
- `tool_call.finished`
- `tool_call.failed`

Tool cards show:
- Icon/name
- Arguments summary
- Status
- Duration
- Expandable result
- Copy/open file actions where relevant

### Notifications

MVP notification events:
- task finished
- task failed
- approval required
- cron job completed/failed

Providers:
- Web Push for installed PWA
- ntfy fallback
- Gotify fallback
- Home Assistant persistent notification optional

### Cron Triggers

The app should not reimplement Hermes cron. It should wrap Hermes cronjob functions/CLI:
- list jobs
- run job
- pause/resume/remove
- create/edit later

### “Prompting itself” / Hermes test harness

Because Hermes Mobile runs next to Hermes Agent, the companion can test connectivity by sending a harmless prompt through:
- Hermes CLI: `hermes chat -q "ping"`
- API server: `/v1/chat/completions`
- Direct Python bridge later

The Settings health page should include a `Test Hermes` button.

## MVP Acceptance Criteria

- Install companion server on same Linux machine as Hermes.
- Open mobile PWA on Android, pair to companion.
- Send text prompt to Hermes and receive answer.
- Upload a PNG or ZIP and include it in prompt context.
- Record a voice note and turn it into a prompt.
- See at least coarse tool activity while Hermes runs.
- Receive push/ntfy notification when task completes.
- View cron jobs and trigger one manually.

## Risks

1. Hermes API server may not expose enough event detail.
   - Mitigation: start with CLI/direct Python bridge and add event callbacks.

2. PWA push notification quirks.
   - Mitigation: support ntfy/Gotify fallback.

3. Mobile audio and lifecycle quirks.
   - Mitigation: use Capacitor from the start; begin with WebView MediaRecorder and move to a native audio plugin if needed.

4. Tool-call streaming requires Hermes integration changes.
   - Mitigation: companion adapter interface; use available callbacks in AIAgent where possible.