Hermes Agent
222 lines
4.4 KiB
Markdown
222 lines
4.4 KiB
Markdown
# Mobile App Plan
|
|
|
|
## App Type
|
|
|
|
Start as a PWA optimized for Android Chrome.
|
|
|
|
Why PWA first:
|
|
- Fast iteration.
|
|
- Easy self-hosting.
|
|
- No app store.
|
|
- Installable to home screen.
|
|
- Web Push support on Android is good.
|
|
- Same frontend can later be wrapped with Capacitor.
|
|
|
|
Potential later native shell:
|
|
- Capacitor Android for native share target, richer notifications, and better file/mic APIs.
|
|
|
|
## Frontend Stack
|
|
|
|
Recommended:
|
|
- Vite + React + TypeScript
|
|
- Tailwind CSS
|
|
- TanStack Query for server data
|
|
- Zustand for local UI/session state
|
|
- Framer Motion for motion where useful
|
|
- Workbox or Vite PWA plugin
|
|
- Zod for API schema validation
|
|
|
|
Alternative:
|
|
- Next.js if SSR/server components become useful, but for a companion-served app Vite is simpler.
|
|
|
|
## App Package Structure
|
|
|
|
```text
|
|
apps/mobile/
|
|
src/
|
|
app/
|
|
App.tsx
|
|
router.tsx
|
|
providers.tsx
|
|
screens/
|
|
AskScreen/
|
|
ActivityScreen/
|
|
SessionsScreen/
|
|
CronScreen/
|
|
FilesScreen/
|
|
SettingsScreen/
|
|
components/
|
|
agent-status-orb/
|
|
composer/
|
|
tool-card/
|
|
event-timeline/
|
|
upload-tray/
|
|
voice-recorder/
|
|
cron-job-card/
|
|
approval-card/
|
|
bottom-nav/
|
|
app-shell/
|
|
features/
|
|
tasks/
|
|
sessions/
|
|
cron/
|
|
files/
|
|
approvals/
|
|
notifications/
|
|
pairing/
|
|
settings/
|
|
lib/
|
|
api-client.ts
|
|
realtime.ts
|
|
push.ts
|
|
audio.ts
|
|
file-utils.ts
|
|
format.ts
|
|
styles/
|
|
globals.css
|
|
tokens.css
|
|
assets/
|
|
icons/
|
|
splash/
|
|
main.tsx
|
|
public/
|
|
manifest.webmanifest
|
|
service-worker.ts
|
|
```
|
|
|
|
## Navigation
|
|
|
|
Bottom tabs:
|
|
- Ask
|
|
- Activity
|
|
- Cron
|
|
- Files
|
|
- Settings
|
|
|
|
Sessions can be either:
|
|
- a sub-screen from Ask, or
|
|
- included in Activity initially.
|
|
|
|
## Ask Screen Behavior
|
|
|
|
States:
|
|
- idle
|
|
- composing
|
|
- uploading
|
|
- sending
|
|
- running
|
|
- waiting_approval
|
|
- finished
|
|
- failed
|
|
|
|
Composer:
|
|
- Text input
|
|
- Attach file button
|
|
- Hold/tap voice recording button
|
|
- Send button
|
|
- Optional mode chips: New / Continue / Background
|
|
|
|
When task starts:
|
|
- Input collapses to bottom.
|
|
- Task timeline appears.
|
|
- Assistant response streams.
|
|
- Tool cards appear in chronological order.
|
|
|
|
## Realtime
|
|
|
|
Use WebSocket first:
|
|
- `/api/realtime?token=...`
|
|
- Subscribe to task/session events.
|
|
|
|
Fallback:
|
|
- SSE `/api/tasks/:id/events/stream`
|
|
- Polling last resort.
|
|
|
|
Frontend event store should append events idempotently by event ID.
|
|
|
|
## Push Notifications
|
|
|
|
PWA flow:
|
|
1. User enables notifications in Settings.
|
|
2. App registers service worker.
|
|
3. App subscribes to PushManager.
|
|
4. Subscription sent to companion server.
|
|
5. Server sends Web Push on completion/approval/cron events.
|
|
|
|
Fallback if Web Push fails:
|
|
- Configure ntfy link in Settings.
|
|
|
|
## Voice Recording
|
|
|
|
Use MediaRecorder.
|
|
|
|
Preferred mime order:
|
|
- `audio/webm;codecs=opus`
|
|
- `audio/ogg;codecs=opus`
|
|
- browser default fallback
|
|
|
|
UI:
|
|
- Press/tap mic to start.
|
|
- Big recording sheet with timer/waveform.
|
|
- Cancel / send.
|
|
- Upload progress.
|
|
|
|
## File Upload UX
|
|
|
|
- Android file picker through `<input type=file multiple>`.
|
|
- Share Target API later for “Share to Hermes”.
|
|
- Preview thumbnails for images.
|
|
- File chips for archives/docs.
|
|
- Upload before task send or as part of task multipart.
|
|
|
|
## Offline/Background Behavior
|
|
|
|
- App shell should load offline.
|
|
- Draft prompt persists locally.
|
|
- Running task state reloads from server after reconnect.
|
|
- No attempt to run Hermes offline.
|
|
|
|
## Settings UX
|
|
|
|
First launch:
|
|
1. Enter companion URL or scan QR/pairing link.
|
|
2. Pair device.
|
|
3. Test connection.
|
|
4. Enable notifications.
|
|
5. Land on Ask.
|
|
|
|
Settings fields:
|
|
- Companion URL
|
|
- Device name
|
|
- Notification status/test
|
|
- Theme
|
|
- Default task mode
|
|
- Upload retention display
|
|
- Hermes health
|
|
|
|
## Android Polish Checklist
|
|
|
|
- `display: standalone` manifest.
|
|
- Proper app icon masks/adaptive icon assets.
|
|
- Theme color matching dark background.
|
|
- Avoid viewport resize bugs with keyboard.
|
|
- Safe-area padding.
|
|
- Disable browser pull-to-refresh where possible and add in-app refresh.
|
|
- Haptics via Vibration API where appropriate.
|
|
- Back button behavior for modals/sheets.
|
|
|
|
## Accessibility
|
|
|
|
- Visible focus states.
|
|
- Reduced motion mode.
|
|
- ARIA labels for icon buttons.
|
|
- Captions/transcript for voice notes.
|
|
|
|
## Future Capacitor Features
|
|
|
|
- Native push via FCM.
|
|
- Native share target.
|
|
- Background upload reliability.
|
|
- Local biometric unlock.
|
|
- Better notification actions: Approve/Deny from notification.
|