docs: make Android native app the primary target

This commit is contained in:
Hermes Agent
2026-07-09 03:50:27 +00:00
parent 2cd4730324
commit deec0c7fa3
6 changed files with 166 additions and 80 deletions
+13 -10
View File
@@ -1,6 +1,6 @@
# Hermes Mobile # Hermes Mobile
Native-feeling Android-first PWA and companion server for [Hermes Agent](https://github.com/NousResearch/hermes-agent). Native Android app, installable PWA fallback, and companion server for [Hermes Agent](https://github.com/NousResearch/hermes-agent).
Hermes Mobile is not a generic LLM chat UI. It is a self-hosted, Hermes-native control surface designed for phone use: prompt by text or voice, upload files, watch tool calls live, approve actions, manage cron jobs, and get notified when long-running agent work is done. Hermes Mobile is not a generic LLM chat UI. It is a self-hosted, Hermes-native control surface designed for phone use: prompt by text or voice, upload files, watch tool calls live, approve actions, manage cron jobs, and get notified when long-running agent work is done.
@@ -22,11 +22,11 @@ Planning scaffold only. No production code yet.
## Planned Components ## Planned Components
```text ```text
Android / mobile browser Android app (Capacitor native shell)
| |
| HTTPS + WebSocket/SSE + Web Push | HTTPS + WebSocket/SSE + native push
v v
Hermes Mobile App (PWA / later Capacitor) Hermes Mobile App (native Android first, PWA fallback)
| |
v v
Hermes Companion Server Hermes Companion Server
@@ -41,9 +41,11 @@ Hermes Companion Server
## MVP Feature Set ## MVP Feature Set
1. Mobile-first app shell 1. Native Android-first app shell
- Installable PWA - Capacitor Android wrapper around the shared web UI
- Android-style bottom navigation - APK/release build installable directly on Android
- PWA/web build retained as fallback and fast dev target
- Native-style bottom navigation, status bar, splash screen, haptics, push, share target later
- Offline-friendly shell - Offline-friendly shell
- Dark, playful visual system inspired by happy.engineering / Happy app - Dark, playful visual system inspired by happy.engineering / Happy app
@@ -103,11 +105,12 @@ Hermes Companion Server
Frontend: Frontend:
- TypeScript - TypeScript
- React + Vite or Next.js static/PWA mode - React + Vite
- Tailwind CSS - Tailwind CSS
- Zustand or TanStack Query for client state - Zustand or TanStack Query for client state
- Service Worker + Web Push - Capacitor Android from the start for genuine phone-app feel
- Capacitor later, only if PWA limitations matter - PWA/service-worker build kept as fallback and for desktop/LAN access
- Native plugins: Push Notifications, Haptics, Filesystem/Share, Status Bar, Splash Screen
Companion server: Companion server:
- TypeScript Node.js initially, because file uploads, WebSocket/SSE, install scripts, and PM2/systemd are straightforward - TypeScript Node.js initially, because file uploads, WebSocket/SSE, install scripts, and PM2/systemd are straightforward
+11 -8
View File
@@ -5,7 +5,8 @@
Hermes Mobile is split into two deployable parts: Hermes Mobile is split into two deployable parts:
1. Mobile app 1. Mobile app
- Installable PWA designed for Android. - Native Android app built with Capacitor around a shared React UI.
- PWA/web build remains available as fallback and dev target.
- Talks only to the companion server. - Talks only to the companion server.
- Does not need direct access to Hermes files or credentials. - Does not need direct access to Hermes files or credentials.
@@ -16,7 +17,7 @@ Hermes Mobile is split into two deployable parts:
```text ```text
+---------------------------+ +---------------------------+
| Android PWA | | Android App |
| - Chat | | - Chat |
| - Tool timeline | | - Tool timeline |
| - Cron UI | | - Cron UI |
@@ -24,7 +25,7 @@ Hermes Mobile is split into two deployable parts:
| - Approvals | | - Approvals |
+-------------+-------------+ +-------------+-------------+
| |
| HTTPS REST + WebSocket/SSE + Web Push | HTTPS REST + WebSocket/SSE + native push/Web Push fallback
v v
+---------------------------+ +---------------------------+
| Hermes Companion Server | | Hermes Companion Server |
@@ -303,9 +304,11 @@ Providers:
- systemd service. - systemd service.
- Better production posture. - Better production posture.
## Future Native Wrapper ## Native Android Shell
If Android PWA is insufficient: Capacitor Android is part of the primary architecture, not a future contingency.
- Add Capacitor app under `apps/android-shell`.
- Reuse same web app. - The shared React UI builds into `apps/mobile/dist`.
- Gain native share target, richer push, microphone/file APIs. - Capacitor packages that build into an Android APK.
- PWA/web access remains useful for dev, desktop, and fallback.
- Native plugins provide share target, richer push, haptics, status bar, splash screen, microphone/file improvements, and Android lifecycle hooks.
+6 -6
View File
@@ -198,13 +198,13 @@ Prefer CSS transitions and Framer Motion if bundle size acceptable.
## Android-native Feeling ## Android-native Feeling
- Installable PWA manifest with proper icons. - Capacitor Android APK with adaptive icon and native splash.
- Standalone display mode. - PWA manifest retained for fallback web install.
- No browser chrome when installed. - No browser chrome in the primary Android app.
- Bottom nav like native Material apps. - Bottom nav like native Material apps.
- Pull-to-refresh disabled; in-app refresh buttons where needed. - Pull-to-refresh disabled in fallback web mode; in-app refresh buttons where needed.
- Back gesture should close panels before navigating away. - Android back gesture/button should close panels before navigating away.
- Use Android share target later so user can share files/text to Hermes. - Add Android share target so user can share files/text to Hermes.
## Accessibility ## Accessibility
+125 -49
View File
@@ -1,43 +1,96 @@
# Mobile App Plan # Mobile App Plan
## App Type ## App Type Decision
Start as a PWA optimized for Android Chrome. Build Hermes Mobile as a native Android app from the start, using Capacitor around a shared React UI.
Why PWA first: PWA will still exist as a fallback/dev target, but it should not be the primary product experience.
- Fast iteration.
- Easy self-hosting.
- No app store.
- Installable to home screen.
- Web Push support on Android is good.
- Same frontend can later be wrapped with Capacitor.
Potential later native shell: ## Why Native Android First
- Capacitor Android for native share target, richer notifications, and better file/mic APIs.
A PWA can get surprisingly far on Android: installable icon, offline shell, camera/mic, file picker, Web Push, and a standalone window. But for this project, “suffices” is not the bar. The goal is that Hermes feels like a genuine phone app designed around agent work.
Native Android via Capacitor gives us:
- Real APK install path.
- Native splash screen and status bar control.
- More reliable push notifications.
- Better haptics.
- Better file/share integration.
- Easier future “Share to Hermes” target from Android apps.
- Better control over permissions and app lifecycle.
- Native-feeling back button behavior.
- More credible “this is an app” feel than browser-installed PWA.
## Role of PWA
Keep PWA support for:
- Fast development loop.
- Desktop/LAN access.
- Emergency fallback if APK is not installed.
- Users who do not want to sideload.
But feature priority should be:
1. Android app / Capacitor
2. PWA fallback
3. Desktop web convenience
## Frontend Stack ## Frontend Stack
Recommended: Recommended:
- Vite + React + TypeScript - Vite + React + TypeScript
- Tailwind CSS - Tailwind CSS
- Capacitor Android
- TanStack Query for server data - TanStack Query for server data
- Zustand for local UI/session state - Zustand for local UI/session state
- Framer Motion for motion where useful - Framer Motion for app-like motion where useful
- Workbox or Vite PWA plugin
- Zod for API schema validation - Zod for API schema validation
- Vite PWA plugin for fallback web install
Alternative: Capacitor plugins:
- Next.js if SSR/server components become useful, but for a companion-served app Vite is simpler. - `@capacitor/android`
- `@capacitor/app` for lifecycle/back button
- `@capacitor/haptics`
- `@capacitor/status-bar`
- `@capacitor/splash-screen`
- `@capacitor/push-notifications`
- `@capacitor/filesystem`
- `@capacitor/share`
- `@capacitor/preferences`
Potential later:
- Biometric auth plugin
- Camera plugin
- Background task/upload plugin if needed
## Build Targets
```text
apps/mobile/ shared React UI
apps/mobile/android/ Capacitor Android project
apps/mobile/dist/ web build consumed by Capacitor and PWA
```
Commands later:
```bash
pnpm mobile:dev # browser dev server
pnpm mobile:build # web build
pnpm android:sync # capacitor sync
pnpm android:apk # debug/release APK
```
## App Package Structure ## App Package Structure
```text ```text
apps/mobile/ apps/mobile/
capacitor.config.ts
android/ # generated Capacitor Android project
src/ src/
app/ app/
App.tsx App.tsx
router.tsx router.tsx
providers.tsx providers.tsx
native.ts # Capacitor runtime helpers
screens/ screens/
AskScreen/ AskScreen/
ActivityScreen/ ActivityScreen/
@@ -65,10 +118,16 @@ apps/mobile/
notifications/ notifications/
pairing/ pairing/
settings/ settings/
native/
haptics.ts
push.ts
share.ts
statusBar.ts
filesystem.ts
backButton.ts
lib/ lib/
api-client.ts api-client.ts
realtime.ts realtime.ts
push.ts
audio.ts audio.ts
file-utils.ts file-utils.ts
format.ts format.ts
@@ -93,9 +152,10 @@ Bottom tabs:
- Files - Files
- Settings - Settings
Sessions can be either: Native back behavior:
- a sub-screen from Ask, or - If a sheet/modal is open, close it.
- included in Activity initially. - Else if not on Ask, go back to previous tab/screen.
- Else prompt/minimize/exit according to Android convention.
## Ask Screen Behavior ## Ask Screen Behavior
@@ -121,6 +181,7 @@ When task starts:
- Task timeline appears. - Task timeline appears.
- Assistant response streams. - Assistant response streams.
- Tool cards appear in chronological order. - Tool cards appear in chronological order.
- Subtle haptic on send, completion, error, and approval required.
## Realtime ## Realtime
@@ -136,35 +197,43 @@ Frontend event store should append events idempotently by event ID.
## Push Notifications ## Push Notifications
PWA flow: Native Android priority:
1. User enables notifications in Settings. - Use Capacitor Push Notifications / FCM where practical.
2. App registers service worker. - Companion server stores device push token.
3. App subscribes to PushManager. - Notifications for task completion, failure, approval required, cron completion.
4. Subscription sent to companion server.
5. Server sends Web Push on completion/approval/cron events.
Fallback if Web Push fails: Fallbacks:
- Configure ntfy link in Settings. - Web Push for PWA.
- ntfy/Gotify if native push is too annoying for self-hosted early builds.
Important: notification provider must be abstracted so the app can start with ntfy/Web Push and later use FCM/native without rewriting task logic.
## Voice Recording ## Voice Recording
Use MediaRecorder. Prefer native-friendly implementation:
- First version can use browser MediaRecorder inside Capacitor WebView.
- If recording quality/lifecycle is poor, move to native audio recording plugin.
Preferred mime order: Preferred mime order for web implementation:
- `audio/webm;codecs=opus` - `audio/webm;codecs=opus`
- `audio/ogg;codecs=opus` - `audio/ogg;codecs=opus`
- browser default fallback - browser default fallback
UI: UI:
- Press/tap mic to start. - Press/tap mic to start.
- Big recording sheet with timer/waveform. - Big native-feeling recording sheet with timer/waveform.
- Haptic on start/stop.
- Cancel / send. - Cancel / send.
- Upload progress. - Upload progress.
## File Upload UX ## File Upload UX
- Android file picker through `<input type=file multiple>`. Priority:
- Share Target API later for “Share to Hermes”. - Use standard file input first inside WebView.
- Add Capacitor Filesystem/Share integration for better Android feel.
- Later add Android share target so user can share files/text/images into Hermes Mobile from other apps.
UI:
- Preview thumbnails for images. - Preview thumbnails for images.
- File chips for archives/docs. - File chips for archives/docs.
- Upload before task send or as part of task multipart. - Upload before task send or as part of task multipart.
@@ -172,14 +241,15 @@ UI:
## Offline/Background Behavior ## Offline/Background Behavior
- App shell should load offline. - App shell should load offline.
- Draft prompt persists locally. - Draft prompt persists locally via Capacitor Preferences/local storage.
- Running task state reloads from server after reconnect. - Running task state reloads from server after reconnect.
- Notifications tell user when background tasks finish.
- No attempt to run Hermes offline. - No attempt to run Hermes offline.
## Settings UX ## Settings UX
First launch: First launch:
1. Enter companion URL or scan QR/pairing link. 1. Enter companion URL or open pairing deep link/QR.
2. Pair device. 2. Pair device.
3. Test connection. 3. Test connection.
4. Enable notifications. 4. Enable notifications.
@@ -193,17 +263,22 @@ Settings fields:
- Default task mode - Default task mode
- Upload retention display - Upload retention display
- Hermes health - Hermes health
- App version/build channel
## Android Polish Checklist ## Native Polish Checklist
- `display: standalone` manifest. - Proper Android package ID, e.g. `cloud.molberg.hermesmobile`.
- Proper app icon masks/adaptive icon assets. - Adaptive icon.
- Theme color matching dark background. - Native splash screen.
- Avoid viewport resize bugs with keyboard. - Status/navigation bar colors match theme.
- Safe-area padding. - Edge-to-edge layout with safe-area handling.
- Disable browser pull-to-refresh where possible and add in-app refresh. - Keyboard does not break composer layout.
- Haptics via Vibration API where appropriate. - Native back button behavior.
- Back button behavior for modals/sheets. - Haptics on important interactions.
- Notification permission onboarding.
- Share target later.
- Deep links for pairing.
- Avoid browser pull-to-refresh in PWA fallback; app should have explicit refresh controls.
## Accessibility ## Accessibility
@@ -211,11 +286,12 @@ Settings fields:
- Reduced motion mode. - Reduced motion mode.
- ARIA labels for icon buttons. - ARIA labels for icon buttons.
- Captions/transcript for voice notes. - Captions/transcript for voice notes.
- Minimum 44px tap targets.
## Future Capacitor Features ## PWA Sufficiency Summary
- Native push via FCM. PWA is enough for a functional prototype.
- Native share target.
- Background upload reliability. Native Android is better for the product Zeb described.
- Local biometric unlock.
- Better notification actions: Approve/Deny from notification. Decision: build shared React app + Capacitor Android from day one, while keeping PWA fallback essentially free.
+4 -2
View File
@@ -32,10 +32,12 @@ Secondary later:
- Tool calls are first-class objects. - Tool calls are first-class objects.
2. Phone-native ergonomics 2. Phone-native ergonomics
- Native Android app first, PWA fallback second.
- Thumb-reachable controls. - Thumb-reachable controls.
- Large tap targets. - Large tap targets.
- Voice and upload actions always nearby. - Voice and upload actions always nearby.
- Background task status survives app close/reopen. - Background task status survives app close/reopen.
- Haptics, native push, status bar/splash, and Android back behavior should be part of the product, not afterthoughts.
3. Private by default 3. Private by default
- Self-hosted companion server. - Self-hosted companion server.
@@ -243,8 +245,8 @@ The Settings health page should include a `Test Hermes` button.
2. PWA push notification quirks. 2. PWA push notification quirks.
- Mitigation: support ntfy/Gotify fallback. - Mitigation: support ntfy/Gotify fallback.
3. Mobile browser audio quirks. 3. Mobile audio and lifecycle quirks.
- Mitigation: test Android Chrome first; Capacitor later if needed. - Mitigation: use Capacitor from the start; begin with WebView MediaRecorder and move to a native audio plugin if needed.
4. Tool-call streaming requires Hermes integration changes. 4. Tool-call streaming requires Hermes integration changes.
- Mitigation: companion adapter interface; use available callbacks in AIAgent where possible. - Mitigation: companion adapter interface; use available callbacks in AIAgent where possible.
+7 -5
View File
@@ -18,7 +18,7 @@ Goal: empty but runnable monorepo.
Deliverables: Deliverables:
- pnpm workspace - pnpm workspace
- `apps/mobile` Vite React PWA - `apps/mobile` Vite React app with Capacitor Android target and PWA fallback
- `apps/companion` Fastify server - `apps/companion` Fastify server
- shared TypeScript package for event/API schemas - shared TypeScript package for event/API schemas
- basic app shell with bottom nav - basic app shell with bottom nav
@@ -66,10 +66,10 @@ Deliverables:
Goal: close phone and get told when done. Goal: close phone and get told when done.
Deliverables: Deliverables:
- PWA service worker - Native Android push path or ntfy/Web Push fallback
- Web Push subscription management - PWA service worker fallback
- notification provider abstraction - notification provider abstraction
- ntfy fallback - notification permission onboarding
- completion/failure/approval notifications - completion/failure/approval notifications
## Phase 6 — Cron + Approvals ## Phase 6 — Cron + Approvals
@@ -102,11 +102,13 @@ Goal: feels native.
Deliverables: Deliverables:
- refined Happy-inspired UI - refined Happy-inspired UI
- animations/microinteractions - animations/microinteractions
- Capacitor Android build pipeline
- adaptive icons/splash - adaptive icons/splash
- native status/navigation bars
- share target - share target
- haptics - haptics
- offline shell polish - offline shell polish
- optional Capacitor wrapper evaluation - APK release/signing flow
## Phase 9 — Deep Hermes Integration ## Phase 9 — Deep Hermes Integration