docs: make Android native app the primary target

This commit is contained in:
Hermes Agent
2026-07-09 03:50:27 +00:00
parent 2cd4730324
commit deec0c7fa3
6 changed files with 166 additions and 80 deletions
+13 -10
View File
@@ -1,6 +1,6 @@
# Hermes Mobile
Native-feeling Android-first PWA and companion server for [Hermes Agent](https://github.com/NousResearch/hermes-agent).
Native Android app, installable PWA fallback, and companion server for [Hermes Agent](https://github.com/NousResearch/hermes-agent).
Hermes Mobile is not a generic LLM chat UI. It is a self-hosted, Hermes-native control surface designed for phone use: prompt by text or voice, upload files, watch tool calls live, approve actions, manage cron jobs, and get notified when long-running agent work is done.
@@ -22,11 +22,11 @@ Planning scaffold only. No production code yet.
## Planned Components
```text
Android / mobile browser
Android app (Capacitor native shell)
|
| HTTPS + WebSocket/SSE + Web Push
| HTTPS + WebSocket/SSE + native push
v
Hermes Mobile App (PWA / later Capacitor)
Hermes Mobile App (native Android first, PWA fallback)
|
v
Hermes Companion Server
@@ -41,9 +41,11 @@ Hermes Companion Server
## MVP Feature Set
1. Mobile-first app shell
- Installable PWA
- Android-style bottom navigation
1. Native Android-first app shell
- Capacitor Android wrapper around the shared web UI
- APK/release build installable directly on Android
- PWA/web build retained as fallback and fast dev target
- Native-style bottom navigation, status bar, splash screen, haptics, push, share target later
- Offline-friendly shell
- Dark, playful visual system inspired by happy.engineering / Happy app
@@ -103,11 +105,12 @@ Hermes Companion Server
Frontend:
- TypeScript
- React + Vite or Next.js static/PWA mode
- React + Vite
- Tailwind CSS
- Zustand or TanStack Query for client state
- Service Worker + Web Push
- Capacitor later, only if PWA limitations matter
- Capacitor Android from the start for genuine phone-app feel
- PWA/service-worker build kept as fallback and for desktop/LAN access
- Native plugins: Push Notifications, Haptics, Filesystem/Share, Status Bar, Splash Screen
Companion server:
- TypeScript Node.js initially, because file uploads, WebSocket/SSE, install scripts, and PM2/systemd are straightforward
+11 -8
View File
@@ -5,7 +5,8 @@
Hermes Mobile is split into two deployable parts:
1. Mobile app
- Installable PWA designed for Android.
- Native Android app built with Capacitor around a shared React UI.
- PWA/web build remains available as fallback and dev target.
- Talks only to the companion server.
- Does not need direct access to Hermes files or credentials.
@@ -16,7 +17,7 @@ Hermes Mobile is split into two deployable parts:
```text
+---------------------------+
| Android PWA |
| Android App |
| - Chat |
| - Tool timeline |
| - Cron UI |
@@ -24,7 +25,7 @@ Hermes Mobile is split into two deployable parts:
| - Approvals |
+-------------+-------------+
|
| HTTPS REST + WebSocket/SSE + Web Push
| HTTPS REST + WebSocket/SSE + native push/Web Push fallback
v
+---------------------------+
| Hermes Companion Server |
@@ -303,9 +304,11 @@ Providers:
- systemd service.
- Better production posture.
## Future Native Wrapper
## Native Android Shell
If Android PWA is insufficient:
- Add Capacitor app under `apps/android-shell`.
- Reuse same web app.
- Gain native share target, richer push, microphone/file APIs.
Capacitor Android is part of the primary architecture, not a future contingency.
- The shared React UI builds into `apps/mobile/dist`.
- Capacitor packages that build into an Android APK.
- PWA/web access remains useful for dev, desktop, and fallback.
- Native plugins provide share target, richer push, haptics, status bar, splash screen, microphone/file improvements, and Android lifecycle hooks.
+6 -6
View File
@@ -198,13 +198,13 @@ Prefer CSS transitions and Framer Motion if bundle size acceptable.
## Android-native Feeling
- Installable PWA manifest with proper icons.
- Standalone display mode.
- No browser chrome when installed.
- Capacitor Android APK with adaptive icon and native splash.
- PWA manifest retained for fallback web install.
- No browser chrome in the primary Android app.
- Bottom nav like native Material apps.
- Pull-to-refresh disabled; in-app refresh buttons where needed.
- Back gesture should close panels before navigating away.
- Use Android share target later so user can share files/text to Hermes.
- Pull-to-refresh disabled in fallback web mode; in-app refresh buttons where needed.
- Android back gesture/button should close panels before navigating away.
- Add Android share target so user can share files/text to Hermes.
## Accessibility
+125 -49
View File
@@ -1,43 +1,96 @@
# Mobile App Plan
## App Type
## App Type Decision
Start as a PWA optimized for Android Chrome.
Build Hermes Mobile as a native Android app from the start, using Capacitor around a shared React UI.
Why PWA first:
- Fast iteration.
- Easy self-hosting.
- No app store.
- Installable to home screen.
- Web Push support on Android is good.
- Same frontend can later be wrapped with Capacitor.
PWA will still exist as a fallback/dev target, but it should not be the primary product experience.
Potential later native shell:
- Capacitor Android for native share target, richer notifications, and better file/mic APIs.
## Why Native Android First
A PWA can get surprisingly far on Android: installable icon, offline shell, camera/mic, file picker, Web Push, and a standalone window. But for this project, “suffices” is not the bar. The goal is that Hermes feels like a genuine phone app designed around agent work.
Native Android via Capacitor gives us:
- Real APK install path.
- Native splash screen and status bar control.
- More reliable push notifications.
- Better haptics.
- Better file/share integration.
- Easier future “Share to Hermes” target from Android apps.
- Better control over permissions and app lifecycle.
- Native-feeling back button behavior.
- More credible “this is an app” feel than browser-installed PWA.
## Role of PWA
Keep PWA support for:
- Fast development loop.
- Desktop/LAN access.
- Emergency fallback if APK is not installed.
- Users who do not want to sideload.
But feature priority should be:
1. Android app / Capacitor
2. PWA fallback
3. Desktop web convenience
## Frontend Stack
Recommended:
- Vite + React + TypeScript
- Tailwind CSS
- Capacitor Android
- TanStack Query for server data
- Zustand for local UI/session state
- Framer Motion for motion where useful
- Workbox or Vite PWA plugin
- Framer Motion for app-like motion where useful
- Zod for API schema validation
- Vite PWA plugin for fallback web install
Alternative:
- Next.js if SSR/server components become useful, but for a companion-served app Vite is simpler.
Capacitor plugins:
- `@capacitor/android`
- `@capacitor/app` for lifecycle/back button
- `@capacitor/haptics`
- `@capacitor/status-bar`
- `@capacitor/splash-screen`
- `@capacitor/push-notifications`
- `@capacitor/filesystem`
- `@capacitor/share`
- `@capacitor/preferences`
Potential later:
- Biometric auth plugin
- Camera plugin
- Background task/upload plugin if needed
## Build Targets
```text
apps/mobile/ shared React UI
apps/mobile/android/ Capacitor Android project
apps/mobile/dist/ web build consumed by Capacitor and PWA
```
Commands later:
```bash
pnpm mobile:dev # browser dev server
pnpm mobile:build # web build
pnpm android:sync # capacitor sync
pnpm android:apk # debug/release APK
```
## App Package Structure
```text
apps/mobile/
capacitor.config.ts
android/ # generated Capacitor Android project
src/
app/
App.tsx
router.tsx
providers.tsx
native.ts # Capacitor runtime helpers
screens/
AskScreen/
ActivityScreen/
@@ -65,10 +118,16 @@ apps/mobile/
notifications/
pairing/
settings/
native/
haptics.ts
push.ts
share.ts
statusBar.ts
filesystem.ts
backButton.ts
lib/
api-client.ts
realtime.ts
push.ts
audio.ts
file-utils.ts
format.ts
@@ -93,9 +152,10 @@ Bottom tabs:
- Files
- Settings
Sessions can be either:
- a sub-screen from Ask, or
- included in Activity initially.
Native back behavior:
- If a sheet/modal is open, close it.
- Else if not on Ask, go back to previous tab/screen.
- Else prompt/minimize/exit according to Android convention.
## Ask Screen Behavior
@@ -121,6 +181,7 @@ When task starts:
- Task timeline appears.
- Assistant response streams.
- Tool cards appear in chronological order.
- Subtle haptic on send, completion, error, and approval required.
## Realtime
@@ -136,35 +197,43 @@ Frontend event store should append events idempotently by event ID.
## Push Notifications
PWA flow:
1. User enables notifications in Settings.
2. App registers service worker.
3. App subscribes to PushManager.
4. Subscription sent to companion server.
5. Server sends Web Push on completion/approval/cron events.
Native Android priority:
- Use Capacitor Push Notifications / FCM where practical.
- Companion server stores device push token.
- Notifications for task completion, failure, approval required, cron completion.
Fallback if Web Push fails:
- Configure ntfy link in Settings.
Fallbacks:
- Web Push for PWA.
- ntfy/Gotify if native push is too annoying for self-hosted early builds.
Important: notification provider must be abstracted so the app can start with ntfy/Web Push and later use FCM/native without rewriting task logic.
## Voice Recording
Use MediaRecorder.
Prefer native-friendly implementation:
- First version can use browser MediaRecorder inside Capacitor WebView.
- If recording quality/lifecycle is poor, move to native audio recording plugin.
Preferred mime order:
Preferred mime order for web implementation:
- `audio/webm;codecs=opus`
- `audio/ogg;codecs=opus`
- browser default fallback
UI:
- Press/tap mic to start.
- Big recording sheet with timer/waveform.
- Big native-feeling recording sheet with timer/waveform.
- Haptic on start/stop.
- Cancel / send.
- Upload progress.
## File Upload UX
- Android file picker through `<input type=file multiple>`.
- Share Target API later for “Share to Hermes”.
Priority:
- Use standard file input first inside WebView.
- Add Capacitor Filesystem/Share integration for better Android feel.
- Later add Android share target so user can share files/text/images into Hermes Mobile from other apps.
UI:
- Preview thumbnails for images.
- File chips for archives/docs.
- Upload before task send or as part of task multipart.
@@ -172,14 +241,15 @@ UI:
## Offline/Background Behavior
- App shell should load offline.
- Draft prompt persists locally.
- Draft prompt persists locally via Capacitor Preferences/local storage.
- Running task state reloads from server after reconnect.
- Notifications tell user when background tasks finish.
- No attempt to run Hermes offline.
## Settings UX
First launch:
1. Enter companion URL or scan QR/pairing link.
1. Enter companion URL or open pairing deep link/QR.
2. Pair device.
3. Test connection.
4. Enable notifications.
@@ -193,17 +263,22 @@ Settings fields:
- Default task mode
- Upload retention display
- Hermes health
- App version/build channel
## Android Polish Checklist
## Native Polish Checklist
- `display: standalone` manifest.
- Proper app icon masks/adaptive icon assets.
- Theme color matching dark background.
- Avoid viewport resize bugs with keyboard.
- Safe-area padding.
- Disable browser pull-to-refresh where possible and add in-app refresh.
- Haptics via Vibration API where appropriate.
- Back button behavior for modals/sheets.
- Proper Android package ID, e.g. `cloud.molberg.hermesmobile`.
- Adaptive icon.
- Native splash screen.
- Status/navigation bar colors match theme.
- Edge-to-edge layout with safe-area handling.
- Keyboard does not break composer layout.
- Native back button behavior.
- Haptics on important interactions.
- Notification permission onboarding.
- Share target later.
- Deep links for pairing.
- Avoid browser pull-to-refresh in PWA fallback; app should have explicit refresh controls.
## Accessibility
@@ -211,11 +286,12 @@ Settings fields:
- Reduced motion mode.
- ARIA labels for icon buttons.
- Captions/transcript for voice notes.
- Minimum 44px tap targets.
## Future Capacitor Features
## PWA Sufficiency Summary
- Native push via FCM.
- Native share target.
- Background upload reliability.
- Local biometric unlock.
- Better notification actions: Approve/Deny from notification.
PWA is enough for a functional prototype.
Native Android is better for the product Zeb described.
Decision: build shared React app + Capacitor Android from day one, while keeping PWA fallback essentially free.
+4 -2
View File
@@ -32,10 +32,12 @@ Secondary later:
- Tool calls are first-class objects.
2. Phone-native ergonomics
- Native Android app first, PWA fallback second.
- Thumb-reachable controls.
- Large tap targets.
- Voice and upload actions always nearby.
- Background task status survives app close/reopen.
- Haptics, native push, status bar/splash, and Android back behavior should be part of the product, not afterthoughts.
3. Private by default
- Self-hosted companion server.
@@ -243,8 +245,8 @@ The Settings health page should include a `Test Hermes` button.
2. PWA push notification quirks.
- Mitigation: support ntfy/Gotify fallback.
3. Mobile browser audio quirks.
- Mitigation: test Android Chrome first; Capacitor later if needed.
3. Mobile audio and lifecycle quirks.
- Mitigation: use Capacitor from the start; begin with WebView MediaRecorder and move to a native audio plugin if needed.
4. Tool-call streaming requires Hermes integration changes.
- Mitigation: companion adapter interface; use available callbacks in AIAgent where possible.
+7 -5
View File
@@ -18,7 +18,7 @@ Goal: empty but runnable monorepo.
Deliverables:
- pnpm workspace
- `apps/mobile` Vite React PWA
- `apps/mobile` Vite React app with Capacitor Android target and PWA fallback
- `apps/companion` Fastify server
- shared TypeScript package for event/API schemas
- basic app shell with bottom nav
@@ -66,10 +66,10 @@ Deliverables:
Goal: close phone and get told when done.
Deliverables:
- PWA service worker
- Web Push subscription management
- Native Android push path or ntfy/Web Push fallback
- PWA service worker fallback
- notification provider abstraction
- ntfy fallback
- notification permission onboarding
- completion/failure/approval notifications
## Phase 6 — Cron + Approvals
@@ -102,11 +102,13 @@ Goal: feels native.
Deliverables:
- refined Happy-inspired UI
- animations/microinteractions
- Capacitor Android build pipeline
- adaptive icons/splash
- native status/navigation bars
- share target
- haptics
- offline shell polish
- optional Capacitor wrapper evaluation
- APK release/signing flow
## Phase 9 — Deep Hermes Integration