diff --git a/README.md b/README.md index 457423f..8ed849e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Hermes Mobile -Native-feeling Android-first PWA and companion server for [Hermes Agent](https://github.com/NousResearch/hermes-agent). +Native Android app, installable PWA fallback, and companion server for [Hermes Agent](https://github.com/NousResearch/hermes-agent). Hermes Mobile is not a generic LLM chat UI. It is a self-hosted, Hermes-native control surface designed for phone use: prompt by text or voice, upload files, watch tool calls live, approve actions, manage cron jobs, and get notified when long-running agent work is done. @@ -22,11 +22,11 @@ Planning scaffold only. No production code yet. ## Planned Components ```text -Android / mobile browser +Android app (Capacitor native shell) | - | HTTPS + WebSocket/SSE + Web Push + | HTTPS + WebSocket/SSE + native push v -Hermes Mobile App (PWA / later Capacitor) +Hermes Mobile App (native Android first, PWA fallback) | v Hermes Companion Server @@ -41,9 +41,11 @@ Hermes Companion Server ## MVP Feature Set -1. Mobile-first app shell - - Installable PWA - - Android-style bottom navigation +1. Native Android-first app shell + - Capacitor Android wrapper around the shared web UI + - APK/release build installable directly on Android + - PWA/web build retained as fallback and fast dev target + - Native-style bottom navigation, status bar, splash screen, haptics, push, share target later - Offline-friendly shell - Dark, playful visual system inspired by happy.engineering / Happy app @@ -103,11 +105,12 @@ Hermes Companion Server Frontend: - TypeScript -- React + Vite or Next.js static/PWA mode +- React + Vite - Tailwind CSS - Zustand or TanStack Query for client state -- Service Worker + Web Push -- Capacitor later, only if PWA limitations matter +- Capacitor Android from the start for genuine phone-app feel +- PWA/service-worker build kept as fallback and for desktop/LAN access +- Native plugins: Push Notifications, Haptics, Filesystem/Share, Status Bar, Splash Screen Companion server: - TypeScript Node.js initially, because file uploads, WebSocket/SSE, install scripts, and PM2/systemd are straightforward diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 87626a5..0d223d7 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -5,7 +5,8 @@ Hermes Mobile is split into two deployable parts: 1. Mobile app - - Installable PWA designed for Android. + - Native Android app built with Capacitor around a shared React UI. + - PWA/web build remains available as fallback and dev target. - Talks only to the companion server. - Does not need direct access to Hermes files or credentials. @@ -16,7 +17,7 @@ Hermes Mobile is split into two deployable parts: ```text +---------------------------+ -| Android PWA | +| Android App | | - Chat | | - Tool timeline | | - Cron UI | @@ -24,7 +25,7 @@ Hermes Mobile is split into two deployable parts: | - Approvals | +-------------+-------------+ | - | HTTPS REST + WebSocket/SSE + Web Push + | HTTPS REST + WebSocket/SSE + native push/Web Push fallback v +---------------------------+ | Hermes Companion Server | @@ -303,9 +304,11 @@ Providers: - systemd service. - Better production posture. -## Future Native Wrapper +## Native Android Shell -If Android PWA is insufficient: -- Add Capacitor app under `apps/android-shell`. -- Reuse same web app. -- Gain native share target, richer push, microphone/file APIs. +Capacitor Android is part of the primary architecture, not a future contingency. + +- The shared React UI builds into `apps/mobile/dist`. +- Capacitor packages that build into an Android APK. +- PWA/web access remains useful for dev, desktop, and fallback. +- Native plugins provide share target, richer push, haptics, status bar, splash screen, microphone/file improvements, and Android lifecycle hooks. diff --git a/docs/DESIGN_SYSTEM.md b/docs/DESIGN_SYSTEM.md index 5943b17..fdc9846 100644 --- a/docs/DESIGN_SYSTEM.md +++ b/docs/DESIGN_SYSTEM.md @@ -198,13 +198,13 @@ Prefer CSS transitions and Framer Motion if bundle size acceptable. ## Android-native Feeling -- Installable PWA manifest with proper icons. -- Standalone display mode. -- No browser chrome when installed. +- Capacitor Android APK with adaptive icon and native splash. +- PWA manifest retained for fallback web install. +- No browser chrome in the primary Android app. - Bottom nav like native Material apps. -- Pull-to-refresh disabled; in-app refresh buttons where needed. -- Back gesture should close panels before navigating away. -- Use Android share target later so user can share files/text to Hermes. +- Pull-to-refresh disabled in fallback web mode; in-app refresh buttons where needed. +- Android back gesture/button should close panels before navigating away. +- Add Android share target so user can share files/text to Hermes. ## Accessibility diff --git a/docs/MOBILE_APP.md b/docs/MOBILE_APP.md index 0b2290b..a680695 100644 --- a/docs/MOBILE_APP.md +++ b/docs/MOBILE_APP.md @@ -1,43 +1,96 @@ # Mobile App Plan -## App Type +## App Type Decision -Start as a PWA optimized for Android Chrome. +Build Hermes Mobile as a native Android app from the start, using Capacitor around a shared React UI. -Why PWA first: -- Fast iteration. -- Easy self-hosting. -- No app store. -- Installable to home screen. -- Web Push support on Android is good. -- Same frontend can later be wrapped with Capacitor. +PWA will still exist as a fallback/dev target, but it should not be the primary product experience. -Potential later native shell: -- Capacitor Android for native share target, richer notifications, and better file/mic APIs. +## Why Native Android First + +A PWA can get surprisingly far on Android: installable icon, offline shell, camera/mic, file picker, Web Push, and a standalone window. But for this project, “suffices” is not the bar. The goal is that Hermes feels like a genuine phone app designed around agent work. + +Native Android via Capacitor gives us: +- Real APK install path. +- Native splash screen and status bar control. +- More reliable push notifications. +- Better haptics. +- Better file/share integration. +- Easier future “Share to Hermes” target from Android apps. +- Better control over permissions and app lifecycle. +- Native-feeling back button behavior. +- More credible “this is an app” feel than browser-installed PWA. + +## Role of PWA + +Keep PWA support for: +- Fast development loop. +- Desktop/LAN access. +- Emergency fallback if APK is not installed. +- Users who do not want to sideload. + +But feature priority should be: +1. Android app / Capacitor +2. PWA fallback +3. Desktop web convenience ## Frontend Stack Recommended: - Vite + React + TypeScript - Tailwind CSS +- Capacitor Android - TanStack Query for server data - Zustand for local UI/session state -- Framer Motion for motion where useful -- Workbox or Vite PWA plugin +- Framer Motion for app-like motion where useful - Zod for API schema validation +- Vite PWA plugin for fallback web install -Alternative: -- Next.js if SSR/server components become useful, but for a companion-served app Vite is simpler. +Capacitor plugins: +- `@capacitor/android` +- `@capacitor/app` for lifecycle/back button +- `@capacitor/haptics` +- `@capacitor/status-bar` +- `@capacitor/splash-screen` +- `@capacitor/push-notifications` +- `@capacitor/filesystem` +- `@capacitor/share` +- `@capacitor/preferences` + +Potential later: +- Biometric auth plugin +- Camera plugin +- Background task/upload plugin if needed + +## Build Targets + +```text +apps/mobile/ shared React UI +apps/mobile/android/ Capacitor Android project +apps/mobile/dist/ web build consumed by Capacitor and PWA +``` + +Commands later: + +```bash +pnpm mobile:dev # browser dev server +pnpm mobile:build # web build +pnpm android:sync # capacitor sync +pnpm android:apk # debug/release APK +``` ## App Package Structure ```text apps/mobile/ + capacitor.config.ts + android/ # generated Capacitor Android project src/ app/ App.tsx router.tsx providers.tsx + native.ts # Capacitor runtime helpers screens/ AskScreen/ ActivityScreen/ @@ -65,10 +118,16 @@ apps/mobile/ notifications/ pairing/ settings/ + native/ + haptics.ts + push.ts + share.ts + statusBar.ts + filesystem.ts + backButton.ts lib/ api-client.ts realtime.ts - push.ts audio.ts file-utils.ts format.ts @@ -93,9 +152,10 @@ Bottom tabs: - Files - Settings -Sessions can be either: -- a sub-screen from Ask, or -- included in Activity initially. +Native back behavior: +- If a sheet/modal is open, close it. +- Else if not on Ask, go back to previous tab/screen. +- Else prompt/minimize/exit according to Android convention. ## Ask Screen Behavior @@ -121,6 +181,7 @@ When task starts: - Task timeline appears. - Assistant response streams. - Tool cards appear in chronological order. +- Subtle haptic on send, completion, error, and approval required. ## Realtime @@ -136,35 +197,43 @@ Frontend event store should append events idempotently by event ID. ## Push Notifications -PWA flow: -1. User enables notifications in Settings. -2. App registers service worker. -3. App subscribes to PushManager. -4. Subscription sent to companion server. -5. Server sends Web Push on completion/approval/cron events. +Native Android priority: +- Use Capacitor Push Notifications / FCM where practical. +- Companion server stores device push token. +- Notifications for task completion, failure, approval required, cron completion. -Fallback if Web Push fails: -- Configure ntfy link in Settings. +Fallbacks: +- Web Push for PWA. +- ntfy/Gotify if native push is too annoying for self-hosted early builds. + +Important: notification provider must be abstracted so the app can start with ntfy/Web Push and later use FCM/native without rewriting task logic. ## Voice Recording -Use MediaRecorder. +Prefer native-friendly implementation: +- First version can use browser MediaRecorder inside Capacitor WebView. +- If recording quality/lifecycle is poor, move to native audio recording plugin. -Preferred mime order: +Preferred mime order for web implementation: - `audio/webm;codecs=opus` - `audio/ogg;codecs=opus` - browser default fallback UI: - Press/tap mic to start. -- Big recording sheet with timer/waveform. +- Big native-feeling recording sheet with timer/waveform. +- Haptic on start/stop. - Cancel / send. - Upload progress. ## File Upload UX -- Android file picker through ``. -- Share Target API later for “Share to Hermes”. +Priority: +- Use standard file input first inside WebView. +- Add Capacitor Filesystem/Share integration for better Android feel. +- Later add Android share target so user can share files/text/images into Hermes Mobile from other apps. + +UI: - Preview thumbnails for images. - File chips for archives/docs. - Upload before task send or as part of task multipart. @@ -172,14 +241,15 @@ UI: ## Offline/Background Behavior - App shell should load offline. -- Draft prompt persists locally. +- Draft prompt persists locally via Capacitor Preferences/local storage. - Running task state reloads from server after reconnect. +- Notifications tell user when background tasks finish. - No attempt to run Hermes offline. ## Settings UX First launch: -1. Enter companion URL or scan QR/pairing link. +1. Enter companion URL or open pairing deep link/QR. 2. Pair device. 3. Test connection. 4. Enable notifications. @@ -193,17 +263,22 @@ Settings fields: - Default task mode - Upload retention display - Hermes health +- App version/build channel -## Android Polish Checklist +## Native Polish Checklist -- `display: standalone` manifest. -- Proper app icon masks/adaptive icon assets. -- Theme color matching dark background. -- Avoid viewport resize bugs with keyboard. -- Safe-area padding. -- Disable browser pull-to-refresh where possible and add in-app refresh. -- Haptics via Vibration API where appropriate. -- Back button behavior for modals/sheets. +- Proper Android package ID, e.g. `cloud.molberg.hermesmobile`. +- Adaptive icon. +- Native splash screen. +- Status/navigation bar colors match theme. +- Edge-to-edge layout with safe-area handling. +- Keyboard does not break composer layout. +- Native back button behavior. +- Haptics on important interactions. +- Notification permission onboarding. +- Share target later. +- Deep links for pairing. +- Avoid browser pull-to-refresh in PWA fallback; app should have explicit refresh controls. ## Accessibility @@ -211,11 +286,12 @@ Settings fields: - Reduced motion mode. - ARIA labels for icon buttons. - Captions/transcript for voice notes. +- Minimum 44px tap targets. -## Future Capacitor Features +## PWA Sufficiency Summary -- Native push via FCM. -- Native share target. -- Background upload reliability. -- Local biometric unlock. -- Better notification actions: Approve/Deny from notification. +PWA is enough for a functional prototype. + +Native Android is better for the product Zeb described. + +Decision: build shared React app + Capacitor Android from day one, while keeping PWA fallback essentially free. diff --git a/docs/PRODUCT_PLAN.md b/docs/PRODUCT_PLAN.md index 85a4194..1718e24 100644 --- a/docs/PRODUCT_PLAN.md +++ b/docs/PRODUCT_PLAN.md @@ -32,10 +32,12 @@ Secondary later: - Tool calls are first-class objects. 2. Phone-native ergonomics + - Native Android app first, PWA fallback second. - Thumb-reachable controls. - Large tap targets. - Voice and upload actions always nearby. - Background task status survives app close/reopen. + - Haptics, native push, status bar/splash, and Android back behavior should be part of the product, not afterthoughts. 3. Private by default - Self-hosted companion server. @@ -243,8 +245,8 @@ The Settings health page should include a `Test Hermes` button. 2. PWA push notification quirks. - Mitigation: support ntfy/Gotify fallback. -3. Mobile browser audio quirks. - - Mitigation: test Android Chrome first; Capacitor later if needed. +3. Mobile audio and lifecycle quirks. + - Mitigation: use Capacitor from the start; begin with WebView MediaRecorder and move to a native audio plugin if needed. 4. Tool-call streaming requires Hermes integration changes. - Mitigation: companion adapter interface; use available callbacks in AIAgent where possible. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 287d147..68a7cdd 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -18,7 +18,7 @@ Goal: empty but runnable monorepo. Deliverables: - pnpm workspace -- `apps/mobile` Vite React PWA +- `apps/mobile` Vite React app with Capacitor Android target and PWA fallback - `apps/companion` Fastify server - shared TypeScript package for event/API schemas - basic app shell with bottom nav @@ -66,10 +66,10 @@ Deliverables: Goal: close phone and get told when done. Deliverables: -- PWA service worker -- Web Push subscription management +- Native Android push path or ntfy/Web Push fallback +- PWA service worker fallback - notification provider abstraction -- ntfy fallback +- notification permission onboarding - completion/failure/approval notifications ## Phase 6 — Cron + Approvals @@ -102,11 +102,13 @@ Goal: feels native. Deliverables: - refined Happy-inspired UI - animations/microinteractions +- Capacitor Android build pipeline - adaptive icons/splash +- native status/navigation bars - share target - haptics - offline shell polish -- optional Capacitor wrapper evaluation +- APK release/signing flow ## Phase 9 — Deep Hermes Integration