bonzi/hermes-mobile
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add companion API and Android APK shell

Browse Source
This commit is contained in:
Hermes Agent
2026-07-09 04:43:00 +00:00
parent bd0cf34aa3
commit 7eb9648eb7
92 changed files with 7907 additions and 877 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/MOBILE_APP.md
+30 -281
View File
@@ -1,297 +1,46 @@
# Mobile App Plan
# Mobile App Structure
## App Type Decision
The mobile app is a Vite React app wrapped by Capacitor for Android. It remains runnable as a browser dev app while producing an installable APK through the generated Android project.
Build Hermes Mobile as a native Android app from the start, using Capacitor around a shared React UI.
## Screens
PWA will still exist as a fallback/dev target, but it should not be the primary product experience.
- **Chat** sends prompts to `POST /api/chat` and displays Hermes replies.
- **Files** browses the companion workspace, opens UTF-8 files, edits content, and saves through HTTP.
- **Terminal** runs allowlisted workspace commands through the companion server.
- **Status** checks companion health, Hermes CLI availability, workspace root, and uptime.
- **Settings** stores the companion URL and access key in `localStorage` and validates pairing.
## Why Native Android First
## Companion Settings
A PWA can get surprisingly far on Android: installable icon, offline shell, camera/mic, file picker, Web Push, and a standalone window. But for this project, “suffices” is not the bar. The goal is that Hermes feels like a genuine phone app designed around agent work.
The default companion URL is `http://10.0.2.2:8787`, which is the standard Android emulator loopback address for a server running on the host computer.
Native Android via Capacitor gives us:
- Real APK install path.
- Native splash screen and status bar control.
- More reliable push notifications.
- Better haptics.
- Better file/share integration.
- Easier future “Share to Hermes” target from Android apps.
- Better control over permissions and app lifecycle.
- Native-feeling back button behavior.
- More credible “this is an app” feel than browser-installed PWA.
For a physical Android device, set the URL to the host machine LAN address, for example `http://192.168.1.50:8787`, and make sure the firewall allows port `8787`.
## Role of PWA
Keep PWA support for:
- Fast development loop.
- Desktop/LAN access.
- Emergency fallback if APK is not installed.
- Users who do not want to sideload.
But feature priority should be:
1. Android app / Capacitor
2. PWA fallback
3. Desktop web convenience
## Frontend Stack
Recommended:
- Vite + React + TypeScript
- Tailwind CSS
- Capacitor Android
- TanStack Query for server data
- Zustand for local UI/session state
- Framer Motion for app-like motion where useful
- Zod for API schema validation
- Vite PWA plugin for fallback web install
Capacitor plugins:
- `@capacitor/android`
- `@capacitor/app` for lifecycle/back button
- `@capacitor/haptics`
- `@capacitor/status-bar`
- `@capacitor/splash-screen`
- `@capacitor/push-notifications`
- `@capacitor/filesystem`
- `@capacitor/share`
- `@capacitor/preferences`
Potential later:
- Biometric auth plugin
- Camera plugin
- Background task/upload plugin if needed
## Build Targets
```text
apps/mobile/ shared React UI
apps/mobile/android/ Capacitor Android project
apps/mobile/dist/ web build consumed by Capacitor and PWA
```
Commands later:
## Development
```bash
pnpm mobile:dev # browser dev server
pnpm mobile:build # web build
pnpm android:sync # capacitor sync
pnpm android:apk # debug/release APK
npm run mobile:dev
```
## App Package Structure
## Android APK Build
After installing dependencies:
```bash
npm run cap:add:android --workspace @hermes-mobile/mobile
npm run android:build:debug --workspace @hermes-mobile/mobile
```
The debug APK is generated by Gradle at:
```text
apps/mobile/
capacitor.config.ts
android/ # generated Capacitor Android project
src/
app/
App.tsx
router.tsx
providers.tsx
native.ts # Capacitor runtime helpers
screens/
AskScreen/
ActivityScreen/
SessionsScreen/
CronScreen/
FilesScreen/
SettingsScreen/
components/
agent-status-orb/
composer/
tool-card/
event-timeline/
upload-tray/
voice-recorder/
cron-job-card/
approval-card/
bottom-nav/
app-shell/
features/
tasks/
sessions/
cron/
files/
approvals/
notifications/
pairing/
settings/
native/
haptics.ts
push.ts
share.ts
statusBar.ts
filesystem.ts
backButton.ts
lib/
api-client.ts
realtime.ts
audio.ts
file-utils.ts
format.ts
styles/
globals.css
tokens.css
assets/
icons/
splash/
main.tsx
public/
manifest.webmanifest
service-worker.ts
apps/mobile/android/app/build/outputs/apk/debug/app-debug.apk
```
## Navigation
If `android/` already exists, use:
Bottom tabs:
- Ask
- Activity
- Cron
- Files
- Settings
Native back behavior:
- If a sheet/modal is open, close it.
- Else if not on Ask, go back to previous tab/screen.
- Else prompt/minimize/exit according to Android convention.
## Ask Screen Behavior
States:
- idle
- composing
- uploading
- sending
- running
- waiting_approval
- finished
- failed
Composer:
- Text input
- Attach file button
- Hold/tap voice recording button
- Send button
- Optional mode chips: New / Continue / Background
When task starts:
- Input collapses to bottom.
- Task timeline appears.
- Assistant response streams.
- Tool cards appear in chronological order.
- Subtle haptic on send, completion, error, and approval required.
## Realtime
Use WebSocket first:
- `/api/realtime?token=...`
- Subscribe to task/session events.
Fallback:
- SSE `/api/tasks/:id/events/stream`
- Polling last resort.
Frontend event store should append events idempotently by event ID.
## Push Notifications
Native Android priority:
- Use Capacitor Push Notifications / FCM where practical.
- Companion server stores device push token.
- Notifications for task completion, failure, approval required, cron completion.
Fallbacks:
- Web Push for PWA.
- ntfy/Gotify if native push is too annoying for self-hosted early builds.
Important: notification provider must be abstracted so the app can start with ntfy/Web Push and later use FCM/native without rewriting task logic.
## Voice Recording
Prefer native-friendly implementation:
- First version can use browser MediaRecorder inside Capacitor WebView.
- If recording quality/lifecycle is poor, move to native audio recording plugin.
Preferred mime order for web implementation:
- `audio/webm;codecs=opus`
- `audio/ogg;codecs=opus`
- browser default fallback
UI:
- Press/tap mic to start.
- Big native-feeling recording sheet with timer/waveform.
- Haptic on start/stop.
- Cancel / send.
- Upload progress.
## File Upload UX
Priority:
- Use standard file input first inside WebView.
- Add Capacitor Filesystem/Share integration for better Android feel.
- Later add Android share target so user can share files/text/images into Hermes Mobile from other apps.
UI:
- Preview thumbnails for images.
- File chips for archives/docs.
- Upload before task send or as part of task multipart.
## Offline/Background Behavior
- App shell should load offline.
- Draft prompt persists locally via Capacitor Preferences/local storage.
- Running task state reloads from server after reconnect.
- Notifications tell user when background tasks finish.
- No attempt to run Hermes offline.
## Settings UX
First launch:
1. Enter companion URL or open pairing deep link/QR.
2. Pair device.
3. Test connection.
4. Enable notifications.
5. Land on Ask.
Settings fields:
- Companion URL
- Device name
- Notification status/test
- Theme
- Default task mode
- Upload retention display
- Hermes health
- App version/build channel
## Native Polish Checklist
- Proper Android package ID, e.g. `cloud.molberg.hermesmobile`.
- Adaptive icon.
- Native splash screen.
- Status/navigation bar colors match theme.
- Edge-to-edge layout with safe-area handling.
- Keyboard does not break composer layout.
- Native back button behavior.
- Haptics on important interactions.
- Notification permission onboarding.
- Share target later.
- Deep links for pairing.
- Avoid browser pull-to-refresh in PWA fallback; app should have explicit refresh controls.
## Accessibility
- Visible focus states.
- Reduced motion mode.
- ARIA labels for icon buttons.
- Captions/transcript for voice notes.
- Minimum 44px tap targets.
## PWA Sufficiency Summary
PWA is enough for a functional prototype.
Native Android is better for the product Zeb described.
Decision: build shared React app + Capacitor Android from day one, while keeping PWA fallback essentially free.
```bash
npm run cap:sync --workspace @hermes-mobile/mobile
cd apps/mobile/android
./gradlew assembleDebug
```