Add allowlisted diagnostic jobs and MG4 ADB plan
This commit is contained in:
@@ -7,3 +7,6 @@ coverage/
|
|||||||
.env
|
.env
|
||||||
.DS_Store
|
.DS_Store
|
||||||
server/data/
|
server/data/
|
||||||
|
|
||||||
|
# Machine-specific Raspberry Pi access notes
|
||||||
|
PI_LOCAL.md
|
||||||
|
|||||||
@@ -40,6 +40,7 @@ Core operation must work without cloud access. Optional remote access is a later
|
|||||||
- generate a sanitized support bundle
|
- generate a sanitized support bundle
|
||||||
- Touch-focused web dashboard
|
- Touch-focused web dashboard
|
||||||
- Native Android WebView shell with editable trusted dashboard URL and reconnect controls
|
- Native Android WebView shell with editable trusted dashboard URL and reconnect controls
|
||||||
|
- Read-only ADB connection health and device/display identity for one explicitly configured head unit
|
||||||
- Raspberry Pi bootstrap, systemd service, health check, update, and uninstall scripts
|
- Raspberry Pi bootstrap, systemd service, health check, update, and uninstall scripts
|
||||||
- Automated tests, production builds, and operator documentation
|
- Automated tests, production builds, and operator documentation
|
||||||
|
|
||||||
@@ -80,6 +81,7 @@ MG4 Android app / browser
|
|||||||
Fastify API + static React app <----> SQLite
|
Fastify API + static React app <----> SQLite
|
||||||
|
|
|
|
||||||
+---- system status collectors
|
+---- system status collectors
|
||||||
|
+---- allowlisted read-only ADB collectors
|
||||||
+---- allowlisted job handlers
|
+---- allowlisted job handlers
|
||||||
+---- controlled artifact storage
|
+---- controlled artifact storage
|
||||||
+---- audit and diagnostic services
|
+---- audit and diagnostic services
|
||||||
@@ -207,6 +209,37 @@ Artifacts live under one configured data directory; API callers never provide fi
|
|||||||
- Preserve Android soft-keyboard behavior and support file chooser/download flows without broad storage permissions.
|
- Preserve Android soft-keyboard behavior and support file chooser/download flows without broad storage permissions.
|
||||||
- Optimize for landscape and offer immersive mode without making navigation recovery difficult.
|
- Optimize for landscape and offer immersive mode without making navigation recovery difficult.
|
||||||
|
|
||||||
|
### MG4 infotainment and ADB boundary
|
||||||
|
|
||||||
|
The hardware baseline is sourced from the captured MG4 Luxury 2022 profile at
|
||||||
|
`https://git.molberg.cloud/alex/mg4-luxury-2022-info` (capture dated 2026-07-30;
|
||||||
|
reference commit `162a6636df89e1df2bba7ffc14550a37b3f9e9d5`). Treat it as a device-specific
|
||||||
|
reference, not proof for other model years, trims, regions, or software builds.
|
||||||
|
|
||||||
|
- The tested head unit is Android 9/API 28 on arm64, with ADB enabled but no root,
|
||||||
|
no `su`, and SELinux enforcing. The companion must not depend on root or system
|
||||||
|
partition changes.
|
||||||
|
- Display 0 is 1920×720 at 160 dpi, but the usable app area is 1778×640 because of
|
||||||
|
an 80 px top status bar and 142 px left navigation bar. Use Android window insets;
|
||||||
|
do not model the navigation bar as a bottom inset.
|
||||||
|
- The physical pixels are non-square. Emulator checks cannot validate the apparent
|
||||||
|
shape of circles, gauges, artwork, or logos; those require the real display.
|
||||||
|
- Car UX restrictions are enforced while moving. Version 1 is parked-use software
|
||||||
|
unless a later native activity is explicitly designed, declared, and validated as
|
||||||
|
distraction optimized.
|
||||||
|
- The Pi is the only intended ADB host. A future bridge must target one configured
|
||||||
|
device, use argument-array process execution, enforce timeouts and output bounds,
|
||||||
|
expose unavailable/unauthorized/offline states, and register every operation in a
|
||||||
|
typed server-side allowlist. No API accepts a shell string, arbitrary ADB arguments,
|
||||||
|
a package/component supplied by a client, or an unrestricted file path.
|
||||||
|
- Begin with read-only device identity, connection health, display, package/version,
|
||||||
|
and carefully selected `dumpsys` collectors. Any app launch, input injection, file
|
||||||
|
transfer, or settings change is a separate state-changing capability requiring an
|
||||||
|
explicit threat model, parked-state policy, confirmation policy, and audit trail.
|
||||||
|
- Do not use ADB or the exported SAIC vehicle service to actuate vehicle hardware in
|
||||||
|
this project. Vehicle telemetry and vendor APIs remain a later, read-only research
|
||||||
|
track until permissions, semantics, privacy, and parked-vehicle tests are documented.
|
||||||
|
|
||||||
### Web experience
|
### Web experience
|
||||||
|
|
||||||
The web app must remain usable for desktop/mobile setup, but the primary runtime layout targets 1920×720 landscape.
|
The web app must remain usable for desktop/mobile setup, but the primary runtime layout targets 1920×720 landscape.
|
||||||
@@ -229,7 +262,7 @@ Primary areas:
|
|||||||
|
|
||||||
## 6. Delivery roadmap
|
## 6. Delivery roadmap
|
||||||
|
|
||||||
The repository currently contains planning documentation only. Status should be updated as work lands: `[ ]` pending, `[~]` in progress, `[x]` complete, `[!]` blocked.
|
The repository contains a working MVP and tracks remaining delivery work here. Status markers are: `[ ]` pending, `[~]` in progress, `[x]` complete, `[!]` blocked.
|
||||||
|
|
||||||
### Milestone 0 — Decisions and scaffold
|
### Milestone 0 — Decisions and scaffold
|
||||||
|
|
||||||
@@ -243,7 +276,7 @@ The repository currently contains planning documentation only. Status should be
|
|||||||
|
|
||||||
### Milestone 1 — Secure authenticated vertical slice
|
### Milestone 1 — Secure authenticated vertical slice
|
||||||
|
|
||||||
- [~] Implement SQLite migrations for users, sessions, settings, audit events, and job runs. User, session, and audit tables are complete; settings and job runs follow with Milestone 2.
|
- [~] Implement SQLite migrations for users, sessions, settings, audit events, and job runs. User, session, audit, and job-run tables are complete; settings remain pending.
|
||||||
- [x] Implement atomic first-admin setup, login, logout, session expiry/revocation, CSRF, and throttling.
|
- [x] Implement atomic first-admin setup, login, logout, session expiry/revocation, CSRF, and throttling.
|
||||||
- [x] Build setup/login screens and authenticated application shell.
|
- [x] Build setup/login screens and authenticated application shell.
|
||||||
- [x] Implement `GET /healthz` and authenticated system status with per-field availability.
|
- [x] Implement `GET /healthz` and authenticated system status with per-field availability.
|
||||||
@@ -254,23 +287,25 @@ The repository currently contains planning documentation only. Status should be
|
|||||||
|
|
||||||
### Milestone 2 — Jobs, audit, and diagnostics
|
### Milestone 2 — Jobs, audit, and diagnostics
|
||||||
|
|
||||||
- [ ] Implement the typed job registry, runner, timeouts, progress events, persistence, and restart recovery.
|
- [x] Implement the typed job registry, runner, timeouts, progress events, persistence, and restart recovery.
|
||||||
- [ ] Add refresh-health, network-diagnostics, and sanitized-support-bundle jobs.
|
- [x] Add refresh-health, network-diagnostics, and sanitized-support-bundle jobs.
|
||||||
- [ ] Implement action-bound confirmation infrastructure for higher-risk future jobs.
|
- [ ] Implement action-bound confirmation infrastructure for higher-risk future jobs.
|
||||||
- [ ] Build the jobs/history UI and audit timeline.
|
- [ ] Build the jobs/history UI and audit timeline.
|
||||||
- [ ] Add bounded output, central redaction, support-bundle manifest, and retention behavior.
|
- [~] Add bounded output, central redaction, support-bundle manifest, and retention behavior. Output bounding, central redaction, and the support-bundle manifest are complete; retention remains pending.
|
||||||
|
|
||||||
**Exit:** the three initial jobs run only through the allowlist, expose live progress, persist safe results, and create attributable audit events.
|
**Exit:** the three initial jobs run only through the allowlist, expose live progress, persist safe results, and create attributable audit events.
|
||||||
|
|
||||||
### Milestone 3 — Android client
|
### Milestone 3 — ADB transport and Android client
|
||||||
|
|
||||||
|
- [ ] Implement one-device ADB configuration, connection/authorization state, timeouts, output bounds, and test fixtures.
|
||||||
|
- [ ] Add allowlisted, read-only device identity and display collectors with per-field availability.
|
||||||
- [ ] Create the Kotlin project with minimum SDK 28 and landscape support.
|
- [ ] Create the Kotlin project with minimum SDK 28 and landscape support.
|
||||||
- [ ] Implement URL setup/storage, trusted-origin navigation, and narrow cleartext policy.
|
- [ ] Implement URL setup/storage, trusted-origin navigation, and narrow cleartext policy.
|
||||||
- [ ] Implement WebView session behavior, soft keyboard, refresh, loading, and offline/error screens.
|
- [ ] Implement WebView session behavior, soft keyboard, refresh, loading, and offline/error screens.
|
||||||
- [ ] Implement safe file chooser/download behavior where supported.
|
- [ ] Implement safe file chooser/download behavior where supported.
|
||||||
- [ ] Produce a debug APK in CI or a documented compatible Android build environment.
|
- [ ] Produce a debug APK in CI or a documented compatible Android build environment.
|
||||||
|
|
||||||
**Exit:** the app builds and can load/authenticate against the local dashboard in an emulator or test device; actual MG4 behavior remains explicitly unverified until hardware testing.
|
**Exit:** the Pi reports truthful ADB/head-unit connection state through read-only allowlisted collectors, and the app builds and can load/authenticate against the local dashboard in an emulator or test device; actual MG4 behavior remains explicitly unverified until hardware testing.
|
||||||
|
|
||||||
### Milestone 4 — Pi deployment
|
### Milestone 4 — Pi deployment
|
||||||
|
|
||||||
@@ -381,6 +416,7 @@ Version 1 is complete only when:
|
|||||||
- real status is visible without fabricated fallback data;
|
- real status is visible without fabricated fallback data;
|
||||||
- the three initial jobs are allowlisted, validated, bounded, streamed, persisted, and audited;
|
- the three initial jobs are allowlisted, validated, bounded, streamed, persisted, and audited;
|
||||||
- a sanitized support bundle can be generated and inspected;
|
- a sanitized support bundle can be generated and inspected;
|
||||||
|
- configured ADB device/display identity is read through bounded, read-only collectors with honest offline and unauthorized states;
|
||||||
- the web UI is usable at 1920×720 and handles connection failure honestly;
|
- the web UI is usable at 1920×720 and handles connection failure honestly;
|
||||||
- the Android app builds and loads only its configured trusted origin;
|
- the Android app builds and loads only its configured trusted origin;
|
||||||
- lint, typecheck, automated tests, server/web builds, and Android build pass;
|
- lint, typecheck, automated tests, server/web builds, and Android build pass;
|
||||||
|
|||||||
@@ -11,12 +11,13 @@ The current MVP slice includes:
|
|||||||
- real host, CPU, memory, disk, network, uptime, and service status;
|
- real host, CPU, memory, disk, network, uptime, and service status;
|
||||||
- field-level unavailable states when host data cannot be collected;
|
- field-level unavailable states when host data cannot be collected;
|
||||||
- authenticated Server-Sent Events for live status updates;
|
- authenticated Server-Sent Events for live status updates;
|
||||||
|
- allowlisted refresh-health, offline network-diagnostics, and sanitized support-bundle jobs;
|
||||||
- a responsive 1920×720-oriented setup, login, and dashboard UI;
|
- a responsive 1920×720-oriented setup, login, and dashboard UI;
|
||||||
- an idempotent Raspberry Pi installer with systemd start-on-boot support;
|
- an idempotent Raspberry Pi installer with systemd start-on-boot support;
|
||||||
- atomic, dashboard-triggered Git updates with automatic verification and rollback safety;
|
- atomic, dashboard-triggered Git updates with automatic verification and rollback safety;
|
||||||
- automated API security tests and reproducible quality commands.
|
- automated API security tests and reproducible quality commands.
|
||||||
|
|
||||||
Jobs, artifacts, the Android companion, and remaining operations work are tracked in [PLAN.md](./PLAN.md).
|
The ADB transport, job UI, artifacts, Android companion, and remaining operations work are tracked in [PLAN.md](./PLAN.md).
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
|
|||||||
+44
-1
@@ -8,6 +8,7 @@ import Fastify, { type FastifyInstance, type FastifyReply, type FastifyRequest }
|
|||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
import type { AppConfig } from "./config.js";
|
import type { AppConfig } from "./config.js";
|
||||||
import { openDatabase, type CompanionDatabase } from "./database.js";
|
import { openDatabase, type CompanionDatabase } from "./database.js";
|
||||||
|
import { JobRunner } from "./jobs.js";
|
||||||
import {
|
import {
|
||||||
clientAddress,
|
clientAddress,
|
||||||
CSRF_COOKIE,
|
CSRF_COOKIE,
|
||||||
@@ -96,13 +97,18 @@ export async function buildApp(config: AppConfig): Promise<FastifyInstance> {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
const database = openDatabase(config.databasePath);
|
const database = openDatabase(config.databasePath);
|
||||||
|
const jobRunner = new JobRunner(database);
|
||||||
|
|
||||||
app.decorate("database", database);
|
app.decorate("database", database);
|
||||||
|
app.decorate("jobRunner", jobRunner);
|
||||||
app.decorateRequest("authUser", null);
|
app.decorateRequest("authUser", null);
|
||||||
await app.register(cookie);
|
await app.register(cookie);
|
||||||
await app.register(rateLimit, { global: false });
|
await app.register(rateLimit, { global: false });
|
||||||
|
|
||||||
app.addHook("onClose", async () => database.close());
|
app.addHook("onClose", async () => {
|
||||||
|
jobRunner.close();
|
||||||
|
database.close();
|
||||||
|
});
|
||||||
app.addHook("onSend", async (_request, reply) => {
|
app.addHook("onSend", async (_request, reply) => {
|
||||||
reply.header("X-Content-Type-Options", "nosniff");
|
reply.header("X-Content-Type-Options", "nosniff");
|
||||||
reply.header("Referrer-Policy", "no-referrer");
|
reply.header("Referrer-Policy", "no-referrer");
|
||||||
@@ -272,6 +278,39 @@ export async function buildApp(config: AppConfig): Promise<FastifyInstance> {
|
|||||||
return collectSystemStatus();
|
return collectSystemStatus();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
app.get("/api/jobs", async (request, reply) => {
|
||||||
|
if (!requireUser(request, reply)) return;
|
||||||
|
return { jobs: jobRunner.listJobs() };
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get("/api/job-runs", async (request, reply) => {
|
||||||
|
if (!requireUser(request, reply)) return;
|
||||||
|
return { runs: jobRunner.listRuns() };
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get<{ Params: { id: string } }>("/api/job-runs/:id", async (request, reply) => {
|
||||||
|
if (!requireUser(request, reply)) return;
|
||||||
|
const run = jobRunner.getRun(request.params.id);
|
||||||
|
if (!run) return reply.code(404).send({ error: { code: "RUN_NOT_FOUND", message: "Job run not found" } });
|
||||||
|
return run;
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post<{ Params: { jobId: string } }>("/api/jobs/:jobId/runs", async (request, reply) => {
|
||||||
|
if (!requireUser(request, reply)) return;
|
||||||
|
if (!jobRunner.has(request.params.jobId)) {
|
||||||
|
return reply.code(404).send({ error: { code: "JOB_NOT_FOUND", message: "Job is not available" } });
|
||||||
|
}
|
||||||
|
const run = jobRunner.enqueue(request.params.jobId, request.authUser!.id);
|
||||||
|
audit(database, {
|
||||||
|
actorUserId: request.authUser!.id,
|
||||||
|
action: "job.enqueue",
|
||||||
|
result: "success",
|
||||||
|
ipAddress: clientAddress(request),
|
||||||
|
details: { jobId: request.params.jobId, runId: run.id }
|
||||||
|
});
|
||||||
|
return reply.code(202).send(run);
|
||||||
|
});
|
||||||
|
|
||||||
app.get("/api/system/update", async (request, reply) => {
|
app.get("/api/system/update", async (request, reply) => {
|
||||||
if (!requireUser(request, reply)) return;
|
if (!requireUser(request, reply)) return;
|
||||||
return readUpdateStatus(config);
|
return readUpdateStatus(config);
|
||||||
@@ -338,9 +377,13 @@ export async function buildApp(config: AppConfig): Promise<FastifyInstance> {
|
|||||||
};
|
};
|
||||||
await sendStatus();
|
await sendStatus();
|
||||||
const interval = setInterval(() => void sendStatus(), 10_000);
|
const interval = setInterval(() => void sendStatus(), 10_000);
|
||||||
|
const stopJobEvents = jobRunner.onRun((run) => {
|
||||||
|
if (!closed) reply.raw.write(`event: job-run\ndata: ${JSON.stringify(run)}\n\n`);
|
||||||
|
});
|
||||||
request.raw.on("close", () => {
|
request.raw.on("close", () => {
|
||||||
closed = true;
|
closed = true;
|
||||||
clearInterval(interval);
|
clearInterval(interval);
|
||||||
|
stopJobEvents();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -43,5 +43,30 @@ function migrate(database: CompanionDatabase): void {
|
|||||||
details_json TEXT NOT NULL DEFAULT '{}',
|
details_json TEXT NOT NULL DEFAULT '{}',
|
||||||
created_at TEXT NOT NULL
|
created_at TEXT NOT NULL
|
||||||
);
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS job_runs (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
job_id TEXT NOT NULL,
|
||||||
|
actor_user_id INTEGER REFERENCES users(id) ON DELETE SET NULL,
|
||||||
|
status TEXT NOT NULL CHECK (status IN (
|
||||||
|
'queued', 'running', 'succeeded', 'failed', 'timed_out', 'interrupted'
|
||||||
|
)),
|
||||||
|
input_json TEXT NOT NULL DEFAULT '{}',
|
||||||
|
output_json TEXT,
|
||||||
|
error_code TEXT,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
started_at TEXT,
|
||||||
|
finished_at TEXT
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS job_runs_created_idx ON job_runs(created_at DESC);
|
||||||
`);
|
`);
|
||||||
|
|
||||||
|
database
|
||||||
|
.prepare(
|
||||||
|
`UPDATE job_runs
|
||||||
|
SET status = 'interrupted', error_code = 'PROCESS_RESTARTED', finished_at = ?
|
||||||
|
WHERE status IN ('queued', 'running')`
|
||||||
|
)
|
||||||
|
.run(new Date().toISOString());
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,284 @@
|
|||||||
|
import { randomUUID } from "node:crypto";
|
||||||
|
import { getServers } from "node:dns";
|
||||||
|
import { hostname, networkInterfaces } from "node:os";
|
||||||
|
import type { CompanionDatabase } from "./database.js";
|
||||||
|
import { collectSystemStatus } from "./status.js";
|
||||||
|
|
||||||
|
const MAX_OUTPUT_BYTES = 32 * 1024;
|
||||||
|
const SECRET_KEY = /authorization|cookie|csrf|password|secret|token|credential/i;
|
||||||
|
const SECRET_VALUE = /(bearer\s+\S+|(?:password|secret|token|authorization|cookie)=\S+)/gi;
|
||||||
|
|
||||||
|
export type JobStatus = "queued" | "running" | "succeeded" | "failed" | "timed_out" | "interrupted";
|
||||||
|
|
||||||
|
export type JobDescriptor = {
|
||||||
|
id: string;
|
||||||
|
title: string;
|
||||||
|
description: string;
|
||||||
|
timeoutMs: number;
|
||||||
|
riskLevel: "low" | "medium" | "high";
|
||||||
|
confirmationRequired: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type JobDefinition = JobDescriptor & {
|
||||||
|
run: (signal: AbortSignal) => Promise<unknown>;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type JobRun = {
|
||||||
|
id: string;
|
||||||
|
jobId: string;
|
||||||
|
status: JobStatus;
|
||||||
|
output: unknown | null;
|
||||||
|
errorCode: string | null;
|
||||||
|
createdAt: string;
|
||||||
|
startedAt: string | null;
|
||||||
|
finishedAt: string | null;
|
||||||
|
};
|
||||||
|
|
||||||
|
function networkDiagnostics() {
|
||||||
|
let localHostname: { available: true; value: string } | { available: false; reason: string };
|
||||||
|
let dnsServers: { available: true; value: string[] } | { available: false; reason: string };
|
||||||
|
let interfaces:
|
||||||
|
| { available: true; value: Array<{ name: string; address: string; family: string; internal: boolean; netmask: string }> }
|
||||||
|
| { available: false; reason: string };
|
||||||
|
try {
|
||||||
|
localHostname = { available: true, value: hostname() };
|
||||||
|
} catch {
|
||||||
|
localHostname = { available: false, reason: "The host name is not exposed by this system" };
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
dnsServers = { available: true, value: getServers() };
|
||||||
|
} catch {
|
||||||
|
dnsServers = { available: false, reason: "Resolver configuration is not exposed by this system" };
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
interfaces = {
|
||||||
|
available: true,
|
||||||
|
value: Object.entries(networkInterfaces()).flatMap(([name, addresses]) =>
|
||||||
|
(addresses ?? []).map(({ address, family, internal, netmask }) => ({ name, address, family, internal, netmask }))
|
||||||
|
)
|
||||||
|
};
|
||||||
|
} catch {
|
||||||
|
interfaces = { available: false, reason: "Network interfaces are not exposed by this system" };
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
hostname: localHostname,
|
||||||
|
dnsServers,
|
||||||
|
interfaces
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function supportBundle(database: CompanionDatabase, generatedAt: string, status: unknown) {
|
||||||
|
const tableCounts = Object.fromEntries(
|
||||||
|
["users", "sessions", "audit_events", "job_runs"].map((table) => {
|
||||||
|
const row = database.prepare(`SELECT COUNT(*) AS count FROM ${table}`).get() as { count: number };
|
||||||
|
return [table, row.count];
|
||||||
|
})
|
||||||
|
);
|
||||||
|
const jobStates = database
|
||||||
|
.prepare("SELECT status, COUNT(*) AS count FROM job_runs GROUP BY status ORDER BY status")
|
||||||
|
.all() as Array<{ status: JobStatus; count: number }>;
|
||||||
|
|
||||||
|
return {
|
||||||
|
manifest: {
|
||||||
|
format: "pi-car-companion-support-bundle",
|
||||||
|
version: 1,
|
||||||
|
generatedAt,
|
||||||
|
entries: [
|
||||||
|
{ name: "system-status", description: "Current host and companion health", classification: "diagnostic" },
|
||||||
|
{ name: "network", description: "Local interface and resolver state", classification: "sensitive-local" },
|
||||||
|
{ name: "database-summary", description: "Record counts only; no account or session values", classification: "diagnostic" }
|
||||||
|
],
|
||||||
|
exclusions: ["password hashes", "session identifiers", "CSRF values", "cookies", "environment variables", "file contents"]
|
||||||
|
},
|
||||||
|
diagnostics: {
|
||||||
|
systemStatus: status,
|
||||||
|
network: networkDiagnostics(),
|
||||||
|
database: { tableCounts, jobStates }
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function defaultDefinitions(database: CompanionDatabase): readonly JobDefinition[] {
|
||||||
|
const safeJob = (definition: Omit<JobDefinition, "riskLevel" | "confirmationRequired">): JobDefinition => ({
|
||||||
|
...definition,
|
||||||
|
riskLevel: "low",
|
||||||
|
confirmationRequired: false
|
||||||
|
});
|
||||||
|
return [
|
||||||
|
safeJob({
|
||||||
|
id: "refresh-health",
|
||||||
|
title: "Refresh system health",
|
||||||
|
description: "Collect a fresh system health snapshot.",
|
||||||
|
timeoutMs: 15_000,
|
||||||
|
run: async () => collectSystemStatus()
|
||||||
|
}),
|
||||||
|
safeJob({
|
||||||
|
id: "network-diagnostics",
|
||||||
|
title: "Network diagnostics",
|
||||||
|
description: "Inspect local interfaces and resolver configuration without contacting the internet.",
|
||||||
|
timeoutMs: 10_000,
|
||||||
|
run: async () => networkDiagnostics()
|
||||||
|
}),
|
||||||
|
safeJob({
|
||||||
|
id: "support-bundle",
|
||||||
|
title: "Generate support bundle",
|
||||||
|
description: "Collect a bounded, sanitized diagnostic manifest without secrets or file contents.",
|
||||||
|
timeoutMs: 20_000,
|
||||||
|
run: async () => {
|
||||||
|
const generatedAt = new Date().toISOString();
|
||||||
|
return supportBundle(database, generatedAt, await collectSystemStatus());
|
||||||
|
}
|
||||||
|
})
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
function sanitize(value: unknown, key = ""): unknown {
|
||||||
|
if (SECRET_KEY.test(key)) return "[REDACTED]";
|
||||||
|
if (typeof value === "string") return value.replace(SECRET_VALUE, "[REDACTED]");
|
||||||
|
if (Array.isArray(value)) return value.map((item) => sanitize(item));
|
||||||
|
if (value && typeof value === "object") {
|
||||||
|
return Object.fromEntries(Object.entries(value).map(([childKey, child]) => [childKey, sanitize(child, childKey)]));
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function boundedJson(value: unknown): string {
|
||||||
|
const json = JSON.stringify(sanitize(value));
|
||||||
|
if (Buffer.byteLength(json) <= MAX_OUTPUT_BYTES) return json;
|
||||||
|
return JSON.stringify({ truncated: true, message: "Job output exceeded the 32 KiB storage limit." });
|
||||||
|
}
|
||||||
|
|
||||||
|
function rowToRun(row: Record<string, unknown>): JobRun {
|
||||||
|
return {
|
||||||
|
id: String(row.id),
|
||||||
|
jobId: String(row.job_id),
|
||||||
|
status: row.status as JobStatus,
|
||||||
|
output: typeof row.output_json === "string" ? JSON.parse(row.output_json) : null,
|
||||||
|
errorCode: typeof row.error_code === "string" ? row.error_code : null,
|
||||||
|
createdAt: String(row.created_at),
|
||||||
|
startedAt: typeof row.started_at === "string" ? row.started_at : null,
|
||||||
|
finishedAt: typeof row.finished_at === "string" ? row.finished_at : null
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export class JobRunner {
|
||||||
|
readonly #jobs: Map<string, JobDefinition>;
|
||||||
|
readonly #listeners = new Set<(run: JobRun) => void>();
|
||||||
|
readonly #controllers = new Set<AbortController>();
|
||||||
|
#closed = false;
|
||||||
|
|
||||||
|
constructor(
|
||||||
|
private readonly database: CompanionDatabase,
|
||||||
|
jobs: readonly JobDefinition[] = defaultDefinitions(database)
|
||||||
|
) {
|
||||||
|
this.#jobs = new Map(jobs.map((job) => [job.id, job]));
|
||||||
|
}
|
||||||
|
|
||||||
|
listJobs(): JobDescriptor[] {
|
||||||
|
return [...this.#jobs.values()].map((job) => ({
|
||||||
|
id: job.id,
|
||||||
|
title: job.title,
|
||||||
|
description: job.description,
|
||||||
|
timeoutMs: job.timeoutMs,
|
||||||
|
riskLevel: job.riskLevel,
|
||||||
|
confirmationRequired: job.confirmationRequired
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
has(jobId: string): boolean {
|
||||||
|
return this.#jobs.has(jobId);
|
||||||
|
}
|
||||||
|
|
||||||
|
onRun(listener: (run: JobRun) => void): () => void {
|
||||||
|
this.#listeners.add(listener);
|
||||||
|
return () => this.#listeners.delete(listener);
|
||||||
|
}
|
||||||
|
|
||||||
|
close(): void {
|
||||||
|
this.#closed = true;
|
||||||
|
for (const controller of this.#controllers) controller.abort();
|
||||||
|
this.database
|
||||||
|
.prepare(
|
||||||
|
`UPDATE job_runs SET status = 'interrupted', error_code = 'PROCESS_STOPPED', finished_at = ?
|
||||||
|
WHERE status IN ('queued', 'running')`
|
||||||
|
)
|
||||||
|
.run(new Date().toISOString());
|
||||||
|
this.#listeners.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
enqueue(jobId: string, actorUserId: number): JobRun {
|
||||||
|
if (this.#closed) throw new Error("RUNNER_CLOSED");
|
||||||
|
const job = this.#jobs.get(jobId);
|
||||||
|
if (!job) throw new Error("UNKNOWN_JOB");
|
||||||
|
const id = randomUUID();
|
||||||
|
const createdAt = new Date().toISOString();
|
||||||
|
this.database
|
||||||
|
.prepare(
|
||||||
|
`INSERT INTO job_runs (id, job_id, actor_user_id, status, created_at)
|
||||||
|
VALUES (?, ?, ?, 'queued', ?)`
|
||||||
|
)
|
||||||
|
.run(id, jobId, actorUserId, createdAt);
|
||||||
|
this.#emit(id);
|
||||||
|
queueMicrotask(() => void this.#execute(id, job));
|
||||||
|
return this.getRun(id)!;
|
||||||
|
}
|
||||||
|
|
||||||
|
getRun(id: string): JobRun | null {
|
||||||
|
const row = this.database.prepare("SELECT * FROM job_runs WHERE id = ?").get(id) as Record<string, unknown> | undefined;
|
||||||
|
return row ? rowToRun(row) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
listRuns(limit = 50): JobRun[] {
|
||||||
|
const rows = this.database
|
||||||
|
.prepare("SELECT * FROM job_runs ORDER BY created_at DESC LIMIT ?")
|
||||||
|
.all(Math.min(Math.max(limit, 1), 100)) as Record<string, unknown>[];
|
||||||
|
return rows.map(rowToRun);
|
||||||
|
}
|
||||||
|
|
||||||
|
async #execute(id: string, job: JobDefinition): Promise<void> {
|
||||||
|
if (this.#closed) return;
|
||||||
|
const startedAt = new Date().toISOString();
|
||||||
|
this.database.prepare("UPDATE job_runs SET status = 'running', started_at = ? WHERE id = ?").run(startedAt, id);
|
||||||
|
this.#emit(id);
|
||||||
|
const controller = new AbortController();
|
||||||
|
this.#controllers.add(controller);
|
||||||
|
let timer: NodeJS.Timeout | undefined;
|
||||||
|
try {
|
||||||
|
const timeout = new Promise<never>((_resolve, reject) => {
|
||||||
|
timer = setTimeout(() => {
|
||||||
|
controller.abort();
|
||||||
|
reject(new Error("JOB_TIMEOUT"));
|
||||||
|
}, job.timeoutMs);
|
||||||
|
timer.unref();
|
||||||
|
});
|
||||||
|
const output = await Promise.race([job.run(controller.signal), timeout]);
|
||||||
|
if (this.#closed) return;
|
||||||
|
this.database
|
||||||
|
.prepare("UPDATE job_runs SET status = 'succeeded', output_json = ?, finished_at = ? WHERE id = ?")
|
||||||
|
.run(boundedJson(output), new Date().toISOString(), id);
|
||||||
|
this.#emit(id);
|
||||||
|
} catch (error) {
|
||||||
|
if (this.#closed) return;
|
||||||
|
const timedOut = error instanceof Error && error.message === "JOB_TIMEOUT";
|
||||||
|
this.database
|
||||||
|
.prepare("UPDATE job_runs SET status = ?, error_code = ?, finished_at = ? WHERE id = ?")
|
||||||
|
.run(timedOut ? "timed_out" : "failed", timedOut ? "JOB_TIMEOUT" : "JOB_FAILED", new Date().toISOString(), id);
|
||||||
|
this.#emit(id);
|
||||||
|
} finally {
|
||||||
|
if (timer) clearTimeout(timer);
|
||||||
|
this.#controllers.delete(controller);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#emit(id: string): void {
|
||||||
|
const run = this.getRun(id);
|
||||||
|
if (!run) return;
|
||||||
|
for (const listener of this.#listeners) {
|
||||||
|
try {
|
||||||
|
listener(run);
|
||||||
|
} catch {
|
||||||
|
// A disconnected event consumer must not change the persisted job result.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,4 +1,5 @@
|
|||||||
import type { CompanionDatabase } from "./database.js";
|
import type { CompanionDatabase } from "./database.js";
|
||||||
|
import type { JobRunner } from "./jobs.js";
|
||||||
|
|
||||||
declare module "fastify" {
|
declare module "fastify" {
|
||||||
interface FastifyRequest {
|
interface FastifyRequest {
|
||||||
@@ -7,6 +8,7 @@ declare module "fastify" {
|
|||||||
|
|
||||||
interface FastifyInstance {
|
interface FastifyInstance {
|
||||||
database: CompanionDatabase;
|
database: CompanionDatabase;
|
||||||
|
jobRunner: JobRunner;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -175,4 +175,51 @@ describe("authentication boundary", () => {
|
|||||||
const response = await app.inject({ method: "GET", url: "/api/system/update" });
|
const response = await app.inject({ method: "GET", url: "/api/system/update" });
|
||||||
expect(response.statusCode).toBe(401);
|
expect(response.statusCode).toBe(401);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("exposes only allowlisted jobs and authenticated run history", async () => {
|
||||||
|
const app = await createApp();
|
||||||
|
const token = await csrf(app);
|
||||||
|
await setup(app, token);
|
||||||
|
const login = await app.inject({
|
||||||
|
method: "POST",
|
||||||
|
url: "/api/auth/login",
|
||||||
|
headers: mutationHeaders(token),
|
||||||
|
payload: { username: "owner", password: "Correct-horse1" }
|
||||||
|
});
|
||||||
|
const session = login.cookies.find((item) => item.name === SESSION_COOKIE)?.value ?? "";
|
||||||
|
const cookie = `${CSRF_COOKIE}=${token}; ${SESSION_COOKIE}=${session}`;
|
||||||
|
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/jobs" })).statusCode).toBe(401);
|
||||||
|
const jobs = await app.inject({ method: "GET", url: "/api/jobs", headers: { cookie } });
|
||||||
|
expect(jobs.statusCode).toBe(200);
|
||||||
|
expect(jobs.json<{ jobs: { id: string }[] }>().jobs.map((job) => job.id)).toEqual([
|
||||||
|
"refresh-health",
|
||||||
|
"network-diagnostics",
|
||||||
|
"support-bundle"
|
||||||
|
]);
|
||||||
|
|
||||||
|
const unknown = await app.inject({
|
||||||
|
method: "POST",
|
||||||
|
url: "/api/jobs/arbitrary-shell/runs",
|
||||||
|
headers: { ...mutationHeaders(token), cookie }
|
||||||
|
});
|
||||||
|
expect(unknown.statusCode).toBe(404);
|
||||||
|
expect(unknown.json()).toMatchObject({ error: { code: "JOB_NOT_FOUND" } });
|
||||||
|
|
||||||
|
const queued = await app.inject({
|
||||||
|
method: "POST",
|
||||||
|
url: "/api/jobs/refresh-health/runs",
|
||||||
|
headers: { ...mutationHeaders(token), cookie }
|
||||||
|
});
|
||||||
|
expect(queued.statusCode).toBe(202);
|
||||||
|
const runId = queued.json<{ id: string }>().id;
|
||||||
|
let run: { status: string } = { status: "queued" };
|
||||||
|
for (let attempt = 0; attempt < 50 && ["queued", "running"].includes(run.status); attempt += 1) {
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 2));
|
||||||
|
const response = await app.inject({ method: "GET", url: `/api/job-runs/${runId}`, headers: { cookie } });
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
run = response.json<{ status: string }>();
|
||||||
|
}
|
||||||
|
expect(run.status).toBe("succeeded");
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,128 @@
|
|||||||
|
import { mkdtempSync, rmSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { afterEach, describe, expect, it } from "vitest";
|
||||||
|
import { openDatabase, type CompanionDatabase } from "../src/database.js";
|
||||||
|
import { JobRunner, type JobDefinition, type JobRun } from "../src/jobs.js";
|
||||||
|
|
||||||
|
const databases: CompanionDatabase[] = [];
|
||||||
|
const directories: string[] = [];
|
||||||
|
|
||||||
|
function database(path = ":memory:"): CompanionDatabase {
|
||||||
|
const instance = openDatabase(path);
|
||||||
|
databases.push(instance);
|
||||||
|
return instance;
|
||||||
|
}
|
||||||
|
|
||||||
|
function runnerWith(run: JobDefinition["run"], timeoutMs = 100): JobRunner {
|
||||||
|
const instance = database();
|
||||||
|
instance
|
||||||
|
.prepare("INSERT INTO users (id, username, password_hash, role, created_at) VALUES (1, 'tester', 'unused', 'admin', ?)")
|
||||||
|
.run(new Date().toISOString());
|
||||||
|
return new JobRunner(instance, [
|
||||||
|
{
|
||||||
|
id: "test-job",
|
||||||
|
title: "Test job",
|
||||||
|
description: "Test only",
|
||||||
|
timeoutMs,
|
||||||
|
riskLevel: "low",
|
||||||
|
confirmationRequired: false,
|
||||||
|
run
|
||||||
|
}
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function settled(runner: JobRunner, id: string): Promise<JobRun> {
|
||||||
|
for (let attempt = 0; attempt < 50; attempt += 1) {
|
||||||
|
const run = runner.getRun(id);
|
||||||
|
if (run && !["queued", "running"].includes(run.status)) return run;
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 2));
|
||||||
|
}
|
||||||
|
throw new Error("Job did not settle during the test");
|
||||||
|
}
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
for (const instance of databases.splice(0)) {
|
||||||
|
if (instance.open) instance.close();
|
||||||
|
}
|
||||||
|
for (const directory of directories.splice(0)) rmSync(directory, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("job runner", () => {
|
||||||
|
it("runs only registered jobs and persists successful output", async () => {
|
||||||
|
const runner = runnerWith(async () => ({ ok: true }));
|
||||||
|
const statuses: string[] = [];
|
||||||
|
const unsubscribe = runner.onRun((run) => statuses.push(run.status));
|
||||||
|
|
||||||
|
expect(runner.has("test-job")).toBe(true);
|
||||||
|
expect(runner.has("shell-command")).toBe(false);
|
||||||
|
expect(() => runner.enqueue("shell-command", 1)).toThrow("UNKNOWN_JOB");
|
||||||
|
|
||||||
|
const queued = runner.enqueue("test-job", 1);
|
||||||
|
const run = await settled(runner, queued.id);
|
||||||
|
expect(run).toMatchObject({ jobId: "test-job", status: "succeeded", output: { ok: true } });
|
||||||
|
expect(statuses).toEqual(["queued", "running", "succeeded"]);
|
||||||
|
unsubscribe();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("redacts likely secrets and bounds stored output", async () => {
|
||||||
|
const secretRunner = runnerWith(async () => ({
|
||||||
|
password: "do-not-store",
|
||||||
|
message: "authorization=abc123",
|
||||||
|
nested: { accessToken: "also-secret" }
|
||||||
|
}));
|
||||||
|
const secretRun = await settled(secretRunner, secretRunner.enqueue("test-job", 1).id);
|
||||||
|
expect(secretRun.output).toEqual({
|
||||||
|
password: "[REDACTED]",
|
||||||
|
message: "[REDACTED]",
|
||||||
|
nested: { accessToken: "[REDACTED]" }
|
||||||
|
});
|
||||||
|
|
||||||
|
const largeRunner = runnerWith(async () => ({ value: "x".repeat(40_000) }));
|
||||||
|
const largeRun = await settled(largeRunner, largeRunner.enqueue("test-job", 1).id);
|
||||||
|
expect(largeRun.output).toMatchObject({ truncated: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("marks jobs that exceed their timeout", async () => {
|
||||||
|
const runner = runnerWith(() => new Promise(() => undefined), 5);
|
||||||
|
const run = await settled(runner, runner.enqueue("test-job", 1).id);
|
||||||
|
expect(run).toMatchObject({ status: "timed_out", errorCode: "JOB_TIMEOUT" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("marks incomplete persisted runs as interrupted on restart", () => {
|
||||||
|
const directory = mkdtempSync(join(tmpdir(), "pi-car-jobs-"));
|
||||||
|
directories.push(directory);
|
||||||
|
const path = join(directory, "companion.db");
|
||||||
|
const first = database(path);
|
||||||
|
first
|
||||||
|
.prepare("INSERT INTO job_runs (id, job_id, status, created_at) VALUES ('run-1', 'test-job', 'running', ?)")
|
||||||
|
.run(new Date().toISOString());
|
||||||
|
first.close();
|
||||||
|
|
||||||
|
const reopened = database(path);
|
||||||
|
const row = reopened.prepare("SELECT status, error_code FROM job_runs WHERE id = 'run-1'").get();
|
||||||
|
expect(row).toEqual({ status: "interrupted", error_code: "PROCESS_RESTARTED" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("generates a manifest-based support bundle without account or session values", async () => {
|
||||||
|
const instance = database();
|
||||||
|
instance
|
||||||
|
.prepare("INSERT INTO users (id, username, password_hash, role, created_at) VALUES (1, 'private-owner', 'private-hash', 'admin', ?)")
|
||||||
|
.run(new Date().toISOString());
|
||||||
|
instance
|
||||||
|
.prepare("INSERT INTO sessions (id_hash, user_id, expires_at, created_at) VALUES ('private-session', 1, ?, ?)")
|
||||||
|
.run(new Date(Date.now() + 60_000).toISOString(), new Date().toISOString());
|
||||||
|
const runner = new JobRunner(instance);
|
||||||
|
|
||||||
|
const run = await settled(runner, runner.enqueue("support-bundle", 1).id);
|
||||||
|
expect(run.status).toBe("succeeded");
|
||||||
|
expect(run.output).toMatchObject({
|
||||||
|
manifest: { format: "pi-car-companion-support-bundle", version: 1 },
|
||||||
|
diagnostics: { database: { tableCounts: { users: 1, sessions: 1 } } }
|
||||||
|
});
|
||||||
|
const serialized = JSON.stringify(run.output);
|
||||||
|
expect(serialized).not.toContain("private-owner");
|
||||||
|
expect(serialized).not.toContain("private-hash");
|
||||||
|
expect(serialized).not.toContain("private-session");
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user