All chapters
Operations
Huntress is meant to be run by people who have other things to do. It updates itself under rules you set, can always be rolled back, and tells you when it is not well.
Updates#
A separate, privileged updater service replaces the program; the program never replaces its own files. It follows a release channel (stable, beta or canary; Updates:Channel) published on GitHub, whose manifest is signed with ECDSA P-256 and lists a SHA-256 for every archive. It refuses a manifest that is unsigned, tampered with, expired, yanked or older than what is installed.
| Step | What happens |
|---|---|
| Check | Every 4 hours (Updates:CheckIntervalMinutes), within the policy below. |
| Download and verify | The archive is checked against the signed manifest. |
| Stage | It is unpacked beside the running release. |
| Preflight | The new program migrates a copy of your database and health-checks it. A problem stops the update here, with no downtime. |
| Back up | A consistent database backup is taken. |
| Switch | The current link moves to the new release and the service restarts, a pause of a few seconds. |
| Health gate | For up to 2 minutes (Updates:HealthTimeoutSeconds) the updater waits for /health/ready to be healthy and /status to report the new version. |
| Keep or roll back | On failure it switches back to the previous release, marks the new one bad and, if the old program refuses the migrated schema, restores the backup. Data written since the update is lost in that last case only. |
Who decides when#
| Setting | Server | Edge |
|---|---|---|
Default mode (Updates:Mode) | Approve: an Owner or Admin of the system organisation approves on the Updates page, and system admins are e-mailed when an update is available, fails or rolls back. | Auto: it updates itself, following its own server and never ahead of it. |
Window (Updates:Window) | 02:00 to 05:00 by default | The same, with a random jitter, and it waits for live viewers for up to 24 hours. |
| Other modes | Off still reports that a version is available; Auto installs inside the window. | — |
Updates:PinVersionholds an installation at a version.Updates:KeepReleases(3) controls how many old releases are kept for rollback.- A private release repository needs
Updates:AuthToken. - With Caddy in front, an update is a pause of a few seconds in which requests wait, not an error.
Backups#
Huntress --backup [dir] [--include-blobs] writes a consistent copy of the database with SQLite's VACUUM INTO and verifies it with PRAGMA quick_check. It is safe to run while the server is running. The updater also takes a backup before every update and keeps the last five. Huntress has no built-in scheduled backup: run the command from cron, a systemd timer or Task Scheduler, and copy the result off the machine.
| What | Where | How |
|---|---|---|
| The database | huntress.db | Huntress --backup |
| Pictures | blobs.db | --include-blobs, which copies it only while the server is stopped |
| Signing keys for sign-in cookies | keys/ | Copy the folder. Losing it signs everyone out. |
| The Zigbee network key | zigbee/ | Copy the folder. Losing it means re-pairing every device. |
| Configuration | /etc/huntress or C:\ProgramData\Huntress\config | Copy the folder. It holds your secrets: protect the backup. |
| Video clips | clips/ | Optional; never included by --backup. |
Retention#
An hourly job deletes old data in small batches. It never touches ongoing incidents, rules, sensors or people. A value of 0 keeps that category forever.
| Data | Kept | Setting |
|---|---|---|
| Readings, events, person events, raw requests, rule firings | 90 days | Retention:EventDays |
| Pictures and their detections | 30 days | Retention:ImageDays |
| Resolved incidents | 365 days | Retention:ResolvedIncidentDays |
| Video clips | 30 days | Retention:ClipDays |
Health#
| Endpoint | Answers |
|---|---|
/health/live | Only that the process is up. |
/health/ready | Whether it is ready to serve: the database, the picture store, the listeners, disk and the write buffer. Use this one for monitoring and load balancers. |
/health | The same checks, with detail for a signed-in user. |
/status | Version and uptime; more for a signed-in user. |
- An anonymous caller gets the status word only.
- Degraded (the service still works): under 1 GB of free disk, a full write buffer, a gap in availability, the detection model failing, an unplugged Zigbee dongle.
- Unhealthy: the database or picture store cannot be read, or a listener could not start. A listener that fails does not stop the server.
- On start-up Huntress checks its own health and refuses to start if it is unhealthy.
Logs#
Logs go to the console (the journal under systemd) and to daily files in <DataDir>/logs/, kept 14 days with a 100 MB cap per file. Tokens appear only as their first eight characters, ffmpeg's output is run through the same redaction because it echoes camera URLs, and every request has an X-Request-ID for correlating a report with a log line.
The database and rollbacks#
Huntress stores its data in an embedded SQLite database (WAL mode) plus a second file for pictures. The schema is migrated automatically on start, and migrations only add (tables, columns, indexes), so the previous release still runs on a migrated database and a rollback needs no restore. A release that must break that promise raises a minimum level; an older program then refuses to start with exit code 3 and the updater restores the backup. After a rollback, rules that use newer features are skipped with a warning rather than failing.
The data directory#
| Item | Holds |
|---|---|
huntress.db | Everything except pictures: places, sensors, readings, events, incidents, rules, users. |
blobs.db | Pictures and their detections. |
keys/ | Keys that sign cookies, antiforgery tokens and the sign-in bot trap. |
zigbee/ | The Zigbee network key and state. |
clips/ | Video clips. |
logs/, backups/, update/ | Log files, database backups, updater state and the offline inbox. |
.edge/, models/ | ffmpeg and the detection model, unpacked from the program on first use. |