Troubleshoot a HypAware installation
Check the installation's status and version:
hyp status
hyp versionRun hyp status and hyp version and tell me what needs attention and the exact commands to fix it.HypAware · Healthy
Daemon Running
Storage 37 MB · 90-day retention
Sharing No organization sync
Clients Status Data
claude Recording (otel) Local only
codex Not recording Local only
To record codex again: hyp client attach codex
No issues detected.
More details: hyp status --verbose
hypaware 1.41.0
node: v26.10.0
platform: darwin arm64hyp status reports the background service, clients, storage, and problems,
with guidance to help resolve them. Use hyp status --verbose for more detail
or hyp status --json for scripts.
No new recordings
hyp client status
hyp daemon status
hyp cache statusHypAware is not recording new sessions. Run hyp client status, hyp daemon status, and hyp cache status, work through the checklist on the troubleshooting page, and tell me what is wrong.- Check that the intended source is configured. Add it through
hyp setupif needed. - Check that the client is attached. Preview and apply its attach using
hyp attach CLIENT --dry-runandhyp attach CLIENT. - Check the daemon is running. Use
hyp daemon startfor an installed service, or follow headless setup when there is no service manager. - Start a new client process and complete a turn so it uses the managed settings.
Ollama attach enables recording and prints recipes without editing client files
or shell/service
OLLAMA_HOST; use its explicit CLI host or SDK constructor. - Check
hyp privacy show /absolute/path/to/projectfor an ignore marking.
For Claude Code, hyp client status claude reports the configured and live
listener endpoints, recent telemetry, and transcript activity. If the endpoints
differ, run hyp attach claude, restart the daemon, and start a new Claude Code
process. Check the Claude Code requirements
for minimum versions.
For transcript-only clients, allow time for the scheduled recovery sweep.
For Factory Droid, restart Droid after attaching so it loads the native hooks.
Check hyp client status droid, then preview recovery with
hyp backfill droid --dry-run. The collector must have access to the same
Factory home as Droid. See Droid troubleshooting and limits
for legacy hook files, missing transcripts, and capture bounds.
Detaching a gateway-routed client restores its own connection settings if a stopped gateway is preventing normal use where that integration manages them. Ollama detach leaves the capture host unchanged. With its collector stopped, launch the CLI at the preserved direct root or construct a new SDK object with that root using direct recovery.
Goose records nothing
Goose capture reads Goose's session database, so hyp status is where its
health shows. hyp client status does not list Goose. hyp status prints a
Goose line only when something is wrong. The ready messages, the last two rows
below, show in hyp status --json as the goose entry's health.message.
| Goose status says | What to do |
|---|---|
cannot read the database at <path> |
The daemon's user cannot read the file. Run HypAware as the same user as Goose. |
failed to open goose sessions.db at <path> |
The file exists but SQLite could not open it, for example because it is damaged. Check that Goose itself can still open its sessions. |
database busy, retrying next poll |
Goose was writing when HypAware read. It retries on the next poll; if this persists, check for a stuck Goose process. |
goose sessions.db has no <table>.<column> column |
Your Goose version uses a database schema this HypAware release does not support. Nothing is recorded until you upgrade HypAware to a release that supports it. |
goose source requires Node >= 22.12 |
Goose capture reads the database with Node's built-in SQLite. Upgrade the Node.js that runs HypAware, then run hyp daemon restart. |
no goose installation detected at <path> |
HypAware found no database at that path. If you set GOOSE_PATH_ROOT or keep the database elsewhere, set sessions_db as shown on the Goose page, then run hyp daemon restart. Sessions from Goose before 1.10.0 are not supported. |
polling <path> every 60000ms, no sessions yet |
Capture is ready. Finish a Goose turn and wait up to a minute. |
New messages arrive within a minute. A change Goose makes to a session it
already wrote, such as rewriting earlier messages, arrives with the scheduled
import, up to 15 minutes later. Run hyp backfill goose to pick it up at once.
Crush records nothing
Crush capture reads each project's Crush database, so hyp status is where
its health shows. hyp client status does not list Crush. hyp status prints
a Crush line only when something is wrong. The ready messages, the last four
rows below, show in hyp status --json as the crush entry's
health.message.
| Crush status says | What to do |
|---|---|
cannot read the crush index at <path> |
projects.json exists but could not be read or parsed. HypAware keeps reading the projects it last found there. Check that Crush itself still starts; Crush rewrites the index when it opens a project. |
<N> crush projects skipped over the 1,024 limit |
HypAware reads at most 1,024 project databases. Remove projects you no longer use from Crush's index, or list the ones you want in data_dirs. |
crush.db has no <table> table (<N> crush databases) or crush.db has no <table>.<column> column (<N> crush databases) |
Your Crush version uses a database schema this HypAware release does not support. A table named goose_db_version is Crush's own migration record, not a Goose problem. Nothing is recorded from those databases until you upgrade HypAware to a release that supports it. |
cannot read <N> crush databases |
The daemon's user cannot open those databases, or the Node.js that runs HypAware has no built-in SQLite. Or a database is damaged; check that Crush still opens that project. Run HypAware as the same user as Crush, and use Node 22.12 or later; hyp backfill crush prints crush capture requires Node >= 22.12 in the second case. Then run hyp daemon restart. |
crush poll failed: <kind> |
HypAware retries on the next poll. If it persists, run hyp daemon restart. |
<N> crush databases busy, retrying next poll |
Crush was writing when HypAware read. It retries on the next poll; if this persists, check for a stuck Crush process. |
<N> sessions appear in more than one crush database |
A project's .crush directory was copied, so two databases hold the same session. HypAware records each such session from the first database it read. Remove the copy, or mark it with hyp privacy ignore <dir>. |
no crush installation detected at <path> |
HypAware found no index at that path. If you set CRUSH_GLOBAL_DATA or XDG_DATA_HOME, set data_dirs as shown on the Crush page, then run hyp daemon restart. |
crush index at <path> lists no projects yet |
Capture is ready. Start Crush in a project. |
polling <N> crush projects every 60000ms, no sessions yet |
Capture is ready. Finish a Crush turn and wait up to a minute. Older sessions, and the first session in a project HypAware has not read before, arrive with the scheduled import within 15 minutes, or now with hyp backfill crush. |
crush ongoing capture disabled (crush.enabled = false) |
Capture is off in your configuration. Set crush.enabled to true, then run hyp daemon restart. |
A status that ends ; <N> listed projects have no crush.db names index
entries or data_dirs whose database is gone, such as a deleted project.
They are skipped and need no action.
New messages arrive within a minute. A change Crush makes to a message it
already finished arrives with the scheduled import, up to 15 minutes later.
Run hyp backfill crush to pick it up at once.
Reading a database can leave an empty crush.db-wal and a 32 KiB
crush.db-shm in the project's .crush/ directory until Crush next closes it
cleanly. These are SQLite's own index files and hold no content.
Mastra Code records nothing
Mastra Code capture reads Mastra Code's mastra.db, so hyp status is where
its health shows. hyp client status does not list Mastra Code. hyp status
prints a Mastra Code line only when something is wrong. The ready messages,
the last three rows below, show in hyp status --json as the mastracode
entry's health.message.
| Mastra Code status says | What to do |
|---|---|
cannot read the mastracode database at <path> |
The daemon's user cannot read the file, or SQLite could not open it, for example because it is damaged. Run HypAware as the same user as Mastra Code, and check that Mastra Code itself still opens its threads. |
mastracode database busy, retrying next poll |
Mastra Code was writing when HypAware read. It retries on the next poll; if this persists, check for a stuck Mastra Code process. |
mastra.db has no <table> table or mastra.db has no <table>.<column> column |
Your Mastra Code version uses a database schema this HypAware release does not support. Nothing is recorded until you upgrade HypAware to a release that supports it. |
SQLite <version> cannot read Mastra Code's JSONB metadata (needs 3.45 or newer) |
The Node.js that runs HypAware has a built-in SQLite too old to read Mastra Code's thread metadata. Upgrade that Node.js, then run hyp daemon restart. |
mastracode source requires Node >= 22.13 (node:sqlite builtin not available) |
Mastra Code capture reads the database with Node's built-in SQLite. Upgrade the Node.js that runs HypAware, then run hyp daemon restart. |
MASTRA_DB_URL is set; remote stores are not supported |
Mastra Code writes to a remote Turso or Postgres store, which HypAware cannot read. Nothing is recorded while MASTRA_DB_URL is set in the daemon's environment. |
mastracode poll failed: <kind> |
HypAware retries on the next poll. If it persists, run hyp daemon restart. |
session exclusions could not be read, session exclusions could not be loaded, or Session exclusions could not be loaded; capture is disabled. ... |
HypAware could not read its saved session exclusions under <HYP_HOME>/hypaware/session-ignores/, so it records nothing rather than risk recording an ignored thread. hyp backfill mastracode fails with Mastra Code session exclusions are unreadable for the same reason. Check that the daemon's user can read that directory and its files, then run hyp daemon restart. |
no mastracode installation detected at <path> or no mastra.db at the resolved path |
HypAware found no database at that path. If you set MASTRA_DB_PATH, MASTRA_APP_DATA_DIR or (on Linux) XDG_DATA_HOME, or moved the database, set db_path as shown on the Mastra Code page, then run hyp daemon restart. |
polling mastracode every 60000ms, no threads yet or polling mastracode every 60000ms |
Capture is ready. The , no threads yet ending drops once the poll has recorded a row since the daemon started, and comes back after a restart until the next new row. With no rows yet, finish a Mastra Code turn and wait up to a minute. |
mastracode ongoing capture disabled (mastracode.enabled = false) |
Capture is off in your configuration. Set mastracode.enabled to true, then run hyp daemon restart. |
New messages arrive within a minute. A change Mastra Code makes to a message
it already wrote arrives with the scheduled import, up to 15 minutes later.
Run hyp backfill mastracode to pick it up at once.
Ollama capture is unconfirmed
hyp ollama setup
hyp client status ollama
hyp status --verboseCheck the Direct upstream, service/model inventory and Capture root (live)
separately. An executable on PATH does not prove a service or model is ready.
Use a model already listed and the supported CLI/SDK recipes in
clients. A configured or stale capture root
is not a live route. After saved endpoint changes with an unconfirmed restart,
run hyp daemon restart and check setup again.
| Result or reason | Next step |
|---|---|
| Ready, no traffic | Route the next client to the printed live /ollama root, or explicitly run hyp ollama verify --model NAME. |
| Observed, awaiting persistence | Give the processor time to finish; use verify for a fresh committed-pair check. |
| Persisted stamp | The append resolved. Verify or query the particular request to establish current visibility. |
| Off or historical evidence | Successful detach keeps the running host forwarding without capture. Use hyp client attach ollama to resume new requests. |
unsupported_shape |
Use native chat/generate with think=False, no nonempty thinking/tools, and supported fields. Unsupported exchanges are omitted in full. |
malformed_stream, missing_terminal, trailing_record |
The stream must finish with exactly one completed terminal and no later nonblank records. Retry the request explicitly after fixing its producer. |
http_error, upstream_unavailable, transport_error |
Check the direct Ollama service and saved custom upstream with setup. Direct inference and capture are separate outcomes. |
capture_limit, processor_unavailable, append_failure |
Check the ordinary daemon log and processing health. Lost capture is not replayed. |
policy_unreadable |
Restore readable, valid policy/configuration. Verification does not bypass policy. |
| Unconfirmed attach/detach | Follow the command's recovery guidance and retry for live confirmation. A saved setting alone is not a completed barrier. |
hyp ollama verify --model NAME --json returns machine-readable status, reason and
whether HTTP completed; its prompt/export disclosure goes to stderr. A
persistence_timeout means inference completed but storage confirmation timed
out. The live collector may finish; settings are unchanged. A later check can
report settlement_busy until the original storage operation actually settles.
The command never retries inference automatically. If the actual collector stops
or its processor is lost, confirmation can be interrupted and remains unconfirmed.
Verification refreshes the shared current gateway spool, including earlier and other-client pending rows, with normal storage/sink hooks. Its cost depends on that backlog. The disposable query worker only reads committed data with normal caller policy and disables legacy query-config migration. A receipt, spool row, HTTP result, old saved stamp or hidden row cannot establish verification success.
Default status carries bounded capture evidence and finite recent failure reasons. These summaries can lag until the next processor status update. Check the ordinary daemon log for a safe failure reason while waiting; it also carries projection, append and lifecycle events. No prompt or response text is needed to diagnose these reasons. Optional dev telemetry can supply structured run/exchange correlation, but is not required for the ordinary status/log path. See diagnostic logs.
When the collector has stopped, use the preserved direct-host CLI command or a
new Client(host=...) from direct recovery.
This restores inference without recording. Off/stopped historical timestamps do
not claim current health. Ollama has no history import or outage replay, and its
unknown directory means directory exclusions cannot protect these API requests.
History or search results are missing
An overview covers the dates printed in its heading. A search may be limited by dates, hit count, searchable columns, privacy, or local retention.
- Confirm the correct
HYP_HOMEand local versus remote target. - Read stderr for withheld-row, freshness, and truncation notices.
- Use
hyp backfill listand a provider-specific backfill dry run to check whether older transcripts are available. Cache refresh is not a backfill. - Refresh the affected local dataset with
hyp cache refresh ai_gateway_messageswhen freshness warnings call for it. - Use
--output ./results.jsonwith--format jsonfor complete query output; this does not lift your query's ownLIMIT.
See querying for searches, session drill-downs, and bounded SQL.
History import failed
hyp status shows this problem when a provider's scheduled history import
failed on its last run:
codex history import failed on its last run, so some history may be missing. Rows read before the failure may have been imported; the command below shows the cause. The next scheduled run retries, and this clears once a run succeeds.Rows read before the failure may have been imported. Every scheduled run retries, and the warning clears on its own once a run succeeds. To see the error, and the failing file if there is one:
hyp backfill codex --dry-runIf the error is a permission problem, fix it and the next run picks the file up. If the same file keeps failing, the file cannot be imported as it is, for example a Codex session file over its size limits. Moving that file out of the session directory clears the warning on the next run, but HypAware will not import it and Codex will no longer list that session, so keep the file somewhere safe.
A Claude import that cannot read OpenCode's session list stops before it
writes anything, so it never records OpenCode's Claude sessions as your own.
hyp backfill claude --dry-run shows why. If it says Node 22.13 or newer is
needed, upgrade Node; otherwise fix the problem it names, such as the
permissions on OpenCode's data directory.
An export or team sync is missing
hyp status
hyp sync --dry-run
hyp remote listRun hyp status, hyp sync --dry-run, and hyp remote list, and tell me why my recordings are not reaching HypAware Cloud.Check that a sink exists, its destination is correct, and the rows are eligible
to leave. An attended first-sync review hold, local-only folder or client
marking, or export retry can explain why local data is not yet in HypAware
Cloud. After reviewing the destination plan, use hyp sync to attempt
delivery and inspect each sink's result. It can partially succeed and still
return failure.
Cloud sync gives each upload chunk a 330-second limit, including authentication,
the request, response reading and retry waits. Retries share that limit;
a healthy sync with several chunks can take longer overall. Timeout or stop
cancels the current Cloud request and waits. Unacknowledged rows remain eligible
for a later export while they remain in the local cache. Check connectivity
and the destination's failure reason, then retry with hyp sync after fixing it.
An export warning clears only when that same configured destination records a full successful completion strictly after its failure was recorded. Starting an upload, acknowledging some rows, queuing a follow-up or succeeding at another destination does not clear it. The warning can remain after 24 hours; equal timestamps do not establish recovery. Ordinary daemon logs record failure and later recovery separately. See diagnostic logs.
The verbose status recent errors count describes retained history from the
last 24 hours, including recovered failures. It is not a count of unsent rows
or every failure the installation has ever had. A warning can clear while this
count stays nonzero. Deleting failure records or letting them age does not
deliver data; retries use the cache and saved export progress. See
storage paths for the diagnostic cap
and cleanup limits.
Remote sign-in with --no-forward grants query access without setting up
sync. hyp leave stops sync but keeps local recordings.
See team setup and privacy controls.
Rows skipped by exports before v1.42.0
Before v1.42.0, a hyp command that refreshed the cache while the daemon was
also writing could commit rows below the point a destination had already
exported to. Those rows stay in the local cache but no scheduled export sends
them (issue 2225). v1.42.0
stops new skips once the daemon restarts on it; hyp daemon restart makes
sure. Nothing finds the rows already skipped, so recovery is a manual step for
each destination. It reaches only rows the local cache still holds: rows that
retention has removed are gone from this host.
For HypAware Cloud, hyp sync --history <client> resends one client's
retained ai_gateway_messages and leaves normal export progress alone. The
server drops rows it already has, so a rerun is safe. Run it for each client
that records on this host, and read the preview first: the client must be
syncing, and the replay also sends rows it held back while it was local-only.
Folders marked local-only stay withheld. Other datasets, such as traces and
logs, have no Cloud replay.
A local folder or S3 destination has no replay, and it does not drop rows it already has. To export one dataset again in full, stop the daemon, move that dataset's earlier export files out of the destination, remove the destination's saved position for the dataset, then start the daemon and sync:
hyp daemon stop
mv <dir>/<dataset> <kept>/<dataset>
rm -r ~/.hyp/hypaware/plugins/@hypaware/local-fs/sink-instances/<instance>/watermarks/<dataset>
hyp daemon start
hyp sync <instance><instance> is the destination's name in hyp sync --dry-run, <dir> is its
dir setting, and <kept> is any folder outside <dir>. The ~/.hyp path
is the default HYP_HOME; use yours if you set another. For S3, move the
objects under <prefix>/<dataset>/ in the destination's bucket to a prefix
outside the destination instead, and the saved position is under
@hypaware/s3 instead of @hypaware/local-fs.
Move the earlier files before the export, don't skip it. A file is named only
by the range of rows it holds, so the re-export can reuse an earlier file's
name and replace it, and if retention has since removed some of that file's
rows from the local cache, they are lost. After the export, the destination
holds every row the local cache still retains, once. The moved files are the
only copy of rows that retention has removed, so keep them; a row in both
places is the same row, and can be dropped where the data is read, for example
by part_id for ai_gateway_messages. Rows held back as local-only stay held
back.
A command is missing or rejects an option
hyp version
hyp --help
hyp plugin list
hyp query --helpA hyp command or flag from the docs is missing on this host. Run hyp version and hyp plugin list and tell me whether it is an inactive plugin or an old version.Plugin-owned commands appear only while the owning plugin is active. Entering a known inactive command prints a repair instruction. If a flag in these docs is unavailable, check the installed version's command help and the update procedure.
The CLI and daemon run different versions
hyp update
hyp statushyp update checks for a newer release and restarts an installed daemon after
updating. If installation succeeded but restart failed, run
hyp daemon restart. A foreground daemon needs to be relaunched by its owner
or supervisor. Source checkouts and npx-cache copies do not self-update.
For a missing global binary or broken service definition, follow recovery.
Disk usage or maintenance problems
hyp cache status
hyp cache maintain ai_gateway_messages --dry-runUse HypAware to show cache size and what a maintenance pass would remove, without applying it.Check retention and storage paths to distinguish the query
cache, exported files, retry state, and logs. Cache retention does not reclaim
exported copies. Inspect a maintenance dry run before applying it, and use
hyp privacy purge only when you intend to delete recorded data.
A query or graph projection refuses for memory
Execution memory is bounded, and work that outgrows its budget refuses instead of returning a partial answer. The refusal names the budget it hit and, where one applies, the variable that raises it. Each is read from the environment of the process doing the work, in MB:
HYP_QUERY_MAX_HEAP_MB(default 1024) bounds a query you run. Work that carries its own budget, such as a graph read or projection, ignores it.HYP_GRAPH_PROJECTION_MAX_HEAP_MB(default 3072) bounds a graph projection, which scans whole source tables.
Narrow the work first: add a WHERE or date filter, a LIMIT, or aggregate
instead of selecting raw rows. Raise a budget only when the work genuinely
needs the memory, and give it a positive value. A value that is blank or not a
number is ignored and the default applies. At zero or below the two differ:
HYP_QUERY_MAX_HEAP_MB turns the query heap guard off rather than tightening
it, while HYP_GRAPH_PROJECTION_MAX_HEAP_MB is ignored and the projection
default applies.
For a command you run yourself, set it in the same shell:
HYP_GRAPH_PROJECTION_MAX_HEAP_MB=6144 hyp graph projectThe installed daemon does not read your shell. It runs scheduled work, such as
the projection after a GitHub poll, in a child of the service process, and that
child inherits the service environment. hyp daemon install writes no
environment block of its own, so set the variable where your service manager
builds that environment, then restart the daemon so it starts with the new
value.
On macOS (launchd):
launchctl setenv HYP_GRAPH_PROJECTION_MAX_HEAP_MB 6144
hyp daemon restartOn Linux (systemd user service):
systemctl --user set-environment HYP_GRAPH_PROJECTION_MAX_HEAP_MB=6144
hyp daemon restartEither setting applies only to processes started afterwards, which is what the
restart is for, and both are forgotten at logout or reboot. To keep a value,
set it where the login session environment is built: an
~/.config/environment.d/hypaware.conf entry under systemd, or a login-time
launchctl setenv under launchd. Remove one with
launchctl unsetenv HYP_GRAPH_PROJECTION_MAX_HEAP_MB or
systemctl --user unset-environment HYP_GRAPH_PROJECTION_MAX_HEAP_MB, then
restart the daemon again. A foreground hyp daemon run takes the environment of
the shell that started it, so it needs neither step.
Find diagnostic logs
With the default HYP_HOME, start with the structured log at
~/.hyp/hypaware/logs/daemon.log. Service stdout and stderr are in
daemon.out.log and daemon.err.log in the same directory. Processing work has
its own ~/.hyp/hypaware/processing/logs/daemon.log. Use the paths reported for
your installation if you configured another home.
On Windows, the Scheduled Task does not capture the daemon's output, so there
is no daemon.out.log or daemon.err.log; daemon.log is still written.
Run hyp daemon status to see whether the daemon is running. To check the
task itself, run this in PowerShell or Command Prompt, replacing <SID> with
the SID that whoami /user prints (it starts with S-1-5-):
schtasks /Query /TN \hypaware\<SID> /V /FO LISTIts Status line shows whether the task is running, and Last Result shows
how the daemon last exited.
An OTLP exporter refused with 421 Misdirected Request is pointed at a name
other than localhost or 127.0.0.1; point it at one of those instead.
Product telemetry has separate controls; it is
automatic for enrolled organizations, defaults off on standalone installations,
and is not the captured conversations or the local diagnostic log.
An OTLP exporter refused with 413 Content Too Large sent a request whose body
is larger than 64 MiB after decompression. HypAware keeps none of that
request's records, and the exporter does not retry it. The daemon log has no
entry for this refusal, so the error appears only on the sending side. Lower the
exporter's batch size so each request stays under the limit: SDKs that read the
standard variables use OTEL_BLRP_MAX_EXPORT_BATCH_SIZE for logs and
OTEL_BSP_MAX_EXPORT_BATCH_SIZE for traces. The limit is fixed and has no
configuration setting.
When reporting a problem, include the version, failing command, exit status, relevant status repair lines, and a short log excerpt around the failure. Review excerpts for credentials, paths, and captured content before sharing.