The daemon ticked every 60 s and ran a scan pass each time: the sweep, then a check-state query
per feed (about 180) to find which were due. In the six hours before, 293 of 362 passes found
nothing due. Now, after each pass, it works out when the earliest feed is due (due_at, shared
with the scan's own check, over Db::http_states, one query) and sleeps until then: at least
30 s, so a feed that never gets a check time cannot spin it, and at most 10 minutes, so what no
command announces, ipx add or a shorter schedule, is picked up. Commands still wake it at once,
and the first pass after starting runs straight away, as the tick's did. The scan reads every
feed's state in one query too.
The prod-check skill says what to expect now: tens of scans in six hours, and pending as the
real queue.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From Dash0's structured logging guide, what applies here:
- Each JSON line inside a traced span ends with its trace_id and span_id, so a line in Loki leads
to its trace in Tempo; the access log is written inside its request's span so it has one too.
The JSON formatter takes no extra fields, so WithTrace appends them to the object it writes.
- A feed or download failure carries error.type (the HTTP status, or dns, redirect_loop,
timeout, ...) and http.response.status_code, from failure_kind beside explain_failure, so
failures group by kind without a regex over msg.
- Each event was logged twice: words under ipx::scan and fields under ipx::io. It is now one
line under ipx::scan with both; the wire copy is at debug, for the admin page's Daemon I/O tab,
and out of production's log. The healthcheck's status reply stays under ipx::io.
- The access log's ms is duration_ms. The dashboard and the prod-check skill follow.
- error fields are Display with the anyhow chain everywhere, not a mix of Debug and Display.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
.claude/skills/ipx-prod-check: what production's JSON log and traces
carry, the queries that find trouble (warnings grouped, failing feeds and
downloads, 5xx and slow routes, whether the worker keeps up, slow and
failed traces), how to tell a publisher's dead feed from an ipx bug, and
filing what is found as issues per CLAUDE.md. query.py beside it runs the
LogQL and TraceQL through a throwaway container on the monitoring
network, since Loki and Tempo publish no query port.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
