Reviewing what production's traces and metrics leave out turned up
twelve gaps; each is an issue.
Metrics. A Prometheus histogram for ipx and the directory both
(ipx_directory::metrics::Histograms), which ipx's request histogram
(#364) now uses too.
- ipx: each statement's time once it has a connection, by its first
word, from SeaORM's metric callback; and the wait for one, for
hand-written SQL in db.rs, measured as the call less the statement
(the callback runs in the poll that returns the result, so the time
read after the await is that statement's) (#367). SeaORM's own queries
are timed but their wait is not seen: a shortcut, noted.
- ipx: counters of what the events say, counted in emit, which every
event passes once: feed failures by error.type, new items, downloads
saved and failed, DeadAir's seconds found (#368).
- ipx: the page's list_shown, play_start and stall timings (#369).
- directory: requests by method, route (a route layer puts the matched
route on the response, or each feed's page was a row) and status;
feeds checked and failed; snapshots taken (#370).
- ipx-web: Caddy's own metrics on METRICS_BIND (:9180), served apart from
the port the tunnel reaches. `servers { metrics }`, which Caddy 2.11
calls deprecated, because the browser tests' Caddy is 2.6 (#371).
Traces.
- A list's sync is list_sync with steps, an OPML upload opml_import
(#372). Doing so showed sync_group reading every account and its
subscription to the list once per listed feed; read once now (#377).
- Each command the worker runs, from the socket, the CLI or the page,
is a command span with its job inside (#373).
- The daemon's start is a trace, from opening the database through the
catalogue's repairs to "daemon started" (#374).
- The directory's lines carry trace_id and span_id: WithTrace moves to
ipx_directory::otel and both use it (#375).
- A tap or key is its trace's root span (#376). The page names a root
span id in each request's traceparent, where the parent was random and
pointed at nothing, and reports the tap (ids, start, its last answer)
when the next begins or the page is hidden. opentelemetry 0.33 gives a
span builder no way to take ids, so the shared tracer provider has an
id generator that takes given ids inside with_ids, on one thread.
Alerts. grafana/alerts.py, Grafana alerting provisioning beside the
dashboard (#365): ipx, the directory or ipx-web down, scans stopped, a
job over 30 minutes, the pool full, 5xx, 502s, Jev refused, Signals
stopped, the directory's refresher stopped, Tower's load and memory, the
monitoring down, Tempo not reading its blocks, iPX's lines not reaching
Loki. No contact point yet: firing shows in Grafana. Prometheus now
scrapes Tempo, Loki, Alloy and itself (#366), in the monitoring project's
config, not this repository's.
The dashboard has database and page-timing panels; the prod check knows
the new metrics, spans and alerts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The style guide imports types and uses them unqualified. 222 places
wrote a path such as std::collections::HashMap or std::time::Duration
inline; each file now imports the type once, one use a module, and
names it bare. Done with a script, file by file, only where the name is
unambiguous there:
- one path for the name throughout the file, and not a name the file
already binds otherwise (its own types, other imports, a function's
local use);
- not a name generic enough to mislead bare (Error, Result, Value,
Mutex, Client, Request, Response, Command, Event, Context, Map, Ok);
- only code a normal build compiles, so no import is unused outside
tests or names a crate only tests depend on.
193 paths stay, those cases above. No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured against /src/styleguide/rustguide.md:
- Each use statement imports from one module, and free functions are
called through theirs (db::step, db::now, jsonwebtoken::decode,
crate::feed::redact, unix::signal) rather than imported bare.
- unsafe_code is forbidden workspace-wide. The one unsafe was a test's
set_var, which is unsound while other tests' threads read the
environment; the test now reads PATH, which every run has.
- Locks say why they may panic: expect("a panic elsewhere poisoned this
lock") in place of 25 unwraps, and unique_slug's endless search says
it cannot end.
- No combinator chains in for headers: each is a named variable first.
- #[must_use] on the 27 functions clippy's must_use_candidate names.
- A literal bool passed where its meaning is not plain gets its name
(/* keep_open */ true, /* piece */ false, ...).
No behaviour changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The code was formatted by hand, and `cargo fmt --check` differed in 866
places. rustfmt.toml keeps the house's long lines (120) and short
expressions on one line, which halves what changes; this commit is only
what rustfmt does with it, and is named in .git-blame-ignore-revs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The exporter setup and the traceparent parser move from ipx into ipx_directory::otel, which ipx
already depends on, so both binaries share them. The directory makes a span a request, a child of
the caller's traceparent, and a trace a refresher round with a span a feed; ipx sends the current
span's traceparent on every call to the directory and wraps each share in a span. The dashboard's
Traces row and the prod-check skill cover all three services.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
