prod-check names each trace's service and an event's fields (#387)

Two 10 s error traces named `poll` read as unexplained in a check: they
are hnopml's, a Hacker News fetch timing out, which the event's own
`error` field said all along. `traces` now prints the root service,
`trace` marks where the service changes and prints an event's fields,
and the skill's error-trace search keeps to ipx's three services.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-06 23:44:51 +00:00
parent 1b2343cf28
commit 8a66f9c7ee
2 changed files with 21 additions and 7 deletions

View File

@@ -185,10 +185,12 @@ a count says something happened, the lines and traces say why.
R='resource.deployment.environment.name="production" && resource.service.name="ipx"'
$Q spans "{$R && span.http.route != \"\" && span.http.route != \"/api/events\" && span.http.route !~ \"/media/.*\" && duration > 1s} | select(span.http.route)"
$Q spans "{$R && name=\"step\" && duration > 500ms} | select(span.of)"
$Q traces '{resource.deployment.environment.name="production" && status = error}'
$Q traces '{resource.deployment.environment.name="production" && resource.service.name=~"ipx|ipx-web|ipx-directory" && status = error}'
$Q trace <id>
```
Tempo keeps about two weeks. A trace id from a log line that Tempo answers 404 for, or a
Other services on Tower send production traces too (hnopml's `poll`, a Hacker News fetch that
times out now and then): `traces` names each trace's service and `trace` prints an event's
fields (its `error`), so say whose a failure is before reading on. Tempo keeps about two weeks. A trace id from a log line that Tempo answers 404 for, or a
search that finds only the last quarter of an hour, is Tempo not reading its stored blocks: see
the monitoring section.