The Signals judge scans the whole entries table on every pass, outside any trace #274

Closed
opened 2026-10-17 06:58:32 -07:00 by rays · 1 comment
Owner

Db::unjudged filters on coalesce(published, first_seen) over every subscribed feed's entries, and no index covers that, so each pass starts with a Parallel Seq Scan of entries (605 MB on production): EXPLAIN (ANALYZE, BUFFERS) took 566 ms on 2026-10-17, for 0 rows. It runs on every wake, which is every scan that found a new item (627 feed_done with new > 0 in 16 hours) and every 5 minutes, and it runs before the signals span starts, so no trace shows it. An index on entries (feed_id, coalesce(published, first_seen)) would let it read only recent items of subscribed feeds.

Db::unjudged filters on coalesce(published, first_seen) over every subscribed feed's entries, and no index covers that, so each pass starts with a Parallel Seq Scan of entries (605 MB on production): EXPLAIN (ANALYZE, BUFFERS) took 566 ms on 2026-10-17, for 0 rows. It runs on every wake, which is every scan that found a new item (627 feed_done with new > 0 in 16 hours) and every 5 minutes, and it runs before the signals span starts, so no trace shows it. An index on entries (feed_id, coalesce(published, first_seen)) would let it read only recent items of subscribed feeds.
rays added the bug label 2026-10-17 06:58:32 -07:00
Author
Owner

Fixed in 2077ea5 and 5130f75: index entries_recent on (feed_id, coalesce(published, first_seen)), and entries analysed once when it has no statistics (without them Postgres still chose the scan, 898 ms). EXPLAIN (ANALYZE) of the judge's query on production now: 19.5 ms, Index Scan using entries_recent, was 566 ms. Deployed in ipx 1.3.1-dev.

Fixed in 2077ea5 and 5130f75: index entries_recent on (feed_id, coalesce(published, first_seen)), and entries analysed once when it has no statistics (without them Postgres still chose the scan, 898 ms). EXPLAIN (ANALYZE) of the judge's query on production now: 19.5 ms, Index Scan using entries_recent, was 566 ms. Deployed in ipx 1.3.1-dev.
rays closed this issue 2026-10-17 07:21:19 -07:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rays/ipx#274