The feed list counts every item of every feed on the server, about 70% of its time #165

Closed
opened 2026-10-08 19:07:17 -07:00 by rays · 1 comment
Owner

Found 2026-10-09 from the traces #164 added. GET /api/feeds took 268 ms in trace 824ff29978858e0cdf7f147386369ba7 and 426 ms in d25064fa325f17502fbdba825133a8d9; feed_list was 259 and 415 ms of that, and within it:

  • entries: 184 ms, 316 ms
  • downloaded: 37 ms, 50 ms
  • unread: 26 ms, 32 ms
  • feeds: 10 ms, 14 ms; the catalogue, subscriptions, subscriber counts, pins: 1-4 ms each

The entries step is SELECT feed_id, count(*) FROM entries WHERE (CAST($1 AS TEXT) IS NULL OR feed_id = $1) GROUP BY feed_id (src/db.rs, Db::feed_list): it counts every item of every feed in the database, 1534 feeds, for a person who subscribes to 146, on every feed list and every live update of one. downloaded does the same over every file row. The unread count already joins subscriptions and costs a fraction.

The Directory (web::popular) calls feed_list too, for titles, artwork and categories, and uses none of the counts.

Found 2026-10-09 from the traces #164 added. GET /api/feeds took 268 ms in trace 824ff29978858e0cdf7f147386369ba7 and 426 ms in d25064fa325f17502fbdba825133a8d9; feed_list was 259 and 415 ms of that, and within it: - entries: 184 ms, 316 ms - downloaded: 37 ms, 50 ms - unread: 26 ms, 32 ms - feeds: 10 ms, 14 ms; the catalogue, subscriptions, subscriber counts, pins: 1-4 ms each The entries step is `SELECT feed_id, count(*) FROM entries WHERE (CAST($1 AS TEXT) IS NULL OR feed_id = $1) GROUP BY feed_id` (src/db.rs, Db::feed_list): it counts every item of every feed in the database, 1534 feeds, for a person who subscribes to 146, on every feed list and every live update of one. downloaded does the same over every file row. The unread count already joins subscriptions and costs a fraction. The Directory (web::popular) calls feed_list too, for titles, artwork and categories, and uses none of the counts.
rays added the bug label 2026-10-08 19:07:17 -07:00
Author
Owner

Fixed in 92cc0cf: the item and file counts join the person's subscriptions, as the unread count did, so they read only their feeds' rows. That took the item count from 184-316 ms to 4-5 ms, but the file count went to 58 ms, reading every file row of 146 feeds for the few with a file; 65ca481 adds a partial index on the rows with a file on disk (enclosures_on_disk), and it is 1 ms. GET /api/feeds in production: 250-430 ms before, 41-54 ms after (trace 998185d1bc24a2a68206d87d24c71a1d: entries 5, downloaded 1, unread 23, feeds 12). The Directory went from 0.3-0.4 s to 0.16 s. Deployed as 0.10.3-dev.

Fixed in 92cc0cf: the item and file counts join the person's subscriptions, as the unread count did, so they read only their feeds' rows. That took the item count from 184-316 ms to 4-5 ms, but the file count went to 58 ms, reading every file row of 146 feeds for the few with a file; 65ca481 adds a partial index on the rows with a file on disk (enclosures_on_disk), and it is 1 ms. GET /api/feeds in production: 250-430 ms before, 41-54 ms after (trace 998185d1bc24a2a68206d87d24c71a1d: entries 5, downloaded 1, unread 23, feeds 12). The Directory went from 0.3-0.4 s to 0.16 s. Deployed as 0.10.3-dev.
rays closed this issue 2026-10-08 19:17:37 -07:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rays/ipx#165