Feeds with no artwork of their own ask their website for an icon on every scan #95

Closed
opened 2026-09-29 09:39:40 -07:00 by rays · 2 comments
Owner

A feed read in full looks up its artwork every time (main.rs scan_one, from #80 and #89): the feed's own image is fetched to check it is an image, and a feed without one asks its website for an icon (site_icon). A feed that sends no ETag or Last-Modified is read in full on every scan, so it asks every scan. The ponytail comment there named this ceiling.

In production: looking-for-group spends 2.0s of its 2.2s feed span in site_icon for https://www.lfg.co/ (trace dc356eb2da00b0fe37865b55df9847b8, 2026-09-29 16:14); antirez ("feed") 0.7s. Every scan.

Fix: skip the check when the feed names the same artwork already stored (it passed when stored), and ask the site again only when nothing usable is stored or the scan was asked for (force), so an icon the site changes still follows on a refresh.

A feed read in full looks up its artwork every time (main.rs scan_one, from #80 and #89): the feed's own image is fetched to check it is an image, and a feed without one asks its website for an icon (site_icon). A feed that sends no ETag or Last-Modified is read in full on every scan, so it asks every scan. The ponytail comment there named this ceiling. In production: looking-for-group spends 2.0s of its 2.2s feed span in site_icon for https://www.lfg.co/ (trace dc356eb2da00b0fe37865b55df9847b8, 2026-09-29 16:14); antirez ("feed") 0.7s. Every scan. Fix: skip the check when the feed names the same artwork already stored (it passed when stored), and ask the site again only when nothing usable is stored or the scan was asked for (force), so an icon the site changes still follows on a refresh.
rays added the bug label 2026-09-29 09:39:40 -07:00
Author
Owner

Fixed in the commit after e8fd3fe (#95, #96): the artwork check and the site-icon lookup run only when the feed names different artwork from what is stored, or the scan was forced by a refresh. Deployed 2026-09-29 17:28 UTC.

Fixed in the commit after e8fd3fe (#95, #96): the artwork check and the site-icon lookup run only when the feed names different artwork from what is stored, or the scan was forced by a refresh. Deployed 2026-09-29 17:28 UTC.
rays closed this issue 2026-09-29 10:32:34 -07:00
Author
Owner

That commit is 2799704.

That commit is 2799704.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rays/ipx#95