Insert only a feed's new items and files on a scan (#96)
The spans added in 2799704 showed it: in a full scan of 134 feeds (trace da9a419b...,
2026-09-29 17:31, 315 s), storing items took 117 s, fetching 44 s and every other database call
about 2 s together. A scan inserted every item and file the feed listed, stored or not, one
round trip of about 10 ms each; Clarkesworld's 1200 items took 13 s. It now reads the feed's
stored guids and file URLs once (Db::stored_items) and inserts only the rest. A file URL not
among the feed's own may still be another feed's, so that one still goes to the insert, which
finds it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -48,6 +48,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- Checking a feed with a long history is much quicker: items already stored are no longer written again on every check.
|
||||
- A scan no longer asks a feed's website for its icon every time; it asks again when the artwork changes or you refresh the feed.
|
||||
- The feed list loads several times faster: it was asking the database six questions per feed.
|
||||
- Artwork a feed offers only over plain http shows on the https site too.
|
||||
|
||||
Reference in New Issue
Block a user