Faster scans and directory rounds, and fetches named by feed (#227, #228, #229, #230)

From the slow traces of 2026-10-13: the directory checked a round's 25 feeds one after
another, so two libsyn feeds timing out at 30 s made a round 70 s; a forced scan looked up
62 site icons in turn, 22 s of 62; mjg59.dreamwidth.org takes the connection and never
answers, so it ran the full 30 s in every scan it was in; and since #225 strips queries,
every YouTube fetch span read the same address, outside any span naming its feed.

The directory now checks five at a time (SQLite stays on one connection, so its writes
queue). A forced scan looks up the icon in the fetch-ahead task, which already runs six
deep, and hands it to scan_one with the body. Both feed clients give up after 15 s of
silence (no successful fetch took over 8 s in a day of traces); FEED_TIMEOUT stays 30 s
for a feed still sending. The directory's client had no connect timeout; it has ipx's 10 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 13:27:19 +00:00
parent cece114d98
commit 8d7a30d1f9
7 changed files with 72 additions and 10 deletions

View File

@@ -12,6 +12,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Changed
- Check every feed looks up the site icons of feeds without artwork a few at a time, alongside the fetches, instead of one after another (#228).
- A feed's server that sends nothing for 15 s is given up on, instead of holding a scan for the full 30 s; a feed that keeps sending still has 30 s (#230).
- A scan's trace names the feed each fetch is for (#229).
## [1.1.0] - 2026-10-13
### Added