A scan fetches six feeds ahead of its turn, so channels next to each
other alphabetically (android-authority-2, android-developers, apple,
architectural-digest-2) went to YouTube together and failed together,
while one asked alone a minute later read. feed::fetch now takes a turn
before each request for a YouTube feed: one at a time per process, two
seconds after the last, its wait a `youtube_turn` span. The retries
take their turn too, in place of their one-second sleep. The directory
builds the same fetch and paces its own checks the same way.
39 channels due together add up to 80 s to that scan.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#382's retries run (each failing fetch takes the two seconds) and still
end 404: YouTube fails about 70% of feed reads from Tower, in streaks
longer than seconds, for ?user= and ?channel_id= addresses alike (142
and 68 failures in 7 days across 26 and 13 feeds). The 39 YouTube
channels are catalogue feeds nobody subscribes to, read once a day, so
each missed about 5 days in 7.
due_at now waits at most an hour for a YouTube feed in the first day of
a run of failures. A channel gone for good costs 24 requests, then backs
off as any feed does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
YouTube answers some requests for a channel's feed with a 404 or 500
in tens of milliseconds, while the next request for the same feed, or
another channel's 200 ms later, reads. Checked once a day, the 39
YouTube channels in the catalogue missed most days: 178 failures in
the week to 2026-11-04, against 74 in the ten days before, and all 39
read from Tower when asked by hand.
feed::fetch now asks a youtube.com/feeds/videos.xml address again, up
to twice, a second apart, when it answers 404 or 5xx. Other hosts'
404s are real and are not retried. tokio moves from ipx-feed's dev
dependencies to its own, for the sleep; both services build it already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Herd with Colin Cowherd's feed is 62 MB, its whole back catalogue, and had failed every check
since the 50 MB cap of #295 came in on 2026-10-19. The cap was there for an MP3 given as a feed;
200 MB still stops that, at four times the largest real feed seen.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lib.rs held 2,000 lines: fetching, parsing, OPML, Patreon, discovery,
artwork, categories, Jev and redaction, and their tests. It is now the
module list and the re-exports, so ipx and the directory call
feed::fetch, feed::parse and the rest as before:
fetch.rs fetch and its limits parse.rs RSS and Atom
outlines.rs OPML (not opml.rs, which would shadow the opml crate)
patreon.rs a creator's shows discover.rs finding a feed
wordpress.rs WordPress.com's API failure.rs failures in words
privacy.rs redact, looks_private artwork.rs https art, icons
category.rs Apple's categories jev.rs Jev's category
Moved verbatim, private items pub(crate) where a sibling calls them,
calls across modules module::function. Every definition and doc comment
line of the old lib.rs is in exactly one of the new files; each test
went with what it tests, and all 40 run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
