A failing YouTube feed is asked again hourly for a day (#386)

#382's retries run (each failing fetch takes the two seconds) and still
end 404: YouTube fails about 70% of feed reads from Tower, in streaks
longer than seconds, for ?user= and ?channel_id= addresses alike (142
and 68 failures in 7 days across 26 and 13 feeds). The 39 YouTube
channels are catalogue feeds nobody subscribes to, read once a day, so
each missed about 5 days in 7.

due_at now waits at most an hour for a YouTube feed in the first day of
a run of failures. A channel gone for good costs 24 requests, then backs
off as any feed does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-06 23:44:51 +00:00
parent d3b8c4cd01
commit bbcf69062e
4 changed files with 31 additions and 6 deletions

View File

@@ -49,9 +49,14 @@ const YOUTUBE_RETRY_AFTER: Duration = Duration::from_secs(1);
/// milliseconds, while the next request for the same feed reads; checked once a day, a channel
/// then missed most days (#382).
fn youtube_spurious(url: &reqwest::Url, status: StatusCode) -> bool {
url.host_str() == Some("www.youtube.com")
&& url.path() == "/feeds/videos.xml"
&& (status == StatusCode::NOT_FOUND || status.is_server_error())
is_youtube_feed(url.as_str()) && (status == StatusCode::NOT_FOUND || status.is_server_error())
}
/// A YouTube channel's or playlist's feed, which fails most reads in streaks that outlast
/// `fetch`'s retries, so a scheduler asks it again sooner than other feeds (#386).
#[must_use]
pub fn is_youtube_feed(url: &str) -> bool {
reqwest::Url::parse(url).is_ok_and(|u| u.host_str() == Some("www.youtube.com") && u.path() == "/feeds/videos.xml")
}
/// A response's body, up to `MAX_BODY`: past it, an error, and the rest is never read.