A failing YouTube feed is asked again hourly for a day (#386)
#382's retries run (each failing fetch takes the two seconds) and still end 404: YouTube fails about 70% of feed reads from Tower, in streaks longer than seconds, for ?user= and ?channel_id= addresses alike (142 and 68 failures in 7 days across 26 and 13 feeds). The 39 YouTube channels are catalogue feeds nobody subscribes to, read once a day, so each missed about 5 days in 7. due_at now waits at most an hour for a YouTube feed in the first day of a run of failures. A channel gone for good costs 24 requests, then backs off as any feed does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -49,9 +49,14 @@ const YOUTUBE_RETRY_AFTER: Duration = Duration::from_secs(1);
|
||||
/// milliseconds, while the next request for the same feed reads; checked once a day, a channel
|
||||
/// then missed most days (#382).
|
||||
fn youtube_spurious(url: &reqwest::Url, status: StatusCode) -> bool {
|
||||
url.host_str() == Some("www.youtube.com")
|
||||
&& url.path() == "/feeds/videos.xml"
|
||||
&& (status == StatusCode::NOT_FOUND || status.is_server_error())
|
||||
is_youtube_feed(url.as_str()) && (status == StatusCode::NOT_FOUND || status.is_server_error())
|
||||
}
|
||||
|
||||
/// A YouTube channel's or playlist's feed, which fails most reads in streaks that outlast
|
||||
/// `fetch`'s retries, so a scheduler asks it again sooner than other feeds (#386).
|
||||
#[must_use]
|
||||
pub fn is_youtube_feed(url: &str) -> bool {
|
||||
reqwest::Url::parse(url).is_ok_and(|u| u.host_str() == Some("www.youtube.com") && u.path() == "/feeds/videos.xml")
|
||||
}
|
||||
|
||||
/// A response's body, up to `MAX_BODY`: past it, an error, and the rest is never read.
|
||||
|
||||
Reference in New Issue
Block a user