From the slow traces of 2026-10-13: the directory checked a round's 25 feeds one after another, so two libsyn feeds timing out at 30 s made a round 70 s; a forced scan looked up 62 site icons in turn, 22 s of 62; mjg59.dreamwidth.org takes the connection and never answers, so it ran the full 30 s in every scan it was in; and since #225 strips queries, every YouTube fetch span read the same address, outside any span naming its feed. The directory now checks five at a time (SQLite stays on one connection, so its writes queue). A forced scan looks up the icon in the fetch-ahead task, which already runs six deep, and hands it to scan_one with the body. Both feed clients give up after 15 s of silence (no successful fetch took over 8 s in a day of traces); FEED_TIMEOUT stays 30 s for a feed still sending. The directory's client had no connect timeout; it has ipx's 10 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -61,6 +61,12 @@ pub enum Fetched {
|
||||
/// downloads, which are not, have no such limit.
|
||||
pub const FEED_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
|
||||
|
||||
/// The longest a feed's server may send nothing, for the clients that fetch feeds to set: one
|
||||
/// that took the connection and never answered ran the whole 30 s, every scan it was in (#230).
|
||||
/// No feed that answered took over 8 s in a day of traces; one that keeps sending, however
|
||||
/// slowly, still has until FEED_TIMEOUT.
|
||||
pub const FEED_SILENCE: std::time::Duration = std::time::Duration::from_secs(15);
|
||||
|
||||
/// Feeds fetched and the bytes of their bodies, since the process started, for /metrics to set
|
||||
/// beside memory (#224): a body is read whole, so a night of large feeds is a night of large
|
||||
/// allocations.
|
||||
|
||||
Reference in New Issue
Block a user