A feed whose server hangs holds up the whole scan: fetches have no timeout #108
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The reqwest client (main.rs, Ctx::client) has no timeout, and feed::fetch sets none, so a fetch waits as long as the far server does. Scans handle feeds in order (#104), so everything waits behind it. Seen 2026-10-01/02: matthew-garrett (https://mjg59.dreamwidth.org/data/rss) answered 504 Gateway Timeout after 60-67 s on every try from 10-01 13:25 to at least 10-02 06:42, and each of those scans took 68-82 s against about 15 s otherwise; the feed span work was 4 s. Traces ccd1d1dc56d5ded62873e9301e6bd35e (82 s), ebfc33c8ba1090be65ae1a81e2b3ded1 (76 s), d662ff3dc9d29c64e3dc0c7ffe78b2d3 (68 s). Dreamwidth answers in 0.3 s again on 10-02. A server that never answers would stall scans indefinitely. Fix: a timeout on feed fetches (connect and overall, e.g. 10 s and 30 s), separate from downloads, which can take minutes.
Fixed in
54d827f: a feed fetch (and a Patreon creator's show list) gets 30 s from connecting to the last byte; the client gets a 10 s connect timeout, which bounds a download's start but not a long download. Deployed 2026-10-02. Not yet seen against a hung server; the next one shows as a 'timeout' feed_error after 30 s.