Sleep until the next feed is due instead of scanning every minute (#114)
The daemon ticked every 60 s and ran a scan pass each time: the sweep, then a check-state query per feed (about 180) to find which were due. In the six hours before, 293 of 362 passes found nothing due. Now, after each pass, it works out when the earliest feed is due (due_at, shared with the scan's own check, over Db::http_states, one query) and sleeps until then: at least 30 s, so a feed that never gets a check time cannot spin it, and at most 10 minutes, so what no command announces, ipx add or a shorter schedule, is picked up. Commands still wake it at once, and the first pass after starting runs straight away, as the tick's did. The scan reads every feed's state in one query too. The prod-check skill says what to expect now: tens of scans in six hours, and pending as the real queue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
30
src/db.rs
30
src/db.rs
@@ -438,19 +438,27 @@ pub struct HttpState {
|
||||
pub error_since: Option<i64>,
|
||||
}
|
||||
|
||||
impl From<feeds::Model> for HttpState {
|
||||
fn from(f: feeds::Model) -> Self {
|
||||
HttpState {
|
||||
etag: f.etag,
|
||||
last_modified: f.last_modified,
|
||||
last_checked: f.last_checked,
|
||||
ttl_mins: f.ttl_mins.map(|t| t.max(0) as u64),
|
||||
error_since: f.error_since,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Db {
|
||||
pub async fn http_state(&self, feed_id: &str) -> Result<HttpState> {
|
||||
Ok(feeds::Entity::find_by_id(feed_id.to_owned())
|
||||
.one(&self.orm)
|
||||
.await?
|
||||
.map(|f| HttpState {
|
||||
etag: f.etag,
|
||||
last_modified: f.last_modified,
|
||||
last_checked: f.last_checked,
|
||||
ttl_mins: f.ttl_mins.map(|t| t.max(0) as u64),
|
||||
error_since: f.error_since,
|
||||
})
|
||||
.unwrap_or_default())
|
||||
Ok(feeds::Entity::find_by_id(feed_id.to_owned()).one(&self.orm).await?.map(HttpState::from).unwrap_or_default())
|
||||
}
|
||||
|
||||
/// Every feed's, in one query: a scan, and the daemon working out when the next is due, asked
|
||||
/// feed by feed, 180 queries a minute (#114).
|
||||
pub async fn http_states(&self) -> Result<std::collections::HashMap<String, HttpState>> {
|
||||
Ok(feeds::Entity::find().all(&self.orm).await?.into_iter().map(|f| (f.id.clone(), HttpState::from(f))).collect())
|
||||
}
|
||||
|
||||
/// Upsert after a successful poll. Clears any previous error.
|
||||
|
||||
Reference in New Issue
Block a user