Retire davewiner's 922 rows, without taking the ones people still read
davewiner's OPML left config.toml before retire_group existed, so its derived rows were skipped by every scan but never cleared. At startup the daemon now retires every group whose parent is gone from config. And retire_group unmanages a feed that has its own config entry instead of dropping it: eleven of davewiner's were promoted without being unmanaged, and dropping them as derived would have deleted their entries. That also covers removing an OPML or Patreon subscription from the page. Closes #3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -27,6 +27,11 @@ The long form, with what was wrong before and how it was found, is in
|
||||
|
||||
### Fixed
|
||||
|
||||
- The feeds left behind by an OPML subscription removed before ipx retired them are cleared at
|
||||
startup: forgotten if nothing was downloaded, kept as orphaned if something was. Feeds from it
|
||||
that have since been given their own settings stay as they are, with their items. Removing an
|
||||
OPML or Patreon subscription no longer deletes the items of a feed inside it that has its own
|
||||
settings.
|
||||
- Titles that arrive as HTML, such as The Verge's, no longer show their entities as text:
|
||||
"Meta’s" reads "Meta’s". Titles already stored are corrected the next time their feed
|
||||
changes.
|
||||
|
||||
@@ -6,6 +6,26 @@ reasoning lives. New write-ups go at the top.
|
||||
|
||||
See [README.md](../README.md) for what the thing is.
|
||||
|
||||
## 2026-09-15 — davewiner's 922 rows, retired at last
|
||||
|
||||
davewiner's OPML subscription left config.toml before `retire_group` existed, so nothing ever
|
||||
retired the 922 feeds derived from it. The backstop in `subscriptions()` kept them from being
|
||||
scanned, but the rows stayed, and 56 of the 75 errors stored in production were theirs: stale, and
|
||||
never going to change.
|
||||
|
||||
Retiring them the obvious way would have done damage. Eleven of the 922 -- xkcd, The Verge,
|
||||
TechCrunch, Hacker News and others -- had since been given config entries of their own and were
|
||||
scanned from there, but their rows still said `managed = 1`. `managed_feeds()` returned them, so
|
||||
`retire_group("davewiner")` would have dropped the ten with no files as derived feeds, deleting
|
||||
the entries people were reading. `unmanage` exists for exactly this and had never been applied.
|
||||
|
||||
`retire_group` now unmanages a feed that has its own config entry instead of dropping it, which
|
||||
also covers removing any OPML or Patreon subscription from the page, and the daemon retires every
|
||||
group whose parent is gone from config when it starts. In production that is davewiner alone: 792
|
||||
rows with nothing downloaded forgotten, 119 with files kept as orphaned, 11 unmanaged. One of the
|
||||
792, `ars-technica-all-content-2`, has a subscriber but neither a config entry nor a file; the
|
||||
backstop had already hidden it, so nobody could see it to lose it.
|
||||
|
||||
## 2026-09-14 — Settings, for everyone with an account
|
||||
|
||||
A user reported that Settings disappeared shortly after they signed in: it showed for a moment,
|
||||
|
||||
54
src/main.rs
54
src/main.rs
@@ -392,6 +392,12 @@ async fn daemon(
|
||||
Err(e) => tracing::warn!(error = ?e, "could not requeue interrupted downloads"),
|
||||
}
|
||||
|
||||
match retire_stranded(&ctx) {
|
||||
Ok(0) => {}
|
||||
Ok(n) => tracing::info!(feeds = n, "retired feeds whose OPML is no longer in config"),
|
||||
Err(e) => tracing::warn!(error = ?e, "could not retire feeds whose OPML is no longer in config"),
|
||||
}
|
||||
|
||||
let (tx_cmd, mut rx_cmd) = mpsc::channel::<Cmd>(64);
|
||||
|
||||
let web = start_web(&ctx, &config_path, web_addr, &tx_cmd, &events).await?;
|
||||
@@ -962,10 +968,16 @@ pub fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
|
||||
/// Patreon feed that listed them: the same rule `sync_group` applies to one the list drops --
|
||||
/// removed if nothing was downloaded, orphaned and kept otherwise. Called once the parent
|
||||
/// itself is removed, since `subscriptions()` would otherwise keep scanning them under a
|
||||
/// fallback policy meant for a feed with no parent at all.
|
||||
/// fallback policy meant for a feed with no parent at all. A feed promoted to config is not
|
||||
/// derived any more, so it is only unmanaged.
|
||||
pub fn retire_group(ctx: &Ctx, parent_id: &str) -> Result<()> {
|
||||
let cfg = ctx.cfg();
|
||||
for m in ctx.db.managed_feeds()?.into_iter().filter(|m| m.group_id == parent_id) {
|
||||
if ctx.db.downloaded_count(&m.id).unwrap_or(1) > 0 {
|
||||
if cfg.feeds.contains_key(&m.id) {
|
||||
// Scanned from its config entry and still read. Dropped as derived, its stored
|
||||
// entries would go with it: davewiner's 11 were promoted without being unmanaged.
|
||||
ctx.db.unmanage(&m.id)?;
|
||||
} else if ctx.db.downloaded_count(&m.id).unwrap_or(1) > 0 {
|
||||
ctx.db.set_orphaned(&m.id, true)?;
|
||||
} else {
|
||||
ctx.db.drop_managed(&m.id)?;
|
||||
@@ -974,6 +986,24 @@ pub fn retire_group(ctx: &Ctx, parent_id: &str) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Retires every group whose parent is gone from config, and returns how many derived rows that
|
||||
/// dropped or unmanaged. An OPML removed before `retire_group` existed left its feeds behind:
|
||||
/// davewiner's 922 were skipped by every scan and never cleared, and their stale errors were
|
||||
/// most of the ones stored.
|
||||
fn retire_stranded(ctx: &Ctx) -> Result<usize> {
|
||||
let cfg = ctx.cfg();
|
||||
let before = ctx.db.managed_feeds()?;
|
||||
let stranded: std::collections::BTreeSet<&str> = before
|
||||
.iter()
|
||||
.map(|m| m.group_id.as_str())
|
||||
.filter(|g| !cfg.feeds.contains_key(*g))
|
||||
.collect();
|
||||
for group in stranded {
|
||||
retire_group(ctx, group)?;
|
||||
}
|
||||
Ok(before.len() - ctx.db.managed_feeds()?.len())
|
||||
}
|
||||
|
||||
/// Seconds to wait before re-checking a feed.
|
||||
///
|
||||
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the
|
||||
@@ -1713,4 +1743,24 @@ mod tests {
|
||||
assert!(managed.iter().any(|m| m.id == "has-file"), "has a file on disk, so it is kept");
|
||||
assert!(ctx.db.feed_summary("has-file").unwrap().orphaned, "and flagged as orphaned");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_stranded_group_is_retired_but_a_promoted_feed_keeps_its_entries() {
|
||||
// davewiner: the OPML left config before retire_group existed, and 11 of its feeds
|
||||
// promoted to config since still said managed = 1.
|
||||
let mut cfg = config::Config::default();
|
||||
cfg.feeds.insert("promoted".into(), feed());
|
||||
cfg.feeds.insert("live-opml".into(), feed());
|
||||
let ctx = test_ctx(cfg);
|
||||
ctx.db.upsert_managed("promoted", "http://x/p.xml", "Promoted", "gone-opml").unwrap();
|
||||
ctx.db.upsert_managed("empty", "http://x/e.xml", "Empty", "gone-opml").unwrap();
|
||||
ctx.db.upsert_managed("listed", "http://x/l.xml", "Listed", "live-opml").unwrap();
|
||||
ctx.db.record_entry("promoted", &feed::Entry { guid: "g1".into(), ..Default::default() }).unwrap();
|
||||
|
||||
assert_eq!(retire_stranded(&ctx).unwrap(), 2, "empty dropped, promoted unmanaged");
|
||||
|
||||
let managed: Vec<String> = ctx.db.managed_feeds().unwrap().into_iter().map(|m| m.id).collect();
|
||||
assert_eq!(managed, ["listed"], "a group still in config is left alone");
|
||||
assert_eq!(ctx.db.feed_summary("promoted").unwrap().entries, 1, "its entries survive");
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user