Retire davewiner's 922 rows, without taking the ones people still read

davewiner's OPML left config.toml before retire_group existed, so its
derived rows were skipped by every scan but never cleared. At startup the
daemon now retires every group whose parent is gone from config. And
retire_group unmanages a feed that has its own config entry instead of
dropping it: eleven of davewiner's were promoted without being unmanaged,
and dropping them as derived would have deleted their entries. That also
covers removing an OPML or Patreon subscription from the page.

Closes #3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-15 17:01:27 +00:00
parent eaea520020
commit 4314b11237
3 changed files with 77 additions and 2 deletions

View File

@@ -27,6 +27,11 @@ The long form, with what was wrong before and how it was found, is in
### Fixed
- The feeds left behind by an OPML subscription removed before ipx retired them are cleared at
startup: forgotten if nothing was downloaded, kept as orphaned if something was. Feeds from it
that have since been given their own settings stay as they are, with their items. Removing an
OPML or Patreon subscription no longer deletes the items of a feed inside it that has its own
settings.
- Titles that arrive as HTML, such as The Verge's, no longer show their entities as text:
"Meta&#8217;s" reads "Metas". Titles already stored are corrected the next time their feed
changes.

View File

@@ -6,6 +6,26 @@ reasoning lives. New write-ups go at the top.
See [README.md](../README.md) for what the thing is.
## 2026-09-15 — davewiner's 922 rows, retired at last
davewiner's OPML subscription left config.toml before `retire_group` existed, so nothing ever
retired the 922 feeds derived from it. The backstop in `subscriptions()` kept them from being
scanned, but the rows stayed, and 56 of the 75 errors stored in production were theirs: stale, and
never going to change.
Retiring them the obvious way would have done damage. Eleven of the 922 -- xkcd, The Verge,
TechCrunch, Hacker News and others -- had since been given config entries of their own and were
scanned from there, but their rows still said `managed = 1`. `managed_feeds()` returned them, so
`retire_group("davewiner")` would have dropped the ten with no files as derived feeds, deleting
the entries people were reading. `unmanage` exists for exactly this and had never been applied.
`retire_group` now unmanages a feed that has its own config entry instead of dropping it, which
also covers removing any OPML or Patreon subscription from the page, and the daemon retires every
group whose parent is gone from config when it starts. In production that is davewiner alone: 792
rows with nothing downloaded forgotten, 119 with files kept as orphaned, 11 unmanaged. One of the
792, `ars-technica-all-content-2`, has a subscriber but neither a config entry nor a file; the
backstop had already hidden it, so nobody could see it to lose it.
## 2026-09-14 — Settings, for everyone with an account
A user reported that Settings disappeared shortly after they signed in: it showed for a moment,

View File

@@ -392,6 +392,12 @@ async fn daemon(
Err(e) => tracing::warn!(error = ?e, "could not requeue interrupted downloads"),
}
match retire_stranded(&ctx) {
Ok(0) => {}
Ok(n) => tracing::info!(feeds = n, "retired feeds whose OPML is no longer in config"),
Err(e) => tracing::warn!(error = ?e, "could not retire feeds whose OPML is no longer in config"),
}
let (tx_cmd, mut rx_cmd) = mpsc::channel::<Cmd>(64);
let web = start_web(&ctx, &config_path, web_addr, &tx_cmd, &events).await?;
@@ -962,10 +968,16 @@ pub fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
/// Patreon feed that listed them: the same rule `sync_group` applies to one the list drops --
/// removed if nothing was downloaded, orphaned and kept otherwise. Called once the parent
/// itself is removed, since `subscriptions()` would otherwise keep scanning them under a
/// fallback policy meant for a feed with no parent at all.
/// fallback policy meant for a feed with no parent at all. A feed promoted to config is not
/// derived any more, so it is only unmanaged.
pub fn retire_group(ctx: &Ctx, parent_id: &str) -> Result<()> {
let cfg = ctx.cfg();
for m in ctx.db.managed_feeds()?.into_iter().filter(|m| m.group_id == parent_id) {
if ctx.db.downloaded_count(&m.id).unwrap_or(1) > 0 {
if cfg.feeds.contains_key(&m.id) {
// Scanned from its config entry and still read. Dropped as derived, its stored
// entries would go with it: davewiner's 11 were promoted without being unmanaged.
ctx.db.unmanage(&m.id)?;
} else if ctx.db.downloaded_count(&m.id).unwrap_or(1) > 0 {
ctx.db.set_orphaned(&m.id, true)?;
} else {
ctx.db.drop_managed(&m.id)?;
@@ -974,6 +986,24 @@ pub fn retire_group(ctx: &Ctx, parent_id: &str) -> Result<()> {
Ok(())
}
/// Retires every group whose parent is gone from config, and returns how many derived rows that
/// dropped or unmanaged. An OPML removed before `retire_group` existed left its feeds behind:
/// davewiner's 922 were skipped by every scan and never cleared, and their stale errors were
/// most of the ones stored.
fn retire_stranded(ctx: &Ctx) -> Result<usize> {
let cfg = ctx.cfg();
let before = ctx.db.managed_feeds()?;
let stranded: std::collections::BTreeSet<&str> = before
.iter()
.map(|m| m.group_id.as_str())
.filter(|g| !cfg.feeds.contains_key(*g))
.collect();
for group in stranded {
retire_group(ctx, group)?;
}
Ok(before.len() - ctx.db.managed_feeds()?.len())
}
/// Seconds to wait before re-checking a feed.
///
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the
@@ -1713,4 +1743,24 @@ mod tests {
assert!(managed.iter().any(|m| m.id == "has-file"), "has a file on disk, so it is kept");
assert!(ctx.db.feed_summary("has-file").unwrap().orphaned, "and flagged as orphaned");
}
#[test]
fn a_stranded_group_is_retired_but_a_promoted_feed_keeps_its_entries() {
// davewiner: the OPML left config before retire_group existed, and 11 of its feeds
// promoted to config since still said managed = 1.
let mut cfg = config::Config::default();
cfg.feeds.insert("promoted".into(), feed());
cfg.feeds.insert("live-opml".into(), feed());
let ctx = test_ctx(cfg);
ctx.db.upsert_managed("promoted", "http://x/p.xml", "Promoted", "gone-opml").unwrap();
ctx.db.upsert_managed("empty", "http://x/e.xml", "Empty", "gone-opml").unwrap();
ctx.db.upsert_managed("listed", "http://x/l.xml", "Listed", "live-opml").unwrap();
ctx.db.record_entry("promoted", &feed::Entry { guid: "g1".into(), ..Default::default() }).unwrap();
assert_eq!(retire_stranded(&ctx).unwrap(), 2, "empty dropped, promoted unmanaged");
let managed: Vec<String> = ctx.db.managed_feeds().unwrap().into_iter().map(|m| m.id).collect();
assert_eq!(managed, ["listed"], "a group still in config is left alone");
assert_eq!(ctx.db.feed_summary("promoted").unwrap().entries, 1, "its entries survive");
}
}