Keep OPML feeds out of config, cap downloads, log daemon work

Writing 82 derived feeds into a hand-edited config.toml made it
unreadable. The OPML is the source of truth, so its feeds are re-derived
each scan and held in the database, inheriting the subscription's
settings; editing one promotes it to a real entry. A migration moves
existing children out -- 611 lines to 38 -- keeping all entries and files.

max_new_per_check defaulted to unlimited, so subscribing to an OPML of 82
feeds pulled whole back catalogues. It now defaults to 3 via [general],
capping every feed that does not set its own, and the pending queue orders
by publish date so a cap of 3 means the three newest.

Scans and downloads travelled as socket events only, so the log view
showed no daemon activity. They are mirrored into tracing, with routine
skips at debug -- at 82 feeds those alone would flush the buffer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
This commit is contained in:
2026-09-10 15:15:54 +00:00
parent c86d698363
commit 665d5b8ecb
9 changed files with 456 additions and 77 deletions

View File

@@ -56,6 +56,35 @@ and until now nothing set them.
---
## 2026-09-10 — OPML feeds out of the config, a real cap, and daemon output in the log
Three reports in quick succession, all fair.
**"Config is a mess."** It was: 611 lines, 85 feeds, 82 of them machine-generated, drowning the three
Ray actually chose. Writing derived data into a hand-edited file was the wrong call. OPML children
now live in the `feeds` table (`managed = 1`, `group_id`), are re-derived every scan, and inherit the
subscription's settings — so there is nothing to store but a URL and a parent. Editing one promotes
it to a real config entry, so `config.toml` only ever holds decisions. A one-time migration moves
existing children out: **611 lines -> 38**, 85 feeds -> 3, with all 3487 entries and 216 files intact.
**"Never download more than 3."** `max_new_per_check` defaulted to *unlimited*, so subscribing to an
OPML of 82 feeds pulled every back catalogue it could reach — 216 files, 22 GB before it was caught.
There is now `[general] max_new_per_check = 3`, used whenever a feed does not set its own, so all 83
uncapped feeds were capped without touching a line of their config. `pending()` also orders by publish
date now: a cap of 3 meant "the three recorded first", not the three newest.
**"No daemon output in the log."** Scans and downloads travel as events to the socket, not through
tracing, so the log view showed only startup and HTTP lines. `Emitter::emit` mirrors them now — and
levelling matters at this scale: at 82 feeds, one INFO line per feed per tick for "not due yet"
flushed the 2000-line buffer of anything useful in minutes, so routine skips and progress are DEBUG,
real activity is INFO, failures WARN.
A splice while refactoring cut `reject` and `fetch_one` out of main.rs, and recovering them from git
over-copied three more. Both caught by the compiler, restored, and verified byte-identical against
`git show HEAD:src/main.rs` rather than eyeballed.
---
## 2026-09-10 — Subscribing to an OPML, not just importing one
Asked whether this version could do what iPodderX did: subscribe to an OPML and get a folder of the