ipodderx-rs/PROGRESS.md

Progress

Running record of what has actually landed. Newest entry first. The full design and step list live in the plan file at /config/.claude/plans/i-want-to-create-playful-quiche.md.

Build order

• 1. Repo skeleton — git init (main), cargo init --name ipx, deps pinned, LICENSE, README, this file.
• 2. config.rs + db.rs — TOML config structs + SQLite schema.
• 3. feed.rs — conditional GET, RSS-then-Atom parse, persist entries.
• 4. download.rs — downloads, filters, dedupe.
• 5. retention.rs — oldest-first quota + age reaper.
• 6. ipc.rs + daemon — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon.
• 7. torrent.rs — librqbit, seed to ratio/time, stall abort. (swarm download unverified — see the step 7 entry)
• 8. OPML + polish — import/export, add/rm/status, tracing setup, systemd units, README.

Smoke tests

1. ipx add <feed> + ipx fetch → file in download_dir/<Show>/, row in enclosures.
2. ipx fetch again → no re-download, feed skipped for TTL.
3. ipx daemon & + nc -U $XDG_RUNTIME_DIR/ipx.sock, send {"cmd":"fetch"} → JSON events; a concurrent ipx fetch proxies to the daemon instead of downloading in parallel.
4. Delete a downloaded file by hand, ipx fetch → NOT re-downloaded.
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
6. ipx reap --dry-run under quota pressure → oldest-first hit list; real run flips rows to reaped.

---

2026-09-09 — Step 8: OPML and polish

ipx add <url> fetches the feed to name it from its own title (Accidental Tech Podcast -> accidental-tech-podcast); a feed that cannot be reached is still added, named from its URL, rather than refused. ipx rm leaves downloads and history alone, so re-adding a feed does not re-pull its back catalogue. ipx import/export walk nested OPML folder outlines and skip URLs already subscribed. tracing logs to stderr, IPX_LOG sets the filter. contrib/ has a systemd user unit for the daemon, plus a timer and one-shot service for the no-daemon style (with the caveat that without a daemon there is no socket for a UI).

Verified: cargo test 27/27. Round-trip — exported 3 feeds with real titles, removed one, re-imported: exactly the missing one came back, no duplicates, config still mode 0600. Quickstart from a genuinely empty home with no env overrides: list on a missing config, add, fetch (1 downloaded, 1 explicit skipped, 1 torrent 404), list; every path resolved under $HOME.

---

2026-09-09 — Step 7: torrent.rs

src/torrent.rs: a librqbit Session started lazily on first torrent (binding ports and starting a DHT for a config that has never seen a torrent would be rude). .torrent URLs and magnets both go through AddTorrent::from_url. Progress is polled once a second and reported through the same Progress events HTTP downloads use.

Downloads go straight into the feed folder rather than staging and moving. The plan said move then seed, which cannot work — seeding serves the files it downloaded, so moving them first breaks it. In-place also removes a copy the original had to do.

Seeding stops at seed_ratio or seed_time_mins, whichever comes first, then the torrent is released from the session (files kept).

Bug the smoke test caught: the stall budget only covered the download loop, but resolving a magnet's metadata happens inside add_torrent, which against a dead swarm never returns — a torrent nobody seeds wedged the scan indefinitely. add_torrent is now wrapped in the same budget.

Verified: cargo test 27/27 (ratio incl. the divide-by-zero case, stall_mins = 0 not meaning "abort instantly"). Routing: with enabled = false a torrent enclosure is marked skipped/torrents disabled and never attempted. Session startup works here. Stall abort measured end to end: with stall_mins = 1, a dead magnet failed at 20:58:23 -> 20:59:24, exactly 61s, and the row recorded error / no metadata after 1 minutes, gave up.

Not verified: an actual successful swarm download. This sandbox has no reachable peers, so smoke 5's happy path — payload lands, seeding stops at the ratio — has not been run. The code paths either side of it are tested; the swarm itself needs a real network. Worth running once against a live torrent feed before trusting it.

---

2026-09-09 — Step 6: ipc.rs + daemon

src/ipc.rs: Event and Command as serde-tagged enums ({"ev":...} / {"cmd":...}), one JSON object per line over a Unix socket. Emitter is the single output path — it broadcasts to socket clients, prints the human rendering to a terminal, or both, so the scan code no longer knows how it is being watched. That replaces printMSG and its ;;1;;1;;100.00;;42.31 sentinels.

main.rs restructured around a Ctx (config, db, client, emitter). ipx daemon binds the socket, serves clients, and runs a 60s ticker that defers to each feed's TTL. Commands from every client funnel through one mpsc queue into a single worker, which is what stops two scans overlapping. Any CLI subcommand with a wire form probes the socket first and proxies to a running daemon; --local forces the work to happen in-process. SIGINT/SIGTERM remove the socket on the way out.

Bug the smoke test caught: fetch runs a retention sweep first, and that sweep was emitting the terminal ReapDone. A UI waiting for its fetch to finish would have stopped reading before the scan started. Only a standalone ipx reap emits it now.

Verified: cargo test 24/24. Smoke 3 in full — daemon starts and binds; a raw socket client sending {"cmd":"fetch","force":true} gets feed_start → 14 throttled progress events → download_done → feed_done → scan_done; {"cmd":"status"} answers {"ev":"status","feeds":1,...}. With the daemon up, ipx fetch from the CLI logged command from a client cmd=Fetch { .. } in the daemon and rendered the streamed events, so it proxied rather than downloading in parallel. Socket removed on SIGTERM; with no daemon the same command runs locally.

Deferred: {"cmd":"cancel","enclosure":N} from the plan's protocol is not implemented — downloads run sequentially in one worker, so there is nothing to cancel concurrently yet. It wants a per-download cancellation token, which is worth doing when downloads go parallel. Say if you want it sooner.

Note: the event stream is a broadcast, so a CLI client seeing a busy daemon also sees that other work. Fine for a terminal; a UI wanting strict request/response would want per-request ids.

Also note: the daemon reads config once at startup — changing config.toml needs a restart.

Next: step 7 — torrent.rs.

---

2026-09-09 — Step 5: retention.rs

src/retention.rs: reconcile pass (rows claiming a file that is gone become reaped, fixing the step-4 wart), age sweep, quota sweep keeping the original's 50 MB headroom pad, and entry pruning. ipx reap [--dry-run]; a sweep also runs before every fetch, as the Python did per download. pick() and aged() are pure so the ordering rules are testable without touching a disk.

Judgement call worth Ray's eye. The Python meant to reap only read = 1 AND flagged = 0 but never managed it — a missing plistlib import and an EntreiesData typo made that filter throw on every candidate, so with a .ipxd present nothing was ever deleted. Requiring read = 1 here would be equally dead, because nothing marks episodes read until a UI exists. So: flagged is the keep-forever marker, and read only decides what goes first (ORDER BY read DESC, downloaded_at ASC). Quota therefore actually reclaims space headless. Say the word if you would rather unread episodes were never touched.

Second call: max_age_days deletes files older than the cutoff, not just fileless entries as the plan's wording had it — "keep 30 days of episodes" is what the setting reads like on a NAS.

Verified: cargo test 21/21, including the two tests encoding the exact bug the Python had — flagged files are never offered, and read sort ahead of unread. Smoke 6 with three 30 MB episodes against a 0.1 GB quota (52.4 MB ceiling after the pad): dry run listed ep1+ep2 and deleted nothing (3 files still on disk), the real run deleted exactly those two oldest, left ep3, flipped both rows to reaped with path = NULL. A full re-parse with the conditional-GET headers cleared then re-downloaded nothing.

Next: step 6 — ipc.rs + daemon.

---

2026-09-09 — Step 4: download.rs

src/download.rs: streaming download to <download_dir>/.ipx-incomplete/ (same filesystem as the destination, so filing it is a rename, not the original's copy-then-unlink), content sniffing, then place(). Filename comes from the URL's last path segment, percent-decoded, unless Content-Disposition names one (RFC 5987 filename*= preferred). The sanitizer keeps UTF-8 — latin1_to_ascii existed because 2004 filesystems demanded ASCII — strips the same characters stringCleaning() did plus control chars, and adds a real 255-byte cap the Python never had, preserving the extension across truncation.

Sniffing replaces detectFileType(), which called a typeFile module that was already missing in 2008 and so always answered 'data'. Two rules survive: an HTML body is a failed download (login wall/error page), and a torrent body is a torrent whatever the MIME claimed.

Filters run once at discovery and are recorded in enclosures.state; the download queue is then just "everything still pending", so an enclosure held back by max_new_per_check is picked up by the next scan instead of being lost. Keywords are OR'd across keywords and AND'd within one — the original's nested loop let a later keyword silently undo an earlier miss.

Verified: cargo test 16/16. Smoke against a local server, five enclosures, each filter path hit: ep1.mp3 -> done, ep2.mp3 -> skipped (explicit), ep2.mp3?v=3 -> skipped (no keyword match), paywall.html -> error (HTML page, not media), ep5.torrent -> torrent (deferred to step 7). Smoke 2 and 4 pass, and because a plain rerun 304s before parsing, dedupe was proved separately by clearing the stored etag/last-modified and re-parsing all five entries: 0 downloaded, hand-deleted file not refetched, .ipx-incomplete left empty.

Known wart: ipx list counts path IS NOT NULL, so a hand-deleted file still reads as downloaded. Reconciling rows against the filesystem belongs in step 5. Next: step 5 — retention.rs.

---

2026-09-09 — Step 3: feed.rs

src/feed.rs: conditional GET (If-None-Match + If-Modified-Since, optional basic auth) and a RSS-first / Atom-fallback parser normalising both into ParsedFeed/Entry/Enclosure. Feed-level itunes:explicit overrides the entry level, as the original did. <ttl> is captured. RSS content:encoded wins over description. Atom enclosures come only from rel="enclosure" links.

GUID: the original hashed the title or description when no guid existed; here the chain is guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5 dependency. An entry with none of them has nothing to download and is dropped.

db.rs gained http_state, record_feed, touch_feed, set_feed_error, record_entry, record_enclosure. A changed title/description flips read back to 0 — what the original's textDiff was ultimately for, minus the <ins>/<del> markup, which belongs in the UI.

main.rs gained ipx fetch [FEED] [--force]. A failing feed records its error and the scan continues.

Verified: cargo test 10/10. Gate met against a local python3 -m http.server serving the fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run showed both skip paths, atomcast: not modified (304) and testcast: not due for 45m (the feed's own ttl=45 beating interval_mins = 0).

Deferred: nothing downloads yet — enclosure rows land in state pending. That is step 4. Next: step 4 — download.rs.

---

2026-09-09 — Step 2: config.rs + db.rs

src/config.rs: serde structs for [general], [torrent] and [feeds.<id>] with defaults, ~ expansion, IPX_CONFIG / IPX_DATA_DIR overrides, save() at mode 0600, Feed::password() (password_env beats a literal password), Torrent::ports() parsing "6881-6889". A missing config file loads as an empty one so a fresh install works. Retention defaults are 0/0 (unlimited, keep forever) — nothing gets deleted until Ray asks for it.

src/db.rs: schema exactly as planned, WAL + busy_timeout, Db::open() idempotent, connection behind a Mutex, feed_summary() for list, now() helper. enclosures.url is UNIQUE — the dedupe key that replaces history.dat.

src/main.rs: clap skeleton with ipx list. Only the subcommands that work exist; the rest arrive with their steps.

Verified: cargo test 5/5 green (config defaults, port-range fallback incl. reversed range, password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — ipx --config <scratch> list printed both feeds and created state.db once across two runs.

Deferred: nothing. Two dead-code warnings (Config::save, Feed::password) are expected; steps 3 and 8 consume them. Next: step 3 — feed.rs.

Also added chrono 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3.

---

2026-09-09 — Step 1: repo skeleton

Repo created at /src/ipodderx-rs, default branch main, cargo init --name ipx (edition 2024, rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this file. Hello-world main.rs builds.

Dependency versions pinned today — the later steps are written against these APIs:

crate	version	notes
tokio	1.53.1	rt-multi-thread, macros, fs, io-util, net, sync, time, signal
reqwest	0.13.5	default-features = false; features rustls (not rustls-tls — renamed in 0.13), http2, gzip, stream, json, charset, system-proxy (env-var proxy pickup is opt-in in 0.13)
rss	2.1.1	default features; the with-syndication feature name in my notes does not exist — Atom is handled by the separate crate
atom_syndication	0.12.10
rusqlite	0.40.2	bundled
librqbit	9.0.1	default-features = false; features rust-tls, http-api-client. The default default-tls feature pulls reqwest/native-tls and an OpenSSL sha1 backend (crypto-hash), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7
serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5
clap	4.6.6	derive
infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6
tracing 0.1.44 / tracing-subscriber 0.3.23		env-filter

Deferred: nothing.

Gotcha worth keeping: three feature names in the plan were wrong against current crate versions — reqwest/rustls-tls is now rustls, env-var proxy support moved behind system-proxy, and rss/with-syndication does not exist. librqbit's default features drag in OpenSSL; rust-tls is the fix. Whole tree is rustls-only now, no C TLS dependency. Next: step 2 — config.rs + db.rs. (done)