ipodderx-rs/PROGRESS.md

Progress

Running record of what has actually landed. Newest entry first. The full design and step list live in the plan file at /config/.claude/plans/i-want-to-create-playful-quiche.md.

Build order

• 1. Repo skeleton — git init (main), cargo init --name ipx, deps pinned, LICENSE, README, this file.
• 2. config.rs + db.rs — TOML config structs + SQLite schema.
• 3. feed.rs — conditional GET, RSS-then-Atom parse, persist entries.
• 4. download.rs — downloads, filters, dedupe.
• 5. retention.rs — oldest-first quota + age reaper.
• 6. ipc.rs + daemon — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon.
• 7. torrent.rs — librqbit, seed to ratio/time, stall abort. Done when: smoke 5 passes.
• 8. OPML + polish — import/export, add/rm/status, tracing setup, systemd unit, README.

Smoke tests

1. ipx add <feed> + ipx fetch → file in download_dir/<Show>/, row in enclosures.
2. ipx fetch again → no re-download, feed skipped for TTL.
3. ipx daemon & + nc -U $XDG_RUNTIME_DIR/ipx.sock, send {"cmd":"fetch"} → JSON events; a concurrent ipx fetch proxies to the daemon instead of downloading in parallel.
4. Delete a downloaded file by hand, ipx fetch → NOT re-downloaded.
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
6. ipx reap --dry-run under quota pressure → oldest-first hit list; real run flips rows to reaped.

---

2026-09-09 — Step 6: ipc.rs + daemon

src/ipc.rs: Event and Command as serde-tagged enums ({"ev":...} / {"cmd":...}), one JSON object per line over a Unix socket. Emitter is the single output path — it broadcasts to socket clients, prints the human rendering to a terminal, or both, so the scan code no longer knows how it is being watched. That replaces printMSG and its ;;1;;1;;100.00;;42.31 sentinels.

main.rs restructured around a Ctx (config, db, client, emitter). ipx daemon binds the socket, serves clients, and runs a 60s ticker that defers to each feed's TTL. Commands from every client funnel through one mpsc queue into a single worker, which is what stops two scans overlapping. Any CLI subcommand with a wire form probes the socket first and proxies to a running daemon; --local forces the work to happen in-process. SIGINT/SIGTERM remove the socket on the way out.

Bug the smoke test caught: fetch runs a retention sweep first, and that sweep was emitting the terminal ReapDone. A UI waiting for its fetch to finish would have stopped reading before the scan started. Only a standalone ipx reap emits it now.

Verified: cargo test 24/24. Smoke 3 in full — daemon starts and binds; a raw socket client sending {"cmd":"fetch","force":true} gets feed_start → 14 throttled progress events → download_done → feed_done → scan_done; {"cmd":"status"} answers {"ev":"status","feeds":1,...}. With the daemon up, ipx fetch from the CLI logged command from a client cmd=Fetch { .. } in the daemon and rendered the streamed events, so it proxied rather than downloading in parallel. Socket removed on SIGTERM; with no daemon the same command runs locally.

Deferred: {"cmd":"cancel","enclosure":N} from the plan's protocol is not implemented — downloads run sequentially in one worker, so there is nothing to cancel concurrently yet. It wants a per-download cancellation token, which is worth doing when downloads go parallel. Say if you want it sooner.

Note: the event stream is a broadcast, so a CLI client seeing a busy daemon also sees that other work. Fine for a terminal; a UI wanting strict request/response would want per-request ids.

Also note: the daemon reads config once at startup — changing config.toml needs a restart.

Next: step 7 — torrent.rs.

---

2026-09-09 — Step 5: retention.rs

src/retention.rs: reconcile pass (rows claiming a file that is gone become reaped, fixing the step-4 wart), age sweep, quota sweep keeping the original's 50 MB headroom pad, and entry pruning. ipx reap [--dry-run]; a sweep also runs before every fetch, as the Python did per download. pick() and aged() are pure so the ordering rules are testable without touching a disk.

Judgement call worth Ray's eye. The Python meant to reap only read = 1 AND flagged = 0 but never managed it — a missing plistlib import and an EntreiesData typo made that filter throw on every candidate, so with a .ipxd present nothing was ever deleted. Requiring read = 1 here would be equally dead, because nothing marks episodes read until a UI exists. So: flagged is the keep-forever marker, and read only decides what goes first (ORDER BY read DESC, downloaded_at ASC). Quota therefore actually reclaims space headless. Say the word if you would rather unread episodes were never touched.

Second call: max_age_days deletes files older than the cutoff, not just fileless entries as the plan's wording had it — "keep 30 days of episodes" is what the setting reads like on a NAS.

Verified: cargo test 21/21, including the two tests encoding the exact bug the Python had — flagged files are never offered, and read sort ahead of unread. Smoke 6 with three 30 MB episodes against a 0.1 GB quota (52.4 MB ceiling after the pad): dry run listed ep1+ep2 and deleted nothing (3 files still on disk), the real run deleted exactly those two oldest, left ep3, flipped both rows to reaped with path = NULL. A full re-parse with the conditional-GET headers cleared then re-downloaded nothing.

Next: step 6 — ipc.rs + daemon.

---

2026-09-09 — Step 4: download.rs

src/download.rs: streaming download to <download_dir>/.ipx-incomplete/ (same filesystem as the destination, so filing it is a rename, not the original's copy-then-unlink), content sniffing, then place(). Filename comes from the URL's last path segment, percent-decoded, unless Content-Disposition names one (RFC 5987 filename*= preferred). The sanitizer keeps UTF-8 — latin1_to_ascii existed because 2004 filesystems demanded ASCII — strips the same characters stringCleaning() did plus control chars, and adds a real 255-byte cap the Python never had, preserving the extension across truncation.

Sniffing replaces detectFileType(), which called a typeFile module that was already missing in 2008 and so always answered 'data'. Two rules survive: an HTML body is a failed download (login wall/error page), and a torrent body is a torrent whatever the MIME claimed.

Filters run once at discovery and are recorded in enclosures.state; the download queue is then just "everything still pending", so an enclosure held back by max_new_per_check is picked up by the next scan instead of being lost. Keywords are OR'd across keywords and AND'd within one — the original's nested loop let a later keyword silently undo an earlier miss.

Verified: cargo test 16/16. Smoke against a local server, five enclosures, each filter path hit: ep1.mp3 -> done, ep2.mp3 -> skipped (explicit), ep2.mp3?v=3 -> skipped (no keyword match), paywall.html -> error (HTML page, not media), ep5.torrent -> torrent (deferred to step 7). Smoke 2 and 4 pass, and because a plain rerun 304s before parsing, dedupe was proved separately by clearing the stored etag/last-modified and re-parsing all five entries: 0 downloaded, hand-deleted file not refetched, .ipx-incomplete left empty.

Known wart: ipx list counts path IS NOT NULL, so a hand-deleted file still reads as downloaded. Reconciling rows against the filesystem belongs in step 5. Next: step 5 — retention.rs.

---

2026-09-09 — Step 3: feed.rs

src/feed.rs: conditional GET (If-None-Match + If-Modified-Since, optional basic auth) and a RSS-first / Atom-fallback parser normalising both into ParsedFeed/Entry/Enclosure. Feed-level itunes:explicit overrides the entry level, as the original did. <ttl> is captured. RSS content:encoded wins over description. Atom enclosures come only from rel="enclosure" links.

GUID: the original hashed the title or description when no guid existed; here the chain is guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5 dependency. An entry with none of them has nothing to download and is dropped.

db.rs gained http_state, record_feed, touch_feed, set_feed_error, record_entry, record_enclosure. A changed title/description flips read back to 0 — what the original's textDiff was ultimately for, minus the <ins>/<del> markup, which belongs in the UI.

main.rs gained ipx fetch [FEED] [--force]. A failing feed records its error and the scan continues.

Verified: cargo test 10/10. Gate met against a local python3 -m http.server serving the fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run showed both skip paths, atomcast: not modified (304) and testcast: not due for 45m (the feed's own ttl=45 beating interval_mins = 0).

Deferred: nothing downloads yet — enclosure rows land in state pending. That is step 4. Next: step 4 — download.rs.

---

2026-09-09 — Step 2: config.rs + db.rs

src/config.rs: serde structs for [general], [torrent] and [feeds.<id>] with defaults, ~ expansion, IPX_CONFIG / IPX_DATA_DIR overrides, save() at mode 0600, Feed::password() (password_env beats a literal password), Torrent::ports() parsing "6881-6889". A missing config file loads as an empty one so a fresh install works. Retention defaults are 0/0 (unlimited, keep forever) — nothing gets deleted until Ray asks for it.

src/db.rs: schema exactly as planned, WAL + busy_timeout, Db::open() idempotent, connection behind a Mutex, feed_summary() for list, now() helper. enclosures.url is UNIQUE — the dedupe key that replaces history.dat.

src/main.rs: clap skeleton with ipx list. Only the subcommands that work exist; the rest arrive with their steps.

Verified: cargo test 5/5 green (config defaults, port-range fallback incl. reversed range, password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — ipx --config <scratch> list printed both feeds and created state.db once across two runs.

Deferred: nothing. Two dead-code warnings (Config::save, Feed::password) are expected; steps 3 and 8 consume them. Next: step 3 — feed.rs.

Also added chrono 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3.

---

2026-09-09 — Step 1: repo skeleton

Repo created at /src/ipodderx-rs, default branch main, cargo init --name ipx (edition 2024, rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this file. Hello-world main.rs builds.

Dependency versions pinned today — the later steps are written against these APIs:

crate	version	notes
tokio	1.53.1	rt-multi-thread, macros, fs, io-util, net, sync, time, signal
reqwest	0.13.5	default-features = false; features rustls (not rustls-tls — renamed in 0.13), http2, gzip, stream, json, charset, system-proxy (env-var proxy pickup is opt-in in 0.13)
rss	2.1.1	default features; the with-syndication feature name in my notes does not exist — Atom is handled by the separate crate
atom_syndication	0.12.10
rusqlite	0.40.2	bundled
librqbit	9.0.1	default-features = false; features rust-tls, http-api-client. The default default-tls feature pulls reqwest/native-tls and an OpenSSL sha1 backend (crypto-hash), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7
serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5
clap	4.6.6	derive
infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6
tracing 0.1.44 / tracing-subscriber 0.3.23		env-filter

Deferred: nothing.

Gotcha worth keeping: three feature names in the plan were wrong against current crate versions — reqwest/rustls-tls is now rustls, env-var proxy support moved behind system-proxy, and rss/with-syndication does not exist. librqbit's default features drag in OpenSSL; rust-tls is the fix. Whole tree is rustls-only now, no C TLS dependency. Next: step 2 — config.rs + db.rs. (done)