# Progress Running record of what has actually landed. Newest entry first. The full design and step list live in the plan file at `/config/.claude/plans/i-want-to-create-playful-quiche.md`. ## Build order - [x] **1. Repo skeleton** — git init (`main`), `cargo init --name ipx`, deps pinned, LICENSE, README, this file. - [x] **2. `config.rs` + `db.rs`** — TOML config structs + SQLite schema. - [x] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries. - [x] **4. `download.rs`** — downloads, filters, dedupe. - [x] **5. `retention.rs`** — oldest-first quota + age reaper. - [x] **6. `ipc.rs` + daemon** — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon. - [x] **7. `torrent.rs`** — librqbit, seed to ratio/time, stall abort. (swarm download unverified — see the step 7 entry) - [x] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd units, README. ## Smoke tests 1. `ipx add ` + `ipx fetch` → file in `download_dir//`, row in `enclosures`. 2. `ipx fetch` again → no re-download, feed skipped for TTL. 3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events; a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel. 4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded. 5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time. 6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to `reaped`. --- ## 2026-09-10 — Tested against a real feed (Patreon / Glass Cannon) First run against a live subscriber feed: 131 items, 6.17 GB total, all `audio/mpeg`, no ``. Parsed clean, titled `Get in the Trunk | Anthology Series | Delta Green`, downloaded a valid MP3 (`file` confirms ID3v2.3, MPEG layer III). A second scan pulled the *next* episode, confirming `max_new_per_check` defers rather than drops. Over the socket: 101 progress events for a 79 MB episode -- whole-percent throttling behaving exactly as intended -- then `download_done`, `feed_done`, `scan_done`. **Fixed: stripped separators left doubled spaces.** The feed title separates words with `|`, a forbidden filename character, so the folder came out `Get in the Trunk Anthology Series Delta Green`. `sanitize()` now collapses runs of whitespace (and maps control characters to a space rather than deleting them). The Python had the same wart. **Worth knowing, not a bug:** - The channel-level `itunes:explicit` is `true`, so with the default `allow_explicit = false` all 131 episodes are skipped. Because filter verdicts are recorded once at discovery, flipping the setting afterwards does not re-evaluate enclosures already marked `skipped` -- they would need a `UPDATE enclosures SET state='pending'`. Worth a `ipx retry ` command if this bites. - Patreon sends neither `etag` nor `last-modified` on a GET (a `last-modified` does appear on HEAD, which is what made it look briefly like a storage bug). So this feed can never 304: every poll re-fetches ~160 KB and re-parses 131 entries, and `interval_mins` is the only thing limiting the rate. Cheap, but it means conditional GET buys nothing here. --- ## 2026-09-09 — Step 8: OPML and polish `ipx add ` fetches the feed to name it from its own title (`Accidental Tech Podcast` -> `accidental-tech-podcast`); a feed that cannot be reached is still added, named from its URL, rather than refused. `ipx rm` leaves downloads and history alone, so re-adding a feed does not re-pull its back catalogue. `ipx import`/`export` walk nested OPML folder outlines and skip URLs already subscribed. `tracing` logs to stderr, `IPX_LOG` sets the filter. `contrib/` has a systemd user unit for the daemon, plus a timer and one-shot service for the no-daemon style (with the caveat that without a daemon there is no socket for a UI). Verified: `cargo test` 27/27. Round-trip — exported 3 feeds with real titles, removed one, re-imported: exactly the missing one came back, no duplicates, config still mode 0600. Quickstart from a genuinely empty home with no env overrides: `list` on a missing config, `add`, `fetch` (1 downloaded, 1 explicit skipped, 1 torrent 404), `list`; every path resolved under `$HOME`. --- ## 2026-09-09 — Step 7: torrent.rs `src/torrent.rs`: a librqbit `Session` started lazily on first torrent (binding ports and starting a DHT for a config that has never seen a torrent would be rude). `.torrent` URLs and magnets both go through `AddTorrent::from_url`. Progress is polled once a second and reported through the same `Progress` events HTTP downloads use. Downloads go **straight into the feed folder** rather than staging and moving. The plan said move then seed, which cannot work — seeding serves the files it downloaded, so moving them first breaks it. In-place also removes a copy the original had to do. Seeding stops at `seed_ratio` or `seed_time_mins`, whichever comes first, then the torrent is released from the session (files kept). **Bug the smoke test caught:** the stall budget only covered the download loop, but resolving a magnet's metadata happens inside `add_torrent`, which against a dead swarm never returns — a torrent nobody seeds wedged the scan indefinitely. `add_torrent` is now wrapped in the same budget. Verified: `cargo test` 27/27 (ratio incl. the divide-by-zero case, stall_mins = 0 not meaning "abort instantly"). Routing: with `enabled = false` a torrent enclosure is marked `skipped/torrents disabled` and never attempted. Session startup works here. Stall abort measured end to end: with `stall_mins = 1`, a dead magnet failed at 20:58:23 -> 20:59:24, exactly 61s, and the row recorded `error / no metadata after 1 minutes, gave up`. **Not verified: an actual successful swarm download.** This sandbox has no reachable peers, so smoke 5's happy path — payload lands, seeding stops at the ratio — has not been run. The code paths either side of it are tested; the swarm itself needs a real network. Worth running once against a live torrent feed before trusting it. --- ## 2026-09-09 — Step 6: ipc.rs + daemon `src/ipc.rs`: `Event` and `Command` as serde-tagged enums (`{"ev":...}` / `{"cmd":...}`), one JSON object per line over a Unix socket. `Emitter` is the single output path — it broadcasts to socket clients, prints the human rendering to a terminal, or both, so the scan code no longer knows how it is being watched. That replaces `printMSG` and its `;;1;;1;;100.00;;42.31` sentinels. `main.rs` restructured around a `Ctx` (config, db, client, emitter). `ipx daemon` binds the socket, serves clients, and runs a 60s ticker that defers to each feed's TTL. Commands from every client funnel through one mpsc queue into a single worker, which is what stops two scans overlapping. Any CLI subcommand with a wire form probes the socket first and proxies to a running daemon; `--local` forces the work to happen in-process. SIGINT/SIGTERM remove the socket on the way out. **Bug the smoke test caught:** `fetch` runs a retention sweep first, and that sweep was emitting the terminal `ReapDone`. A UI waiting for its fetch to finish would have stopped reading before the scan started. Only a standalone `ipx reap` emits it now. Verified: `cargo test` 24/24. Smoke 3 in full — daemon starts and binds; a raw socket client sending `{"cmd":"fetch","force":true}` gets `feed_start` → 14 throttled `progress` events → `download_done` → `feed_done` → `scan_done`; `{"cmd":"status"}` answers `{"ev":"status","feeds":1,...}`. With the daemon up, `ipx fetch` from the CLI logged `command from a client cmd=Fetch { .. }` in the daemon and rendered the streamed events, so it proxied rather than downloading in parallel. Socket removed on SIGTERM; with no daemon the same command runs locally. Deferred: `{"cmd":"cancel","enclosure":N}` from the plan's protocol is **not implemented** — downloads run sequentially in one worker, so there is nothing to cancel concurrently yet. It wants a per-download cancellation token, which is worth doing when downloads go parallel. Say if you want it sooner. Note: the event stream is a broadcast, so a CLI client seeing a busy daemon also sees that other work. Fine for a terminal; a UI wanting strict request/response would want per-request ids. Also note: the daemon reads config once at startup — changing `config.toml` needs a restart. Next: step 7 — `torrent.rs`. --- ## 2026-09-09 — Step 5: retention.rs `src/retention.rs`: reconcile pass (rows claiming a file that is gone become `reaped`, fixing the step-4 wart), age sweep, quota sweep keeping the original's 50 MB headroom pad, and entry pruning. `ipx reap [--dry-run]`; a sweep also runs before every `fetch`, as the Python did per download. `pick()` and `aged()` are pure so the ordering rules are testable without touching a disk. **Judgement call worth Ray's eye.** The Python meant to reap only `read = 1 AND flagged = 0` but never managed it — a missing `plistlib` import and an `EntreiesData` typo made that filter throw on every candidate, so with a `.ipxd` present nothing was ever deleted. Requiring `read = 1` here would be equally dead, because nothing marks episodes read until a UI exists. So: **`flagged` is the keep-forever marker, and `read` only decides what goes first** (`ORDER BY read DESC, downloaded_at ASC`). Quota therefore actually reclaims space headless. Say the word if you would rather unread episodes were never touched. Second call: `max_age_days` deletes *files* older than the cutoff, not just fileless entries as the plan's wording had it — "keep 30 days of episodes" is what the setting reads like on a NAS. Verified: `cargo test` 21/21, including the two tests encoding the exact bug the Python had — flagged files are never offered, and read sort ahead of unread. Smoke 6 with three 30 MB episodes against a 0.1 GB quota (52.4 MB ceiling after the pad): dry run listed ep1+ep2 and deleted nothing (3 files still on disk), the real run deleted exactly those two oldest, left ep3, flipped both rows to `reaped` with `path = NULL`. A full re-parse with the conditional-GET headers cleared then re-downloaded nothing. Next: step 6 — `ipc.rs` + daemon. --- ## 2026-09-09 — Step 4: download.rs `src/download.rs`: streaming download to `/.ipx-incomplete/` (same filesystem as the destination, so filing it is a rename, not the original's copy-then-unlink), content sniffing, then `place()`. Filename comes from the URL's last path segment, percent-decoded, unless `Content-Disposition` names one (RFC 5987 `filename*=` preferred). The sanitizer keeps UTF-8 — `latin1_to_ascii` existed because 2004 filesystems demanded ASCII — strips the same characters `stringCleaning()` did plus control chars, and adds a real 255-byte cap the Python never had, preserving the extension across truncation. Sniffing replaces `detectFileType()`, which called a `typeFile` module that was already missing in 2008 and so always answered `'data'`. Two rules survive: an HTML body is a failed download (login wall/error page), and a torrent body is a torrent whatever the MIME claimed. Filters run once at discovery and are recorded in `enclosures.state`; the download queue is then just "everything still `pending`", so an enclosure held back by `max_new_per_check` is picked up by the next scan instead of being lost. Keywords are OR'd across keywords and AND'd within one — the original's nested loop let a later keyword silently undo an earlier miss. Verified: `cargo test` 16/16. Smoke against a local server, five enclosures, each filter path hit: `ep1.mp3 -> done`, `ep2.mp3 -> skipped (explicit)`, `ep2.mp3?v=3 -> skipped (no keyword match)`, `paywall.html -> error (HTML page, not media)`, `ep5.torrent -> torrent (deferred to step 7)`. Smoke 2 and 4 pass, and because a plain rerun 304s before parsing, dedupe was proved separately by clearing the stored etag/last-modified and re-parsing all five entries: 0 downloaded, hand-deleted file not refetched, `.ipx-incomplete` left empty. Known wart: `ipx list` counts `path IS NOT NULL`, so a hand-deleted file still reads as downloaded. Reconciling rows against the filesystem belongs in step 5. Next: step 5 — `retention.rs`. --- ## 2026-09-09 — Step 3: feed.rs `src/feed.rs`: conditional GET (`If-None-Match` + `If-Modified-Since`, optional basic auth) and a RSS-first / Atom-fallback parser normalising both into `ParsedFeed`/`Entry`/`Enclosure`. Feed-level `itunes:explicit` overrides the entry level, as the original did. `` is captured. RSS `content:encoded` wins over `description`. Atom enclosures come only from `rel="enclosure"` links. GUID: the original hashed the title or description when no guid existed; here the chain is guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5 dependency. An entry with none of them has nothing to download and is dropped. `db.rs` gained `http_state`, `record_feed`, `touch_feed`, `set_feed_error`, `record_entry`, `record_enclosure`. A changed title/description flips `read` back to 0 — what the original's textDiff was ultimately for, minus the ``/`` markup, which belongs in the UI. `main.rs` gained `ipx fetch [FEED] [--force]`. A failing feed records its error and the scan continues. Verified: `cargo test` 10/10. Gate met against a local `python3 -m http.server` serving the fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run showed both skip paths, `atomcast: not modified` (304) and `testcast: not due for 45m` (the feed's own ttl=45 beating `interval_mins = 0`). Deferred: nothing downloads yet — enclosure rows land in state `pending`. That is step 4. Next: step 4 — `download.rs`. --- ## 2026-09-09 — Step 2: config.rs + db.rs `src/config.rs`: serde structs for `[general]`, `[torrent]` and `[feeds.]` with defaults, `~` expansion, `IPX_CONFIG` / `IPX_DATA_DIR` overrides, `save()` at mode 0600, `Feed::password()` (`password_env` beats a literal `password`), `Torrent::ports()` parsing `"6881-6889"`. A missing config file loads as an empty one so a fresh install works. Retention defaults are 0/0 (unlimited, keep forever) — nothing gets deleted until Ray asks for it. `src/db.rs`: schema exactly as planned, WAL + `busy_timeout`, `Db::open()` idempotent, connection behind a `Mutex`, `feed_summary()` for `list`, `now()` helper. `enclosures.url` is UNIQUE — the dedupe key that replaces `history.dat`. `src/main.rs`: clap skeleton with `ipx list`. Only the subcommands that work exist; the rest arrive with their steps. Verified: `cargo test` 5/5 green (config defaults, port-range fallback incl. reversed range, password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — `ipx --config list` printed both feeds and created `state.db` once across two runs. Deferred: nothing. Two dead-code warnings (`Config::save`, `Feed::password`) are expected; steps 3 and 8 consume them. Next: step 3 — `feed.rs`. Also added `chrono` 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3. --- ## 2026-09-09 — Step 1: repo skeleton Repo created at `/src/ipodderx-rs`, default branch `main`, `cargo init --name ipx` (edition 2024, rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this file. Hello-world `main.rs` builds. Dependency versions pinned today — the later steps are written against these APIs: | crate | version | notes | |---|---|---| | tokio | 1.53.1 | rt-multi-thread, macros, fs, io-util, net, sync, time, signal | | reqwest | 0.13.5 | `default-features = false`; features **`rustls`** (not `rustls-tls` — renamed in 0.13), http2, gzip, stream, json, charset, **`system-proxy`** (env-var proxy pickup is opt-in in 0.13) | | rss | 2.1.1 | default features; the `with-syndication` feature name in my notes does not exist — Atom is handled by the separate crate | | atom_syndication | 0.12.10 | | | rusqlite | 0.40.2 | `bundled` | | librqbit | 9.0.1 | `default-features = false`; features `rust-tls`, `http-api-client`. The default `default-tls` feature pulls reqwest/native-tls **and** an OpenSSL sha1 backend (`crypto-hash`), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7 | | serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5 | | | | clap | 4.6.6 | derive | | infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6 | | | | tracing 0.1.44 / tracing-subscriber 0.3.23 | | env-filter | Deferred: nothing. Gotcha worth keeping: three feature names in the plan were wrong against current crate versions — `reqwest/rustls-tls` is now `rustls`, env-var proxy support moved behind `system-proxy`, and `rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls` is the fix. Whole tree is rustls-only now, no C TLS dependency. Next: step 2 — `config.rs` + `db.rs`. (done)