ipodderx-rs/PROGRESS.md

Progress

Running record of what has actually landed. Newest entry first. The full design and step list live in the plan file at /config/.claude/plans/i-want-to-create-playful-quiche.md.

Build order

• 1. Repo skeleton — git init (main), cargo init --name ipx, deps pinned, LICENSE, README, this file.
• 2. config.rs + db.rs — TOML config structs + SQLite schema.
• 3. feed.rs — conditional GET, RSS-then-Atom parse, persist entries.
• 4. download.rs — downloads, filters, dedupe.
• 5. retention.rs — oldest-first quota + age reaper.
• 6. ipc.rs + daemon — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon.
• 7. torrent.rs — librqbit, seed to ratio/time, stall abort. (swarm download unverified — see the step 7 entry)
• 8. OPML + polish — import/export, add/rm/status, tracing setup, systemd units, README.

Phase 2 — web front end

Decided with Ray: axum serving plain HTML/JS (no WASM toolchain), running inside the daemon process so it reads SQLite and the event bus directly, LAN-bindable with a shared token.

• 9. Config hot-reload + web skeleton. Ctx.cfg becomes RwLock<Arc<Config>> so the UI can edit feeds without a daemon restart. [web] config section (enabled/bind/token, token auto-generated and saved on first run). axum server started by ipx daemon, token checked by middleware, ?token= sets a cookie so <audio> requests authenticate too. Done when: ipx daemon serves a page on the configured bind, and a wrong token gets 401.
• 10. Browsing. /api/feeds, /api/feeds/:id/entries, entry detail. Descriptions are untrusted feed HTML — sanitized with ammonia before they reach the page. Done when: the Glass Cannon feed's 131 entries browse and read correctly.
• 11. Media actions. Range-request audio streaming (tower-http ServeFile) so seeking works, download-on-demand for a pending enclosure, delete a file, mark read/flagged. Done when: an episode plays and seeks in a browser, and delete reaps the row.
• 12. Feed configuration. Add/remove feeds and edit folder, keywords, allow_explicit, auto_download, max_new_per_check from the UI, written back to config.toml and hot-reloaded. Done when: flipping allow_explicit in the UI takes effect on the next scan with no restart.
• 13. Live progress + polish. SSE from the existing broadcast bus so downloads show live. README section, screenshot-free usage notes. Done when: starting a fetch from the UI shows progress advancing without a reload.

Note: read/flagged finally get a writer here. Retention orders by them (see the step 5 entry), and until now nothing set them.

Smoke tests

1. ipx add <feed> + ipx fetch → file in download_dir/<Show>/, row in enclosures.
2. ipx fetch again → no re-download, feed skipped for TTL.
3. ipx daemon & + nc -U $XDG_RUNTIME_DIR/ipx.sock, send {"cmd":"fetch"} → JSON events; a concurrent ipx fetch proxies to the daemon instead of downloading in parallel.
4. Delete a downloaded file by hand, ipx fetch → NOT re-downloaded.
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
6. ipx reap --dry-run under quota pressure → oldest-first hit list; real run flips rows to reaped.

---

2026-09-10 — Phase 3: full-featured UI

Rewrite of web/index.html (~714 lines) plus the backend it needed.

New stored metadata, all of which the real feed carries on all 131 episodes and none of which was being captured: feed and per-episode artwork (itunes:image), duration (itunes:duration, parsed from either raw seconds or a 1:34:09 clock), season/episode numbers, and a playback position so episodes resume.

Schema migration. Db::open now runs migrate(): PRAGMA table_info then ALTER TABLE ADD COLUMN for anything missing, since CREATE TABLE IF NOT EXISTS does nothing to an existing table. Tested on a copy of the live database first. Accidentally SIGPIPE'd it partway through (a head -4 on the output) which usefully proved it is idempotent and self-heals: the next run added the remaining columns, 131 entries and 6 downloads intact.

New endpoints: entry filters (all/unread/downloaded/flagged), case-insensitive search over title and notes, pagination totals, POST .../position, read-all, download-latest, and OPML import/export over HTTP.

UI: persistent player bar surviving navigation (play/pause, ±15s/30s, 0.8-2.5x speed, volume, scrubber, all remembered in localStorage), resume playback via sendBeacon every 10s, MediaSession for lock-screen controls, keyboard shortcuts (space, arrows, /, Esc), artwork everywhere with generated-initials fallback, filter tabs, episode search, load-more pagination, live progress bars off SSE, real modals, toasts, light/dark toggle, and a mobile layout with a collapsing sidebar.

Bug found and fixed in review: filtering was broken while searching worked. count_entries dropped the search clause when no term was given but still bound ?2; rusqlite rejects a parameter the statement never mentions -- "Wrong number of parameters passed to query. Got 2, needed 1". So ?q=... worked and a plain Unread/Downloaded/Flagged filter 500'd. The clause is now always present with ?2 = '' short-circuiting it. Regression test runs all four filters with and without a search term and asserts the page and the count agree.

A non-bug worth recording: artwork looked broken on the live DB (0/131) while a fresh DB filled all 131. The parser was fine -- the live DB simply had not been rescanned yet, since backfill happens through record_entry's update path. After a forced scan: 131/131 image, duration and season.

Verified live: all four filters return sane totals (131/129/10/0), search works (session zero -> 2, trunk -> 10), /media/1 with a Range header returns 206, index serves 34 KB. cargo test 32/32.

---

2026-09-10 — Fixed: the UI's Download button downloaded the wrong episodes

Reported from the running instance: clicking Download on Music from a Darkened Room | Session Zero showed progress, then left the episode at pending.

Cause, and it was a design error in step 11, not a glitch. POST /api/enclosures/{id}/download requeued the row to pending and asked for an ordinary scan, justified at the time as "needed no new machinery — the queue is the table". But a scan means take the lowest-id pending rows, up to max_new_per_check. Against a 125-item backlog with a cap of 2, that is ids 7 and 8; the requested row 11 was never a candidate. The progress on screen was two other episodes downloading past it.

The queue can express what is outstanding but not what was asked for. Command::Download { enclosure } is now its own command: it fetches that one row immediately, ignoring both queue order and the per-scan cap, still through the single worker so it cannot overlap a scan. Torrent enclosures route to torrent_one the same way a scan would.

Verified live: enclosure 11 went pending -> done, 90.3 MB on disk, and nothing else was pulled in its place. cargo test 30/30, with the new command's wire form pinned.

Worth remembering: the daemon scans immediately on its first tick, so every restart costs max_new_per_check episodes on the normal path.

---

2026-09-10 — Phase 2: web front end (steps 9-13)

axum in the daemon process, plain HTML/JS in web/index.html (embedded with include_str!), no WASM toolchain. One binary still.

Config hot-reload. Ctx.cfg is now RwLock<Arc<Config>>; ctx.cfg() hands out a snapshot, so no guard is ever held across an await. The UI rewrites config.toml and calls reload_cfg, and a running daemon picks the change up on its next scan. The CLI mutation commands (add/rm/import) now clone a snapshot, edit, and save.

Auth. [web] enabled/bind/token; the token is minted from /dev/urandom on first run, written back to config.toml, and the URL printed. ?token= sets a year-long cookie — it has to be a cookie because an <audio> element cannot send a header. Comparison is constant-time. An empty token makes the server refuse to serve rather than serve open.

Endpoints. /api/feeds (GET/POST), /api/feeds/{id} (PATCH/DELETE), /api/feeds/{id}/entries, /api/entries/{feed}/{guid}/flags, /api/enclosures/{id}/download, /api/enclosures/{id} (DELETE), /api/fetch, /api/events (SSE off the existing broadcast bus), /media/{id} (tower-http ServeFile, so Range works).

read/flagged finally have a writer — the UI sets them, and playing an episode marks it read. Retention has ordered by these since step 5 with nothing to set them.

Download-on-demand needed no new machinery: requeue the row to pending and kick a scan, since the queue is the table. That also un-skips an enclosure a filter rejected under older settings.

Verified against the live Glass Cannon feed, daemon on 127.0.0.1:8749:

• auth: no token 401, wrong token 401, right token 200 + cookie, API then works on the cookie alone; /media/1 unauthenticated is 401.
• browsing: 131 entries page with enclosures attached.
• XSS: injected <script>alert(1)</script><img src=x onerror=alert(2)> into a stored description; it reaches the page as <p>hi</p><img src="x"> — script tag and handler both gone.
• media: 200 with accept-ranges: bytes; Range: bytes=1000000-1000999 returns 206 with the right content-range, so seeking works.
• config: PATCH -> 204, written to config.toml, and the running daemon reports the new values with no restart.
• SSE: 105 events for one download (101 progress), then download_done/feed_done/scan_done.
• flags: 204, unread count dropped 131 -> 130. Delete: file gone, row reaped/path=NULL.

cargo test 30/30.

Caveats. It is plain HTTP — on a LAN bind the token crosses the network in the clear, and a feed URL may itself carry a credential (Patreon's does), which the /api/feeds response includes. A reverse proxy with TLS is the answer if that matters. There is no per-user anything; the token is all-or-nothing access.

---

2026-09-10 — Tested against a real feed (Patreon / Glass Cannon)

First run against a live subscriber feed: 131 items, 6.17 GB total, all audio/mpeg, no <ttl>. Parsed clean, titled Get in the Trunk | Anthology Series | Delta Green, downloaded a valid MP3 (file confirms ID3v2.3, MPEG layer III). A second scan pulled the next episode, confirming max_new_per_check defers rather than drops. Over the socket: 101 progress events for a 79 MB episode -- whole-percent throttling behaving exactly as intended -- then download_done, feed_done, scan_done.

Fixed: stripped separators left doubled spaces. The feed title separates words with |, a forbidden filename character, so the folder came out Get in the Trunk Anthology Series Delta Green. The Python had the same wart.

Forbidden characters are now split in two. Separators (/ \\ | :) become -; the rest (? * < > " ') are simply dropped. A run of dashes and spaces then collapses to " - " when the run contained whitespace and to a bare - when it did not, so:

input	output
Get in the Trunk | Anthology Series | Delta Green	Get in the Trunk - Anthology Series - Delta Green
Ep 12: The One	Ep 12 - The One
AC/DC	AC-DC
well-known.mp3	well-known.mp3
../../etc/passwd	etc-passwd

Leading dashes and dots are trimmed too -- a filename starting with - trips up CLI tools.

Worth knowing, not a bug:

• The channel-level itunes:explicit is true, so with the default allow_explicit = false all 131 episodes are skipped. Because filter verdicts are recorded once at discovery, flipping the setting afterwards does not re-evaluate enclosures already marked skipped -- they would need a UPDATE enclosures SET state='pending'. Worth a ipx retry <feed> command if this bites.
• Patreon sends neither etag nor last-modified on a GET (a last-modified does appear on HEAD, which is what made it look briefly like a storage bug). So this feed can never 304: every poll re-fetches ~160 KB and re-parses 131 entries, and interval_mins is the only thing limiting the rate. Cheap, but it means conditional GET buys nothing here.

---

2026-09-09 — Step 8: OPML and polish

ipx add <url> fetches the feed to name it from its own title (Accidental Tech Podcast -> accidental-tech-podcast); a feed that cannot be reached is still added, named from its URL, rather than refused. ipx rm leaves downloads and history alone, so re-adding a feed does not re-pull its back catalogue. ipx import/export walk nested OPML folder outlines and skip URLs already subscribed. tracing logs to stderr, IPX_LOG sets the filter. contrib/ has a systemd user unit for the daemon, plus a timer and one-shot service for the no-daemon style (with the caveat that without a daemon there is no socket for a UI).

Verified: cargo test 27/27. Round-trip — exported 3 feeds with real titles, removed one, re-imported: exactly the missing one came back, no duplicates, config still mode 0600. Quickstart from a genuinely empty home with no env overrides: list on a missing config, add, fetch (1 downloaded, 1 explicit skipped, 1 torrent 404), list; every path resolved under $HOME.

---

2026-09-09 — Step 7: torrent.rs

src/torrent.rs: a librqbit Session started lazily on first torrent (binding ports and starting a DHT for a config that has never seen a torrent would be rude). .torrent URLs and magnets both go through AddTorrent::from_url. Progress is polled once a second and reported through the same Progress events HTTP downloads use.

Downloads go straight into the feed folder rather than staging and moving. The plan said move then seed, which cannot work — seeding serves the files it downloaded, so moving them first breaks it. In-place also removes a copy the original had to do.

Seeding stops at seed_ratio or seed_time_mins, whichever comes first, then the torrent is released from the session (files kept).

Bug the smoke test caught: the stall budget only covered the download loop, but resolving a magnet's metadata happens inside add_torrent, which against a dead swarm never returns — a torrent nobody seeds wedged the scan indefinitely. add_torrent is now wrapped in the same budget.

Verified: cargo test 27/27 (ratio incl. the divide-by-zero case, stall_mins = 0 not meaning "abort instantly"). Routing: with enabled = false a torrent enclosure is marked skipped/torrents disabled and never attempted. Session startup works here. Stall abort measured end to end: with stall_mins = 1, a dead magnet failed at 20:58:23 -> 20:59:24, exactly 61s, and the row recorded error / no metadata after 1 minutes, gave up.

Not verified: an actual successful swarm download. This sandbox has no reachable peers, so smoke 5's happy path — payload lands, seeding stops at the ratio — has not been run. The code paths either side of it are tested; the swarm itself needs a real network. Worth running once against a live torrent feed before trusting it.

---

2026-09-09 — Step 6: ipc.rs + daemon

src/ipc.rs: Event and Command as serde-tagged enums ({"ev":...} / {"cmd":...}), one JSON object per line over a Unix socket. Emitter is the single output path — it broadcasts to socket clients, prints the human rendering to a terminal, or both, so the scan code no longer knows how it is being watched. That replaces printMSG and its ;;1;;1;;100.00;;42.31 sentinels.

main.rs restructured around a Ctx (config, db, client, emitter). ipx daemon binds the socket, serves clients, and runs a 60s ticker that defers to each feed's TTL. Commands from every client funnel through one mpsc queue into a single worker, which is what stops two scans overlapping. Any CLI subcommand with a wire form probes the socket first and proxies to a running daemon; --local forces the work to happen in-process. SIGINT/SIGTERM remove the socket on the way out.

Bug the smoke test caught: fetch runs a retention sweep first, and that sweep was emitting the terminal ReapDone. A UI waiting for its fetch to finish would have stopped reading before the scan started. Only a standalone ipx reap emits it now.

Verified: cargo test 24/24. Smoke 3 in full — daemon starts and binds; a raw socket client sending {"cmd":"fetch","force":true} gets feed_start → 14 throttled progress events → download_done → feed_done → scan_done; {"cmd":"status"} answers {"ev":"status","feeds":1,...}. With the daemon up, ipx fetch from the CLI logged command from a client cmd=Fetch { .. } in the daemon and rendered the streamed events, so it proxied rather than downloading in parallel. Socket removed on SIGTERM; with no daemon the same command runs locally.

Deferred: {"cmd":"cancel","enclosure":N} from the plan's protocol is not implemented — downloads run sequentially in one worker, so there is nothing to cancel concurrently yet. It wants a per-download cancellation token, which is worth doing when downloads go parallel. Say if you want it sooner.

Note: the event stream is a broadcast, so a CLI client seeing a busy daemon also sees that other work. Fine for a terminal; a UI wanting strict request/response would want per-request ids.

Also note: the daemon reads config once at startup — changing config.toml needs a restart.

Next: step 7 — torrent.rs.

---

2026-09-09 — Step 5: retention.rs

src/retention.rs: reconcile pass (rows claiming a file that is gone become reaped, fixing the step-4 wart), age sweep, quota sweep keeping the original's 50 MB headroom pad, and entry pruning. ipx reap [--dry-run]; a sweep also runs before every fetch, as the Python did per download. pick() and aged() are pure so the ordering rules are testable without touching a disk.

Judgement call worth Ray's eye. The Python meant to reap only read = 1 AND flagged = 0 but never managed it — a missing plistlib import and an EntreiesData typo made that filter throw on every candidate, so with a .ipxd present nothing was ever deleted. Requiring read = 1 here would be equally dead, because nothing marks episodes read until a UI exists. So: flagged is the keep-forever marker, and read only decides what goes first (ORDER BY read DESC, downloaded_at ASC). Quota therefore actually reclaims space headless. Say the word if you would rather unread episodes were never touched.

Second call: max_age_days deletes files older than the cutoff, not just fileless entries as the plan's wording had it — "keep 30 days of episodes" is what the setting reads like on a NAS.

Verified: cargo test 21/21, including the two tests encoding the exact bug the Python had — flagged files are never offered, and read sort ahead of unread. Smoke 6 with three 30 MB episodes against a 0.1 GB quota (52.4 MB ceiling after the pad): dry run listed ep1+ep2 and deleted nothing (3 files still on disk), the real run deleted exactly those two oldest, left ep3, flipped both rows to reaped with path = NULL. A full re-parse with the conditional-GET headers cleared then re-downloaded nothing.

Next: step 6 — ipc.rs + daemon.

---

2026-09-09 — Step 4: download.rs

src/download.rs: streaming download to <download_dir>/.ipx-incomplete/ (same filesystem as the destination, so filing it is a rename, not the original's copy-then-unlink), content sniffing, then place(). Filename comes from the URL's last path segment, percent-decoded, unless Content-Disposition names one (RFC 5987 filename*= preferred). The sanitizer keeps UTF-8 — latin1_to_ascii existed because 2004 filesystems demanded ASCII — strips the same characters stringCleaning() did plus control chars, and adds a real 255-byte cap the Python never had, preserving the extension across truncation.

Sniffing replaces detectFileType(), which called a typeFile module that was already missing in 2008 and so always answered 'data'. Two rules survive: an HTML body is a failed download (login wall/error page), and a torrent body is a torrent whatever the MIME claimed.

Filters run once at discovery and are recorded in enclosures.state; the download queue is then just "everything still pending", so an enclosure held back by max_new_per_check is picked up by the next scan instead of being lost. Keywords are OR'd across keywords and AND'd within one — the original's nested loop let a later keyword silently undo an earlier miss.

Verified: cargo test 16/16. Smoke against a local server, five enclosures, each filter path hit: ep1.mp3 -> done, ep2.mp3 -> skipped (explicit), ep2.mp3?v=3 -> skipped (no keyword match), paywall.html -> error (HTML page, not media), ep5.torrent -> torrent (deferred to step 7). Smoke 2 and 4 pass, and because a plain rerun 304s before parsing, dedupe was proved separately by clearing the stored etag/last-modified and re-parsing all five entries: 0 downloaded, hand-deleted file not refetched, .ipx-incomplete left empty.

Known wart: ipx list counts path IS NOT NULL, so a hand-deleted file still reads as downloaded. Reconciling rows against the filesystem belongs in step 5. Next: step 5 — retention.rs.

---

2026-09-09 — Step 3: feed.rs

src/feed.rs: conditional GET (If-None-Match + If-Modified-Since, optional basic auth) and a RSS-first / Atom-fallback parser normalising both into ParsedFeed/Entry/Enclosure. Feed-level itunes:explicit overrides the entry level, as the original did. <ttl> is captured. RSS content:encoded wins over description. Atom enclosures come only from rel="enclosure" links.

GUID: the original hashed the title or description when no guid existed; here the chain is guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5 dependency. An entry with none of them has nothing to download and is dropped.

db.rs gained http_state, record_feed, touch_feed, set_feed_error, record_entry, record_enclosure. A changed title/description flips read back to 0 — what the original's textDiff was ultimately for, minus the <ins>/<del> markup, which belongs in the UI.

main.rs gained ipx fetch [FEED] [--force]. A failing feed records its error and the scan continues.

Verified: cargo test 10/10. Gate met against a local python3 -m http.server serving the fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run showed both skip paths, atomcast: not modified (304) and testcast: not due for 45m (the feed's own ttl=45 beating interval_mins = 0).

Deferred: nothing downloads yet — enclosure rows land in state pending. That is step 4. Next: step 4 — download.rs.

---

2026-09-09 — Step 2: config.rs + db.rs

src/config.rs: serde structs for [general], [torrent] and [feeds.<id>] with defaults, ~ expansion, IPX_CONFIG / IPX_DATA_DIR overrides, save() at mode 0600, Feed::password() (password_env beats a literal password), Torrent::ports() parsing "6881-6889". A missing config file loads as an empty one so a fresh install works. Retention defaults are 0/0 (unlimited, keep forever) — nothing gets deleted until Ray asks for it.

src/db.rs: schema exactly as planned, WAL + busy_timeout, Db::open() idempotent, connection behind a Mutex, feed_summary() for list, now() helper. enclosures.url is UNIQUE — the dedupe key that replaces history.dat.

src/main.rs: clap skeleton with ipx list. Only the subcommands that work exist; the rest arrive with their steps.

Verified: cargo test 5/5 green (config defaults, port-range fallback incl. reversed range, password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — ipx --config <scratch> list printed both feeds and created state.db once across two runs.

Deferred: nothing. Two dead-code warnings (Config::save, Feed::password) are expected; steps 3 and 8 consume them. Next: step 3 — feed.rs.

Also added chrono 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3.

---

2026-09-09 — Step 1: repo skeleton

Repo created at /src/ipodderx-rs, default branch main, cargo init --name ipx (edition 2024, rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this file. Hello-world main.rs builds.

Dependency versions pinned today — the later steps are written against these APIs:

crate	version	notes
tokio	1.53.1	rt-multi-thread, macros, fs, io-util, net, sync, time, signal
reqwest	0.13.5	default-features = false; features rustls (not rustls-tls — renamed in 0.13), http2, gzip, stream, json, charset, system-proxy (env-var proxy pickup is opt-in in 0.13)
rss	2.1.1	default features; the with-syndication feature name in my notes does not exist — Atom is handled by the separate crate
atom_syndication	0.12.10
rusqlite	0.40.2	bundled
librqbit	9.0.1	default-features = false; features rust-tls, http-api-client. The default default-tls feature pulls reqwest/native-tls and an OpenSSL sha1 backend (crypto-hash), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7
serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5
clap	4.6.6	derive
infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6
tracing 0.1.44 / tracing-subscriber 0.3.23		env-filter

Deferred: nothing.

Gotcha worth keeping: three feature names in the plan were wrong against current crate versions — reqwest/rustls-tls is now rustls, env-var proxy support moved behind system-proxy, and rss/with-syndication does not exist. librqbit's default features drag in OpenSSL; rust-tls is the fix. Whole tree is rustls-only now, no C TLS dependency. Next: step 2 — config.rs + db.rs. (done)