Files

rays 2c34a144ba History: the icon pass, sorting and the double play; fix a stale changelog line

docs/history.md gets the long-form entry CLAUDE.md asks for: what the
UI pass found, the three bugs it turned up (Escape inside a dialog's
text box, the white password box, the stray dot), why sorting is done
on the server, and how the Files pane came to play a file twice.
CHANGELOG's Added line for the item table named the old columns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV

2026-09-11 17:32:41 +00:00

81 KiB

Raw Permalink Blame History

How ipx was built

The long-form record, newest first: what landed each day, what was wrong before, and what it cost to find out. CHANGELOG.md has the short version, by release; this is where the reasoning lives. New write-ups go at the top.

See README.md for what the thing is.

2026-09-11 — One meaning per icon, sortable columns, and one player

Ray asked for a pass over the whole UI: consistent icons, and buttons placed next to what they act on. The pass started from screenshots of every view and dialog, taken against the browser suite's own daemon and fixtures. Several things did not agree:

Unsubscribe was a minus in the toolbar but an x in the feed header. There it read as "close this page", and x also meant Cancel in every dialog.
Half the dialogs still had word buttons: Log, Add feed, Users, Unsubscribe, OPML, Popular's Subscribe, the Copy buttons and Sign out.
The player bar and the folder arrow were font characters, and so was the row's "playing" marker. Those came out thin and varied from font to font.
The OPML subscription page laid out its header differently from a feed's, and All Subscriptions had no buttons at all.

The rule now is one meaning per icon. Minus unsubscribes, x closes or cancels, plus adds, subscribes or imports, and a dialog's confirm button carries the icon of what it does. The words go in the tooltip. The toolbar is grouped by what it acts on: add, unsubscribe and scan act on feeds; play, read and keep act on the selected item. Its read and keep buttons now show that item's state, as the item's own buttons already did. A browser test opens all eight dialogs and fails on any button without an icon and a tooltip. It also checks that the toolbar and the page use the same icon for the same action.

The pass turned up three bugs:

Escape ignored inside a dialog's text box. The keydown handler returned early for any key typed in a text box, and Escape sat below that check. So Add feed, which opens with the cursor in its URL box, could not be closed with Escape. It showed up because the screenshot run stalled: the dialog stayed open, and every later click landed on its overlay until the run timed out.
A white password box in the dark theme. The input rule listed text, search and number fields, but not password ones.
A stray "• •" under an item with no date. Each detail carried its own trailing dot, whether or not the next one was there.

All Subscriptions gained Mark everything read, as POST /api/read-all. It marks the feeds from the person's subscriptions, which is the same set the view's query is scoped by. It asks first, since one click covers every feed.

Sorting happens on the server. A list arrives fifty at a time, so sorting in the page would only have reordered what was loaded. The column name picks one of a fixed set of SQL expressions (order_sql), so nothing the browser sends reaches the query, and an unknown name means newest first. Ties also fall back to newest first, so "Load more" does not skip or repeat items at a page boundary. A database test runs every column both ways. Size came out of the File column into its own, and small files show KB, since whole megabytes made every article image "0 MB".

The double play. Ray found that pressing play in the Files pane played the file twice at once. The pane drew its own <audio controls>, and its onplay handler also started the player bar on the same file. That meant two audio elements from one click. The pane now has a play button that hands that exact file to the player bar, the only player. play() now takes the file. Its check for resuming compares the file as well as the item, so another of an item's files starts from its beginning instead of carrying on with the first. A test plays from the pane and checks the page has one <audio> element.

2026-09-11 — The original's layout

Ray pointed at a screenshot of the Mac app (techpp.com, 2012) and asked for its panes, its toolbar, and a Directory that lives in the feed list rather than behind a button. What it had, and what ipx now does:

A toolbar across the window: subscribe and unsubscribe, play, flag, refresh, and a search box scoped to the feed on show. ipx's has the same groups, acting on the selected feed and item, with Settings and Log at the right end, for admins only.
A source list opening with Directory, Playlist Builder and All Subscriptions above the feeds. ipx has Directory, Popular and All Subscriptions there, opening in the main pane. Playlist Builder is left out, since nothing here builds playlists.
The entries as a table, with a Files pane beside it and the entry below. The columns are unread, kept, the item, its feed, its file and when. Sorting by column is not done yet.
A status bar with the totals for what is on show.

All Subscriptions needed one new endpoint, GET /api/entries. It is the per-feed query with feed_id = ? swapped for the person's subscriptions. The enclosure lookup that follows it used to filter by feed as well; a page can now span feeds, so each file is matched to its row by feed and guid instead.

A phone has no room for a pane beside the table, so there the files follow the item's text in the full-screen reader, and the table drops to title and date.

2026-09-11 — Popular on this server

The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how the list was fed. After every successful download, iPXClass.py:1249 called doPing, which sent the enclosure URL and the feed URL, unencoded and over plain HTTP, to directory.iPodderX.com/feedData/survey/files. It was on by default, and only the Mac app's preferences could switch it off (anonFeedback). The directory itself lived on that server and in the Cocoa app, and both are gone.

ipx keeps the idea and drops the ping. Add feed lists what other accounts on this server subscribe to, ranked by subscriber count. The old ping would have sent a Patreon feed's auth= key along with everything else, so this is built around not doing that:

A row carries an id, a title, artwork and a count. No URL reaches the page, so no key can.
Subscribing goes by id through POST /api/popular/{id}, which checks the id against the same list. Guessing a private feed's id gets a 400.
Anything that looks private is never listed: a login configured for the feed, credentials in the URL, or a query key containing auth, token, key, secret, pass, sig, session, user or uid. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed unlisted flag is the upgrade if that happens.

It happened the same day. The first screenshot of the new Directory listed "Glass Cannon Live! Ascension (for Ray Slakinski)", a Supercast feed at feeds.supercast.com/feeds/<key>. Treating any long path segment as a key would have hidden public feeds too: acast's show ids look the same. So paid-feed services are named instead (Patreon, Supercast, Supporting Cast, Glow, Memberful), and any feed from one of them is private whatever its URL looks like. The per-feed flag is still the upgrade for a service not on that list.
Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so they would bury everything anyone chose on purpose.

2026-09-11 — Import an OPML file by uploading it

The import screen now has a file picker as well as the paste box. The page reads the file and checks it looks like OPML before sending anything. If it doesn't, the page says so and clears the picker, and it also clears the picker after an import. The file travels as text and is never written to disk on the server, so there is nothing to clean up there.

The server now parses the OPML before touching anything, and returns 400 "that is not an OPML file" instead of a 500. subscribe_opml takes a parsed document, so ipx import also refuses a non-OPML file by name before changing anything.

2026-09-11 — OPML import and export are per person

Importing an OPML now subscribes you to every feed in it. Feeds already in the catalogue cost nothing, and unknown ones are added under the OPML's title. Before this, import only added URLs missing from config.toml and subscribed nobody. So importing an export from another account did nothing at all, and a genuinely new feed had no subscriber and was never scanned. The page said "Imported 0 feed(s)". ipx import had the same gap; it now subscribes the first admin. Both go through subscribe_opml.

Export now lists only your own subscriptions. It used to write out the whole catalogue to anyone signed in, including other people's feeds and any private URLs in them.

Tests now cover every way in and out:

Settings' Export OPML download and paste-to-import screen.
A round trip that imports the admin's actual export into a second account and checks both exports match. That round trip is exactly what failed.
ipx import and ipx export, run in their own scratch config and database.

2026-09-11 — The log is admin-only

GET /api/logs now returns 403 to anyone who is not an admin, and the page hides the Log button from them. Before this, every signed-in person could read the whole log. That includes every account's name, every feed anyone subscribes to, and every failed sign-in. /api/events stays open to everyone, because it carries the scan progress each person's page shows.

2026-09-11 — Managing users from the web

Settings has a Manage users… screen for an admin. From it you can add someone, with a password or with none for someone the proxy signs in, tick or untick Admin, and remove an account. It is backed by GET/POST /api/users and PATCH/DELETE /api/users/{id}, which return 403 for anyone who is not an admin. The only admin cannot be demoted or removed, because nobody would then be able to manage accounts except from the CLI on the box. Before this, accounts could only be managed with ipx user.

2026-09-11 — Unread feeds first inside an OPML

An OPML subscription's feeds, both in the sidebar folder and on its own page, now list the ones with unread items first. They were listed alphabetically, so with dozens of feeds the few with anything new were scattered through the list. Within each half the order is still alphabetical. The browser suite's fixture OPML gained a second feed, Aardvark Radio, which sorts first by name and by position, so the new test only passes if unread wins.

2026-09-11 — Deploying is a Docker image

Production moved from a hand-started daemon in code-server to the iPodderX container in the Arcane project content. CLAUDE.md now deploys by pushing to the registry at 192.168.1.130:5000 and recreating that one service with docker compose. The old instructions copied a binary over a process nobody supervised, so it did not come back after a reboot. Two known gaps are gone: the image does support accounts, and the entrypoint drops to 99:100, so downloads are no longer owned by root. The README's Docker section said docker compose up -d builds the image and named the service ipx; the compose file pulls from the registry and the service is ipodderx.

2026-09-11 — Documentation

PROGRESS.md became this changelog; the finished step lists moved to an appendix. The README is now an overview that points at docs/: configuration (every key, path and environment variable), cli, users (what several people share), sso (refreshed for accounts and the admin-only settings), and architecture (modules, the scan, the schema, the socket protocol, the HTTP API). CLAUDE.md collects what someone working on the code needs to know and has already been learned the hard way here -- pkill -x not -f, the page being compiled in, the dead columns on entries, the Playwright worker that deleted its own database.

2026-09-11 — Pruning respects a star from anyone

Same defect as the reaper, one function along: prune_entries guarded on entries.flagged, which nothing has written since read state moved to entry_state. Starring a text item with no file would not have saved it from the age sweep. It follows the reaper's rule now, and takes orphaned read state with whatever it deletes.

2026-09-11 — Step D: one file, and everyone who wants it

The last of it, which is all about telling the truth before acting:

Delete says whose file it is. A feed with other subscribers labels the button Delete for everyone, and the confirmation names how many people share it. The server then has the last word: if anyone else has starred the item or not played it yet, DELETE /api/enclosures/{id} returns 409 with the reason in plain words, and only ?force=true goes through. So the rule is enforced where it matters rather than in the page that happens to be asking.
A feed's header says it is shared -- "shared with 1 other person" -- which is the answer to "why is there a file here I never asked for": someone else's subscription fetched it, and one copy serves you both.
Retention already respects the same rule from the entry before this one: starred by anyone keeps it, read by everyone releases it.

others_wanting is tested with three subscribers disagreeing, and a browser test walks the whole delete flow: the label, both prompts, declining the second, and the file still being there.

2026-09-11 — Retention caught up with per-user state

Moving read and starred into entry_state left reap_candidates reading entries.read and entries.flagged, which nothing writes any more: starring stopped protecting a file, and the "delete the ones already read first" ordering was dead. Harmless while the quota and age limits are 0 -- the reaper deletes nothing at all then -- but it would have bitten the moment one was set.

One file serves every subscriber, so both tests are now about all of them: anyone starring it keeps it, and it only counts as read once everyone subscribed has read it. A file whose feed nobody subscribes to has nobody left to keep it, so it sorts with the read ones. Tested with two subscribers disagreeing.

2026-09-11 — Steps B and C: what is yours, what is everyone's

Read, starred and playback position moved out of entries into entry_state (user_id, feed_id, guid, ...); subscriptions became rows in subscriptions (user_id, feed_id, ...) carrying your keywords, auto-download, explicit and per-scan limit. The feed list, unread counts, filters and mark-all-read are all per person now. On first start the existing library is adopted by the admin: 2438 read/starred items and all 86 feeds, so nothing was lost.

The split follows from the file being shared:

Yours: read state, starred, position, keywords, auto-download, explicit, per-scan limit, and which feeds you see at all.
Everyone's: the feed URL, its download folder, and when it is scanned -- there is one copy of a file however many people subscribe, so those describe the file, not a preference. Admin-only, refused with a 403 for anyone else rather than merely hidden.

Scanning merges the subscribers' wants, because one fetch and one file serve them all: an item is downloaded if anyone wants it (any one person's keyword set matching is enough, and one person taking everything removes the filter), auto-download is on if anyone has it on, and the per-scan cap is the largest anyone asked for. merge_policy is a pure function with a test covering each of those. Subscribing to a feed someone already has costs no second fetch and no second copy on disk; unsubscribing takes it off your list alone, and only when the last subscriber leaves does the feed stop being scanned.

A test-harness bug worth naming: Playwright imports the config in every worker, so the fixture's prepare() ran again mid-run and deleted the data directory out from under the daemon. The daemon kept serving from the unlinked inode while the CLI and any query opened a fresh empty database at the same path -- which looked exactly like sign-in being broken. Only the launching process wipes now (a worker has TEST_WORKER_INDEX).

2026-09-10 — Scanning is the operator's decision

The per-feed Check schedule picker is gone from feed settings, and the global Settings page is hidden from anyone who is not an admin. Both are enforced in the handlers rather than merely hidden: PATCH /api/settings and a schedule in PATCH /api/feeds/{id} return 403 for an ordinary user. Polling costs bandwidth, is what a publisher notices, and one impatient setting affects everyone reading that feed -- it belongs in config.toml.

Folders, keywords, per-feed download limits and the feed URL stay editable by anyone signed in.

docs/sso.md covers putting Cloudflare Zero Trust or Authentik in front of ipx: the tunnel and Access application, the Authentik proxy provider and its forward-auth nginx block, the three lines of ipx config each needs, and why trusted_proxies must name the proxy rather than a subnet -- with the command to prove the refusal works.

2026-09-10 — Marking an item read

Two bugs in one place. epAction's redraw closure called itself when it had a row to update -- if(el) redraw() where it meant to swap the row -- so the Mark read button in the text pane recursed until the stack blew. It now swaps that one row in place and refreshes the text below only when it is the item being read.

And opening an item now marks it read, which is what clicking a thing to read it means. The row is redrawn where it stands rather than the list reloaded, so an item does not vanish from under the pointer on the Unread tab.

users and sessions tables, Argon2id hashing, a session cookie, and ipx user add|list|passwd|rm (passwords come in on stdin, so they miss the shell history and any ps listing).

Three ways in, in order of how specific the claim is:

A proxy header naming the user -- Cf-Access-Authenticated-User-Email for the Cloudflare Zero Trust in front of ipodderx.sdf1.net. Honoured only from an address in trusted_proxies (loopback by default): a header is worth exactly as much as the hop that set it, and the LAN port would otherwise let anyone claim to be anyone. Verified both ways -- a spoof from an untrusted address is refused.
A session cookie from signing in at /login.
The shared token, which is the admin, so the healthcheck and existing links keep working.

A database with no accounts creates admin / ipodderx and says so loudly in the log. The UI shows who is signed in above the sidebar footer, with a sign-out, and a 401 sends the page to /login.

Nothing is per-user yet: everyone still sees the same feeds and read state. That is step B.

2026-09-10 — Items, not episodes

Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search box, the empty detail pane, the phone back button, the retention and per-feed settings, and the download dialog all say item now. The sidebar's 20 eps reads 20 items.

S1E1 badges stay -- those come from itunes:episode/itunes:season and only appear when a feed actually publishes them -- as does the episode column, which is that same field.

2026-09-10 — Scanning stopped strobing

A scan of 85 feeds fires 85 feed_done events, and the UI refreshed on every one: a full sidebar rebuild plus an episode reload each time, which read as flicker. Bursts now collapse into one refresh (500ms trailing), the per-feed "N new" toasts add up into a single summary, and both lists keep their scroll position across a rebuild instead of jumping to the top.

Measured over a full scan: 11 feed refreshes and 2 episode reloads in 45 seconds, down from one per event.

Settings and Log moved out of the header and toolbar into a footer under the feed list -- they are housekeeping, not daily controls. The theme toggle stays in the header.

2026-09-10 — Defaults and chrome

All is the default episode filter and the first tab; Unread, Downloaded and Flagged follow. Opening a feed and seeing nothing because everything in it was read is a poor first impression.
OPML import/export moved into Settings, under a Subscriptions heading, and the OPML button is gone from the sidebar -- it is a thing you do once in a while, not a daily control.
The feed actions are pills with icons, and Unsubscribe is pushed to the far end in a quiet style: it sat next to Settings looking exactly like it, one slip away from losing a feed.

2026-09-10 — Phone layout

The UI was unusable on a phone, starting with the worst of it: the ☰ button lived inside the player bar, which is hidden until something plays, so there was no way to reach the feed list at all. It now sits in a small bar at the top of the main pane that is always there, and the sidebar slides over the page with a scrim to tap away.

The rest:

One pane at a time. The three-pane split becomes a list, with the item text taking the whole screen over it and a ← Episodes button back. The divider is hidden.
No sideways scrolling. A grid column is min-content wide by default, so one long headline ("davewiner/hackerNewsStars") dragged the entire page off the right edge -- min-width:0 down the shell/main/wrap chain, overflow-wrap:anywhere on headings, and an explicit minmax(0,1fr) column for the OPML child list.
The player stacks: title row on top, transport and seek bar below, and it paints above the reading pane so it stays reachable.
Header artwork, buttons and log rows shrink to fit; tap targets go to 38px.

A Playwright case at 390x844 locks in the three things that actually broke: the burger is visible with nothing playing, the page does not scroll sideways, and an item opens and closes over the list.

2026-09-10 — Marking a subscription read

An OPML subscription's page now has Mark all read, sitting where every other feed keeps it -- before Unsubscribe. The fix is in the handler rather than the button: read-all resolves the feeds whose group is the given id (through subscriptions(), so a child promoted to config counts too) and marks those, since the subscription's own row holds no entries and marking it did nothing.

2026-09-10 — A folder counts what it holds

An OPML subscription has no entries of its own, so its row always read 0 unread no matter how much was waiting inside it. Unread and saved are now summed from the feeds it holds -- from all of them, not just the ones a search filter left showing, so the number doesn't move as you type. The badge caps at 999+ (the real figure is in its tooltip); a four-digit count ate the title beside it.

2026-09-10 — Sidebar alignment

The feed list had four different left edges: a row with no children skipped the chevron entirely, so its artwork sat a chevron-width left of a folder's; children then used a different indent and a smaller icon. Nothing lined up with anything.

Every row now reserves the chevron slot whether or not it opens (an empty one is pointer-events:none, so the click falls through to the row), all artwork is one size, and nesting reads from the indent alone. Both label lines are display:block on a shared line-height instead of an inline baseline, and the unread count has a min-width so three-digit feeds don't shove the title. Rows came out shorter, so more feeds fit without scrolling.

2026-09-10 — An item's picture

An item's artwork now resolves in order of how deliberate the source is: itunes:image, then Media RSS media:thumbnail, then a media:content that declares itself an image, and finally an image enclosure. That last one matters here -- Substack puts each article's header picture in an <enclosure>, which is why those blog entries had no artwork despite carrying one all along. Audio enclosures are never mistaken for pictures.

Backfilling needed the validators cleared first: record_entry fills a missing image on update, but a 304 skips parsing entirely, so the feeds would have kept their blank squares. (The self-heal added earlier only fires when a feed has zero entries, which was not the case here.)

Result across the library: 325 of 3470 entries now carry their own picture, 13 feeds where every entry has one, 69 feeds that publish no per-item image at all -- those fall back to the feed's artwork, which is the intended behaviour rather than a gap.

2026-09-10 — Database cleanup, and the 304 trap it walked into

Cleaned up on request: removed the CT Log Archive Torrents folder (123 preallocated files from the abandoned 1.6 TB tuscolo torrent, 19 GB real on disk, no row referencing any of it, feed no longer subscribed), the unsubscribed ct-log feed's rows, and every OPML-derived entry/enclosure that held no file so a rescan could rebuild them with the current parser. Kept every row holding a file, so nothing on disk was orphaned, and kept the skipped/reaped history, without which the next scan would re-download the 149 images just deleted. Disk 22 GB -> 3.8 GB, database 22.8 MB -> 5.2 MB.

Correction worth recording: mid-download I checked that torrent folder, saw du -sh report 8 KB, and told Ray it was sparse with nothing written. By the time it was abandoned it had allocated 19 GB. The reassurance had a shelf life I did not mention.

Then "Abort Retry Fail is empty". The cleanup deleted entries but left each feed's ETag/Last-Modified. The rescan sent them, servers answered 304 (120 times in the log), the daemon skipped parsing, and 57 feeds stayed empty -- and would have until a publisher happened to change something. Cleared the validators on the empty feeds and rescanned: entries 1524 -> 3009, feeds with content 19 -> 64, Abort Retry Fail back to 20.

Fixed in code so it cannot recur: a 304 arriving while the feed has zero stored entries means the validator has outlived the data -- a restore, a manual edit, a cleanup. The daemon now believes the database over the validator, drops it and asks again. Proved live by deleting a feed's entries, leaving its ETag, and rescanning: 20 entries rebuilt, self-heal logged once.

"not a wanted media type" was internal jargon reaching the UI, and it was stored in last_error so an ordinary filter decision rendered in red as though something had failed. Reworded to "not audio or video", 144 existing rows updated, and the UI now paints a reason red only when the state is actually error.

2026-09-10 — Multiple enclosures per item, and viewing without downloading

View without downloading. A non-audio/video enclosure now carries a View link opening in a new tab: the publisher's own URL when nothing is downloaded, the local copy at /media/<id> when it is. Deliberately a direct link rather than a proxy — relaying arbitrary URLs through the daemon would make it a fetch-anything service.

Multiple enclosures. Probed before assuming, and the result was worse than expected: the rss crate models an item as having at most one enclosure (which is what RSS 2.0 says) and when a feed carries several it silently keeps the last, dropping the rest. So a two-file item lost its first file entirely.

enclosures_by_item() reads them straight from the XML with quick-xml, in document order, unescaping attribute values — a feed URL's & arrives as &, so skipping that would corrupt every query string. It falls back to the parsed enclosure if the scan and the parser disagree on item count. Atom already collected all rel="enclosure" links. The row now summarises the enclosure you would act on (playable, else downloaded, else first) and says "+N more files"; the pane below lists them all.

A real limitation surfaced by a broken fixture. Two browser tests failed with zero enclosures in the detail pane. Not the new scanner — verified by running it against the fixture files directly, which was right every time. The cause: my fixtures pointed two feeds at the same enclosure URL, and enclosures.url is UNIQUE across the whole database, so whichever feed is scanned first claims it and the other's entry gets nothing. That is the dedupe working as designed, but it means two feeds legitimately sharing a media URL will only ever show it under one of them — worth knowing, and worth revisiting if a network feed and a show feed ever overlap.

2026-09-10 — A file is not the same as a playable file

Reported: "Abort Retry Fail still shows downloaded and audio playback UI". The media-type filter stopped new image enclosures being fetched, but those 20 JPEGs were already on disk from before it existed — and the UI gave a play button and an <audio> element to anything with a path. An audio element pointed at a JPEG is just a broken player.

isPlayable() now separates having a file from being playable: audio/* or video/*, falling back to the file extension when a feed declares no type. Four places used the wrong test — the row's play button, the artwork's play overlay, encBox, and play() itself, which picked the first downloaded enclosure regardless of what it was. A downloaded non-media file now shows as image · downloaded · Save · Delete file, so it is still there and still retrievable, just not pretending to be an episode.

Covered by a new browser test with a fixture blog whose entry carries a JPEG enclosure and a feed configured to download it: no play button on the row, no <audio> in the detail pane, and the Save button still present.

2026-09-10 — Three-pane layout

Ray asked for feeds beside, items above, and the selected item's text with its enclosures below — the layout iPodderX used. Went looking for a screenshot to work from: the archived ipodderx.com kept only marketing panels and 130px feature icons, no full window, and the review screenshots lived on MacMerc which the archive query did not surface. Said so rather than pretending to have reference art; the description is a precise spec on its own.

#split is a grid of list / divider / detail. Selecting a row no longer expands it inline — it highlights and fills the pane below with the title, metadata, read/keep buttons, the sanitized notes and one block per enclosure (a player when the file is here, otherwise what it is and a Download button). The divider drags, and the split is remembered in localStorage. The OPML group page keeps the old scrolling layout via a plain class.

Process note, because it cost a dozen calls. Four anchor-patches in a row failed on text I had guessed rather than read: return el not return div, open:new Set() mid-line rather than starting one. Each assert aborted before writing, so the file kept reverting to a half-applied state, and one earlier patch did land while broken and left a syntax error. What worked was reverting to the committed copy and then reading each block verbatim before touching it, with regex for the whitespace-sensitive parts.

The suite paid for itself again: the new pane test failed on #detail audio because max_new_per_check = 1 plus newest-first means the daemon fetches Second Episode, so First has no player. My assumption, not a bug — the same mistake as the earlier S1E1 one. The test now finds the downloaded row by its chip instead of assuming which episode it is. 9 browser tests pass.

2026-09-10 — Image enclosures were being treated as episodes

Reported as "Abort Retry Fail shows downloads but there are none". It had 20 enclosures, all image/jpeg: Substack puts each article's header image in the RSS <enclosure> tag, so ipx downloaded 20 JPEGs and counted them as episodes. Across the OPML subscription: 149 images, 74 MB, 11 feeds.

[general] media_types defaults to ["audio", "video"], with a per-feed override. An enclosure whose top-level type is not wanted is recorded and shown but not auto-downloaded.

Clarified mid-change: those enclosures should still be visible, just not fetched automatically. So nothing is hidden — the row names what it is ("image", "pdf", "torrent") instead of a bare "skipped", and the download button still works if you want that file.

Two judgement calls in wanted_media: an unknown or absent type is allowed, because the real type is only known after downloading and refusing everything untyped would drop feeds that simply omit the attribute; and a torrent is allowed, being a container rather than media, judged once unpacked.

The enclosure-less case Ray also described was already correct — every download affordance in the row is gated on the enclosure existing. Checked before changing anything.

2026-09-10 — Daemon I/O log tab, and Playwright

Log tabs. All / Daemon I/O / Scans / HTTP. "Daemon I/O" is the control protocol itself: every command arriving (-> {"cmd":...}) and every event leaving (<- {"ev":...}), logged under ipx::io at the one point every command funnels through, so it catches socket clients, the CLI proxying and the web UI alike. Rendered in that tab as in/out rather than level+target.

The two logging destinations now have separate filters: stderr follows IPX_LOG (default ipx=info), the in-app buffer follows IPX_UI_LOG (default ipx=debug) and holds 5000 lines. The UI can therefore show protocol traffic and routine skips that would be noise on a terminal.

Playwright. npm install plus npx playwright install --with-deps chromium, added to /src/install.sh. playwright.config.js starts a fixture feed server and a daemon on a scratch config, and eight tests drive a real browser. Each maps to a bug that actually reached Ray.

Two things the harness taught, both worth keeping:

Playwright starts webServer before globalSetup. Writing the scratch config in globalSetup meant the daemon launched with no config, fell back to defaults, and tried to seize the live daemon's socket. The config is written at config-load time instead.
Ports 8097/8771 looked free but ss showed 8097 held by a process outside this container (shared host network). Moved to 8791/8792.

The suite found a real bug on its first green-ish run: OPML folders rendered expanded by default. The code stored the set of closed groups, so a folder never toggled — every folder in a fresh browser — counted as open, the exact opposite of the comment above it. It stores the open ones now. Two other failures were my tests' fault, not the code's: asserting S1E1 on .ep.first() when newest-first put a different episode there, and an unscoped getByText matching both the sidebar entry and the page heading. The SE in that first failure was the artwork placeholder's initials, which I nearly misread as a broken chip.

2026-09-10 — Regression: derived feeds looked "unsubscribed" to half the code

Reported as error: enclosure 235 belongs to unsubscribed feed "abort-retry-fail". Moving OPML feeds out of config.toml meant a config-only lookup no longer finds them, and download_one still did exactly that — so Download on any episode from an OPML subscription failed outright.

Worse than the one bug: my first grep for the pattern gave false confidence. grep -n 'cfg\.feeds\.get' returned three hits, all legitimate, so it looked clean — but the failing call was split across lines, cfg then .feeds then .get, and never matched. Re-searching with newlines collapsed found sixteen .feeds accesses, several of them wrong:

download_one — the reported failure.
Status and the daemon's startup line counted config feeds only: 3 reported where there are 85.
add deduped and slugged against config only, so adding a URL an OPML already lists would have duplicated it, and a new feed could collide with a derived feed's id.
rm on a derived feed said "no feed with id".
The web add endpoint had the same duplicate hole.

All now go through subscriptions(). Verified against the exact failure: enclosure 235 went pending -> done.

Lesson recorded because it will recur: when a lookup moves, a single-line grep is not a survey. Collapse newlines before searching Rust method chains.

2026-09-10 — OPML feeds out of the config, a real cap, and daemon output in the log

Three reports in quick succession, all fair.

"Config is a mess." It was: 611 lines, 85 feeds, 82 of them machine-generated, drowning the three Ray actually chose. Writing derived data into a hand-edited file was the wrong call. OPML children now live in the feeds table (managed = 1, group_id), are re-derived every scan, and inherit the subscription's settings — so there is nothing to store but a URL and a parent. Editing one promotes it to a real config entry, so config.toml only ever holds decisions. A one-time migration moves existing children out: 611 lines -> 38, 85 feeds -> 3, with all 3487 entries and 216 files intact.

"Never download more than 3." max_new_per_check defaulted to unlimited, so subscribing to an OPML of 82 feeds pulled every back catalogue it could reach — 216 files, 22 GB before it was caught. There is now [general] max_new_per_check = 3, used whenever a feed does not set its own, so all 83 uncapped feeds were capped without touching a line of their config. pending() also orders by publish date now: a cap of 3 meant "the three recorded first", not the three newest.

"No daemon output in the log." Scans and downloads travel as events to the socket, not through tracing, so the log view showed only startup and HTTP lines. Emitter::emit mirrors them now — and levelling matters at this scale: at 82 feeds, one INFO line per feed per tick for "not due yet" flushed the 2000-line buffer of anything useful in minutes, so routine skips and progress are DEBUG, real activity is INFO, failures WARN.

A splice while refactoring cut reject and fetch_one out of main.rs, and recovering them from git over-copied three more. Both caught by the compiler, restored, and verified byte-identical against git show HEAD:src/main.rs rather than eyeballed.

2026-09-10 — Subscribing to an OPML, not just importing one

Asked whether this version could do what iPodderX did: subscribe to an OPML and get a folder of the feeds inside it. It could not — ipx import was one-shot from a file and nothing ever re-read it. The original checked for a URL ending in .opml and re-fetched it on every scan (iPXClass.py:34-40).

Now: a feed whose body sniffs as OPML is a subscription list. Sniffing the body rather than the URL extension also catches an OPML served from a .rss or extension-less URL, which the original missed. Every scan re-reads it; listed feeds become real config entries with group = "<opml-id>" inheriting the parent's settings, land in one nested folder, and are scanned in the same run rather than waiting an interval.

Removal policy, Ray's rule: a downloaded file is never orphaned. Gone from the OPML with nothing downloaded -> unsubscribed and removed. Gone but with downloads -> kept and flagged orphaned, with the UI saying why. Verified end to end on a local fixture: the feed with 1 download was kept and flagged, the one with 0 was removed, the file untouched.

folder_for had to change: it sanitized the whole folder string, so sub/Show would have collapsed into one directory. Each path segment is sanitized separately now, with a test that ../../etc/Show cannot climb out of the download directory.

Also: Db::memory() now runs migrate() like a real open. It did not, so a column added only in the migration passed the tests while being absent in production — which is exactly backwards.

Live result: Ray's lists.opml.org subscription expanded into 82 child feeds, all grouped, 1431 entries, no feed errors, worker still responsive mid-scan.

2026-09-10 — Log view in the app, and Docker

Log view. A ring buffer (2000 lines) fed by a tracing layer, exposed at /api/logs with a sequence cursor so the UI polls for "everything after N" without duplicates. Tailing a file was the obvious approach and the wrong one: under Docker the logs go to stdout and there is no file. An access_log middleware adds a line per HTTP request, so the daemon and the web side share one stream — which is what Ray asked for. That middleware skips /api/logs itself, or the panel's 2-second poll would generate a line per poll forever, a log of nothing but its own requests.

UI: sidebar Log button, level and text filters, follow-tail toggle, copy button, colour-coded levels.

Restart safety. Detached torrents introduced a downloading state, and a row left in it by a restart would sit there forever — the pending queue skips it and nothing else revisits it. requeue_interrupted() resets those at startup, leaving alone any row that already has a file.

Docker. Multi-stage build (deps cached separately from source), debian:bookworm-slim runtime, 114 MB. Entrypoint writes a starter config bound to 0.0.0.0 on first run, because a container's loopback is unreachable from outside, and drops to PUID:PGID via gosu. Healthcheck is ipx status, which goes through the control socket to the worker and so catches a wedged daemon, not just a dead one.

Built and actually run, not just written: image builds, container starts, writes its config, serves the UI (401 without a token, 200 with), /api/logs answers, and the process runs as uid=99(ipx) gid=100(users) with files owned 99:100 and the token file at mode 600.

Worth noting for future testing here: this session's Docker socket belongs to the Unraid host, so -v /tmp/...:/config binds a path on the host, not one visible from inside this container. The mount looks empty from here while being perfectly correct — verify inside the container instead.

2026-09-10 — Torrents: they work, and one was freezing the whole daemon

Asked "is torrents working?". The honest answer had been "unverified since step 7" — so it got tested properly.

They work. End to end against a real swarm: Debian 13.6.0 netinst, 791,674,880 bytes, completed, file confirms a bootable ISO, row marked done. Metadata resolution, transfer, placement and completion all good.

But the daemon was wedged. Ray's CT-log feed had 47 pending torrents and nothing was progressing. The worker was blocked: a status command over the control socket got no reply in 15s. Torrents ran inline on the single command worker, so one torrent stopped every feed scan, every HTTP download and every status command behind it — for up to stall_mins (30 default) waiting on metadata, and worse, Torrents::fetch blocks on seeding for up to seed_time_mins (60 default) after finishing. One successful torrent could have frozen podcast fetching for an hour.

Torrents are now detached: spawn_torrent runs the job on its own task behind a 2-permit semaphore, the row is marked downloading so a rescan does not queue it twice, and the worker moves straight on. A one-shot CLI run still runs them inline, or the process would exit mid-download. Verified: the worker now answers status in 0.0s.

A diagnostic misstep worth recording: /api/settings returning 200 was taken as proof the worker was alive. It is not — the web server is a separate task, so HTTP stays responsive while the queue is completely stuck. Probing the control socket is the real test.

Those CT-log torrents are ~557 GB each, 47 of them. The RSS advertises length="0", so nothing warns you; the size only appears once metadata resolves. auto_download on that feed is set to false as a hold, since restarting would otherwise have begun a half-terabyte transfer immediately (the scheduler's first tick fires at startup). Flip it back in the feed's Settings when wanted.

Also noted: an aborted torrent leaves librqbit's pre-created file placeholders behind in the destination folder (zero-length). Not cleaned up yet.

2026-09-10 — Schedule pickers, and progress painting every row

Pickers. "Check feeds every" is a number input plus a unit dropdown (minutes/hours/days/weeks) in both global and per-feed Settings, replacing free text. parse_interval gained weeks first — it only knew m/h/d, so the backend would have rejected the new option. The per-feed dropdown's first entry is "Use the default — every N", which disables the number and sends null; that rides on the null-clearing path fixed earlier today, and would not have worked before it. splitEvery picks the largest unit that divides evenly, so 120 reads "2 hours" rather than "120 min". The API now returns schedule_mins so the page does not re-implement the parser.

Bug: one download painted every pending row's progress bar. Reported after adding the TWiT feed. The progress event carried feed/url/file but no enclosure id, so the handler had nothing to target and set the width on all of them — the code even carried a comment admitting it applied "to whatever is downloading now". Adding a feed therefore looked like it was downloading the entire back catalogue.

Event::Progress, DownloadDone and DownloadError now carry enclosure, threaded through both the HTTP and torrent paths (db::Pending gained its row id to make that possible), and the handler targets .dlbar[data-bar="<id>"] alone. A test asserts the id is on the wire. Idle bars are transparent and only get a track while live, so a row that is not downloading shows nothing.

tests/page-smoke.js also drives each modal with live-shaped data now — load-time smoke never reaches code that only runs when a dialog opens, which is exactly where these changes landed.

Verified live: weeks round-trips (every 2w -> 20160 min), both pickers render, clearing an override returns to the global default. cargo test 37/37.

2026-09-10 — The whole UI was dead, and server-side tests could not see it

Reported as "my feeds seem to have disappeared", then "settings and dark/light mode don't do anything either". One cause for all three.

$('#prefs').onclick = prefsModal referenced a function that did not exist. An uncaught ReferenceError stops the entire script, and that line sits above the theme toggle, the feed filter, the SSE connection and the loadFeeds() call that fills the sidebar — so everything below it silently never ran.

Cause: a patch anchored on something already deleted. The scheduling UI was inserted with str.replace anchored on function toggleSettings(){, which belonged to the old basic UI that the full rewrite had already removed. Python's replace matches nothing and says nothing, and there was no assert — so prefsModal, everyText, due and globalEvery were never added, while the line calling prefsModal went in fine via a different anchor that did match.

Why it got through. Every check was server-side: curl for status codes, JSON shape, config contents. All passed, because the server was fine. node --check also passed — it parses, and a ReferenceError is a runtime failure. The page was never executed.

tests/page-smoke.js now runs the real page script against a stub DOM, fails on anything thrown, and flags handlers wired to elements that do not exist. Confirmed non-vacuous by reintroducing the exact bug: exit 1 pointing at the offending line, clean once restored. Run it with node tests/page-smoke.js.

Wiring is also defensive now — on(sel, ev, fn) logs and skips rather than throwing, so one dead reference cannot blank the app again.

2026-09-10 — Scheduling, and two bugs it uncovered

Scheduling. The original engine had none — it only skipped feeds on <ttl>; the schedule lived in the Cocoa GUI that was never open-sourced, so there was no original behaviour to match. Scope agreed with Ray: interval only, global default plus per-feed override, no quiet hours.

general.schedule takes "every 30m", "every 4h", "1d", or a bare number of minutes; feeds.<id>.schedule overrides it. The legacy interval_mins is still read so existing configs work, and saving from the UI migrates it. A malformed value in the file warns and falls back rather than stopping the daemon; a malformed value over the API is a 400.

Precedence, which is a judgement call: an explicit per-feed schedule wins outright, including over the publisher's <ttl>. With no per-feed schedule, <ttl> still raises the interval when the publisher asks to be polled less often. Visible in practice — TWiT resolves to 720 min from its own ttl against a 60 min global.

UI: a gear opens global Settings (schedule, quota, max age); each feed's Settings gains a schedule field; the feed header shows "every 4 hours · next in 37m".

Bug 1: null never cleared anything. For Option<Option<T>>, serde maps JSON null onto the outer None, which the handler reads as "field absent, leave alone" — so Some(None) was unreachable and every clear was a no-op returning 204. That silently affected folder, schedule and max_new_per_check; unsetting any of them from the UI did nothing and looked like it worked. Fixed with a double_option deserializer, and pinned by a test covering absent / null / value.

Bug 2: the daemon ignored SIGTERM while working. tokio::select! races its branches only at selection time; once inside run(...).await the shutdown future was not polled at all, so a signal queued behind a 90 MB download. Under systemd every restart would have hit TimeoutStopSec and been SIGKILLed mid-transfer.

This one cost real time: pkill appeared to work but didn't, a stale daemon kept port 8099, and because the restart also rm -f'd the socket it defeated the "already listening" guard and a second daemon started. Every PATCH "verified" after that went to the old binary — so bug 1 was reported fixed while completely untested. Lesson: confirm the process actually died and the socket guard is doing its job, rather than trusting the restart.

The stop signal now lives in a watch channel raced inside each job. Measured: SIGTERM mid- download exits in 1s (was: blocked until the transfer finished). An abandoned download leaves a partial in .ipx-incomplete, discarded on the next attempt.

cargo test 37/37.

2026-09-10 — App name, and a transparent icon

Title is iPodderX in the browser tab and the sidebar. The heading's text-transform: uppercase had to go with it, or the camel case would have rendered as IPODDERX.

The icon's white background is gone. A blanket -transparent white would have punched holes straight through the device, whose body is also white — so the background is flood-filled inward from all four corners at 12% fuzz, which only clears the connected region and stops at the dark outline. Then trimmed and scaled to 128px.

Checked by compositing onto the real #0e131b rather than trusting the alpha channel: no white plate, and no pale fringe from the JPEG artifacts along the outline. The .logo rule lost its background:#fff plate and switched to object-fit:contain.

Both files are tracked: ipodderx-icon.jpg is the untouched archival original, ipodderx-icon.png the transparent version derived from it, so the provenance chain survives. The PNG is embedded as a data URI for the sidebar mark and the favicon.

2026-09-10 — Colour scheme derived from the icon

Palette sampled from web/ipodderx-icon.jpg with ImageMagick rather than eyeballed. The icon gives three families: the silver device ramp (#1A1A1A #606060 #929292 #ADADAD #D6D6D6 #F5F5F5), the screen blues (#2D5391 #5574A9 #7595CA #92B2E6 #B4CAED), one amber from the EQ bars (#F49E2C), and a blue-grey #95A0B1.

Dark theme: --bg is the screen navy taken right down, --fg the device highlight, --dim the blue-grey verbatim, --accent the screen blue, --accent2 the amber. Light theme flips to the device body — white panels, #D6D6D6 edges, #1A1A1A text — with the deeper #2D5391 as accent, since the pale screen blue vanishes on white. Text sitting on an accent fill moved from a hardcoded #0b0e14 to a --ink token.

Two calls: the amber doubles as the pending state, being the only warm note in the icon and exactly where the eye should go. And since the icon has no green or red, the downloaded/error colours are tuned to the palette's saturation and temperature rather than invented from scratch.

Contrast was checked, not assumed, and it failed first time. --faint — which carries dates, durations and file sizes at small size — came out at 3.92 dark and 3.11 light, both under AA. Moved along the icon's own grey ramp to #7a8799 and #767676. Every pair in both themes now clears WCAG AA; the lowest anywhere is 4.54.

2026-09-10 — The real iPodderX icon

The sidebar mark was an invented ix placeholder. The original icon is not in this repo or the legacy one: only the Python engine was open-sourced, never the Cocoa bundle whose Resources held the .icns (the legacy repo's history contains nothing but .py/.so/.po/.mo/LICENSE/README).

Recovered from the Internet Archive's capture of ipodderx.com: _graphics/iPXicon.jpg, 173x165 -- the white iPod-style device with the antenna and EQ display. That is the largest surviving copy; the site's 2005 favicon.ico is far too small to use. Saved as web/ipodderx-icon.jpg so the file lives in the tree, and embedded as a base64 data URI (12.7 KB) for both the sidebar mark and the favicon, which avoids an extra asset route.

.logo changed from a gradient chip to a white rounded tile, since the icon is photo-style on a light ground.

Provenance is Ray's own: the app was Slakinski & Trometer / Thunderstone Media, and this is its successor.

2026-09-10 — Feed URL is editable, with a copy button

The URL in a feed's Settings is now an editable field paired with Copy. Entries and download history are keyed by feed id, not URL, so changing it keeps everything — which is the point, since a Patreon feed URL carries an auth token that gets rotated.

Two things that would otherwise have bitten:

navigator.clipboard does not exist here. It requires a secure context, and this is served over plain HTTP on a LAN address, so the button would have silently done nothing. Falls back to a hidden textarea plus execCommand('copy'), and reports Copied/Failed either way.
Changing the URL clears the stored ETag/Last-Modified. Those validators belong to the old URL; carrying them over could yield a bogus 304 against the new one and make a working feed look empty.

check_url is a tested pure function: rejects empty, unparseable and non-http(s) URLs (so file:///etc/passwd is refused), rejects a URL another feed already uses, and allows a feed to keep its own URL unchanged. Validation failures and unknown feed ids now return 400, not 500 — bad input is the caller's mistake, and only genuine server faults are logged as errors.

Verified live: the three rejection cases return 400 with readable messages, a no-op save returns 204, and the feed still reports 131 entries / 11 downloaded. cargo test 34/34.

2026-09-10 — Fixed: pressing play made an episode vanish from the list

Reported as "where did Session Zero go, and why do parts share a number?".

Two separate things, one of them a real bug.

The bug. play() marked an episode read the moment playback started. The default view is the Unread tab, so pressing play removed the episode from the list you were looking at — indistinguishable from it going missing. Confirmed from the data: all four read entries had position = 0, meaning playback never passed the 10-second save threshold. They had been started, never listened to.

Marking read now happens when an episode is actually consumed — on ended, or once playback passes 90% — never on play. Two earlier PROGRESS entries described mark-on-play as a deliberate feature; it was a design error, and this supersedes them.

Also restored the four affected rows to unread, since their read flag was purely an artifact of the bug (UPDATE entries SET read=0 WHERE read=1 AND position=0 — no genuinely-played episode could match, since playing one writes a position).

Not a bug: repeated episode numbers. Each story part ships as two items — the main episode and a shorter Junk in the Trunk companion — with distinct GUIDs, durations and files, and the publisher gives both the same itunes:episode. Zero duplicated titles across all 131 entries. Session Zero additionally carries a season with no episode number and drops the "Part N" naming, so it does not match the pattern its siblings follow.

Verified: the arc shows 11/11 items under the default Unread view again.

2026-09-10 — Titles: RSS `<title>` is the only source

Confirmed against the live feed rather than assumed: 131 feed items, 131 stored, 0 mismatches — entry titles are byte-for-byte what the RSS <title> publishes, separators included. feed.rs never consults itunes:title, and season/episode live in their own columns, rendered as chips rather than folded into the title.

Pinned with a test: a fixture whose itunes:title differs from its <title> must still yield the RSS title, and an item with itunes:season but no itunes:episode keeps a null episode instead of inventing one.

That null-episode case is real in the wild — this feed's Music from a Darkened Room | Session Zero carries S8 with no episode number, while Parts 1-5 get E1-E5, and it drops the "Part N" convention its siblings use. The arc is 11 items: Session Zero plus five parts, each part being a main episode and its shorter Junk in the Trunk companion.

2026-09-10 — Phase 3: full-featured UI

Rewrite of web/index.html (~714 lines) plus the backend it needed.

New stored metadata, all of which the real feed carries on all 131 episodes and none of which was being captured: feed and per-episode artwork (itunes:image), duration (itunes:duration, parsed from either raw seconds or a 1:34:09 clock), season/episode numbers, and a playback position so episodes resume.

Schema migration. Db::open now runs migrate(): PRAGMA table_info then ALTER TABLE ADD COLUMN for anything missing, since CREATE TABLE IF NOT EXISTS does nothing to an existing table. Tested on a copy of the live database first. Accidentally SIGPIPE'd it partway through (a head -4 on the output) which usefully proved it is idempotent and self-heals: the next run added the remaining columns, 131 entries and 6 downloads intact.

New endpoints: entry filters (all/unread/downloaded/flagged), case-insensitive search over title and notes, pagination totals, POST .../position, read-all, download-latest, and OPML import/export over HTTP.

UI: persistent player bar surviving navigation (play/pause, ±15s/30s, 0.8-2.5x speed, volume, scrubber, all remembered in localStorage), resume playback via sendBeacon every 10s, MediaSession for lock-screen controls, keyboard shortcuts (space, arrows, /, Esc), artwork everywhere with generated-initials fallback, filter tabs, episode search, load-more pagination, live progress bars off SSE, real modals, toasts, light/dark toggle, and a mobile layout with a collapsing sidebar.

Bug found and fixed in review: filtering was broken while searching worked. count_entries dropped the search clause when no term was given but still bound ?2; rusqlite rejects a parameter the statement never mentions -- "Wrong number of parameters passed to query. Got 2, needed 1". So ?q=... worked and a plain Unread/Downloaded/Flagged filter 500'd. The clause is now always present with ?2 = '' short-circuiting it. Regression test runs all four filters with and without a search term and asserts the page and the count agree.

A non-bug worth recording: artwork looked broken on the live DB (0/131) while a fresh DB filled all 131. The parser was fine -- the live DB simply had not been rescanned yet, since backfill happens through record_entry's update path. After a forced scan: 131/131 image, duration and season.

Verified live: all four filters return sane totals (131/129/10/0), search works (session zero -> 2, trunk -> 10), /media/1 with a Range header returns 206, index serves 34 KB. cargo test 32/32.

2026-09-10 — Fixed: the UI's Download button downloaded the wrong episodes

Reported from the running instance: clicking Download on Music from a Darkened Room | Session Zero showed progress, then left the episode at pending.

Cause, and it was a design error in step 11, not a glitch. POST /api/enclosures/{id}/download requeued the row to pending and asked for an ordinary scan, justified at the time as "needed no new machinery — the queue is the table". But a scan means take the lowest-id pending rows, up to max_new_per_check. Against a 125-item backlog with a cap of 2, that is ids 7 and 8; the requested row 11 was never a candidate. The progress on screen was two other episodes downloading past it.

The queue can express what is outstanding but not what was asked for. Command::Download { enclosure } is now its own command: it fetches that one row immediately, ignoring both queue order and the per-scan cap, still through the single worker so it cannot overlap a scan. Torrent enclosures route to torrent_one the same way a scan would.

Verified live: enclosure 11 went pending -> done, 90.3 MB on disk, and nothing else was pulled in its place. cargo test 30/30, with the new command's wire form pinned.

Worth remembering: the daemon scans immediately on its first tick, so every restart costs max_new_per_check episodes on the normal path.

2026-09-10 — Phase 2: web front end (steps 9-13)

axum in the daemon process, plain HTML/JS in web/index.html (embedded with include_str!), no WASM toolchain. One binary still.

Config hot-reload. Ctx.cfg is now RwLock<Arc<Config>>; ctx.cfg() hands out a snapshot, so no guard is ever held across an await. The UI rewrites config.toml and calls reload_cfg, and a running daemon picks the change up on its next scan. The CLI mutation commands (add/rm/import) now clone a snapshot, edit, and save.

Auth. [web] enabled/bind/token; the token is minted from /dev/urandom on first run, written back to config.toml, and the URL printed. ?token= sets a year-long cookie — it has to be a cookie because an <audio> element cannot send a header. Comparison is constant-time. An empty token makes the server refuse to serve rather than serve open.

Endpoints. /api/feeds (GET/POST), /api/feeds/{id} (PATCH/DELETE), /api/feeds/{id}/entries, /api/entries/{feed}/{guid}/flags, /api/enclosures/{id}/download, /api/enclosures/{id} (DELETE), /api/fetch, /api/events (SSE off the existing broadcast bus), /media/{id} (tower-http ServeFile, so Range works).

read/flagged finally have a writer — the UI sets them, and playing an episode marks it read. Retention has ordered by these since step 5 with nothing to set them.

Download-on-demand needed no new machinery: requeue the row to pending and kick a scan, since the queue is the table. That also un-skips an enclosure a filter rejected under older settings.

Verified against the live Glass Cannon feed, daemon on 127.0.0.1:8749:

auth: no token 401, wrong token 401, right token 200 + cookie, API then works on the cookie alone; /media/1 unauthenticated is 401.
browsing: 131 entries page with enclosures attached.
XSS: injected <script>alert(1)</script><img src=x onerror=alert(2)> into a stored description; it reaches the page as <p>hi</p><img src="x"> — script tag and handler both gone.
media: 200 with accept-ranges: bytes; Range: bytes=1000000-1000999 returns 206 with the right content-range, so seeking works.
config: PATCH -> 204, written to config.toml, and the running daemon reports the new values with no restart.
SSE: 105 events for one download (101 progress), then download_done/feed_done/scan_done.
flags: 204, unread count dropped 131 -> 130. Delete: file gone, row reaped/path=NULL.

cargo test 30/30.

Caveats. It is plain HTTP — on a LAN bind the token crosses the network in the clear, and a feed URL may itself carry a credential (Patreon's does), which the /api/feeds response includes. A reverse proxy with TLS is the answer if that matters. There is no per-user anything; the token is all-or-nothing access.

2026-09-10 — Tested against a real feed (Patreon / Glass Cannon)

First run against a live subscriber feed: 131 items, 6.17 GB total, all audio/mpeg, no <ttl>. Parsed clean, titled Get in the Trunk | Anthology Series | Delta Green, downloaded a valid MP3 (file confirms ID3v2.3, MPEG layer III). A second scan pulled the next episode, confirming max_new_per_check defers rather than drops. Over the socket: 101 progress events for a 79 MB episode -- whole-percent throttling behaving exactly as intended -- then download_done, feed_done, scan_done.

Fixed: stripped separators left doubled spaces. The feed title separates words with |, a forbidden filename character, so the folder came out Get in the Trunk Anthology Series Delta Green. The Python had the same wart.

Forbidden characters are now split in two. Separators (/ \\ | :) become -; the rest (? * < > " ') are simply dropped. A run of dashes and spaces then collapses to " - " when the run contained whitespace and to a bare - when it did not, so:

input	output
`Get in the Trunk \| Anthology Series \| Delta Green`	`Get in the Trunk - Anthology Series - Delta Green`
`Ep 12: The One`	`Ep 12 - The One`
`AC/DC`	`AC-DC`
`well-known.mp3`	`well-known.mp3`
`../../etc/passwd`	`etc-passwd`

Leading dashes and dots are trimmed too -- a filename starting with - trips up CLI tools.

Worth knowing, not a bug:

The channel-level itunes:explicit is true, so with the default allow_explicit = false all 131 episodes are skipped. Because filter verdicts are recorded once at discovery, flipping the setting afterwards does not re-evaluate enclosures already marked skipped -- they would need a UPDATE enclosures SET state='pending'. Worth a ipx retry <feed> command if this bites.
Patreon sends neither etag nor last-modified on a GET (a last-modified does appear on HEAD, which is what made it look briefly like a storage bug). So this feed can never 304: every poll re-fetches ~160 KB and re-parses 131 entries, and interval_mins is the only thing limiting the rate. Cheap, but it means conditional GET buys nothing here.

2026-09-09 — Step 8: OPML and polish

ipx add <url> fetches the feed to name it from its own title (Accidental Tech Podcast -> accidental-tech-podcast); a feed that cannot be reached is still added, named from its URL, rather than refused. ipx rm leaves downloads and history alone, so re-adding a feed does not re-pull its back catalogue. ipx import/export walk nested OPML folder outlines and skip URLs already subscribed. tracing logs to stderr, IPX_LOG sets the filter. contrib/ has a systemd user unit for the daemon, plus a timer and one-shot service for the no-daemon style (with the caveat that without a daemon there is no socket for a UI).

Verified: cargo test 27/27. Round-trip — exported 3 feeds with real titles, removed one, re-imported: exactly the missing one came back, no duplicates, config still mode 0600. Quickstart from a genuinely empty home with no env overrides: list on a missing config, add, fetch (1 downloaded, 1 explicit skipped, 1 torrent 404), list; every path resolved under $HOME.

2026-09-09 — Step 7: torrent.rs

src/torrent.rs: a librqbit Session started lazily on first torrent (binding ports and starting a DHT for a config that has never seen a torrent would be rude). .torrent URLs and magnets both go through AddTorrent::from_url. Progress is polled once a second and reported through the same Progress events HTTP downloads use.

Downloads go straight into the feed folder rather than staging and moving. The plan said move then seed, which cannot work — seeding serves the files it downloaded, so moving them first breaks it. In-place also removes a copy the original had to do.

Seeding stops at seed_ratio or seed_time_mins, whichever comes first, then the torrent is released from the session (files kept).

Bug the smoke test caught: the stall budget only covered the download loop, but resolving a magnet's metadata happens inside add_torrent, which against a dead swarm never returns — a torrent nobody seeds wedged the scan indefinitely. add_torrent is now wrapped in the same budget.

Verified: cargo test 27/27 (ratio incl. the divide-by-zero case, stall_mins = 0 not meaning "abort instantly"). Routing: with enabled = false a torrent enclosure is marked skipped/torrents disabled and never attempted. Session startup works here. Stall abort measured end to end: with stall_mins = 1, a dead magnet failed at 20:58:23 -> 20:59:24, exactly 61s, and the row recorded error / no metadata after 1 minutes, gave up.

Not verified: an actual successful swarm download. This sandbox has no reachable peers, so smoke 5's happy path — payload lands, seeding stops at the ratio — has not been run. The code paths either side of it are tested; the swarm itself needs a real network. Worth running once against a live torrent feed before trusting it.

2026-09-09 — Step 6: ipc.rs + daemon

src/ipc.rs: Event and Command as serde-tagged enums ({"ev":...} / {"cmd":...}), one JSON object per line over a Unix socket. Emitter is the single output path — it broadcasts to socket clients, prints the human rendering to a terminal, or both, so the scan code no longer knows how it is being watched. That replaces printMSG and its ;;1;;1;;100.00;;42.31 sentinels.

main.rs restructured around a Ctx (config, db, client, emitter). ipx daemon binds the socket, serves clients, and runs a 60s ticker that defers to each feed's TTL. Commands from every client funnel through one mpsc queue into a single worker, which is what stops two scans overlapping. Any CLI subcommand with a wire form probes the socket first and proxies to a running daemon; --local forces the work to happen in-process. SIGINT/SIGTERM remove the socket on the way out.

Bug the smoke test caught: fetch runs a retention sweep first, and that sweep was emitting the terminal ReapDone. A UI waiting for its fetch to finish would have stopped reading before the scan started. Only a standalone ipx reap emits it now.

Verified: cargo test 24/24. Smoke 3 in full — daemon starts and binds; a raw socket client sending {"cmd":"fetch","force":true} gets feed_start → 14 throttled progress events → download_done → feed_done → scan_done; {"cmd":"status"} answers {"ev":"status","feeds":1,...}. With the daemon up, ipx fetch from the CLI logged command from a client cmd=Fetch { .. } in the daemon and rendered the streamed events, so it proxied rather than downloading in parallel. Socket removed on SIGTERM; with no daemon the same command runs locally.

Deferred: {"cmd":"cancel","enclosure":N} from the plan's protocol is not implemented — downloads run sequentially in one worker, so there is nothing to cancel concurrently yet. It wants a per-download cancellation token, which is worth doing when downloads go parallel. Say if you want it sooner.

Note: the event stream is a broadcast, so a CLI client seeing a busy daemon also sees that other work. Fine for a terminal; a UI wanting strict request/response would want per-request ids.

Also note: the daemon reads config once at startup — changing config.toml needs a restart.

Next: step 7 — torrent.rs.

2026-09-09 — Step 5: retention.rs

src/retention.rs: reconcile pass (rows claiming a file that is gone become reaped, fixing the step-4 wart), age sweep, quota sweep keeping the original's 50 MB headroom pad, and entry pruning. ipx reap [--dry-run]; a sweep also runs before every fetch, as the Python did per download. pick() and aged() are pure so the ordering rules are testable without touching a disk.

Judgement call worth Ray's eye. The Python meant to reap only read = 1 AND flagged = 0 but never managed it — a missing plistlib import and an EntreiesData typo made that filter throw on every candidate, so with a .ipxd present nothing was ever deleted. Requiring read = 1 here would be equally dead, because nothing marks episodes read until a UI exists. So: flagged is the keep-forever marker, and read only decides what goes first (ORDER BY read DESC, downloaded_at ASC). Quota therefore actually reclaims space headless. Say the word if you would rather unread episodes were never touched.

Second call: max_age_days deletes files older than the cutoff, not just fileless entries as the plan's wording had it — "keep 30 days of episodes" is what the setting reads like on a NAS.

Verified: cargo test 21/21, including the two tests encoding the exact bug the Python had — flagged files are never offered, and read sort ahead of unread. Smoke 6 with three 30 MB episodes against a 0.1 GB quota (52.4 MB ceiling after the pad): dry run listed ep1+ep2 and deleted nothing (3 files still on disk), the real run deleted exactly those two oldest, left ep3, flipped both rows to reaped with path = NULL. A full re-parse with the conditional-GET headers cleared then re-downloaded nothing.

Next: step 6 — ipc.rs + daemon.

2026-09-09 — Step 4: download.rs

src/download.rs: streaming download to <download_dir>/.ipx-incomplete/ (same filesystem as the destination, so filing it is a rename, not the original's copy-then-unlink), content sniffing, then place(). Filename comes from the URL's last path segment, percent-decoded, unless Content-Disposition names one (RFC 5987 filename*= preferred). The sanitizer keeps UTF-8 — latin1_to_ascii existed because 2004 filesystems demanded ASCII — strips the same characters stringCleaning() did plus control chars, and adds a real 255-byte cap the Python never had, preserving the extension across truncation.

Sniffing replaces detectFileType(), which called a typeFile module that was already missing in 2008 and so always answered 'data'. Two rules survive: an HTML body is a failed download (login wall/error page), and a torrent body is a torrent whatever the MIME claimed.

Filters run once at discovery and are recorded in enclosures.state; the download queue is then just "everything still pending", so an enclosure held back by max_new_per_check is picked up by the next scan instead of being lost. Keywords are OR'd across keywords and AND'd within one — the original's nested loop let a later keyword silently undo an earlier miss.

Verified: cargo test 16/16. Smoke against a local server, five enclosures, each filter path hit: ep1.mp3 -> done, ep2.mp3 -> skipped (explicit), ep2.mp3?v=3 -> skipped (no keyword match), paywall.html -> error (HTML page, not media), ep5.torrent -> torrent (deferred to step 7). Smoke 2 and 4 pass, and because a plain rerun 304s before parsing, dedupe was proved separately by clearing the stored etag/last-modified and re-parsing all five entries: 0 downloaded, hand-deleted file not refetched, .ipx-incomplete left empty.

Known wart: ipx list counts path IS NOT NULL, so a hand-deleted file still reads as downloaded. Reconciling rows against the filesystem belongs in step 5. Next: step 5 — retention.rs.

2026-09-09 — Step 3: feed.rs

src/feed.rs: conditional GET (If-None-Match + If-Modified-Since, optional basic auth) and a RSS-first / Atom-fallback parser normalising both into ParsedFeed/Entry/Enclosure. Feed-level itunes:explicit overrides the entry level, as the original did. <ttl> is captured. RSS content:encoded wins over description. Atom enclosures come only from rel="enclosure" links.

GUID: the original hashed the title or description when no guid existed; here the chain is guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5 dependency. An entry with none of them has nothing to download and is dropped.

db.rs gained http_state, record_feed, touch_feed, set_feed_error, record_entry, record_enclosure. A changed title/description flips read back to 0 — what the original's textDiff was ultimately for, minus the <ins>/<del> markup, which belongs in the UI.

main.rs gained ipx fetch [FEED] [--force]. A failing feed records its error and the scan continues.

Verified: cargo test 10/10. Gate met against a local python3 -m http.server serving the fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run showed both skip paths, atomcast: not modified (304) and testcast: not due for 45m (the feed's own ttl=45 beating interval_mins = 0).

Deferred: nothing downloads yet — enclosure rows land in state pending. That is step 4. Next: step 4 — download.rs.

2026-09-09 — Step 2: config.rs + db.rs

src/config.rs: serde structs for [general], [torrent] and [feeds.<id>] with defaults, ~ expansion, IPX_CONFIG / IPX_DATA_DIR overrides, save() at mode 0600, Feed::password() (password_env beats a literal password), Torrent::ports() parsing "6881-6889". A missing config file loads as an empty one so a fresh install works. Retention defaults are 0/0 (unlimited, keep forever) — nothing gets deleted until Ray asks for it.

src/db.rs: schema exactly as planned, WAL + busy_timeout, Db::open() idempotent, connection behind a Mutex, feed_summary() for list, now() helper. enclosures.url is UNIQUE — the dedupe key that replaces history.dat.

src/main.rs: clap skeleton with ipx list. Only the subcommands that work exist; the rest arrive with their steps.

Verified: cargo test 5/5 green (config defaults, port-range fallback incl. reversed range, password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — ipx --config <scratch> list printed both feeds and created state.db once across two runs.

Deferred: nothing. Two dead-code warnings (Config::save, Feed::password) are expected; steps 3 and 8 consume them. Next: step 3 — feed.rs.

Also added chrono 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3.

2026-09-09 — Step 1: repo skeleton

Repo created at /src/ipodderx-rs, default branch main, cargo init --name ipx (edition 2024, rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this file. Hello-world main.rs builds.

Dependency versions pinned today — the later steps are written against these APIs:

crate	version	notes
tokio	1.53.1	rt-multi-thread, macros, fs, io-util, net, sync, time, signal
reqwest	0.13.5	`default-features = false`; features `rustls` (not `rustls-tls` — renamed in 0.13), http2, gzip, stream, json, charset, `system-proxy` (env-var proxy pickup is opt-in in 0.13)
rss	2.1.1	default features; the `with-syndication` feature name in my notes does not exist — Atom is handled by the separate crate
atom_syndication	0.12.10
rusqlite	0.40.2	`bundled`
librqbit	9.0.1	`default-features = false`; features `rust-tls`, `http-api-client`. The default `default-tls` feature pulls reqwest/native-tls and an OpenSSL sha1 backend (`crypto-hash`), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7
serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5
clap	4.6.6	derive
infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6
tracing 0.1.44 / tracing-subscriber 0.3.23		env-filter

Deferred: nothing.

Gotcha worth keeping: three feature names in the plan were wrong against current crate versions — reqwest/rustls-tls is now rustls, env-var proxy support moved behind system-proxy, and rss/with-syndication does not exist. librqbit's default features drag in OpenSSL; rust-tls is the fix. Whole tree is rustls-only now, no C TLS dependency. Next: step 2 — config.rs + db.rs. (done)

How this was built

Three planned phases, all complete. The dated entries above are the record of each.

The engine (steps 1-8) — repo skeleton with pinned dependencies; config.rs + db.rs (TOML config, SQLite schema); feed.rs (conditional GET, RSS-then-Atom parsing); download.rs (streaming downloads, filters, URL dedupe); retention.rs (oldest-first quota and age reaper); ipc.rs plus the daemon (Unix-socket JSON lines, TTL scheduler, CLI proxying to a running daemon); torrent.rs (librqbit, seeding to a ratio or a time, stall abort); OPML import/export and polish.

The web front end (steps 9-13) — axum serving plain HTML and JS, no WASM toolchain, running inside the daemon so it reads SQLite and the event bus directly: config hot-reload and the [web] section; browsing feeds and entries with ammonia-sanitized show notes; media actions (Range streaming, download on demand, delete, read/starred); feed configuration written back to config.toml; live progress over SSE from the existing broadcast bus.

Multi-user (steps A-D) — accounts, sessions and sign-in (Argon2id, session cookie, a trusted proxy header for Cloudflare Zero Trust); per-user read state; per-user subscriptions; and the rules that follow from one file serving everyone. SQLite stayed: Postgres was considered and dropped, as nothing here contends for writes.

The live checks each step had to pass

ipx add <feed> + ipx fetch → file in download_dir/<Show>/, row in enclosures.
ipx fetch again → no re-download, feed skipped for TTL.
ipx daemon & + nc -U $XDG_RUNTIME_DIR/ipx.sock, send {"cmd":"fetch"} → JSON events; a concurrent ipx fetch proxies to the daemon instead of downloading in parallel.
Delete a downloaded file by hand, ipx fetch → NOT re-downloaded.
Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
ipx reap --dry-run under quota pressure → oldest-first hit list; real run flips rows to reaped.

81 KiB Raw Permalink Blame History