rays
8b8b48302b
- Add feed lists what other accounts subscribe to, most subscribers
first, and subscribes you by id (GET /api/popular, POST
/api/popular/{id}). Rows never carry a URL. Feeds from an OPML and
anything that looks private (a login, credentials in the URL, a key
such as auth= or token=) are never listed, and the subscribe route
checks the id against the same list.
- CHANGELOG.md follows Keep a Changelog 1.1.0: 0.1.0 (2026-09-09, the
CLI), 0.2.0 (2026-09-10, the web UI), 0.3.0 (2026-09-11, accounts and
sharing). The long-form entries moved unchanged to docs/history.md.
- Cargo.toml is 0.3.0. CLAUDE.md says how to add an entry and cut a
release.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
1332 lines
76 KiB
Markdown
1332 lines
76 KiB
Markdown
# How ipx was built
|
|
|
|
The long-form record, newest first: what landed each day, what was wrong before, and what it cost
|
|
to find out. [CHANGELOG.md](../CHANGELOG.md) has the short version, by release; this is where the
|
|
reasoning lives. New write-ups go at the top.
|
|
|
|
See [README.md](../README.md) for what the thing is.
|
|
|
|
## 2026-09-11 — Popular on this server
|
|
|
|
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
|
|
the list was fed. After every successful download, `iPXClass.py:1249` called `doPing`, which sent
|
|
the enclosure URL and the feed URL, unencoded and over plain HTTP, to
|
|
`directory.iPodderX.com/feedData/survey/files`. It was on by default, and only the Mac app's
|
|
preferences could switch it off (`anonFeedback`). The directory itself lived on that server and in
|
|
the Cocoa app, and both are gone.
|
|
|
|
ipx keeps the idea and drops the ping. Add feed lists what other accounts on this server subscribe
|
|
to, ranked by subscriber count. The old ping would have sent a Patreon feed's `auth=` key along
|
|
with everything else, so this is built around not doing that:
|
|
|
|
- A row carries an id, a title, artwork and a count. No URL reaches the page, so no key can.
|
|
- Subscribing goes by id through `POST /api/popular/{id}`, which checks the id against the same
|
|
list. Guessing a private feed's id gets a 400.
|
|
- Anything that looks private is never listed: a login configured for the feed, credentials in
|
|
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
|
|
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
|
|
`unlisted` flag is the upgrade if that happens.
|
|
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
|
|
they would bury everything anyone chose on purpose.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Import an OPML file by uploading it
|
|
|
|
The import screen now has a file picker as well as the paste box. The page reads the file and
|
|
checks it looks like OPML before sending anything. If it doesn't, the page says so and clears the
|
|
picker, and it also clears the picker after an import. The file travels as text and is never
|
|
written to disk on the server, so there is nothing to clean up there.
|
|
|
|
The server now parses the OPML before touching anything, and returns `400` "that is not an OPML
|
|
file" instead of a `500`. `subscribe_opml` takes a parsed document, so `ipx import` also refuses a
|
|
non-OPML file by name before changing anything.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — OPML import and export are per person
|
|
|
|
Importing an OPML now subscribes you to every feed in it. Feeds already in the catalogue cost
|
|
nothing, and unknown ones are added under the OPML's title. Before this, import only added URLs
|
|
missing from `config.toml` and subscribed nobody. So importing an export from another account did
|
|
nothing at all, and a genuinely new feed had no subscriber and was never scanned. The page said
|
|
"Imported 0 feed(s)". `ipx import` had the same gap; it now subscribes the first admin. Both go
|
|
through `subscribe_opml`.
|
|
|
|
Export now lists only your own subscriptions. It used to write out the whole catalogue to anyone
|
|
signed in, including other people's feeds and any private URLs in them.
|
|
|
|
Tests now cover every way in and out:
|
|
- Settings' Export OPML download and paste-to-import screen.
|
|
- A round trip that imports the admin's actual export into a second account and checks both
|
|
exports match. That round trip is exactly what failed.
|
|
- `ipx import` and `ipx export`, run in their own scratch config and database.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — The log is admin-only
|
|
|
|
`GET /api/logs` now returns `403` to anyone who is not an admin, and the page hides the Log button
|
|
from them. Before this, every signed-in person could read the whole log. That includes every
|
|
account's name, every feed anyone subscribes to, and every failed sign-in. `/api/events` stays open
|
|
to everyone, because it carries the scan progress each person's page shows.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Managing users from the web
|
|
|
|
Settings has a **Manage users…** screen for an admin. From it you can add someone, with a password
|
|
or with none for someone the proxy signs in, tick or untick Admin, and remove an account. It is
|
|
backed by `GET/POST /api/users` and `PATCH/DELETE /api/users/{id}`, which return `403` for anyone
|
|
who is not an admin. The only admin cannot be demoted or removed, because nobody would then be
|
|
able to manage accounts except from the CLI on the box. Before this, accounts could only be managed
|
|
with `ipx user`.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Unread feeds first inside an OPML
|
|
|
|
An OPML subscription's feeds, both in the sidebar folder and on its own page, now list the ones with
|
|
unread items first. They were listed alphabetically, so with dozens of feeds the few with anything
|
|
new were scattered through the list. Within each half the order is still alphabetical. The browser
|
|
suite's fixture OPML gained a second feed, Aardvark Radio, which sorts first by name and by position,
|
|
so the new test only passes if unread wins.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Deploying is a Docker image
|
|
|
|
Production moved from a hand-started daemon in code-server to the `iPodderX` container in the Arcane
|
|
project `content`. `CLAUDE.md` now deploys by pushing to the registry at `192.168.1.130:5000` and
|
|
recreating that one service with `docker compose`. The old instructions copied a binary over a
|
|
process nobody supervised, so it did not come back after a reboot. Two known gaps are gone: the
|
|
image does support accounts, and the entrypoint drops to `99:100`, so downloads are no longer
|
|
owned by root. The README's Docker section said `docker compose up -d` builds the image and named the
|
|
service `ipx`; the compose file pulls from the registry and the service is `ipodderx`.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Documentation
|
|
|
|
`PROGRESS.md` became this changelog; the finished step lists moved to an appendix. The README is now
|
|
an overview that points at `docs/`: [configuration](configuration.md) (every key, path and
|
|
environment variable), [cli](cli.md), [users](users.md) (what several people share),
|
|
[sso](sso.md) (refreshed for accounts and the admin-only settings), and
|
|
[architecture](architecture.md) (modules, the scan, the schema, the socket protocol, the HTTP
|
|
API). `CLAUDE.md` collects what someone working on the code needs to know and has already been
|
|
learned the hard way here -- `pkill -x` not `-f`, the page being compiled in, the dead columns on
|
|
`entries`, the Playwright worker that deleted its own database.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Pruning respects a star from anyone
|
|
|
|
Same defect as the reaper, one function along: `prune_entries` guarded on `entries.flagged`, which
|
|
nothing has written since read state moved to `entry_state`. Starring a text item with no file would
|
|
not have saved it from the age sweep. It follows the reaper's rule now, and takes orphaned read
|
|
state with whatever it deletes.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Step D: one file, and everyone who wants it
|
|
|
|
The last of it, which is all about telling the truth before acting:
|
|
|
|
* **Delete says whose file it is.** A feed with other subscribers labels the button *Delete for
|
|
everyone*, and the confirmation names how many people share it. The server then has the last
|
|
word: if anyone else has starred the item or not played it yet, `DELETE /api/enclosures/{id}`
|
|
returns **409** with the reason in plain words, and only `?force=true` goes through. So the
|
|
rule is enforced where it matters rather than in the page that happens to be asking.
|
|
* **A feed's header says it is shared** -- "shared with 1 other person" -- which is the answer to
|
|
"why is there a file here I never asked for": someone else's subscription fetched it, and one
|
|
copy serves you both.
|
|
* Retention already respects the same rule from the entry before this one: starred by anyone keeps
|
|
it, read by everyone releases it.
|
|
|
|
`others_wanting` is tested with three subscribers disagreeing, and a browser test walks the whole
|
|
delete flow: the label, both prompts, declining the second, and the file still being there.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Retention caught up with per-user state
|
|
|
|
Moving read and starred into `entry_state` left `reap_candidates` reading `entries.read` and
|
|
`entries.flagged`, which nothing writes any more: **starring stopped protecting a file**, and the
|
|
"delete the ones already read first" ordering was dead. Harmless while the quota and age limits are
|
|
0 -- the reaper deletes nothing at all then -- but it would have bitten the moment one was set.
|
|
|
|
One file serves every subscriber, so both tests are now about all of them: **anyone** starring it
|
|
keeps it, and it only counts as read once **everyone** subscribed has read it. A file whose feed
|
|
nobody subscribes to has nobody left to keep it, so it sorts with the read ones. Tested with two
|
|
subscribers disagreeing.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Steps B and C: what is yours, what is everyone's
|
|
|
|
Read, starred and playback position moved out of `entries` into `entry_state (user_id, feed_id,
|
|
guid, ...)`; subscriptions became rows in `subscriptions (user_id, feed_id, ...)` carrying **your**
|
|
keywords, auto-download, explicit and per-scan limit. The feed list, unread counts, filters and
|
|
mark-all-read are all per person now. On first start the existing library is adopted by the admin:
|
|
2438 read/starred items and all 86 feeds, so nothing was lost.
|
|
|
|
The split follows from the file being shared:
|
|
|
|
* **Yours**: read state, starred, position, keywords, auto-download, explicit, per-scan limit,
|
|
and which feeds you see at all.
|
|
* **Everyone's**: the feed URL, its download folder, and when it is scanned -- there is one copy of
|
|
a file however many people subscribe, so those describe the file, not a preference. Admin-only,
|
|
refused with a 403 for anyone else rather than merely hidden.
|
|
|
|
Scanning merges the subscribers' wants, because one fetch and one file serve them all: an item is
|
|
downloaded if **anyone** wants it (any one person's keyword set matching is enough, and one person
|
|
taking everything removes the filter), auto-download is on if anyone has it on, and the per-scan cap
|
|
is the largest anyone asked for. `merge_policy` is a pure function with a test covering each of
|
|
those. Subscribing to a feed someone already has costs no second fetch and no second copy on disk;
|
|
unsubscribing takes it off your list alone, and only when the last subscriber leaves does the feed
|
|
stop being scanned.
|
|
|
|
**A test-harness bug worth naming**: Playwright imports the config in every worker, so the fixture's
|
|
`prepare()` ran again mid-run and deleted the data directory out from under the daemon. The daemon
|
|
kept serving from the unlinked inode while the CLI and any query opened a fresh empty database at
|
|
the same path -- which looked exactly like sign-in being broken. Only the launching process wipes
|
|
now (a worker has `TEST_WORKER_INDEX`).
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Scanning is the operator's decision
|
|
|
|
The per-feed **Check schedule** picker is gone from feed settings, and the global Settings page is
|
|
hidden from anyone who is not an admin. Both are enforced in the handlers rather than merely hidden:
|
|
`PATCH /api/settings` and a `schedule` in `PATCH /api/feeds/{id}` return 403 for an ordinary user.
|
|
Polling costs bandwidth, is what a publisher notices, and one impatient setting affects everyone
|
|
reading that feed -- it belongs in config.toml.
|
|
|
|
Folders, keywords, per-feed download limits and the feed URL stay editable by anyone signed in.
|
|
|
|
`docs/sso.md` covers putting Cloudflare Zero Trust or Authentik in front of ipx: the tunnel and
|
|
Access application, the Authentik proxy provider and its forward-auth nginx block, the three lines
|
|
of ipx config each needs, and why `trusted_proxies` must name the proxy rather than a subnet --
|
|
with the command to prove the refusal works.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Marking an item read
|
|
|
|
Two bugs in one place. `epAction`'s `redraw` closure called *itself* when it had a row to
|
|
update -- `if(el) redraw()` where it meant to swap the row -- so the Mark read button in the
|
|
text pane recursed until the stack blew. It now swaps that one row in place and refreshes the
|
|
text below only when it is the item being read.
|
|
|
|
And opening an item now marks it read, which is what clicking a thing to read it means. The row
|
|
is redrawn where it stands rather than the list reloaded, so an item does not vanish from under
|
|
the pointer on the Unread tab.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Step A: accounts and sign-in
|
|
|
|
`users` and `sessions` tables, Argon2id hashing, a session cookie, and `ipx user add|list|passwd|rm`
|
|
(passwords come in on stdin, so they miss the shell history and any `ps` listing).
|
|
|
|
Three ways in, in order of how specific the claim is:
|
|
|
|
1. **A proxy header** naming the user -- `Cf-Access-Authenticated-User-Email` for the Cloudflare
|
|
Zero Trust in front of `ipodderx.sdf1.net`. Honoured **only** from an address in
|
|
`trusted_proxies` (loopback by default): a header is worth exactly as much as the hop that set
|
|
it, and the LAN port would otherwise let anyone claim to be anyone. Verified both ways -- a
|
|
spoof from an untrusted address is refused.
|
|
2. **A session cookie** from signing in at `/login`.
|
|
3. **The shared token**, which is the admin, so the healthcheck and existing links keep working.
|
|
|
|
A database with no accounts creates **admin / ipodderx** and says so loudly in the log. The UI shows
|
|
who is signed in above the sidebar footer, with a sign-out, and a 401 sends the page to `/login`.
|
|
|
|
Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B.
|
|
|
|
---
|
|
## 2026-09-10 — Items, not episodes
|
|
|
|
Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search
|
|
box, the empty detail pane, the phone back button, the retention and per-feed settings, and the
|
|
download dialog all say **item** now. The sidebar's `20 eps` reads `20 items`.
|
|
|
|
`S1E1` badges stay -- those come from `itunes:episode`/`itunes:season` and only appear when a feed
|
|
actually publishes them -- as does the `episode` column, which is that same field.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Scanning stopped strobing
|
|
|
|
A scan of 85 feeds fires 85 `feed_done` events, and the UI refreshed on every one: a full sidebar
|
|
rebuild plus an episode reload each time, which read as flicker. Bursts now collapse into one
|
|
refresh (500ms trailing), the per-feed "N new" toasts add up into a single summary, and both lists
|
|
keep their scroll position across a rebuild instead of jumping to the top.
|
|
|
|
Measured over a full scan: 11 feed refreshes and 2 episode reloads in 45 seconds, down from one per
|
|
event.
|
|
|
|
Settings and Log moved out of the header and toolbar into a footer under the feed list -- they are
|
|
housekeeping, not daily controls. The theme toggle stays in the header.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Defaults and chrome
|
|
|
|
* **All** is the default episode filter and the first tab; Unread, Downloaded and Flagged follow.
|
|
Opening a feed and seeing nothing because everything in it was read is a poor first impression.
|
|
* **OPML import/export moved into Settings**, under a Subscriptions heading, and the OPML button is
|
|
gone from the sidebar -- it is a thing you do once in a while, not a daily control.
|
|
* **The feed actions are pills with icons**, and Unsubscribe is pushed to the far end in a quiet
|
|
style: it sat next to Settings looking exactly like it, one slip away from losing a feed.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Phone layout
|
|
|
|
The UI was unusable on a phone, starting with the worst of it: the ☰ button lived inside the player
|
|
bar, which is hidden until something plays, so there was no way to reach the feed list at all. It
|
|
now sits in a small bar at the top of the main pane that is always there, and the sidebar slides
|
|
over the page with a scrim to tap away.
|
|
|
|
The rest:
|
|
|
|
* **One pane at a time.** The three-pane split becomes a list, with the item text taking the whole
|
|
screen over it and a `← Episodes` button back. The divider is hidden.
|
|
* **No sideways scrolling.** A grid column is min-content wide by default, so one long headline
|
|
("davewiner/hackerNewsStars") dragged the entire page off the right edge -- `min-width:0` down the
|
|
shell/main/wrap chain, `overflow-wrap:anywhere` on headings, and an explicit
|
|
`minmax(0,1fr)` column for the OPML child list.
|
|
* **The player stacks**: title row on top, transport and seek bar below, and it paints above the
|
|
reading pane so it stays reachable.
|
|
* Header artwork, buttons and log rows shrink to fit; tap targets go to 38px.
|
|
|
|
A Playwright case at 390x844 locks in the three things that actually broke: the burger is visible
|
|
with nothing playing, the page does not scroll sideways, and an item opens and closes over the list.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Marking a subscription read
|
|
|
|
An OPML subscription's page now has **Mark all read**, sitting where every other feed keeps it --
|
|
before Unsubscribe. The fix is in the handler rather than the button: `read-all` resolves the feeds
|
|
whose group is the given id (through `subscriptions()`, so a child promoted to config counts too)
|
|
and marks those, since the subscription's own row holds no entries and marking it did nothing.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — A folder counts what it holds
|
|
|
|
An OPML subscription has no entries of its own, so its row always read `0 unread` no matter how much
|
|
was waiting inside it. Unread and saved are now summed from the feeds it holds -- from all of them,
|
|
not just the ones a search filter left showing, so the number doesn't move as you type. The badge
|
|
caps at `999+` (the real figure is in its tooltip); a four-digit count ate the title beside it.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Sidebar alignment
|
|
|
|
The feed list had four different left edges: a row with no children skipped the chevron entirely, so
|
|
its artwork sat a chevron-width left of a folder's; children then used a different indent *and* a
|
|
smaller icon. Nothing lined up with anything.
|
|
|
|
Every row now reserves the chevron slot whether or not it opens (an empty one is
|
|
`pointer-events:none`, so the click falls through to the row), all artwork is one size, and nesting
|
|
reads from the indent alone. Both label lines are `display:block` on a shared line-height instead of
|
|
an inline baseline, and the unread count has a `min-width` so three-digit feeds don't shove the
|
|
title. Rows came out shorter, so more feeds fit without scrolling.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — An item's picture
|
|
|
|
An item's artwork now resolves in order of how deliberate the source is: `itunes:image`, then Media
|
|
RSS `media:thumbnail`, then a `media:content` that declares itself an image, and finally an image
|
|
**enclosure**. That last one matters here -- Substack puts each article's header picture in an
|
|
`<enclosure>`, which is why those blog entries had no artwork despite carrying one all along. Audio
|
|
enclosures are never mistaken for pictures.
|
|
|
|
Backfilling needed the validators cleared first: `record_entry` fills a missing image on update, but
|
|
a 304 skips parsing entirely, so the feeds would have kept their blank squares. (The self-heal added
|
|
earlier only fires when a feed has *zero* entries, which was not the case here.)
|
|
|
|
Result across the library: 325 of 3470 entries now carry their own picture, 13 feeds where every
|
|
entry has one, 69 feeds that publish no per-item image at all -- those fall back to the feed's
|
|
artwork, which is the intended behaviour rather than a gap.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Database cleanup, and the 304 trap it walked into
|
|
|
|
Cleaned up on request: removed the `CT Log Archive Torrents` folder (123 preallocated files from the
|
|
abandoned 1.6 TB tuscolo torrent, **19 GB real** on disk, no row referencing any of it, feed no
|
|
longer subscribed), the unsubscribed ct-log feed's rows, and every OPML-derived entry/enclosure that
|
|
held no file so a rescan could rebuild them with the current parser. Kept every row holding a file,
|
|
so nothing on disk was orphaned, and kept the skipped/reaped history, without which the next scan
|
|
would re-download the 149 images just deleted. Disk 22 GB -> 3.8 GB, database 22.8 MB -> 5.2 MB.
|
|
|
|
Correction worth recording: mid-download I checked that torrent folder, saw `du -sh` report 8 KB,
|
|
and told Ray it was sparse with nothing written. By the time it was abandoned it had allocated 19 GB.
|
|
The reassurance had a shelf life I did not mention.
|
|
|
|
**Then "Abort Retry Fail is empty".** The cleanup deleted entries but left each feed's
|
|
ETag/Last-Modified. The rescan sent them, servers answered 304 (120 times in the log), the daemon
|
|
skipped parsing, and 57 feeds stayed empty -- and would have until a publisher happened to change
|
|
something. Cleared the validators on the empty feeds and rescanned: entries 1524 -> 3009, feeds with
|
|
content 19 -> 64, Abort Retry Fail back to 20.
|
|
|
|
Fixed in code so it cannot recur: a 304 arriving while the feed has **zero stored entries** means the
|
|
validator has outlived the data -- a restore, a manual edit, a cleanup. The daemon now believes the
|
|
database over the validator, drops it and asks again. Proved live by deleting a feed's entries,
|
|
leaving its ETag, and rescanning: 20 entries rebuilt, self-heal logged once.
|
|
|
|
**"not a wanted media type"** was internal jargon reaching the UI, and it was stored in `last_error`
|
|
so an ordinary filter decision rendered in red as though something had failed. Reworded to "not
|
|
audio or video", 144 existing rows updated, and the UI now paints a reason red only when the state
|
|
is actually `error`.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Multiple enclosures per item, and viewing without downloading
|
|
|
|
**View without downloading.** A non-audio/video enclosure now carries a View link opening in a new
|
|
tab: the publisher's own URL when nothing is downloaded, the local copy at `/media/<id>` when it is.
|
|
Deliberately a direct link rather than a proxy — relaying arbitrary URLs through the daemon would
|
|
make it a fetch-anything service.
|
|
|
|
**Multiple enclosures.** Probed before assuming, and the result was worse than expected: the `rss`
|
|
crate models an item as having at most one enclosure (which is what RSS 2.0 says) and when a feed
|
|
carries several it silently keeps **the last**, dropping the rest. So a two-file item lost its first
|
|
file entirely.
|
|
|
|
`enclosures_by_item()` reads them straight from the XML with quick-xml, in document order,
|
|
unescaping attribute values — a feed URL's `&` arrives as `&`, so skipping that would corrupt
|
|
every query string. It falls back to the parsed enclosure if the scan and the parser disagree on
|
|
item count. Atom already collected all `rel="enclosure"` links. The row now summarises the enclosure
|
|
you would act on (playable, else downloaded, else first) and says "+N more files"; the pane below
|
|
lists them all.
|
|
|
|
**A real limitation surfaced by a broken fixture.** Two browser tests failed with zero enclosures in
|
|
the detail pane. Not the new scanner — verified by running it against the fixture files directly,
|
|
which was right every time. The cause: my fixtures pointed two feeds at the *same* enclosure URL,
|
|
and `enclosures.url` is UNIQUE across the whole database, so whichever feed is scanned first claims
|
|
it and the other's entry gets nothing. That is the dedupe working as designed, but it means **two
|
|
feeds legitimately sharing a media URL will only ever show it under one of them** — worth knowing,
|
|
and worth revisiting if a network feed and a show feed ever overlap.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — A file is not the same as a playable file
|
|
|
|
Reported: "Abort Retry Fail still shows downloaded and audio playback UI". The media-type filter
|
|
stopped *new* image enclosures being fetched, but those 20 JPEGs were already on disk from before
|
|
it existed — and the UI gave a play button and an `<audio>` element to anything with a `path`. An
|
|
audio element pointed at a JPEG is just a broken player.
|
|
|
|
`isPlayable()` now separates having a file from being playable: audio/* or video/*, falling back to
|
|
the file extension when a feed declares no type. Four places used the wrong test — the row's play
|
|
button, the artwork's play overlay, `encBox`, and `play()` itself, which picked the first downloaded
|
|
enclosure regardless of what it was. A downloaded non-media file now shows as
|
|
`image · downloaded · Save · Delete file`, so it is still there and still retrievable, just not
|
|
pretending to be an episode.
|
|
|
|
Covered by a new browser test with a fixture blog whose entry carries a JPEG enclosure and a feed
|
|
configured to download it: no play button on the row, no `<audio>` in the detail pane, and the Save
|
|
button still present.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Three-pane layout
|
|
|
|
Ray asked for feeds beside, items above, and the selected item's text with its enclosures below —
|
|
the layout iPodderX used. Went looking for a screenshot to work from: the archived ipodderx.com kept
|
|
only marketing panels and 130px feature icons, no full window, and the review screenshots lived on
|
|
MacMerc which the archive query did not surface. Said so rather than pretending to have reference
|
|
art; the description is a precise spec on its own.
|
|
|
|
`#split` is a grid of list / divider / detail. Selecting a row no longer expands it inline — it
|
|
highlights and fills the pane below with the title, metadata, read/keep buttons, the sanitized notes
|
|
and one block per enclosure (a player when the file is here, otherwise what it is and a Download
|
|
button). The divider drags, and the split is remembered in localStorage. The OPML group page keeps
|
|
the old scrolling layout via a `plain` class.
|
|
|
|
**Process note, because it cost a dozen calls.** Four anchor-patches in a row failed on text I had
|
|
guessed rather than read: `return el` not `return div`, `open:new Set()` mid-line rather than
|
|
starting one. Each assert aborted before writing, so the file kept reverting to a half-applied
|
|
state, and one earlier patch did land while broken and left a syntax error. What worked was
|
|
reverting to the committed copy and then reading each block verbatim before touching it, with regex
|
|
for the whitespace-sensitive parts.
|
|
|
|
The suite paid for itself again: the new pane test failed on `#detail audio` because
|
|
`max_new_per_check = 1` plus newest-first means the daemon fetches *Second* Episode, so First has no
|
|
player. My assumption, not a bug — the same mistake as the earlier `S1E1` one. The test now finds
|
|
the downloaded row by its chip instead of assuming which episode it is. 9 browser tests pass.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Image enclosures were being treated as episodes
|
|
|
|
Reported as "Abort Retry Fail shows downloads but there are none". It had 20 enclosures, all
|
|
`image/jpeg`: Substack puts each article's header image in the RSS `<enclosure>` tag, so ipx
|
|
downloaded 20 JPEGs and counted them as episodes. Across the OPML subscription: 149 images, 74 MB,
|
|
11 feeds.
|
|
|
|
`[general] media_types` defaults to `["audio", "video"]`, with a per-feed override. An enclosure
|
|
whose top-level type is not wanted is recorded and shown but not auto-downloaded.
|
|
|
|
Clarified mid-change: those enclosures should still be *visible*, just not fetched automatically.
|
|
So nothing is hidden — the row names what it is ("image", "pdf", "torrent") instead of a bare
|
|
"skipped", and the download button still works if you want that file.
|
|
|
|
Two judgement calls in `wanted_media`: an **unknown or absent** type is allowed, because the real
|
|
type is only known after downloading and refusing everything untyped would drop feeds that simply
|
|
omit the attribute; and a **torrent** is allowed, being a container rather than media, judged once
|
|
unpacked.
|
|
|
|
The enclosure-less case Ray also described was already correct — every download affordance in the
|
|
row is gated on the enclosure existing. Checked before changing anything.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Daemon I/O log tab, and Playwright
|
|
|
|
**Log tabs.** All / Daemon I/O / Scans / HTTP. "Daemon I/O" is the control protocol itself: every
|
|
command arriving (`-> {"cmd":...}`) and every event leaving (`<- {"ev":...}`), logged under
|
|
`ipx::io` at the one point every command funnels through, so it catches socket clients, the CLI
|
|
proxying and the web UI alike. Rendered in that tab as in/out rather than level+target.
|
|
|
|
The two logging destinations now have **separate filters**: stderr follows `IPX_LOG` (default
|
|
`ipx=info`), the in-app buffer follows `IPX_UI_LOG` (default `ipx=debug`) and holds 5000 lines. The
|
|
UI can therefore show protocol traffic and routine skips that would be noise on a terminal.
|
|
|
|
**Playwright.** `npm install` plus `npx playwright install --with-deps chromium`, added to
|
|
`/src/install.sh`. `playwright.config.js` starts a fixture feed server and a daemon on a scratch
|
|
config, and eight tests drive a real browser. Each maps to a bug that actually reached Ray.
|
|
|
|
Two things the harness taught, both worth keeping:
|
|
|
|
- **Playwright starts `webServer` before `globalSetup`.** Writing the scratch config in globalSetup
|
|
meant the daemon launched with no config, fell back to defaults, and tried to seize the live
|
|
daemon's socket. The config is written at config-load time instead.
|
|
- Ports 8097/8771 looked free but `ss` showed 8097 held by a process outside this container (shared
|
|
host network). Moved to 8791/8792.
|
|
|
|
**The suite found a real bug on its first green-ish run:** OPML folders rendered *expanded* by
|
|
default. The code stored the set of closed groups, so a folder never toggled — every folder in a
|
|
fresh browser — counted as open, the exact opposite of the comment above it. It stores the open ones
|
|
now. Two other failures were my tests' fault, not the code's: asserting `S1E1` on `.ep.first()` when
|
|
newest-first put a different episode there, and an unscoped `getByText` matching both the sidebar
|
|
entry and the page heading. The `SE` in that first failure was the artwork placeholder's initials,
|
|
which I nearly misread as a broken chip.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Regression: derived feeds looked "unsubscribed" to half the code
|
|
|
|
Reported as `error: enclosure 235 belongs to unsubscribed feed "abort-retry-fail"`. Moving OPML
|
|
feeds out of config.toml meant a config-only lookup no longer finds them, and `download_one` still
|
|
did exactly that — so Download on any episode from an OPML subscription failed outright.
|
|
|
|
Worse than the one bug: **my first grep for the pattern gave false confidence.** `grep -n
|
|
'cfg\.feeds\.get'` returned three hits, all legitimate, so it looked clean — but the failing call
|
|
was split across lines, `cfg` then `.feeds` then `.get`, and never matched. Re-searching with
|
|
newlines collapsed found sixteen `.feeds` accesses, several of them wrong:
|
|
|
|
- `download_one` — the reported failure.
|
|
- `Status` and the daemon's startup line counted config feeds only: 3 reported where there are 85.
|
|
- `add` deduped and slugged against config only, so adding a URL an OPML already lists would have
|
|
duplicated it, and a new feed could collide with a derived feed's id.
|
|
- `rm` on a derived feed said "no feed with id".
|
|
- The web add endpoint had the same duplicate hole.
|
|
|
|
All now go through `subscriptions()`. Verified against the exact failure: enclosure 235 went
|
|
`pending` -> `done`.
|
|
|
|
Lesson recorded because it will recur: when a lookup moves, a single-line grep is not a survey.
|
|
Collapse newlines before searching Rust method chains.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — OPML feeds out of the config, a real cap, and daemon output in the log
|
|
|
|
Three reports in quick succession, all fair.
|
|
|
|
**"Config is a mess."** It was: 611 lines, 85 feeds, 82 of them machine-generated, drowning the three
|
|
Ray actually chose. Writing derived data into a hand-edited file was the wrong call. OPML children
|
|
now live in the `feeds` table (`managed = 1`, `group_id`), are re-derived every scan, and inherit the
|
|
subscription's settings — so there is nothing to store but a URL and a parent. Editing one promotes
|
|
it to a real config entry, so `config.toml` only ever holds decisions. A one-time migration moves
|
|
existing children out: **611 lines -> 38**, 85 feeds -> 3, with all 3487 entries and 216 files intact.
|
|
|
|
**"Never download more than 3."** `max_new_per_check` defaulted to *unlimited*, so subscribing to an
|
|
OPML of 82 feeds pulled every back catalogue it could reach — 216 files, 22 GB before it was caught.
|
|
There is now `[general] max_new_per_check = 3`, used whenever a feed does not set its own, so all 83
|
|
uncapped feeds were capped without touching a line of their config. `pending()` also orders by publish
|
|
date now: a cap of 3 meant "the three recorded first", not the three newest.
|
|
|
|
**"No daemon output in the log."** Scans and downloads travel as events to the socket, not through
|
|
tracing, so the log view showed only startup and HTTP lines. `Emitter::emit` mirrors them now — and
|
|
levelling matters at this scale: at 82 feeds, one INFO line per feed per tick for "not due yet"
|
|
flushed the 2000-line buffer of anything useful in minutes, so routine skips and progress are DEBUG,
|
|
real activity is INFO, failures WARN.
|
|
|
|
A splice while refactoring cut `reject` and `fetch_one` out of main.rs, and recovering them from git
|
|
over-copied three more. Both caught by the compiler, restored, and verified byte-identical against
|
|
`git show HEAD:src/main.rs` rather than eyeballed.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Subscribing to an OPML, not just importing one
|
|
|
|
Asked whether this version could do what iPodderX did: subscribe to an OPML and get a folder of the
|
|
feeds inside it. It could not — `ipx import` was one-shot from a file and nothing ever re-read it.
|
|
The original checked for a URL ending in `.opml` and re-fetched it on every scan
|
|
(`iPXClass.py:34-40`).
|
|
|
|
Now: a feed whose body sniffs as OPML is a subscription list. Sniffing the body rather than the URL
|
|
extension also catches an OPML served from a `.rss` or extension-less URL, which the original missed.
|
|
Every scan re-reads it; listed feeds become real config entries with `group = "<opml-id>"` inheriting
|
|
the parent's settings, land in one nested folder, and are scanned in the same run rather than waiting
|
|
an interval.
|
|
|
|
Removal policy, Ray's rule: **a downloaded file is never orphaned.** Gone from the OPML with nothing
|
|
downloaded -> unsubscribed and removed. Gone but with downloads -> kept and flagged `orphaned`, with
|
|
the UI saying why. Verified end to end on a local fixture: the feed with 1 download was kept and
|
|
flagged, the one with 0 was removed, the file untouched.
|
|
|
|
`folder_for` had to change: it sanitized the whole folder string, so `sub/Show` would have collapsed
|
|
into one directory. Each path segment is sanitized separately now, with a test that `../../etc/Show`
|
|
cannot climb out of the download directory.
|
|
|
|
Also: `Db::memory()` now runs `migrate()` like a real open. It did not, so a column added only in the
|
|
migration passed the tests while being absent in production — which is exactly backwards.
|
|
|
|
Live result: Ray's `lists.opml.org` subscription expanded into 82 child feeds, all grouped, 1431
|
|
entries, no feed errors, worker still responsive mid-scan.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Log view in the app, and Docker
|
|
|
|
**Log view.** A ring buffer (2000 lines) fed by a `tracing` layer, exposed at `/api/logs` with a
|
|
sequence cursor so the UI polls for "everything after N" without duplicates. Tailing a file was the
|
|
obvious approach and the wrong one: under Docker the logs go to stdout and there is no file. An
|
|
`access_log` middleware adds a line per HTTP request, so the daemon and the web side share one
|
|
stream — which is what Ray asked for. That middleware skips `/api/logs` itself, or the panel's
|
|
2-second poll would generate a line per poll forever, a log of nothing but its own requests.
|
|
|
|
UI: sidebar **Log** button, level and text filters, follow-tail toggle, copy button, colour-coded
|
|
levels.
|
|
|
|
**Restart safety.** Detached torrents introduced a `downloading` state, and a row left in it by a
|
|
restart would sit there forever — the pending queue skips it and nothing else revisits it.
|
|
`requeue_interrupted()` resets those at startup, leaving alone any row that already has a file.
|
|
|
|
**Docker.** Multi-stage build (deps cached separately from source), `debian:bookworm-slim` runtime,
|
|
114 MB. Entrypoint writes a starter config bound to `0.0.0.0` on first run, because a container's
|
|
loopback is unreachable from outside, and drops to `PUID:PGID` via gosu. Healthcheck is
|
|
`ipx status`, which goes through the control socket to the worker and so catches a *wedged* daemon,
|
|
not just a dead one.
|
|
|
|
Built and actually run, not just written: image builds, container starts, writes its config, serves
|
|
the UI (401 without a token, 200 with), `/api/logs` answers, and the process runs as
|
|
`uid=99(ipx) gid=100(users)` with files owned `99:100` and the token file at mode 600.
|
|
|
|
Worth noting for future testing here: this session's Docker socket belongs to the Unraid host, so
|
|
`-v /tmp/...:/config` binds a path on the *host*, not one visible from inside this container. The
|
|
mount looks empty from here while being perfectly correct — verify inside the container instead.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Torrents: they work, and one was freezing the whole daemon
|
|
|
|
Asked "is torrents working?". The honest answer had been "unverified since step 7" — so it got tested
|
|
properly.
|
|
|
|
**They work.** End to end against a real swarm: Debian 13.6.0 netinst, 791,674,880 bytes, completed,
|
|
`file` confirms a bootable ISO, row marked `done`. Metadata resolution, transfer, placement and
|
|
completion all good.
|
|
|
|
**But the daemon was wedged.** Ray's CT-log feed had 47 pending torrents and nothing was progressing.
|
|
The worker was blocked: a `status` command over the control socket got no reply in 15s. Torrents ran
|
|
*inline on the single command worker*, so one torrent stopped every feed scan, every HTTP download
|
|
and every status command behind it — for up to `stall_mins` (30 default) waiting on metadata, and
|
|
worse, `Torrents::fetch` blocks on seeding for up to `seed_time_mins` (60 default) **after**
|
|
finishing. One successful torrent could have frozen podcast fetching for an hour.
|
|
|
|
Torrents are now detached: `spawn_torrent` runs the job on its own task behind a 2-permit semaphore,
|
|
the row is marked `downloading` so a rescan does not queue it twice, and the worker moves straight
|
|
on. A one-shot CLI run still runs them inline, or the process would exit mid-download. Verified: the
|
|
worker now answers `status` in 0.0s.
|
|
|
|
A diagnostic misstep worth recording: `/api/settings` returning 200 was taken as proof the worker was
|
|
alive. It is not — the web server is a separate task, so HTTP stays responsive while the queue is
|
|
completely stuck. Probing the control socket is the real test.
|
|
|
|
**Those CT-log torrents are ~557 GB each, 47 of them.** The RSS advertises `length="0"`, so nothing
|
|
warns you; the size only appears once metadata resolves. `auto_download` on that feed is set to
|
|
false as a hold, since restarting would otherwise have begun a half-terabyte transfer immediately
|
|
(the scheduler's first tick fires at startup). Flip it back in the feed's Settings when wanted.
|
|
|
|
Also noted: an aborted torrent leaves librqbit's pre-created file placeholders behind in the
|
|
destination folder (zero-length). Not cleaned up yet.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Schedule pickers, and progress painting every row
|
|
|
|
**Pickers.** "Check feeds every" is a number input plus a unit dropdown (minutes/hours/days/weeks)
|
|
in both global and per-feed Settings, replacing free text. `parse_interval` gained weeks first — it
|
|
only knew m/h/d, so the backend would have rejected the new option. The per-feed dropdown's first
|
|
entry is "Use the default — every N", which disables the number and sends `null`; that rides on the
|
|
null-clearing path fixed earlier today, and would not have worked before it. `splitEvery` picks the
|
|
largest unit that divides evenly, so 120 reads "2 hours" rather than "120 min". The API now returns
|
|
`schedule_mins` so the page does not re-implement the parser.
|
|
|
|
**Bug: one download painted every pending row's progress bar.** Reported after adding the TWiT feed.
|
|
The `progress` event carried feed/url/file but no enclosure id, so the handler had nothing to target
|
|
and set the width on all of them — the code even carried a comment admitting it applied "to whatever
|
|
is downloading now". Adding a feed therefore looked like it was downloading the entire back
|
|
catalogue.
|
|
|
|
`Event::Progress`, `DownloadDone` and `DownloadError` now carry `enclosure`, threaded through both
|
|
the HTTP and torrent paths (`db::Pending` gained its row id to make that possible), and the handler
|
|
targets `.dlbar[data-bar="<id>"]` alone. A test asserts the id is on the wire. Idle bars are
|
|
transparent and only get a track while live, so a row that is not downloading shows nothing.
|
|
|
|
`tests/page-smoke.js` also drives each modal with live-shaped data now — load-time smoke never
|
|
reaches code that only runs when a dialog opens, which is exactly where these changes landed.
|
|
|
|
Verified live: weeks round-trips (`every 2w` -> 20160 min), both pickers render, clearing an
|
|
override returns to the global default. `cargo test` 37/37.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — The whole UI was dead, and server-side tests could not see it
|
|
|
|
Reported as "my feeds seem to have disappeared", then "settings and dark/light mode don't do
|
|
anything either". One cause for all three.
|
|
|
|
`$('#prefs').onclick = prefsModal` referenced a function that did not exist. An uncaught
|
|
`ReferenceError` stops the entire script, and that line sits above the theme toggle, the feed
|
|
filter, the SSE connection and the `loadFeeds()` call that fills the sidebar — so everything below
|
|
it silently never ran.
|
|
|
|
**Cause: a patch anchored on something already deleted.** The scheduling UI was inserted with
|
|
`str.replace` anchored on `function toggleSettings(){`, which belonged to the *old* basic UI that
|
|
the full rewrite had already removed. Python's replace matches nothing and says nothing, and there
|
|
was no assert — so `prefsModal`, `everyText`, `due` and `globalEvery` were never added, while the
|
|
line *calling* `prefsModal` went in fine via a different anchor that did match.
|
|
|
|
**Why it got through.** Every check was server-side: curl for status codes, JSON shape, config
|
|
contents. All passed, because the server was fine. `node --check` also passed — it parses, and a
|
|
ReferenceError is a runtime failure. The page was never executed.
|
|
|
|
`tests/page-smoke.js` now runs the real page script against a stub DOM, fails on anything thrown,
|
|
and flags handlers wired to elements that do not exist. Confirmed non-vacuous by reintroducing the
|
|
exact bug: exit 1 pointing at the offending line, clean once restored. Run it with
|
|
`node tests/page-smoke.js`.
|
|
|
|
Wiring is also defensive now — `on(sel, ev, fn)` logs and skips rather than throwing, so one dead
|
|
reference cannot blank the app again.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Scheduling, and two bugs it uncovered
|
|
|
|
**Scheduling.** The original engine had none — it only skipped feeds on `<ttl>`; the schedule lived
|
|
in the Cocoa GUI that was never open-sourced, so there was no original behaviour to match. Scope
|
|
agreed with Ray: interval only, global default plus per-feed override, no quiet hours.
|
|
|
|
`general.schedule` takes "every 30m", "every 4h", "1d", or a bare number of minutes;
|
|
`feeds.<id>.schedule` overrides it. The legacy `interval_mins` is still read so existing configs
|
|
work, and saving from the UI migrates it. A malformed value in the file warns and falls back rather
|
|
than stopping the daemon; a malformed value over the API is a 400.
|
|
|
|
Precedence, which is a judgement call: an explicit per-feed schedule wins outright, including over
|
|
the publisher's `<ttl>`. With no per-feed schedule, `<ttl>` still raises the interval when the
|
|
publisher asks to be polled less often. Visible in practice — TWiT resolves to 720 min from its own
|
|
ttl against a 60 min global.
|
|
|
|
UI: a gear opens global Settings (schedule, quota, max age); each feed's Settings gains a schedule
|
|
field; the feed header shows "every 4 hours · next in 37m".
|
|
|
|
**Bug 1: `null` never cleared anything.** For `Option<Option<T>>`, serde maps JSON `null` onto the
|
|
*outer* `None`, which the handler reads as "field absent, leave alone" — so `Some(None)` was
|
|
unreachable and every clear was a no-op returning 204. That silently affected `folder`, `schedule`
|
|
and `max_new_per_check`; unsetting any of them from the UI did nothing and looked like it worked.
|
|
Fixed with a `double_option` deserializer, and pinned by a test covering absent / null / value.
|
|
|
|
**Bug 2: the daemon ignored SIGTERM while working.** `tokio::select!` races its branches only at
|
|
selection time; once inside `run(...).await` the shutdown future was not polled at all, so a signal
|
|
queued behind a 90 MB download. Under systemd every restart would have hit `TimeoutStopSec` and been
|
|
SIGKILLed mid-transfer.
|
|
|
|
This one cost real time: `pkill` appeared to work but didn't, a stale daemon kept port 8099, and
|
|
because the restart also `rm -f`'d the socket it defeated the "already listening" guard and a second
|
|
daemon started. Every PATCH "verified" after that went to the *old* binary — so bug 1 was reported
|
|
fixed while completely untested. Lesson: confirm the process actually died and the socket guard is
|
|
doing its job, rather than trusting the restart.
|
|
|
|
The stop signal now lives in a `watch` channel raced *inside* each job. Measured: SIGTERM mid-
|
|
download exits in 1s (was: blocked until the transfer finished). An abandoned download leaves a
|
|
partial in `.ipx-incomplete`, discarded on the next attempt.
|
|
|
|
`cargo test` 37/37.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — App name, and a transparent icon
|
|
|
|
Title is `iPodderX` in the browser tab and the sidebar. The heading's `text-transform: uppercase`
|
|
had to go with it, or the camel case would have rendered as `IPODDERX`.
|
|
|
|
The icon's white background is gone. A blanket `-transparent white` would have punched holes
|
|
straight through the device, whose body is also white — so the background is flood-filled inward
|
|
from all four corners at 12% fuzz, which only clears the *connected* region and stops at the dark
|
|
outline. Then trimmed and scaled to 128px.
|
|
|
|
Checked by compositing onto the real `#0e131b` rather than trusting the alpha channel: no white
|
|
plate, and no pale fringe from the JPEG artifacts along the outline. The `.logo` rule lost its
|
|
`background:#fff` plate and switched to `object-fit:contain`.
|
|
|
|
Both files are tracked: `ipodderx-icon.jpg` is the untouched archival original, `ipodderx-icon.png`
|
|
the transparent version derived from it, so the provenance chain survives. The PNG is embedded as a
|
|
data URI for the sidebar mark and the favicon.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Colour scheme derived from the icon
|
|
|
|
Palette sampled from `web/ipodderx-icon.jpg` with ImageMagick rather than eyeballed. The icon gives
|
|
three families: the silver device ramp (`#1A1A1A` `#606060` `#929292` `#ADADAD` `#D6D6D6` `#F5F5F5`),
|
|
the screen blues (`#2D5391` `#5574A9` `#7595CA` `#92B2E6` `#B4CAED`), one amber from the EQ bars
|
|
(`#F49E2C`), and a blue-grey `#95A0B1`.
|
|
|
|
Dark theme: `--bg` is the screen navy taken right down, `--fg` the device highlight, `--dim` the
|
|
blue-grey verbatim, `--accent` the screen blue, `--accent2` the amber. Light theme flips to the
|
|
device body — white panels, `#D6D6D6` edges, `#1A1A1A` text — with the deeper `#2D5391` as accent,
|
|
since the pale screen blue vanishes on white. Text sitting on an accent fill moved from a hardcoded
|
|
`#0b0e14` to a `--ink` token.
|
|
|
|
Two calls: the amber doubles as the pending state, being the only warm note in the icon and exactly
|
|
where the eye should go. And since the icon has no green or red, the downloaded/error colours are
|
|
tuned to the palette's saturation and temperature rather than invented from scratch.
|
|
|
|
**Contrast was checked, not assumed, and it failed first time.** `--faint` — which carries dates,
|
|
durations and file sizes at small size — came out at 3.92 dark and 3.11 light, both under AA. Moved
|
|
along the icon's own grey ramp to `#7a8799` and `#767676`. Every pair in both themes now clears
|
|
WCAG AA; the lowest anywhere is 4.54.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — The real iPodderX icon
|
|
|
|
The sidebar mark was an invented `ix` placeholder. The original icon is not in this repo or the
|
|
legacy one: only the Python engine was open-sourced, never the Cocoa bundle whose Resources held the
|
|
`.icns` (the legacy repo's history contains nothing but .py/.so/.po/.mo/LICENSE/README).
|
|
|
|
Recovered from the Internet Archive's capture of ipodderx.com: `_graphics/iPXicon.jpg`, 173x165 --
|
|
the white iPod-style device with the antenna and EQ display. That is the largest surviving copy; the
|
|
site's 2005 `favicon.ico` is far too small to use. Saved as `web/ipodderx-icon.jpg` so the file lives
|
|
in the tree, and embedded as a base64 data URI (12.7 KB) for both the sidebar mark and the favicon,
|
|
which avoids an extra asset route.
|
|
|
|
`.logo` changed from a gradient chip to a white rounded tile, since the icon is photo-style on a
|
|
light ground.
|
|
|
|
Provenance is Ray's own: the app was Slakinski & Trometer / Thunderstone Media, and this is its
|
|
successor.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Feed URL is editable, with a copy button
|
|
|
|
The URL in a feed's Settings is now an editable field paired with Copy. Entries and download
|
|
history are keyed by feed id, not URL, so changing it keeps everything — which is the point, since
|
|
a Patreon feed URL carries an auth token that gets rotated.
|
|
|
|
Two things that would otherwise have bitten:
|
|
|
|
- **`navigator.clipboard` does not exist here.** It requires a secure context, and this is served
|
|
over plain HTTP on a LAN address, so the button would have silently done nothing. Falls back to a
|
|
hidden textarea plus `execCommand('copy')`, and reports Copied/Failed either way.
|
|
- **Changing the URL clears the stored ETag/Last-Modified.** Those validators belong to the old URL;
|
|
carrying them over could yield a bogus 304 against the new one and make a working feed look empty.
|
|
|
|
`check_url` is a tested pure function: rejects empty, unparseable and non-http(s) URLs (so
|
|
`file:///etc/passwd` is refused), rejects a URL another feed already uses, and allows a feed to keep
|
|
its own URL unchanged. Validation failures and unknown feed ids now return **400**, not 500 — bad
|
|
input is the caller's mistake, and only genuine server faults are logged as errors.
|
|
|
|
Verified live: the three rejection cases return 400 with readable messages, a no-op save returns 204,
|
|
and the feed still reports 131 entries / 11 downloaded. `cargo test` 34/34.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Fixed: pressing play made an episode vanish from the list
|
|
|
|
Reported as "where did Session Zero go, and why do parts share a number?".
|
|
|
|
Two separate things, one of them a real bug.
|
|
|
|
**The bug.** `play()` marked an episode read the moment playback started. The default view is the
|
|
Unread tab, so pressing play removed the episode from the list you were looking at — indistinguishable
|
|
from it going missing. Confirmed from the data: all four read entries had `position = 0`, meaning
|
|
playback never passed the 10-second save threshold. They had been started, never listened to.
|
|
|
|
Marking read now happens when an episode is actually consumed — on `ended`, or once playback passes
|
|
90% — never on play. Two earlier PROGRESS entries described mark-on-play as a deliberate feature;
|
|
it was a design error, and this supersedes them.
|
|
|
|
Also restored the four affected rows to unread, since their read flag was purely an artifact of the
|
|
bug (`UPDATE entries SET read=0 WHERE read=1 AND position=0` — no genuinely-played episode could
|
|
match, since playing one writes a position).
|
|
|
|
**Not a bug: repeated episode numbers.** Each story part ships as two items — the main episode and a
|
|
shorter `Junk in the Trunk` companion — with distinct GUIDs, durations and files, and the publisher
|
|
gives both the same `itunes:episode`. Zero duplicated titles across all 131 entries. `Session Zero`
|
|
additionally carries a season with no episode number and drops the "Part N" naming, so it does not
|
|
match the pattern its siblings follow.
|
|
|
|
Verified: the arc shows 11/11 items under the default Unread view again.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Titles: RSS `<title>` is the only source
|
|
|
|
Confirmed against the live feed rather than assumed: 131 feed items, 131 stored, **0 mismatches** —
|
|
entry titles are byte-for-byte what the RSS `<title>` publishes, separators included. `feed.rs` never
|
|
consults `itunes:title`, and season/episode live in their own columns, rendered as chips rather than
|
|
folded into the title.
|
|
|
|
Pinned with a test: a fixture whose `itunes:title` differs from its `<title>` must still yield the
|
|
RSS title, and an item with `itunes:season` but no `itunes:episode` keeps a null episode instead of
|
|
inventing one.
|
|
|
|
That null-episode case is real in the wild — this feed's `Music from a Darkened Room | Session Zero`
|
|
carries S8 with no episode number, while Parts 1-5 get E1-E5, and it drops the "Part N" convention
|
|
its siblings use. The arc is 11 items: Session Zero plus five parts, each part being a main episode
|
|
and its shorter `Junk in the Trunk` companion.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Phase 3: full-featured UI
|
|
|
|
Rewrite of `web/index.html` (~714 lines) plus the backend it needed.
|
|
|
|
**New stored metadata**, all of which the real feed carries on all 131 episodes and none of which
|
|
was being captured: feed and per-episode artwork (`itunes:image`), duration (`itunes:duration`,
|
|
parsed from either raw seconds or a `1:34:09` clock), season/episode numbers, and a playback
|
|
`position` so episodes resume.
|
|
|
|
**Schema migration.** `Db::open` now runs `migrate()`: `PRAGMA table_info` then `ALTER TABLE ADD
|
|
COLUMN` for anything missing, since `CREATE TABLE IF NOT EXISTS` does nothing to an existing table.
|
|
Tested on a *copy* of the live database first. Accidentally SIGPIPE'd it partway through (a `head -4`
|
|
on the output) which usefully proved it is idempotent and self-heals: the next run added the
|
|
remaining columns, 131 entries and 6 downloads intact.
|
|
|
|
**New endpoints:** entry filters (all/unread/downloaded/flagged), case-insensitive search over title
|
|
and notes, pagination totals, `POST .../position`, `read-all`, `download-latest`, and OPML
|
|
import/export over HTTP.
|
|
|
|
**UI:** persistent player bar surviving navigation (play/pause, ±15s/30s, 0.8-2.5x speed, volume,
|
|
scrubber, all remembered in localStorage), resume playback via `sendBeacon` every 10s, MediaSession
|
|
for lock-screen controls, keyboard shortcuts (space, arrows, `/`, Esc), artwork everywhere with
|
|
generated-initials fallback, filter tabs, episode search, load-more pagination, live progress bars
|
|
off SSE, real modals, toasts, light/dark toggle, and a mobile layout with a collapsing sidebar.
|
|
|
|
**Bug found and fixed in review: filtering was broken while searching worked.** `count_entries`
|
|
dropped the search clause when no term was given but still bound `?2`; rusqlite rejects a parameter
|
|
the statement never mentions -- *"Wrong number of parameters passed to query. Got 2, needed 1"*. So
|
|
`?q=...` worked and a plain Unread/Downloaded/Flagged filter 500'd. The clause is now always present
|
|
with `?2 = ''` short-circuiting it. Regression test runs all four filters with and without a search
|
|
term and asserts the page and the count agree.
|
|
|
|
**A non-bug worth recording:** artwork looked broken on the live DB (0/131) while a fresh DB filled
|
|
all 131. The parser was fine -- the live DB simply had not been rescanned yet, since backfill happens
|
|
through `record_entry`'s update path. After a forced scan: 131/131 image, duration and season.
|
|
|
|
Verified live: all four filters return sane totals (131/129/10/0), search works (`session zero` -> 2,
|
|
`trunk` -> 10), `/media/1` with a Range header returns 206, index serves 34 KB. `cargo test` 32/32.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Fixed: the UI's Download button downloaded the wrong episodes
|
|
|
|
Reported from the running instance: clicking Download on *Music from a Darkened Room | Session Zero*
|
|
showed progress, then left the episode at `pending`.
|
|
|
|
**Cause, and it was a design error in step 11, not a glitch.** `POST /api/enclosures/{id}/download`
|
|
requeued the row to `pending` and asked for an ordinary scan, justified at the time as "needed no
|
|
new machinery — the queue is the table". But a scan means *take the lowest-id pending rows, up to
|
|
`max_new_per_check`*. Against a 125-item backlog with a cap of 2, that is ids 7 and 8; the requested
|
|
row 11 was never a candidate. The progress on screen was two other episodes downloading past it.
|
|
|
|
The queue can express what is outstanding but not what was *asked for*. `Command::Download {
|
|
enclosure }` is now its own command: it fetches that one row immediately, ignoring both queue order
|
|
and the per-scan cap, still through the single worker so it cannot overlap a scan. Torrent
|
|
enclosures route to `torrent_one` the same way a scan would.
|
|
|
|
Verified live: enclosure 11 went `pending` -> `done`, 90.3 MB on disk, and nothing else was pulled
|
|
in its place. `cargo test` 30/30, with the new command's wire form pinned.
|
|
|
|
Worth remembering: **the daemon scans immediately on its first tick**, so every restart costs
|
|
`max_new_per_check` episodes on the normal path.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Phase 2: web front end (steps 9-13)
|
|
|
|
axum in the daemon process, plain HTML/JS in `web/index.html` (embedded with `include_str!`), no
|
|
WASM toolchain. One binary still.
|
|
|
|
**Config hot-reload.** `Ctx.cfg` is now `RwLock<Arc<Config>>`; `ctx.cfg()` hands out a snapshot, so
|
|
no guard is ever held across an await. The UI rewrites config.toml and calls `reload_cfg`, and a
|
|
running daemon picks the change up on its next scan. The CLI mutation commands (`add`/`rm`/`import`)
|
|
now clone a snapshot, edit, and save.
|
|
|
|
**Auth.** `[web] enabled/bind/token`; the token is minted from `/dev/urandom` on first run, written
|
|
back to config.toml, and the URL printed. `?token=` sets a year-long cookie — it has to be a cookie
|
|
because an `<audio>` element cannot send a header. Comparison is constant-time. An empty token makes
|
|
the server refuse to serve rather than serve open.
|
|
|
|
**Endpoints.** `/api/feeds` (GET/POST), `/api/feeds/{id}` (PATCH/DELETE),
|
|
`/api/feeds/{id}/entries`, `/api/entries/{feed}/{guid}/flags`, `/api/enclosures/{id}/download`,
|
|
`/api/enclosures/{id}` (DELETE), `/api/fetch`, `/api/events` (SSE off the existing broadcast bus),
|
|
`/media/{id}` (tower-http `ServeFile`, so Range works).
|
|
|
|
**`read`/`flagged` finally have a writer** — the UI sets them, and playing an episode marks it read.
|
|
Retention has ordered by these since step 5 with nothing to set them.
|
|
|
|
**Download-on-demand needed no new machinery**: requeue the row to `pending` and kick a scan, since
|
|
the queue is the table. That also un-skips an enclosure a filter rejected under older settings.
|
|
|
|
Verified against the live Glass Cannon feed, daemon on 127.0.0.1:8749:
|
|
|
|
- auth: no token 401, wrong token 401, right token 200 + cookie, API then works on the cookie alone;
|
|
`/media/1` unauthenticated is 401.
|
|
- browsing: 131 entries page with enclosures attached.
|
|
- **XSS**: injected `<script>alert(1)</script><img src=x onerror=alert(2)>` into a stored
|
|
description; it reaches the page as `<p>hi</p><img src="x">` — script tag and handler both gone.
|
|
- media: 200 with `accept-ranges: bytes`; `Range: bytes=1000000-1000999` returns 206 with the right
|
|
`content-range`, so seeking works.
|
|
- config: PATCH -> 204, written to config.toml, and the running daemon reports the new values with
|
|
no restart.
|
|
- SSE: 105 events for one download (101 progress), then `download_done`/`feed_done`/`scan_done`.
|
|
- flags: 204, unread count dropped 131 -> 130. Delete: file gone, row `reaped`/`path=NULL`.
|
|
|
|
`cargo test` 30/30.
|
|
|
|
**Caveats.** It is plain HTTP — on a LAN bind the token crosses the network in the clear, and a feed
|
|
URL may itself carry a credential (Patreon's does), which the `/api/feeds` response includes. A
|
|
reverse proxy with TLS is the answer if that matters. There is no per-user anything; the token is
|
|
all-or-nothing access.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Tested against a real feed (Patreon / Glass Cannon)
|
|
|
|
First run against a live subscriber feed: 131 items, 6.17 GB total, all `audio/mpeg`, no `<ttl>`.
|
|
Parsed clean, titled `Get in the Trunk | Anthology Series | Delta Green`, downloaded a valid MP3
|
|
(`file` confirms ID3v2.3, MPEG layer III). A second scan pulled the *next* episode, confirming
|
|
`max_new_per_check` defers rather than drops. Over the socket: 101 progress events for a 79 MB
|
|
episode -- whole-percent throttling behaving exactly as intended -- then `download_done`,
|
|
`feed_done`, `scan_done`.
|
|
|
|
**Fixed: stripped separators left doubled spaces.** The feed title separates words with `|`, a
|
|
forbidden filename character, so the folder came out `Get in the Trunk Anthology Series Delta
|
|
Green`. The Python had the same wart.
|
|
|
|
Forbidden characters are now split in two. Separators (`/ \\ | :`) become `-`; the rest
|
|
(`? * < > " '`) are simply dropped. A run of dashes and spaces then collapses to `" - "` when the
|
|
run contained whitespace and to a bare `-` when it did not, so:
|
|
|
|
| input | output |
|
|
|---|---|
|
|
| `Get in the Trunk \| Anthology Series \| Delta Green` | `Get in the Trunk - Anthology Series - Delta Green` |
|
|
| `Ep 12: The One` | `Ep 12 - The One` |
|
|
| `AC/DC` | `AC-DC` |
|
|
| `well-known.mp3` | `well-known.mp3` |
|
|
| `../../etc/passwd` | `etc-passwd` |
|
|
|
|
Leading dashes and dots are trimmed too -- a filename starting with `-` trips up CLI tools.
|
|
|
|
**Worth knowing, not a bug:**
|
|
|
|
- The channel-level `itunes:explicit` is `true`, so with the default `allow_explicit = false` all
|
|
131 episodes are skipped. Because filter verdicts are recorded once at discovery, flipping the
|
|
setting afterwards does not re-evaluate enclosures already marked `skipped` -- they would need a
|
|
`UPDATE enclosures SET state='pending'`. Worth a `ipx retry <feed>` command if this bites.
|
|
- Patreon sends neither `etag` nor `last-modified` on a GET (a `last-modified` does appear on HEAD,
|
|
which is what made it look briefly like a storage bug). So this feed can never 304: every poll
|
|
re-fetches ~160 KB and re-parses 131 entries, and `interval_mins` is the only thing limiting the
|
|
rate. Cheap, but it means conditional GET buys nothing here.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 8: OPML and polish
|
|
|
|
`ipx add <url>` fetches the feed to name it from its own title (`Accidental Tech Podcast` ->
|
|
`accidental-tech-podcast`); a feed that cannot be reached is still added, named from its URL, rather
|
|
than refused. `ipx rm` leaves downloads and history alone, so re-adding a feed does not re-pull its
|
|
back catalogue. `ipx import`/`export` walk nested OPML folder outlines and skip URLs already
|
|
subscribed. `tracing` logs to stderr, `IPX_LOG` sets the filter. `contrib/` has a systemd user unit
|
|
for the daemon, plus a timer and one-shot service for the no-daemon style (with the caveat that
|
|
without a daemon there is no socket for a UI).
|
|
|
|
Verified: `cargo test` 27/27. Round-trip — exported 3 feeds with real titles, removed one,
|
|
re-imported: exactly the missing one came back, no duplicates, config still mode 0600. Quickstart
|
|
from a genuinely empty home with no env overrides: `list` on a missing config, `add`, `fetch`
|
|
(1 downloaded, 1 explicit skipped, 1 torrent 404), `list`; every path resolved under `$HOME`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 7: torrent.rs
|
|
|
|
`src/torrent.rs`: a librqbit `Session` started lazily on first torrent (binding ports and starting a
|
|
DHT for a config that has never seen a torrent would be rude). `.torrent` URLs and magnets both go
|
|
through `AddTorrent::from_url`. Progress is polled once a second and reported through the same
|
|
`Progress` events HTTP downloads use.
|
|
|
|
Downloads go **straight into the feed folder** rather than staging and moving. The plan said move
|
|
then seed, which cannot work — seeding serves the files it downloaded, so moving them first breaks
|
|
it. In-place also removes a copy the original had to do.
|
|
|
|
Seeding stops at `seed_ratio` or `seed_time_mins`, whichever comes first, then the torrent is
|
|
released from the session (files kept).
|
|
|
|
**Bug the smoke test caught:** the stall budget only covered the download loop, but resolving a
|
|
magnet's metadata happens inside `add_torrent`, which against a dead swarm never returns — a
|
|
torrent nobody seeds wedged the scan indefinitely. `add_torrent` is now wrapped in the same budget.
|
|
|
|
Verified: `cargo test` 27/27 (ratio incl. the divide-by-zero case, stall_mins = 0 not meaning
|
|
"abort instantly"). Routing: with `enabled = false` a torrent enclosure is marked
|
|
`skipped/torrents disabled` and never attempted. Session startup works here. Stall abort measured
|
|
end to end: with `stall_mins = 1`, a dead magnet failed at 20:58:23 -> 20:59:24, exactly 61s, and
|
|
the row recorded `error / no metadata after 1 minutes, gave up`.
|
|
|
|
**Not verified: an actual successful swarm download.** This sandbox has no reachable peers, so
|
|
smoke 5's happy path — payload lands, seeding stops at the ratio — has not been run. The code paths
|
|
either side of it are tested; the swarm itself needs a real network. Worth running once against a
|
|
live torrent feed before trusting it.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 6: ipc.rs + daemon
|
|
|
|
`src/ipc.rs`: `Event` and `Command` as serde-tagged enums (`{"ev":...}` / `{"cmd":...}`), one JSON
|
|
object per line over a Unix socket. `Emitter` is the single output path — it broadcasts to socket
|
|
clients, prints the human rendering to a terminal, or both, so the scan code no longer knows how it
|
|
is being watched. That replaces `printMSG` and its `;;1;;1;;100.00;;42.31` sentinels.
|
|
|
|
`main.rs` restructured around a `Ctx` (config, db, client, emitter). `ipx daemon` binds the socket,
|
|
serves clients, and runs a 60s ticker that defers to each feed's TTL. Commands from every client
|
|
funnel through one mpsc queue into a single worker, which is what stops two scans overlapping.
|
|
Any CLI subcommand with a wire form probes the socket first and proxies to a running daemon;
|
|
`--local` forces the work to happen in-process. SIGINT/SIGTERM remove the socket on the way out.
|
|
|
|
**Bug the smoke test caught:** `fetch` runs a retention sweep first, and that sweep was emitting the
|
|
terminal `ReapDone`. A UI waiting for its fetch to finish would have stopped reading before the scan
|
|
started. Only a standalone `ipx reap` emits it now.
|
|
|
|
Verified: `cargo test` 24/24. Smoke 3 in full — daemon starts and binds; a raw socket client sending
|
|
`{"cmd":"fetch","force":true}` gets `feed_start` → 14 throttled `progress` events → `download_done`
|
|
→ `feed_done` → `scan_done`; `{"cmd":"status"}` answers `{"ev":"status","feeds":1,...}`. With the
|
|
daemon up, `ipx fetch` from the CLI logged `command from a client cmd=Fetch { .. }` in the daemon
|
|
and rendered the streamed events, so it proxied rather than downloading in parallel. Socket removed
|
|
on SIGTERM; with no daemon the same command runs locally.
|
|
|
|
Deferred: `{"cmd":"cancel","enclosure":N}` from the plan's protocol is **not implemented** —
|
|
downloads run sequentially in one worker, so there is nothing to cancel concurrently yet. It wants
|
|
a per-download cancellation token, which is worth doing when downloads go parallel. Say if you want
|
|
it sooner.
|
|
|
|
Note: the event stream is a broadcast, so a CLI client seeing a busy daemon also sees that other
|
|
work. Fine for a terminal; a UI wanting strict request/response would want per-request ids.
|
|
|
|
Also note: the daemon reads config once at startup — changing `config.toml` needs a restart.
|
|
|
|
Next: step 7 — `torrent.rs`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 5: retention.rs
|
|
|
|
`src/retention.rs`: reconcile pass (rows claiming a file that is gone become `reaped`, fixing the
|
|
step-4 wart), age sweep, quota sweep keeping the original's 50 MB headroom pad, and entry pruning.
|
|
`ipx reap [--dry-run]`; a sweep also runs before every `fetch`, as the Python did per download.
|
|
`pick()` and `aged()` are pure so the ordering rules are testable without touching a disk.
|
|
|
|
**Judgement call worth Ray's eye.** The Python meant to reap only `read = 1 AND flagged = 0` but
|
|
never managed it — a missing `plistlib` import and an `EntreiesData` typo made that filter throw on
|
|
every candidate, so with a `.ipxd` present nothing was ever deleted. Requiring `read = 1` here would
|
|
be equally dead, because nothing marks episodes read until a UI exists. So: **`flagged` is the
|
|
keep-forever marker, and `read` only decides what goes first** (`ORDER BY read DESC, downloaded_at
|
|
ASC`). Quota therefore actually reclaims space headless. Say the word if you would rather unread
|
|
episodes were never touched.
|
|
|
|
Second call: `max_age_days` deletes *files* older than the cutoff, not just fileless entries as the
|
|
plan's wording had it — "keep 30 days of episodes" is what the setting reads like on a NAS.
|
|
|
|
Verified: `cargo test` 21/21, including the two tests encoding the exact bug the Python had —
|
|
flagged files are never offered, and read sort ahead of unread. Smoke 6 with three 30 MB episodes
|
|
against a 0.1 GB quota (52.4 MB ceiling after the pad): dry run listed ep1+ep2 and deleted nothing
|
|
(3 files still on disk), the real run deleted exactly those two oldest, left ep3, flipped both rows
|
|
to `reaped` with `path = NULL`. A full re-parse with the conditional-GET headers cleared then
|
|
re-downloaded nothing.
|
|
|
|
Next: step 6 — `ipc.rs` + daemon.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 4: download.rs
|
|
|
|
`src/download.rs`: streaming download to `<download_dir>/.ipx-incomplete/` (same filesystem as the
|
|
destination, so filing it is a rename, not the original's copy-then-unlink), content sniffing,
|
|
then `place()`. Filename comes from the URL's last path segment, percent-decoded, unless
|
|
`Content-Disposition` names one (RFC 5987 `filename*=` preferred). The sanitizer keeps UTF-8 —
|
|
`latin1_to_ascii` existed because 2004 filesystems demanded ASCII — strips the same characters
|
|
`stringCleaning()` did plus control chars, and adds a real 255-byte cap the Python never had,
|
|
preserving the extension across truncation.
|
|
|
|
Sniffing replaces `detectFileType()`, which called a `typeFile` module that was already missing in
|
|
2008 and so always answered `'data'`. Two rules survive: an HTML body is a failed download (login
|
|
wall/error page), and a torrent body is a torrent whatever the MIME claimed.
|
|
|
|
Filters run once at discovery and are recorded in `enclosures.state`; the download queue is then
|
|
just "everything still `pending`", so an enclosure held back by `max_new_per_check` is picked up by
|
|
the next scan instead of being lost. Keywords are OR'd across keywords and AND'd within one — the
|
|
original's nested loop let a later keyword silently undo an earlier miss.
|
|
|
|
Verified: `cargo test` 16/16. Smoke against a local server, five enclosures, each filter path hit:
|
|
`ep1.mp3 -> done`, `ep2.mp3 -> skipped (explicit)`, `ep2.mp3?v=3 -> skipped (no keyword match)`,
|
|
`paywall.html -> error (HTML page, not media)`, `ep5.torrent -> torrent (deferred to step 7)`.
|
|
Smoke 2 and 4 pass, and because a plain rerun 304s before parsing, dedupe was proved separately by
|
|
clearing the stored etag/last-modified and re-parsing all five entries: 0 downloaded, hand-deleted
|
|
file not refetched, `.ipx-incomplete` left empty.
|
|
|
|
Known wart: `ipx list` counts `path IS NOT NULL`, so a hand-deleted file still reads as downloaded.
|
|
Reconciling rows against the filesystem belongs in step 5.
|
|
Next: step 5 — `retention.rs`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 3: feed.rs
|
|
|
|
`src/feed.rs`: conditional GET (`If-None-Match` + `If-Modified-Since`, optional basic auth) and a
|
|
RSS-first / Atom-fallback parser normalising both into `ParsedFeed`/`Entry`/`Enclosure`. Feed-level
|
|
`itunes:explicit` overrides the entry level, as the original did. `<ttl>` is captured. RSS
|
|
`content:encoded` wins over `description`. Atom enclosures come only from `rel="enclosure"` links.
|
|
|
|
GUID: the original hashed the title or description when no guid existed; here the chain is
|
|
guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5
|
|
dependency. An entry with none of them has nothing to download and is dropped.
|
|
|
|
`db.rs` gained `http_state`, `record_feed`, `touch_feed`, `set_feed_error`, `record_entry`,
|
|
`record_enclosure`. A changed title/description flips `read` back to 0 — what the original's
|
|
textDiff was ultimately for, minus the `<ins>`/`<del>` markup, which belongs in the UI.
|
|
|
|
`main.rs` gained `ipx fetch [FEED] [--force]`. A failing feed records its error and the scan
|
|
continues.
|
|
|
|
Verified: `cargo test` 10/10. Gate met against a local `python3 -m http.server` serving the
|
|
fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run
|
|
showed both skip paths, `atomcast: not modified` (304) and `testcast: not due for 45m` (the feed's
|
|
own ttl=45 beating `interval_mins = 0`).
|
|
|
|
Deferred: nothing downloads yet — enclosure rows land in state `pending`. That is step 4.
|
|
Next: step 4 — `download.rs`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 2: config.rs + db.rs
|
|
|
|
`src/config.rs`: serde structs for `[general]`, `[torrent]` and `[feeds.<id>]` with defaults, `~`
|
|
expansion, `IPX_CONFIG` / `IPX_DATA_DIR` overrides, `save()` at mode 0600, `Feed::password()`
|
|
(`password_env` beats a literal `password`), `Torrent::ports()` parsing `"6881-6889"`. A missing
|
|
config file loads as an empty one so a fresh install works. Retention defaults are 0/0
|
|
(unlimited, keep forever) — nothing gets deleted until Ray asks for it.
|
|
|
|
`src/db.rs`: schema exactly as planned, WAL + `busy_timeout`, `Db::open()` idempotent, connection
|
|
behind a `Mutex`, `feed_summary()` for `list`, `now()` helper. `enclosures.url` is UNIQUE — the
|
|
dedupe key that replaces `history.dat`.
|
|
|
|
`src/main.rs`: clap skeleton with `ipx list`. Only the subcommands that work exist; the rest arrive
|
|
with their steps.
|
|
|
|
Verified: `cargo test` 5/5 green (config defaults, port-range fallback incl. reversed range,
|
|
password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — `ipx --config
|
|
<scratch> list` printed both feeds and created `state.db` once across two runs.
|
|
|
|
Deferred: nothing. Two dead-code warnings (`Config::save`, `Feed::password`) are expected; steps 3
|
|
and 8 consume them.
|
|
Next: step 3 — `feed.rs`.
|
|
|
|
Also added `chrono` 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 1: repo skeleton
|
|
|
|
Repo created at `/src/ipodderx-rs`, default branch `main`, `cargo init --name ipx` (edition 2024,
|
|
rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this
|
|
file. Hello-world `main.rs` builds.
|
|
|
|
Dependency versions pinned today — the later steps are written against these APIs:
|
|
|
|
| crate | version | notes |
|
|
|---|---|---|
|
|
| tokio | 1.53.1 | rt-multi-thread, macros, fs, io-util, net, sync, time, signal |
|
|
| reqwest | 0.13.5 | `default-features = false`; features **`rustls`** (not `rustls-tls` — renamed in 0.13), http2, gzip, stream, json, charset, **`system-proxy`** (env-var proxy pickup is opt-in in 0.13) |
|
|
| rss | 2.1.1 | default features; the `with-syndication` feature name in my notes does not exist — Atom is handled by the separate crate |
|
|
| atom_syndication | 0.12.10 | |
|
|
| rusqlite | 0.40.2 | `bundled` |
|
|
| librqbit | 9.0.1 | `default-features = false`; features `rust-tls`, `http-api-client`. The default `default-tls` feature pulls reqwest/native-tls **and** an OpenSSL sha1 backend (`crypto-hash`), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7 |
|
|
| serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5 | | |
|
|
| clap | 4.6.6 | derive |
|
|
| infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6 | | |
|
|
| tracing 0.1.44 / tracing-subscriber 0.3.23 | | env-filter |
|
|
|
|
Deferred: nothing.
|
|
|
|
Gotcha worth keeping: three feature names in the plan were wrong against current crate versions —
|
|
`reqwest/rustls-tls` is now `rustls`, env-var proxy support moved behind `system-proxy`, and
|
|
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
|
|
is the fix. Whole tree is rustls-only now, no C TLS dependency.
|
|
Next: step 2 — `config.rs` + `db.rs`. (done)
|
|
|
|
---
|
|
|
|
## How this was built
|
|
|
|
Three planned phases, all complete. The dated entries above are the record of each.
|
|
|
|
**The engine (steps 1-8)** — repo skeleton with pinned dependencies; `config.rs` + `db.rs` (TOML
|
|
config, SQLite schema); `feed.rs` (conditional GET, RSS-then-Atom parsing); `download.rs`
|
|
(streaming downloads, filters, URL dedupe); `retention.rs` (oldest-first quota and age reaper);
|
|
`ipc.rs` plus the daemon (Unix-socket JSON lines, TTL scheduler, CLI proxying to a running daemon);
|
|
`torrent.rs` (librqbit, seeding to a ratio or a time, stall abort); OPML import/export and polish.
|
|
|
|
**The web front end (steps 9-13)** — axum serving plain HTML and JS, no WASM toolchain, running
|
|
*inside* the daemon so it reads SQLite and the event bus directly: config hot-reload and the `[web]`
|
|
section; browsing feeds and entries with `ammonia`-sanitized show notes; media actions (Range
|
|
streaming, download on demand, delete, read/starred); feed configuration written back to
|
|
config.toml; live progress over SSE from the existing broadcast bus.
|
|
|
|
**Multi-user (steps A-D)** — accounts, sessions and sign-in (Argon2id, session cookie, a trusted
|
|
proxy header for Cloudflare Zero Trust); per-user read state; per-user subscriptions; and the rules
|
|
that follow from one file serving everyone. SQLite stayed: Postgres was considered and dropped, as
|
|
nothing here contends for writes.
|
|
|
|
### The live checks each step had to pass
|
|
|
|
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
|
|
2. `ipx fetch` again → no re-download, feed skipped for TTL.
|
|
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
|
|
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
|
|
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
|
|
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
|
|
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
|
|
`reaped`.
|