Opening an episode marks it read, so filtering on read hid every episode anyone had started. The player now also reports the length it measured, filling in one the feed left out. Fixes #14. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1706 lines
102 KiB
Markdown
1706 lines
102 KiB
Markdown
# How ipx was built
|
|
|
|
The long-form record, newest first: what landed each day, what was wrong before, and what it cost
|
|
to find out. [CHANGELOG.md](../CHANGELOG.md) has the short version, by release; this is where the
|
|
reasoning lives. New write-ups go at the top.
|
|
|
|
See [README.md](../README.md) for what the thing is.
|
|
|
|
## 2026-09-15 — Currently Listening, empty for anyone who opens what they play
|
|
|
|
Issue #14: an episode 32 minutes into 41 was missing from Currently Listening, which said nothing
|
|
was in progress. The filter was "position past five seconds and unread", on the reasoning that
|
|
`markPlayed` marks an episode read at 90%. But opening an item marks it read too, and you open an
|
|
episode to play it. In production every one of the twelve episodes with a saved position was read,
|
|
so the list was empty for everyone. The browser test for it had set the episode unread before
|
|
checking, to cope with an earlier test having opened it, and so tested around exactly this.
|
|
|
|
Finished now means the saved position is 90% of the episode's length or more, the same line
|
|
`markPlayed` draws, and read plays no part. Three of those twelve episodes had no length in their
|
|
feed, and with no length there is no telling finished from started, so they would have stayed
|
|
listed for good. The player now sends the length it measured with each saved position, and that
|
|
fills in a missing one, never replacing a length the feed gave.
|
|
|
|
## 2026-09-15 — davewiner's 922 rows, retired at last
|
|
|
|
davewiner's OPML subscription left config.toml before `retire_group` existed, so nothing ever
|
|
retired the 922 feeds derived from it. The backstop in `subscriptions()` kept them from being
|
|
scanned, but the rows stayed, and 56 of the 75 errors stored in production were theirs: stale, and
|
|
never going to change.
|
|
|
|
Retiring them the obvious way would have done damage. Eleven of the 922 -- xkcd, The Verge,
|
|
TechCrunch, Hacker News and others -- had since been given config entries of their own and were
|
|
scanned from there, but their rows still said `managed = 1`. `managed_feeds()` returned them, so
|
|
`retire_group("davewiner")` would have dropped the ten with no files as derived feeds, deleting
|
|
the entries people were reading. `unmanage` exists for exactly this and had never been applied.
|
|
|
|
`retire_group` now unmanages a feed that has its own config entry instead of dropping it, which
|
|
also covers removing any OPML or Patreon subscription from the page, and the daemon retires every
|
|
group whose parent is gone from config when it starts. In production that is davewiner alone: 792
|
|
rows with nothing downloaded forgotten, 119 with files kept as orphaned, 11 unmanaged. One of the
|
|
792, `ars-technica-all-content-2`, has a subscriber but neither a config entry nor a file; the
|
|
backstop had already hidden it, so nobody could see it to lose it.
|
|
|
|
## 2026-09-14 — Settings, for everyone with an account
|
|
|
|
A user reported that Settings disappeared shortly after they signed in: it showed for a moment,
|
|
then was gone. `#prefs` sat inside the same `.tgroup` as `#logs`, and `api('/api/me')` hid the
|
|
whole group -- `$('#admintools').hidden=true` -- the moment it learned the account was not an
|
|
admin. Nothing wrong with that check timing; it was hiding the wrong thing.
|
|
|
|
The Settings modal is not actually all-or-nothing. `GET /api/settings`, and Export and Import
|
|
OPML, carry no admin check server-side -- `export_opml` and `import_opml` work from a user's own
|
|
subscriptions, and the schedule/quota page is read-only information, not a control. Only the
|
|
`PATCH` that changes those settings, and the Users screen behind it, return 403 for anyone but an
|
|
admin. The comment above the old hide -- "scanning, quotas, accounts and the log are the
|
|
operator's business" -- was wrong about quotas and half wrong about accounts: reading them is
|
|
everyone's; changing them is the operator's.
|
|
|
|
`prefsModal()` now branches on `S.me.admin` the way the per-feed settings modal already does for
|
|
its URL field: a non-admin gets the schedule and quota as text, Subscriptions (Export/Import)
|
|
in full, and no Users section or Save button. Only `#logs` stays hidden, since the log names every
|
|
account and every failed sign-in. The browser test for a second account asserted the old
|
|
behaviour outright (`#prefs` hidden, not an admin) rather than what the server actually allows;
|
|
fixing the UI meant fixing the test's premise too, not just the assertion.
|
|
|
|
## 2026-09-12 — Healthy while busy
|
|
|
|
After a deploy the container sat at "starting" for a minute, and Docker's health log showed two
|
|
`ipx status` probes exceeding their 5-second timeout. The daemon's own log explained it. The first
|
|
scan after the start fetched 23 feeds, from 14:10:41 to 14:11:35, and both probes' `status`
|
|
commands waited in the job queue behind it; they were answered together at 14:11:35, straight after
|
|
`scan_done`. The worker runs one job at a time and `status` was one of its jobs, so any scan or
|
|
download longer than about a minute and a half, three 30-second probes, would have had Docker call
|
|
a working daemon unhealthy.
|
|
|
|
The socket now answers `status` itself, from two short queries, and only real work goes through the
|
|
queue. The trade is that healthy now means the daemon answers on its socket and can read its
|
|
database; a worker stuck on one job would still pass. Asking a daemon that downloads hour-long
|
|
podcasts to be idle within five seconds was never a fair test of whether it was alive. A test holds
|
|
the queue full and checks `status` still comes back.
|
|
|
|
The first version broadcast the answer, as the queued one had been. Timing `status` during a forced
|
|
scan in production showed the catch: `status` is a terminal event, so the `ipx fetch` watching that
|
|
scan stopped reading at the first probe and printed the status line as its last, while the scan
|
|
carried on. When `status` waited behind the scan it could never arrive first, so this had never
|
|
shown. The answer now goes only to the client that asked, and the test checks that another client
|
|
hears nothing.
|
|
|
|
## 2026-09-12 — Signing in through Authentik, for real
|
|
|
|
Ray could not get Authentik's sign-in to reach ipx, following `docs/sso.md`, which had been written
|
|
without ever being tried. Looking at the Cloudflare account through its API showed that side was
|
|
already complete. Authentik is Zero Trust's OpenID Connect identity provider; the Access application
|
|
`ipodderx` allows only it and a list of five addresses; the tunnel `rays-unraid` routes
|
|
`ipodderx.sdf1.net` to `192.168.1.130:8099`; DNS is a proxied CNAME to the tunnel. Access's log
|
|
showed `rays@sdf1.net` signing in through it. Nothing on Cloudflare was changed, so no other site
|
|
was touched.
|
|
|
|
The gaps were all at ipx's end: `trusted_header` was empty, `trusted_proxies` held only loopback,
|
|
and the account was called `rays` while the header carries `rays@sdf1.net`.
|
|
|
|
Finding the address to trust took the most time. The page said `127.0.0.1`, but `cloudflared` runs in
|
|
its own container and reaches ipx through the host's published port. ipx logs no peer addresses, so
|
|
the address was read from `/proc/net/tcp` inside the ipx container: `192.168.16.1`, the gateway of
|
|
`content_default`, where Docker's masquerade puts traffic crossing from another bridge. A request
|
|
from Tower's own shell arrived as `192.168.1.130` instead, and a throwaway `busybox` on the default
|
|
bridge as `192.168.16.1`: the first was refused with the header, the second believed. LAN machines
|
|
keep their own addresses, since Docker forwards published ports with iptables (the userland proxy
|
|
only handles loopback).
|
|
|
|
Every change, in order, with how to undo it:
|
|
|
|
1. **Code**, commit `586d2c0`: `ipx user rename`, deployed. Revert the commit and redeploy to
|
|
remove it; nothing depends on it once used.
|
|
2. **Account**: `docker exec iPodderX ipx user rename rays rays@sdf1.net`. Same id, so its feeds,
|
|
read state, password and admin rights stayed. Undo: `docker exec iPodderX ipx user rename
|
|
rays@sdf1.net rays`. Signing in at `/login` now takes the new name.
|
|
3. **Config**, `/mnt/fast/appdata/ipodderx/config.toml`, `[web]`: `trusted_header` from `""` to
|
|
`"Cf-Access-Authenticated-User-Email"`, and `"192.168.16.1"` added to `trusted_proxies`. The
|
|
file as it was is `config.toml.2026-09-12-sso.bak` beside it. Undo: copy the backup back and
|
|
`docker compose -f /mnt/fast/arcane/projects/content/compose.yaml restart ipodderx`.
|
|
4. **Cloudflare, Docker networks and other containers**: unchanged. The `busybox` test container
|
|
was removed when it exited, and its image afterwards.
|
|
5. **Authentik**, later the same day, because ipodderx had no tile in its library while Outline
|
|
did: a bookmark application `ipodderx` (pk `5854a98e-816a-4c4f-9f27-63e69dc29d1d`), made
|
|
through the API with a token of Ray's. No provider and no policy bindings, like Outline's, the
|
|
iPodderX icon, and a link to `https://ipodderx.sdf1.net`. It changes nothing about who can sign
|
|
in. Undo: delete it under Applications → Applications, or
|
|
`DELETE /api/v3/core/applications/ipodderx/`.
|
|
6. **Signing out**, later again. Sign out landed on ipx's password page while Access still vouched
|
|
for Ray, so it signed nothing out, and the page looked like the wrong login. Cloudflare's
|
|
`/cdn-cgi/access/logout` ends the Access session for every Access application at once (there is
|
|
no per-application sign-out, and it takes no redirect), and Authentik's end-session only ends
|
|
one application's session unless single logout is set up there. Ray chose Access's sign-out. New
|
|
`[web] sign_out_url`, set to `/cdn-cgi/access/logout` in production (the file as it was is
|
|
`config.toml.2026-09-12-signout.bak`), and `/login` now sends anyone the proxy vouches for on to
|
|
`/`. Undo: take the key out and restart; the code does nothing without it.
|
|
|
|
What the address trusts is any container on Tower that connects through the host's port, not only
|
|
`cloudflared`. Verifying Cloudflare's signed `Cf-Access-Jwt-Assertion` would remove that, and is
|
|
the upgrade if it matters.
|
|
|
|
## 2026-09-12 — Trimming the state database
|
|
|
|
An audit of the database layer, with a read-only copy of production to check it against. The
|
|
data was already clean: no tables or indexes left from older versions, 47 free pages after the
|
|
column drops earlier the same day, and one stray `entry_state` row. The code had five things:
|
|
|
|
- `migrate()` still added eight columns to any table missing them. All eight shipped in 0.2.0 and
|
|
upgrades now start from 0.3.0 at the oldest, so the list and its loop went; the `retired` drop
|
|
list stays, since a database coming from 0.4.0 still has the old read columns.
|
|
- `created` on `users`, `subscriptions` and `sessions` was written by every insert and read by
|
|
nothing. They joined `retired`. The old-database test now builds all three tables, foreign keys
|
|
included, since `DROP COLUMN` on a table that references another was the part worth proving.
|
|
- `Db::subscribed_feed_ids` had no callers, though its doc said the scanner walked it.
|
|
`Db::subscriber_count` had one caller asking whether it was above zero, which
|
|
`subscriber_counts().contains_key` answers. `Managed.orphaned` was selected and never read.
|
|
- `users.created` came back the same afternoon, with `last_login` beside it. Nothing read it, but
|
|
when an account was made and when it last signed in is what you want to know when tidying
|
|
accounts, and it cannot be recovered later. Both existing accounts got their creation times back
|
|
from the backup taken before the drop, and a last sign-in from their newest session in it.
|
|
`last_login` is kept to the hour, because the proxy vouches for every request and that would
|
|
otherwise be a write each time.
|
|
|
|
## 2026-09-12 — Cutting what had outlived its reason
|
|
|
|
A whole-repo audit for over-engineering listed twelve things to cut, and all of them went.
|
|
|
|
- **Upgrades from before accounts.** `migrate_opml_children` moved OPML feeds that old versions
|
|
wrote into `config.toml` out to the database, and ran at every daemon start to do nothing after
|
|
the first. Production ran it in 0.3.0; anything older has to pass through 0.4.0.
|
|
- **Half of the adoption, and not the other half.** The audit called `adopt_existing_library` a
|
|
one-time migration and it was cut whole. It did two jobs: copy the old read state into
|
|
`entry_state`, which was dead, and subscribe the first admin to the whole catalogue while nobody
|
|
subscribed to anything, which is how a fresh install's first account gets `config.toml`'s feeds.
|
|
The browser suite caught it at once, signing in to an empty sidebar; `cargo test` had no idea.
|
|
The second job is back as `adopt_catalogue`, with a unit test of its own.
|
|
- **The dead `entries` columns.** `read`, `flagged` and `position` moved to `entry_state` with
|
|
accounts. The adoption's copy was their last reader, but `record_entry` still wrote them, and
|
|
still reset `read` when a title changed, which nothing looked at. Two bugs came from queries
|
|
reading them. `migrate()` now drops them from an existing database (SQLite has had `DROP COLUMN`
|
|
since 3.35), and a test builds an old table to prove it. On production each drop rewrote the
|
|
66 MB `entries` table, about four seconds apiece, so the first start took thirteen. An
|
|
`ipx status` run in that window failed with `migrating schema`: every `ipx` command migrates when
|
|
it opens the database, and it collided with the daemon doing the same. A failed `ALTER TABLE`
|
|
changes nothing, and the database had been copied to `backup/` first anyway.
|
|
- **`interval_mins`**, which `schedule` replaced. An old config that still has the key loads; the
|
|
key is ignored, and the config test carries it to keep that true.
|
|
- **Three dependencies.** `infer` was only asked whether a file is a torrent, and the check after
|
|
it already looked for `d8:announce`, which is what `infer` looks for. `dirs` was three lookups of
|
|
`XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME`. `tokio-stream` wrapped the broadcast receiver for
|
|
the event stream; `futures_util::stream::unfold` does the same, lagging clients included.
|
|
- **Two token generators.** The web token came from a copy of the session-token code, with a
|
|
clock fallback on top. It uses `auth::new_session_token` now, and is 64 characters.
|
|
- **The icon inlined four times**, 23 KB of base64 each, into both pages. It is `/icon.png` now,
|
|
outside the sign-in wall with `/login`, since the sign-in page shows it.
|
|
- Also: the `contrib/` systemd units from before the container, `Db::entries` and
|
|
`Db::count_entries` that only the tests called, three `logbuf` visitors that repeated the trait's
|
|
defaults, and unused state, a helper and dead CSS in the page.
|
|
|
|
## 2026-09-11 — A design pass on the web UI
|
|
|
|
A review against screenshots of every view in all three themes found that Dark and Light read as a
|
|
generic dark dashboard: one pale blue did every job, most labels were bold, and nothing led. The
|
|
list it produced, in `TODO.md`, was worked through in one go. What was worth knowing:
|
|
|
|
- **Amber means new.** Badges, unread dots and download bars take the icon's EQ amber; blue is left
|
|
for the primary action and links. Light's amber was `#b06f10`, which gives white text 4.1:1,
|
|
short of AA for 11 px bold. It is `#9a5f0a` now, 5.2:1.
|
|
- **EQ bars mark what is playing.** Three `<i>` bars stand at 60, 100 and 40 % and animate only
|
|
while `body.playing` is set. The first version left the animation on but paused, expecting each
|
|
bar to hold a different frame. The frames it held were within a pixel of each other, and on
|
|
screen the bars read as three dots. Under reduced motion one rule drops every animation and
|
|
transition, which leaves the bars standing.
|
|
- **The focus ring was clipped.** `.tgroup` and `#topbar` both set `overflow:hidden`, so a ring
|
|
drawn outside a toolbar button was cut off. Rings inside clipping parents are inset instead.
|
|
- **The feed list could not be used from the keyboard at all.** Rows were `<div>`s and the triangle
|
|
a `<span>`, so Tab went from the feed filter to Sign out. Rows now take focus, the triangle is a
|
|
`<button aria-expanded>`, and `renderFeeds` puts focus back on the same row after redrawing,
|
|
since every live update replaces every row. The list's own key handler stops Space and the
|
|
arrows from reaching the player's shortcuts on the document.
|
|
- **The triangle hangs in the margin.** Every row used to reserve an 18 px slot for it, pushing a
|
|
hundred feeds 28 px right of the places above for the sake of two folders. It is now absolutely
|
|
placed in the row's left padding, the full height of the row, so a near miss no longer opens
|
|
the folder's page.
|
|
- **A selected tile vanished** because the initials tile and the selected row were both `--raise`.
|
|
Tiles now mix their tint into `--bg`, which no row uses.
|
|
- **The Files pane hides itself** with `#split:has(>#files[hidden])`, which collapses its column.
|
|
The phone layout already hides the pane, and a zero-width extra track there is harmless.
|
|
- **Flagged became Kept** in the tab, and in the server's refusal to delete a file someone else
|
|
kept. The filter value and the column stay `flagged`; renaming those buys nothing.
|
|
|
|
## 2026-09-11 — A Patreon creator is a list of shows
|
|
|
|
Ray asked whether ipx could sync with Patreon. Not in full. The documented API (v2, the
|
|
`identity.memberships` scope) lists the creators you back and whether each has a feed (`has_rss`),
|
|
but no resource carries the `auth` token that makes a feed URL work. That token only comes from the
|
|
creator's page. It is also one per membership, not one per account: techpod's differs from Glass
|
|
Cannon's, so no single token finds everything you back.
|
|
|
|
What does work is one creator at a time, which is what Ray wanted for Glass Cannon and its 33 shows:
|
|
|
|
- `patreon.com/rss?auth=<token>`, with no creator named, returns that token's creator. Its self link,
|
|
about 660 bytes in, gives the campaign by number (`/rss/369921`). Patreon ignores `Range` here, so
|
|
ipx reads the stream until the number appears and hangs up, instead of taking all 2.8 MB.
|
|
- A show's `show=` number is a Patreon collection. Asked anonymously, the collection listing
|
|
(`/api/collection?filter[campaign_id]=`) and a post's `collections` both hide patron-only ones: you
|
|
get "FAQ". `/api/campaigns/<id>?include=shows` lists every show, anonymously, in one response.
|
|
- Every spelling works: `rss/glasscannon?auth=…&show=N`, `rss/369921?…` and `rss?auth=…&show=N` all
|
|
serve the same 131 items. Enclosure URLs are the same in the creator feed and the show feed, and
|
|
stable between fetches.
|
|
|
|
That last point shaped the design. `enclosures.url` is unique, so whichever feed is scanned first owns
|
|
the file. The first cut only asked a creator for its shows while it had no entries of its own, so that a
|
|
creator already read as a plain feed, holding every show's episodes, would never be split into shows
|
|
that came up empty. Within the hour that was the wrong call: Glass Cannon had gone into production on
|
|
the build before this one, been read as one feed of 2,385 items, and the rule kept it that way. Finding
|
|
anything in that heap was the problem Ray wanted solved.
|
|
|
|
So a creator with more than one show is always a group, run through the same sync as an OPML
|
|
(`sync_group`, split out of `sync_opml`). When it becomes one, its items are cleared and each show
|
|
takes over the enclosures the creator holds as the show lists them (`Db::adopt`), downloaded files and
|
|
everyone's read state included. One show leaves it a plain feed, which is what techpod already was. If
|
|
the shows cannot be listed, a creator already split fails the scan rather than being read as one heap;
|
|
one that never was is read as one feed until they can be. An answer without a `shows` list is an error,
|
|
not "no shows".
|
|
|
|
**Filter verdicts follow the settings.** Ray also reported that turning on Allow explicit and
|
|
rescanning brought nothing back. An item was judged once, when first seen, and `skipped` was final. The
|
|
2026-09-10 entry below saw it coming ("worth a `ipx retry <feed>` command if this bites"). It bit:
|
|
2,166 Glass Cannon items and all 88 of Shadowdark's sat at `skipped: explicit` with the setting on.
|
|
Every scan now runs the filters again over what they skipped (not over `torrents disabled`, which is not
|
|
a filter's call) and requeues what they now let through. Only that direction: a queued item is never
|
|
pulled back, because Download latest and a manual download both work by queueing.
|
|
|
|
**Two gaps beside it.** Add feed had no explicit box, so every new feed's first scan skipped all its
|
|
explicit items; it has one now, stored on your subscription like the feed dialog's. And a feed inside a
|
|
group ignored your settings on the group, though the group's dialog said they were inherited: settings
|
|
live on each person's subscription, and nothing read the group's. `Db::subscribers` now fills what you
|
|
have not set on the feed from your subscription to the group, and the feed list shows the same.
|
|
|
|
**A name that was already used.** Replaying the split on a copy of the production database left one
|
|
show with a Supercast episode in it. "Glass Cannon Live! Ascension | Pathfinder 2E" slugs to
|
|
`glass-cannon-live-ascension-pathfinder-2`, the id of a Supercast feed of the same show that had been
|
|
removed. Removing a feed keeps its rows on purpose, so that re-adding it does not fetch the back
|
|
catalogue again, but choosing a new id only checked config.toml and derived feeds. The Patreon show took
|
|
the old id and everything still filed under it. An id is now also taken when the database has a feed by
|
|
that id at a different URL; the same URL may still have it back, which is the re-add case.
|
|
|
|
Shows already added by hand are matched by token and show number, not by exact URL (`same_feed`), so a
|
|
bare token does not add Get in the Trunk and Shadowdark a second time under another spelling.
|
|
|
|
The show listing is Patreon's own undocumented web API. If it changes, only finding new shows stops.
|
|
|
|
## 2026-09-11 — One meaning per icon, sortable columns, and one player
|
|
|
|
Ray asked for a pass over the whole UI: consistent icons, and buttons placed next to what they act
|
|
on. The pass started from screenshots of every view and dialog, taken against the browser suite's
|
|
own daemon and fixtures. Several things did not agree:
|
|
|
|
- Unsubscribe was a minus in the toolbar but an x in the feed header. There it read as "close this
|
|
page", and x also meant Cancel in every dialog.
|
|
- Half the dialogs still had word buttons: Log, Add feed, Users, Unsubscribe, OPML, Popular's
|
|
Subscribe, the Copy buttons and Sign out.
|
|
- The player bar and the folder arrow were font characters, and so was the row's "playing" marker.
|
|
Those came out thin and varied from font to font.
|
|
- The OPML subscription page laid out its header differently from a feed's, and All Subscriptions
|
|
had no buttons at all.
|
|
|
|
The rule now is one meaning per icon. Minus unsubscribes, x closes or cancels, plus adds, subscribes
|
|
or imports, and a dialog's confirm button carries the icon of what it does. The words go in the
|
|
tooltip. The toolbar is grouped by what it acts on: add, unsubscribe and scan act on feeds; play,
|
|
read and keep act on the selected item. Its read and keep buttons now show that item's state, as
|
|
the item's own buttons already did. A browser test opens all eight dialogs and fails on any button
|
|
without an icon and a tooltip. It also checks that the toolbar and the page use the same icon for
|
|
the same action.
|
|
|
|
The pass turned up three bugs:
|
|
|
|
- **Escape ignored inside a dialog's text box.** The keydown handler returned early for any key
|
|
typed in a text box, and Escape sat below that check. So Add feed, which opens with the cursor in
|
|
its URL box, could not be closed with Escape. It showed up because the screenshot run stalled:
|
|
the dialog stayed open, and every later click landed on its overlay until the run timed out.
|
|
- **A white password box in the dark theme.** The input rule listed text, search and number
|
|
fields, but not password ones.
|
|
- **A stray "• •" under an item with no date.** Each detail carried its own trailing dot, whether
|
|
or not the next one was there.
|
|
|
|
All Subscriptions gained Mark everything read, as `POST /api/read-all`. It marks the feeds from the
|
|
person's subscriptions, which is the same set the view's query is scoped by. It asks first, since
|
|
one click covers every feed.
|
|
|
|
**Sorting** happens on the server. A list arrives fifty at a time, so sorting in the page would
|
|
only have reordered what was loaded. The column name picks one of a fixed set of SQL expressions
|
|
(`order_sql`), so nothing the browser sends reaches the query, and an unknown name means newest
|
|
first. Ties also fall back to newest first, so "Load more" does not skip or repeat items at a page
|
|
boundary. A database test runs every column both ways. Size came out of the File column into its
|
|
own, and small files show KB, since whole megabytes made every article image "0 MB".
|
|
|
|
**The double play.** Ray found that pressing play in the Files pane played the file twice at once.
|
|
The pane drew its own `<audio controls>`, and its `onplay` handler also started the player bar on
|
|
the same file. That meant two audio elements from one click. The pane now has a play button that
|
|
hands that exact file to the player bar, the only player. `play()` now takes the file. Its check
|
|
for resuming compares the file as well as the item, so another of an item's files starts from its
|
|
beginning instead of carrying on with the first. A test plays from the pane and checks the page
|
|
has one `<audio>` element.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — The original's layout
|
|
|
|
Ray pointed at a screenshot of the Mac app (techpp.com, 2012) and asked for its panes, its
|
|
toolbar, and a Directory that lives in the feed list rather than behind a button. What it had, and
|
|
what ipx now does:
|
|
|
|
- A toolbar across the window: subscribe and unsubscribe, play, flag, refresh, and a search box
|
|
scoped to the feed on show. ipx's has the same groups, acting on the selected feed and item, with
|
|
Settings and Log at the right end, for admins only.
|
|
- A source list opening with Directory, Playlist Builder and All Subscriptions above the feeds.
|
|
ipx has Directory, Popular and All Subscriptions there, opening in the main pane. Playlist
|
|
Builder is left out, since nothing here builds playlists.
|
|
- The entries as a table, with a Files pane beside it and the entry below. The columns are unread,
|
|
kept, the item, its feed, its file and when. Sorting by column is not done yet.
|
|
- A status bar with the totals for what is on show.
|
|
|
|
All Subscriptions needed one new endpoint, `GET /api/entries`. It is the per-feed query with
|
|
`feed_id = ?` swapped for the person's subscriptions. The enclosure lookup that follows it used to
|
|
filter by feed as well; a page can now span feeds, so each file is matched to its row by feed and
|
|
guid instead.
|
|
|
|
A phone has no room for a pane beside the table, so there the files follow the item's text in the
|
|
full-screen reader, and the table drops to title and date.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Popular on this server
|
|
|
|
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
|
|
the list was fed. After every successful download, `iPXClass.py:1249` called `doPing`, which sent
|
|
the enclosure URL and the feed URL, unencoded and over plain HTTP, to
|
|
`directory.iPodderX.com/feedData/survey/files`. It was on by default, and only the Mac app's
|
|
preferences could switch it off (`anonFeedback`). The directory itself lived on that server and in
|
|
the Cocoa app, and both are gone.
|
|
|
|
ipx keeps the idea and drops the ping. Add feed lists what other accounts on this server subscribe
|
|
to, ranked by subscriber count. The old ping would have sent a Patreon feed's `auth=` key along
|
|
with everything else, so this is built around not doing that:
|
|
|
|
- A row carries an id, a title, artwork and a count. No URL reaches the page, so no key can.
|
|
- Subscribing goes by id through `POST /api/popular/{id}`, which checks the id against the same
|
|
list. Guessing a private feed's id gets a 400.
|
|
- Anything that looks private is never listed: a login configured for the feed, credentials in
|
|
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
|
|
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
|
|
`unlisted` flag is the upgrade if that happens.
|
|
|
|
It happened the same day. The first screenshot of the new Directory listed "Glass Cannon Live!
|
|
Ascension (for Ray Slakinski)", a Supercast feed at `feeds.supercast.com/feeds/<key>`. Treating
|
|
any long path segment as a key would have hidden public feeds too: acast's show ids look the
|
|
same. So paid-feed services are named instead (Patreon, Supercast, Supporting Cast, Glow,
|
|
Memberful), and any feed from one of them is private whatever its URL looks like. The per-feed
|
|
flag is still the upgrade for a service not on that list.
|
|
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
|
|
they would bury everything anyone chose on purpose.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Import an OPML file by uploading it
|
|
|
|
The import screen now has a file picker as well as the paste box. The page reads the file and
|
|
checks it looks like OPML before sending anything. If it doesn't, the page says so and clears the
|
|
picker, and it also clears the picker after an import. The file travels as text and is never
|
|
written to disk on the server, so there is nothing to clean up there.
|
|
|
|
The server now parses the OPML before touching anything, and returns `400` "that is not an OPML
|
|
file" instead of a `500`. `subscribe_opml` takes a parsed document, so `ipx import` also refuses a
|
|
non-OPML file by name before changing anything.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — OPML import and export are per person
|
|
|
|
Importing an OPML now subscribes you to every feed in it. Feeds already in the catalogue cost
|
|
nothing, and unknown ones are added under the OPML's title. Before this, import only added URLs
|
|
missing from `config.toml` and subscribed nobody. So importing an export from another account did
|
|
nothing at all, and a genuinely new feed had no subscriber and was never scanned. The page said
|
|
"Imported 0 feed(s)". `ipx import` had the same gap; it now subscribes the first admin. Both go
|
|
through `subscribe_opml`.
|
|
|
|
Export now lists only your own subscriptions. It used to write out the whole catalogue to anyone
|
|
signed in, including other people's feeds and any private URLs in them.
|
|
|
|
Tests now cover every way in and out:
|
|
- Settings' Export OPML download and paste-to-import screen.
|
|
- A round trip that imports the admin's actual export into a second account and checks both
|
|
exports match. That round trip is exactly what failed.
|
|
- `ipx import` and `ipx export`, run in their own scratch config and database.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — The log is admin-only
|
|
|
|
`GET /api/logs` now returns `403` to anyone who is not an admin, and the page hides the Log button
|
|
from them. Before this, every signed-in person could read the whole log. That includes every
|
|
account's name, every feed anyone subscribes to, and every failed sign-in. `/api/events` stays open
|
|
to everyone, because it carries the scan progress each person's page shows.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Managing users from the web
|
|
|
|
Settings has a **Manage users…** screen for an admin. From it you can add someone, with a password
|
|
or with none for someone the proxy signs in, tick or untick Admin, and remove an account. It is
|
|
backed by `GET/POST /api/users` and `PATCH/DELETE /api/users/{id}`, which return `403` for anyone
|
|
who is not an admin. The only admin cannot be demoted or removed, because nobody would then be
|
|
able to manage accounts except from the CLI on the box. Before this, accounts could only be managed
|
|
with `ipx user`.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Unread feeds first inside an OPML
|
|
|
|
An OPML subscription's feeds, both in the sidebar folder and on its own page, now list the ones with
|
|
unread items first. They were listed alphabetically, so with dozens of feeds the few with anything
|
|
new were scattered through the list. Within each half the order is still alphabetical. The browser
|
|
suite's fixture OPML gained a second feed, Aardvark Radio, which sorts first by name and by position,
|
|
so the new test only passes if unread wins.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Deploying is a Docker image
|
|
|
|
Production moved from a hand-started daemon in code-server to the `iPodderX` container in the Arcane
|
|
project `content`. `CLAUDE.md` now deploys by pushing to the registry at `192.168.1.130:5000` and
|
|
recreating that one service with `docker compose`. The old instructions copied a binary over a
|
|
process nobody supervised, so it did not come back after a reboot. Two known gaps are gone: the
|
|
image does support accounts, and the entrypoint drops to `99:100`, so downloads are no longer
|
|
owned by root. The README's Docker section said `docker compose up -d` builds the image and named the
|
|
service `ipx`; the compose file pulls from the registry and the service is `ipodderx`.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Documentation
|
|
|
|
`PROGRESS.md` became this changelog; the finished step lists moved to an appendix. The README is now
|
|
an overview that points at `docs/`: [configuration](configuration.md) (every key, path and
|
|
environment variable), [cli](cli.md), [users](users.md) (what several people share),
|
|
[sso](sso.md) (refreshed for accounts and the admin-only settings), and
|
|
[architecture](architecture.md) (modules, the scan, the schema, the socket protocol, the HTTP
|
|
API). `CLAUDE.md` collects what someone working on the code needs to know and has already been
|
|
learned the hard way here -- `pkill -x` not `-f`, the page being compiled in, the dead columns on
|
|
`entries`, the Playwright worker that deleted its own database.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Pruning respects a star from anyone
|
|
|
|
Same defect as the reaper, one function along: `prune_entries` guarded on `entries.flagged`, which
|
|
nothing has written since read state moved to `entry_state`. Starring a text item with no file would
|
|
not have saved it from the age sweep. It follows the reaper's rule now, and takes orphaned read
|
|
state with whatever it deletes.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Step D: one file, and everyone who wants it
|
|
|
|
The last of it, which is all about telling the truth before acting:
|
|
|
|
* **Delete says whose file it is.** A feed with other subscribers labels the button *Delete for
|
|
everyone*, and the confirmation names how many people share it. The server then has the last
|
|
word: if anyone else has starred the item or not played it yet, `DELETE /api/enclosures/{id}`
|
|
returns **409** with the reason in plain words, and only `?force=true` goes through. So the
|
|
rule is enforced where it matters rather than in the page that happens to be asking.
|
|
* **A feed's header says it is shared** -- "shared with 1 other person" -- which is the answer to
|
|
"why is there a file here I never asked for": someone else's subscription fetched it, and one
|
|
copy serves you both.
|
|
* Retention already respects the same rule from the entry before this one: starred by anyone keeps
|
|
it, read by everyone releases it.
|
|
|
|
`others_wanting` is tested with three subscribers disagreeing, and a browser test walks the whole
|
|
delete flow: the label, both prompts, declining the second, and the file still being there.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Retention caught up with per-user state
|
|
|
|
Moving read and starred into `entry_state` left `reap_candidates` reading `entries.read` and
|
|
`entries.flagged`, which nothing writes any more: **starring stopped protecting a file**, and the
|
|
"delete the ones already read first" ordering was dead. Harmless while the quota and age limits are
|
|
0 -- the reaper deletes nothing at all then -- but it would have bitten the moment one was set.
|
|
|
|
One file serves every subscriber, so both tests are now about all of them: **anyone** starring it
|
|
keeps it, and it only counts as read once **everyone** subscribed has read it. A file whose feed
|
|
nobody subscribes to has nobody left to keep it, so it sorts with the read ones. Tested with two
|
|
subscribers disagreeing.
|
|
|
|
---
|
|
|
|
## 2026-09-11 — Steps B and C: what is yours, what is everyone's
|
|
|
|
Read, starred and playback position moved out of `entries` into `entry_state (user_id, feed_id,
|
|
guid, ...)`; subscriptions became rows in `subscriptions (user_id, feed_id, ...)` carrying **your**
|
|
keywords, auto-download, explicit and per-scan limit. The feed list, unread counts, filters and
|
|
mark-all-read are all per person now. On first start the existing library is adopted by the admin:
|
|
2438 read/starred items and all 86 feeds, so nothing was lost.
|
|
|
|
The split follows from the file being shared:
|
|
|
|
* **Yours**: read state, starred, position, keywords, auto-download, explicit, per-scan limit,
|
|
and which feeds you see at all.
|
|
* **Everyone's**: the feed URL, its download folder, and when it is scanned -- there is one copy of
|
|
a file however many people subscribe, so those describe the file, not a preference. Admin-only,
|
|
refused with a 403 for anyone else rather than merely hidden.
|
|
|
|
Scanning merges the subscribers' wants, because one fetch and one file serve them all: an item is
|
|
downloaded if **anyone** wants it (any one person's keyword set matching is enough, and one person
|
|
taking everything removes the filter), auto-download is on if anyone has it on, and the per-scan cap
|
|
is the largest anyone asked for. `merge_policy` is a pure function with a test covering each of
|
|
those. Subscribing to a feed someone already has costs no second fetch and no second copy on disk;
|
|
unsubscribing takes it off your list alone, and only when the last subscriber leaves does the feed
|
|
stop being scanned.
|
|
|
|
**A test-harness bug worth naming**: Playwright imports the config in every worker, so the fixture's
|
|
`prepare()` ran again mid-run and deleted the data directory out from under the daemon. The daemon
|
|
kept serving from the unlinked inode while the CLI and any query opened a fresh empty database at
|
|
the same path -- which looked exactly like sign-in being broken. Only the launching process wipes
|
|
now (a worker has `TEST_WORKER_INDEX`).
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Scanning is the operator's decision
|
|
|
|
The per-feed **Check schedule** picker is gone from feed settings, and the global Settings page is
|
|
hidden from anyone who is not an admin. Both are enforced in the handlers rather than merely hidden:
|
|
`PATCH /api/settings` and a `schedule` in `PATCH /api/feeds/{id}` return 403 for an ordinary user.
|
|
Polling costs bandwidth, is what a publisher notices, and one impatient setting affects everyone
|
|
reading that feed -- it belongs in config.toml.
|
|
|
|
Folders, keywords, per-feed download limits and the feed URL stay editable by anyone signed in.
|
|
|
|
`docs/sso.md` covers putting Cloudflare Zero Trust or Authentik in front of ipx: the tunnel and
|
|
Access application, the Authentik proxy provider and its forward-auth nginx block, the three lines
|
|
of ipx config each needs, and why `trusted_proxies` must name the proxy rather than a subnet --
|
|
with the command to prove the refusal works.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Marking an item read
|
|
|
|
Two bugs in one place. `epAction`'s `redraw` closure called *itself* when it had a row to
|
|
update -- `if(el) redraw()` where it meant to swap the row -- so the Mark read button in the
|
|
text pane recursed until the stack blew. It now swaps that one row in place and refreshes the
|
|
text below only when it is the item being read.
|
|
|
|
And opening an item now marks it read, which is what clicking a thing to read it means. The row
|
|
is redrawn where it stands rather than the list reloaded, so an item does not vanish from under
|
|
the pointer on the Unread tab.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Step A: accounts and sign-in
|
|
|
|
`users` and `sessions` tables, Argon2id hashing, a session cookie, and `ipx user add|list|passwd|rm`
|
|
(passwords come in on stdin, so they miss the shell history and any `ps` listing).
|
|
|
|
Three ways in, in order of how specific the claim is:
|
|
|
|
1. **A proxy header** naming the user -- `Cf-Access-Authenticated-User-Email` for the Cloudflare
|
|
Zero Trust in front of `ipodderx.sdf1.net`. Honoured **only** from an address in
|
|
`trusted_proxies` (loopback by default): a header is worth exactly as much as the hop that set
|
|
it, and the LAN port would otherwise let anyone claim to be anyone. Verified both ways -- a
|
|
spoof from an untrusted address is refused.
|
|
2. **A session cookie** from signing in at `/login`.
|
|
3. **The shared token**, which is the admin, so the healthcheck and existing links keep working.
|
|
|
|
A database with no accounts creates **admin / ipodderx** and says so loudly in the log. The UI shows
|
|
who is signed in above the sidebar footer, with a sign-out, and a 401 sends the page to `/login`.
|
|
|
|
Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B.
|
|
|
|
---
|
|
## 2026-09-10 — Items, not episodes
|
|
|
|
Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search
|
|
box, the empty detail pane, the phone back button, the retention and per-feed settings, and the
|
|
download dialog all say **item** now. The sidebar's `20 eps` reads `20 items`.
|
|
|
|
`S1E1` badges stay -- those come from `itunes:episode`/`itunes:season` and only appear when a feed
|
|
actually publishes them -- as does the `episode` column, which is that same field.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Scanning stopped strobing
|
|
|
|
A scan of 85 feeds fires 85 `feed_done` events, and the UI refreshed on every one: a full sidebar
|
|
rebuild plus an episode reload each time, which read as flicker. Bursts now collapse into one
|
|
refresh (500ms trailing), the per-feed "N new" toasts add up into a single summary, and both lists
|
|
keep their scroll position across a rebuild instead of jumping to the top.
|
|
|
|
Measured over a full scan: 11 feed refreshes and 2 episode reloads in 45 seconds, down from one per
|
|
event.
|
|
|
|
Settings and Log moved out of the header and toolbar into a footer under the feed list -- they are
|
|
housekeeping, not daily controls. The theme toggle stays in the header.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Defaults and chrome
|
|
|
|
* **All** is the default episode filter and the first tab; Unread, Downloaded and Flagged follow.
|
|
Opening a feed and seeing nothing because everything in it was read is a poor first impression.
|
|
* **OPML import/export moved into Settings**, under a Subscriptions heading, and the OPML button is
|
|
gone from the sidebar -- it is a thing you do once in a while, not a daily control.
|
|
* **The feed actions are pills with icons**, and Unsubscribe is pushed to the far end in a quiet
|
|
style: it sat next to Settings looking exactly like it, one slip away from losing a feed.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Phone layout
|
|
|
|
The UI was unusable on a phone, starting with the worst of it: the ☰ button lived inside the player
|
|
bar, which is hidden until something plays, so there was no way to reach the feed list at all. It
|
|
now sits in a small bar at the top of the main pane that is always there, and the sidebar slides
|
|
over the page with a scrim to tap away.
|
|
|
|
The rest:
|
|
|
|
* **One pane at a time.** The three-pane split becomes a list, with the item text taking the whole
|
|
screen over it and a `← Episodes` button back. The divider is hidden.
|
|
* **No sideways scrolling.** A grid column is min-content wide by default, so one long headline
|
|
("davewiner/hackerNewsStars") dragged the entire page off the right edge -- `min-width:0` down the
|
|
shell/main/wrap chain, `overflow-wrap:anywhere` on headings, and an explicit
|
|
`minmax(0,1fr)` column for the OPML child list.
|
|
* **The player stacks**: title row on top, transport and seek bar below, and it paints above the
|
|
reading pane so it stays reachable.
|
|
* Header artwork, buttons and log rows shrink to fit; tap targets go to 38px.
|
|
|
|
A Playwright case at 390x844 locks in the three things that actually broke: the burger is visible
|
|
with nothing playing, the page does not scroll sideways, and an item opens and closes over the list.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Marking a subscription read
|
|
|
|
An OPML subscription's page now has **Mark all read**, sitting where every other feed keeps it --
|
|
before Unsubscribe. The fix is in the handler rather than the button: `read-all` resolves the feeds
|
|
whose group is the given id (through `subscriptions()`, so a child promoted to config counts too)
|
|
and marks those, since the subscription's own row holds no entries and marking it did nothing.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — A folder counts what it holds
|
|
|
|
An OPML subscription has no entries of its own, so its row always read `0 unread` no matter how much
|
|
was waiting inside it. Unread and saved are now summed from the feeds it holds -- from all of them,
|
|
not just the ones a search filter left showing, so the number doesn't move as you type. The badge
|
|
caps at `999+` (the real figure is in its tooltip); a four-digit count ate the title beside it.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Sidebar alignment
|
|
|
|
The feed list had four different left edges: a row with no children skipped the chevron entirely, so
|
|
its artwork sat a chevron-width left of a folder's; children then used a different indent *and* a
|
|
smaller icon. Nothing lined up with anything.
|
|
|
|
Every row now reserves the chevron slot whether or not it opens (an empty one is
|
|
`pointer-events:none`, so the click falls through to the row), all artwork is one size, and nesting
|
|
reads from the indent alone. Both label lines are `display:block` on a shared line-height instead of
|
|
an inline baseline, and the unread count has a `min-width` so three-digit feeds don't shove the
|
|
title. Rows came out shorter, so more feeds fit without scrolling.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — An item's picture
|
|
|
|
An item's artwork now resolves in order of how deliberate the source is: `itunes:image`, then Media
|
|
RSS `media:thumbnail`, then a `media:content` that declares itself an image, and finally an image
|
|
**enclosure**. That last one matters here -- Substack puts each article's header picture in an
|
|
`<enclosure>`, which is why those blog entries had no artwork despite carrying one all along. Audio
|
|
enclosures are never mistaken for pictures.
|
|
|
|
Backfilling needed the validators cleared first: `record_entry` fills a missing image on update, but
|
|
a 304 skips parsing entirely, so the feeds would have kept their blank squares. (The self-heal added
|
|
earlier only fires when a feed has *zero* entries, which was not the case here.)
|
|
|
|
Result across the library: 325 of 3470 entries now carry their own picture, 13 feeds where every
|
|
entry has one, 69 feeds that publish no per-item image at all -- those fall back to the feed's
|
|
artwork, which is the intended behaviour rather than a gap.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Database cleanup, and the 304 trap it walked into
|
|
|
|
Cleaned up on request: removed the `CT Log Archive Torrents` folder (123 preallocated files from the
|
|
abandoned 1.6 TB tuscolo torrent, **19 GB real** on disk, no row referencing any of it, feed no
|
|
longer subscribed), the unsubscribed ct-log feed's rows, and every OPML-derived entry/enclosure that
|
|
held no file so a rescan could rebuild them with the current parser. Kept every row holding a file,
|
|
so nothing on disk was orphaned, and kept the skipped/reaped history, without which the next scan
|
|
would re-download the 149 images just deleted. Disk 22 GB -> 3.8 GB, database 22.8 MB -> 5.2 MB.
|
|
|
|
Correction worth recording: mid-download I checked that torrent folder, saw `du -sh` report 8 KB,
|
|
and told Ray it was sparse with nothing written. By the time it was abandoned it had allocated 19 GB.
|
|
The reassurance had a shelf life I did not mention.
|
|
|
|
**Then "Abort Retry Fail is empty".** The cleanup deleted entries but left each feed's
|
|
ETag/Last-Modified. The rescan sent them, servers answered 304 (120 times in the log), the daemon
|
|
skipped parsing, and 57 feeds stayed empty -- and would have until a publisher happened to change
|
|
something. Cleared the validators on the empty feeds and rescanned: entries 1524 -> 3009, feeds with
|
|
content 19 -> 64, Abort Retry Fail back to 20.
|
|
|
|
Fixed in code so it cannot recur: a 304 arriving while the feed has **zero stored entries** means the
|
|
validator has outlived the data -- a restore, a manual edit, a cleanup. The daemon now believes the
|
|
database over the validator, drops it and asks again. Proved live by deleting a feed's entries,
|
|
leaving its ETag, and rescanning: 20 entries rebuilt, self-heal logged once.
|
|
|
|
**"not a wanted media type"** was internal jargon reaching the UI, and it was stored in `last_error`
|
|
so an ordinary filter decision rendered in red as though something had failed. Reworded to "not
|
|
audio or video", 144 existing rows updated, and the UI now paints a reason red only when the state
|
|
is actually `error`.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Multiple enclosures per item, and viewing without downloading
|
|
|
|
**View without downloading.** A non-audio/video enclosure now carries a View link opening in a new
|
|
tab: the publisher's own URL when nothing is downloaded, the local copy at `/media/<id>` when it is.
|
|
Deliberately a direct link rather than a proxy — relaying arbitrary URLs through the daemon would
|
|
make it a fetch-anything service.
|
|
|
|
**Multiple enclosures.** Probed before assuming, and the result was worse than expected: the `rss`
|
|
crate models an item as having at most one enclosure (which is what RSS 2.0 says) and when a feed
|
|
carries several it silently keeps **the last**, dropping the rest. So a two-file item lost its first
|
|
file entirely.
|
|
|
|
`enclosures_by_item()` reads them straight from the XML with quick-xml, in document order,
|
|
unescaping attribute values — a feed URL's `&` arrives as `&`, so skipping that would corrupt
|
|
every query string. It falls back to the parsed enclosure if the scan and the parser disagree on
|
|
item count. Atom already collected all `rel="enclosure"` links. The row now summarises the enclosure
|
|
you would act on (playable, else downloaded, else first) and says "+N more files"; the pane below
|
|
lists them all.
|
|
|
|
**A real limitation surfaced by a broken fixture.** Two browser tests failed with zero enclosures in
|
|
the detail pane. Not the new scanner — verified by running it against the fixture files directly,
|
|
which was right every time. The cause: my fixtures pointed two feeds at the *same* enclosure URL,
|
|
and `enclosures.url` is UNIQUE across the whole database, so whichever feed is scanned first claims
|
|
it and the other's entry gets nothing. That is the dedupe working as designed, but it means **two
|
|
feeds legitimately sharing a media URL will only ever show it under one of them** — worth knowing,
|
|
and worth revisiting if a network feed and a show feed ever overlap.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — A file is not the same as a playable file
|
|
|
|
Reported: "Abort Retry Fail still shows downloaded and audio playback UI". The media-type filter
|
|
stopped *new* image enclosures being fetched, but those 20 JPEGs were already on disk from before
|
|
it existed — and the UI gave a play button and an `<audio>` element to anything with a `path`. An
|
|
audio element pointed at a JPEG is just a broken player.
|
|
|
|
`isPlayable()` now separates having a file from being playable: audio/* or video/*, falling back to
|
|
the file extension when a feed declares no type. Four places used the wrong test — the row's play
|
|
button, the artwork's play overlay, `encBox`, and `play()` itself, which picked the first downloaded
|
|
enclosure regardless of what it was. A downloaded non-media file now shows as
|
|
`image · downloaded · Save · Delete file`, so it is still there and still retrievable, just not
|
|
pretending to be an episode.
|
|
|
|
Covered by a new browser test with a fixture blog whose entry carries a JPEG enclosure and a feed
|
|
configured to download it: no play button on the row, no `<audio>` in the detail pane, and the Save
|
|
button still present.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Three-pane layout
|
|
|
|
Ray asked for feeds beside, items above, and the selected item's text with its enclosures below —
|
|
the layout iPodderX used. Went looking for a screenshot to work from: the archived ipodderx.com kept
|
|
only marketing panels and 130px feature icons, no full window, and the review screenshots lived on
|
|
MacMerc which the archive query did not surface. Said so rather than pretending to have reference
|
|
art; the description is a precise spec on its own.
|
|
|
|
`#split` is a grid of list / divider / detail. Selecting a row no longer expands it inline — it
|
|
highlights and fills the pane below with the title, metadata, read/keep buttons, the sanitized notes
|
|
and one block per enclosure (a player when the file is here, otherwise what it is and a Download
|
|
button). The divider drags, and the split is remembered in localStorage. The OPML group page keeps
|
|
the old scrolling layout via a `plain` class.
|
|
|
|
**Process note, because it cost a dozen calls.** Four anchor-patches in a row failed on text I had
|
|
guessed rather than read: `return el` not `return div`, `open:new Set()` mid-line rather than
|
|
starting one. Each assert aborted before writing, so the file kept reverting to a half-applied
|
|
state, and one earlier patch did land while broken and left a syntax error. What worked was
|
|
reverting to the committed copy and then reading each block verbatim before touching it, with regex
|
|
for the whitespace-sensitive parts.
|
|
|
|
The suite paid for itself again: the new pane test failed on `#detail audio` because
|
|
`max_new_per_check = 1` plus newest-first means the daemon fetches *Second* Episode, so First has no
|
|
player. My assumption, not a bug — the same mistake as the earlier `S1E1` one. The test now finds
|
|
the downloaded row by its chip instead of assuming which episode it is. 9 browser tests pass.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Image enclosures were being treated as episodes
|
|
|
|
Reported as "Abort Retry Fail shows downloads but there are none". It had 20 enclosures, all
|
|
`image/jpeg`: Substack puts each article's header image in the RSS `<enclosure>` tag, so ipx
|
|
downloaded 20 JPEGs and counted them as episodes. Across the OPML subscription: 149 images, 74 MB,
|
|
11 feeds.
|
|
|
|
`[general] media_types` defaults to `["audio", "video"]`, with a per-feed override. An enclosure
|
|
whose top-level type is not wanted is recorded and shown but not auto-downloaded.
|
|
|
|
Clarified mid-change: those enclosures should still be *visible*, just not fetched automatically.
|
|
So nothing is hidden — the row names what it is ("image", "pdf", "torrent") instead of a bare
|
|
"skipped", and the download button still works if you want that file.
|
|
|
|
Two judgement calls in `wanted_media`: an **unknown or absent** type is allowed, because the real
|
|
type is only known after downloading and refusing everything untyped would drop feeds that simply
|
|
omit the attribute; and a **torrent** is allowed, being a container rather than media, judged once
|
|
unpacked.
|
|
|
|
The enclosure-less case Ray also described was already correct — every download affordance in the
|
|
row is gated on the enclosure existing. Checked before changing anything.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Daemon I/O log tab, and Playwright
|
|
|
|
**Log tabs.** All / Daemon I/O / Scans / HTTP. "Daemon I/O" is the control protocol itself: every
|
|
command arriving (`-> {"cmd":...}`) and every event leaving (`<- {"ev":...}`), logged under
|
|
`ipx::io` at the one point every command funnels through, so it catches socket clients, the CLI
|
|
proxying and the web UI alike. Rendered in that tab as in/out rather than level+target.
|
|
|
|
The two logging destinations now have **separate filters**: stderr follows `IPX_LOG` (default
|
|
`ipx=info`), the in-app buffer follows `IPX_UI_LOG` (default `ipx=debug`) and holds 5000 lines. The
|
|
UI can therefore show protocol traffic and routine skips that would be noise on a terminal.
|
|
|
|
**Playwright.** `npm install` plus `npx playwright install --with-deps chromium`, added to
|
|
`/src/install.sh`. `playwright.config.js` starts a fixture feed server and a daemon on a scratch
|
|
config, and eight tests drive a real browser. Each maps to a bug that actually reached Ray.
|
|
|
|
Two things the harness taught, both worth keeping:
|
|
|
|
- **Playwright starts `webServer` before `globalSetup`.** Writing the scratch config in globalSetup
|
|
meant the daemon launched with no config, fell back to defaults, and tried to seize the live
|
|
daemon's socket. The config is written at config-load time instead.
|
|
- Ports 8097/8771 looked free but `ss` showed 8097 held by a process outside this container (shared
|
|
host network). Moved to 8791/8792.
|
|
|
|
**The suite found a real bug on its first green-ish run:** OPML folders rendered *expanded* by
|
|
default. The code stored the set of closed groups, so a folder never toggled — every folder in a
|
|
fresh browser — counted as open, the exact opposite of the comment above it. It stores the open ones
|
|
now. Two other failures were my tests' fault, not the code's: asserting `S1E1` on `.ep.first()` when
|
|
newest-first put a different episode there, and an unscoped `getByText` matching both the sidebar
|
|
entry and the page heading. The `SE` in that first failure was the artwork placeholder's initials,
|
|
which I nearly misread as a broken chip.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Regression: derived feeds looked "unsubscribed" to half the code
|
|
|
|
Reported as `error: enclosure 235 belongs to unsubscribed feed "abort-retry-fail"`. Moving OPML
|
|
feeds out of config.toml meant a config-only lookup no longer finds them, and `download_one` still
|
|
did exactly that — so Download on any episode from an OPML subscription failed outright.
|
|
|
|
Worse than the one bug: **my first grep for the pattern gave false confidence.** `grep -n
|
|
'cfg\.feeds\.get'` returned three hits, all legitimate, so it looked clean — but the failing call
|
|
was split across lines, `cfg` then `.feeds` then `.get`, and never matched. Re-searching with
|
|
newlines collapsed found sixteen `.feeds` accesses, several of them wrong:
|
|
|
|
- `download_one` — the reported failure.
|
|
- `Status` and the daemon's startup line counted config feeds only: 3 reported where there are 85.
|
|
- `add` deduped and slugged against config only, so adding a URL an OPML already lists would have
|
|
duplicated it, and a new feed could collide with a derived feed's id.
|
|
- `rm` on a derived feed said "no feed with id".
|
|
- The web add endpoint had the same duplicate hole.
|
|
|
|
All now go through `subscriptions()`. Verified against the exact failure: enclosure 235 went
|
|
`pending` -> `done`.
|
|
|
|
Lesson recorded because it will recur: when a lookup moves, a single-line grep is not a survey.
|
|
Collapse newlines before searching Rust method chains.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — OPML feeds out of the config, a real cap, and daemon output in the log
|
|
|
|
Three reports in quick succession, all fair.
|
|
|
|
**"Config is a mess."** It was: 611 lines, 85 feeds, 82 of them machine-generated, drowning the three
|
|
Ray actually chose. Writing derived data into a hand-edited file was the wrong call. OPML children
|
|
now live in the `feeds` table (`managed = 1`, `group_id`), are re-derived every scan, and inherit the
|
|
subscription's settings — so there is nothing to store but a URL and a parent. Editing one promotes
|
|
it to a real config entry, so `config.toml` only ever holds decisions. A one-time migration moves
|
|
existing children out: **611 lines -> 38**, 85 feeds -> 3, with all 3487 entries and 216 files intact.
|
|
|
|
**"Never download more than 3."** `max_new_per_check` defaulted to *unlimited*, so subscribing to an
|
|
OPML of 82 feeds pulled every back catalogue it could reach — 216 files, 22 GB before it was caught.
|
|
There is now `[general] max_new_per_check = 3`, used whenever a feed does not set its own, so all 83
|
|
uncapped feeds were capped without touching a line of their config. `pending()` also orders by publish
|
|
date now: a cap of 3 meant "the three recorded first", not the three newest.
|
|
|
|
**"No daemon output in the log."** Scans and downloads travel as events to the socket, not through
|
|
tracing, so the log view showed only startup and HTTP lines. `Emitter::emit` mirrors them now — and
|
|
levelling matters at this scale: at 82 feeds, one INFO line per feed per tick for "not due yet"
|
|
flushed the 2000-line buffer of anything useful in minutes, so routine skips and progress are DEBUG,
|
|
real activity is INFO, failures WARN.
|
|
|
|
A splice while refactoring cut `reject` and `fetch_one` out of main.rs, and recovering them from git
|
|
over-copied three more. Both caught by the compiler, restored, and verified byte-identical against
|
|
`git show HEAD:src/main.rs` rather than eyeballed.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Subscribing to an OPML, not just importing one
|
|
|
|
Asked whether this version could do what iPodderX did: subscribe to an OPML and get a folder of the
|
|
feeds inside it. It could not — `ipx import` was one-shot from a file and nothing ever re-read it.
|
|
The original checked for a URL ending in `.opml` and re-fetched it on every scan
|
|
(`iPXClass.py:34-40`).
|
|
|
|
Now: a feed whose body sniffs as OPML is a subscription list. Sniffing the body rather than the URL
|
|
extension also catches an OPML served from a `.rss` or extension-less URL, which the original missed.
|
|
Every scan re-reads it; listed feeds become real config entries with `group = "<opml-id>"` inheriting
|
|
the parent's settings, land in one nested folder, and are scanned in the same run rather than waiting
|
|
an interval.
|
|
|
|
Removal policy, Ray's rule: **a downloaded file is never orphaned.** Gone from the OPML with nothing
|
|
downloaded -> unsubscribed and removed. Gone but with downloads -> kept and flagged `orphaned`, with
|
|
the UI saying why. Verified end to end on a local fixture: the feed with 1 download was kept and
|
|
flagged, the one with 0 was removed, the file untouched.
|
|
|
|
`folder_for` had to change: it sanitized the whole folder string, so `sub/Show` would have collapsed
|
|
into one directory. Each path segment is sanitized separately now, with a test that `../../etc/Show`
|
|
cannot climb out of the download directory.
|
|
|
|
Also: `Db::memory()` now runs `migrate()` like a real open. It did not, so a column added only in the
|
|
migration passed the tests while being absent in production — which is exactly backwards.
|
|
|
|
Live result: Ray's `lists.opml.org` subscription expanded into 82 child feeds, all grouped, 1431
|
|
entries, no feed errors, worker still responsive mid-scan.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Log view in the app, and Docker
|
|
|
|
**Log view.** A ring buffer (2000 lines) fed by a `tracing` layer, exposed at `/api/logs` with a
|
|
sequence cursor so the UI polls for "everything after N" without duplicates. Tailing a file was the
|
|
obvious approach and the wrong one: under Docker the logs go to stdout and there is no file. An
|
|
`access_log` middleware adds a line per HTTP request, so the daemon and the web side share one
|
|
stream — which is what Ray asked for. That middleware skips `/api/logs` itself, or the panel's
|
|
2-second poll would generate a line per poll forever, a log of nothing but its own requests.
|
|
|
|
UI: sidebar **Log** button, level and text filters, follow-tail toggle, copy button, colour-coded
|
|
levels.
|
|
|
|
**Restart safety.** Detached torrents introduced a `downloading` state, and a row left in it by a
|
|
restart would sit there forever — the pending queue skips it and nothing else revisits it.
|
|
`requeue_interrupted()` resets those at startup, leaving alone any row that already has a file.
|
|
|
|
**Docker.** Multi-stage build (deps cached separately from source), `debian:bookworm-slim` runtime,
|
|
114 MB. Entrypoint writes a starter config bound to `0.0.0.0` on first run, because a container's
|
|
loopback is unreachable from outside, and drops to `PUID:PGID` via gosu. Healthcheck is
|
|
`ipx status`, which goes through the control socket to the worker and so catches a *wedged* daemon,
|
|
not just a dead one.
|
|
|
|
Built and actually run, not just written: image builds, container starts, writes its config, serves
|
|
the UI (401 without a token, 200 with), `/api/logs` answers, and the process runs as
|
|
`uid=99(ipx) gid=100(users)` with files owned `99:100` and the token file at mode 600.
|
|
|
|
Worth noting for future testing here: this session's Docker socket belongs to the Unraid host, so
|
|
`-v /tmp/...:/config` binds a path on the *host*, not one visible from inside this container. The
|
|
mount looks empty from here while being perfectly correct — verify inside the container instead.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Torrents: they work, and one was freezing the whole daemon
|
|
|
|
Asked "is torrents working?". The honest answer had been "unverified since step 7" — so it got tested
|
|
properly.
|
|
|
|
**They work.** End to end against a real swarm: Debian 13.6.0 netinst, 791,674,880 bytes, completed,
|
|
`file` confirms a bootable ISO, row marked `done`. Metadata resolution, transfer, placement and
|
|
completion all good.
|
|
|
|
**But the daemon was wedged.** Ray's CT-log feed had 47 pending torrents and nothing was progressing.
|
|
The worker was blocked: a `status` command over the control socket got no reply in 15s. Torrents ran
|
|
*inline on the single command worker*, so one torrent stopped every feed scan, every HTTP download
|
|
and every status command behind it — for up to `stall_mins` (30 default) waiting on metadata, and
|
|
worse, `Torrents::fetch` blocks on seeding for up to `seed_time_mins` (60 default) **after**
|
|
finishing. One successful torrent could have frozen podcast fetching for an hour.
|
|
|
|
Torrents are now detached: `spawn_torrent` runs the job on its own task behind a 2-permit semaphore,
|
|
the row is marked `downloading` so a rescan does not queue it twice, and the worker moves straight
|
|
on. A one-shot CLI run still runs them inline, or the process would exit mid-download. Verified: the
|
|
worker now answers `status` in 0.0s.
|
|
|
|
A diagnostic misstep worth recording: `/api/settings` returning 200 was taken as proof the worker was
|
|
alive. It is not — the web server is a separate task, so HTTP stays responsive while the queue is
|
|
completely stuck. Probing the control socket is the real test.
|
|
|
|
**Those CT-log torrents are ~557 GB each, 47 of them.** The RSS advertises `length="0"`, so nothing
|
|
warns you; the size only appears once metadata resolves. `auto_download` on that feed is set to
|
|
false as a hold, since restarting would otherwise have begun a half-terabyte transfer immediately
|
|
(the scheduler's first tick fires at startup). Flip it back in the feed's Settings when wanted.
|
|
|
|
Also noted: an aborted torrent leaves librqbit's pre-created file placeholders behind in the
|
|
destination folder (zero-length). Not cleaned up yet.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Schedule pickers, and progress painting every row
|
|
|
|
**Pickers.** "Check feeds every" is a number input plus a unit dropdown (minutes/hours/days/weeks)
|
|
in both global and per-feed Settings, replacing free text. `parse_interval` gained weeks first — it
|
|
only knew m/h/d, so the backend would have rejected the new option. The per-feed dropdown's first
|
|
entry is "Use the default — every N", which disables the number and sends `null`; that rides on the
|
|
null-clearing path fixed earlier today, and would not have worked before it. `splitEvery` picks the
|
|
largest unit that divides evenly, so 120 reads "2 hours" rather than "120 min". The API now returns
|
|
`schedule_mins` so the page does not re-implement the parser.
|
|
|
|
**Bug: one download painted every pending row's progress bar.** Reported after adding the TWiT feed.
|
|
The `progress` event carried feed/url/file but no enclosure id, so the handler had nothing to target
|
|
and set the width on all of them — the code even carried a comment admitting it applied "to whatever
|
|
is downloading now". Adding a feed therefore looked like it was downloading the entire back
|
|
catalogue.
|
|
|
|
`Event::Progress`, `DownloadDone` and `DownloadError` now carry `enclosure`, threaded through both
|
|
the HTTP and torrent paths (`db::Pending` gained its row id to make that possible), and the handler
|
|
targets `.dlbar[data-bar="<id>"]` alone. A test asserts the id is on the wire. Idle bars are
|
|
transparent and only get a track while live, so a row that is not downloading shows nothing.
|
|
|
|
`tests/page-smoke.js` also drives each modal with live-shaped data now — load-time smoke never
|
|
reaches code that only runs when a dialog opens, which is exactly where these changes landed.
|
|
|
|
Verified live: weeks round-trips (`every 2w` -> 20160 min), both pickers render, clearing an
|
|
override returns to the global default. `cargo test` 37/37.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — The whole UI was dead, and server-side tests could not see it
|
|
|
|
Reported as "my feeds seem to have disappeared", then "settings and dark/light mode don't do
|
|
anything either". One cause for all three.
|
|
|
|
`$('#prefs').onclick = prefsModal` referenced a function that did not exist. An uncaught
|
|
`ReferenceError` stops the entire script, and that line sits above the theme toggle, the feed
|
|
filter, the SSE connection and the `loadFeeds()` call that fills the sidebar — so everything below
|
|
it silently never ran.
|
|
|
|
**Cause: a patch anchored on something already deleted.** The scheduling UI was inserted with
|
|
`str.replace` anchored on `function toggleSettings(){`, which belonged to the *old* basic UI that
|
|
the full rewrite had already removed. Python's replace matches nothing and says nothing, and there
|
|
was no assert — so `prefsModal`, `everyText`, `due` and `globalEvery` were never added, while the
|
|
line *calling* `prefsModal` went in fine via a different anchor that did match.
|
|
|
|
**Why it got through.** Every check was server-side: curl for status codes, JSON shape, config
|
|
contents. All passed, because the server was fine. `node --check` also passed — it parses, and a
|
|
ReferenceError is a runtime failure. The page was never executed.
|
|
|
|
`tests/page-smoke.js` now runs the real page script against a stub DOM, fails on anything thrown,
|
|
and flags handlers wired to elements that do not exist. Confirmed non-vacuous by reintroducing the
|
|
exact bug: exit 1 pointing at the offending line, clean once restored. Run it with
|
|
`node tests/page-smoke.js`.
|
|
|
|
Wiring is also defensive now — `on(sel, ev, fn)` logs and skips rather than throwing, so one dead
|
|
reference cannot blank the app again.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Scheduling, and two bugs it uncovered
|
|
|
|
**Scheduling.** The original engine had none — it only skipped feeds on `<ttl>`; the schedule lived
|
|
in the Cocoa GUI that was never open-sourced, so there was no original behaviour to match. Scope
|
|
agreed with Ray: interval only, global default plus per-feed override, no quiet hours.
|
|
|
|
`general.schedule` takes "every 30m", "every 4h", "1d", or a bare number of minutes;
|
|
`feeds.<id>.schedule` overrides it. The legacy `interval_mins` is still read so existing configs
|
|
work, and saving from the UI migrates it. A malformed value in the file warns and falls back rather
|
|
than stopping the daemon; a malformed value over the API is a 400.
|
|
|
|
Precedence, which is a judgement call: an explicit per-feed schedule wins outright, including over
|
|
the publisher's `<ttl>`. With no per-feed schedule, `<ttl>` still raises the interval when the
|
|
publisher asks to be polled less often. Visible in practice — TWiT resolves to 720 min from its own
|
|
ttl against a 60 min global.
|
|
|
|
UI: a gear opens global Settings (schedule, quota, max age); each feed's Settings gains a schedule
|
|
field; the feed header shows "every 4 hours · next in 37m".
|
|
|
|
**Bug 1: `null` never cleared anything.** For `Option<Option<T>>`, serde maps JSON `null` onto the
|
|
*outer* `None`, which the handler reads as "field absent, leave alone" — so `Some(None)` was
|
|
unreachable and every clear was a no-op returning 204. That silently affected `folder`, `schedule`
|
|
and `max_new_per_check`; unsetting any of them from the UI did nothing and looked like it worked.
|
|
Fixed with a `double_option` deserializer, and pinned by a test covering absent / null / value.
|
|
|
|
**Bug 2: the daemon ignored SIGTERM while working.** `tokio::select!` races its branches only at
|
|
selection time; once inside `run(...).await` the shutdown future was not polled at all, so a signal
|
|
queued behind a 90 MB download. Under systemd every restart would have hit `TimeoutStopSec` and been
|
|
SIGKILLed mid-transfer.
|
|
|
|
This one cost real time: `pkill` appeared to work but didn't, a stale daemon kept port 8099, and
|
|
because the restart also `rm -f`'d the socket it defeated the "already listening" guard and a second
|
|
daemon started. Every PATCH "verified" after that went to the *old* binary — so bug 1 was reported
|
|
fixed while completely untested. Lesson: confirm the process actually died and the socket guard is
|
|
doing its job, rather than trusting the restart.
|
|
|
|
The stop signal now lives in a `watch` channel raced *inside* each job. Measured: SIGTERM mid-
|
|
download exits in 1s (was: blocked until the transfer finished). An abandoned download leaves a
|
|
partial in `.ipx-incomplete`, discarded on the next attempt.
|
|
|
|
`cargo test` 37/37.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — App name, and a transparent icon
|
|
|
|
Title is `iPodderX` in the browser tab and the sidebar. The heading's `text-transform: uppercase`
|
|
had to go with it, or the camel case would have rendered as `IPODDERX`.
|
|
|
|
The icon's white background is gone. A blanket `-transparent white` would have punched holes
|
|
straight through the device, whose body is also white — so the background is flood-filled inward
|
|
from all four corners at 12% fuzz, which only clears the *connected* region and stops at the dark
|
|
outline. Then trimmed and scaled to 128px.
|
|
|
|
Checked by compositing onto the real `#0e131b` rather than trusting the alpha channel: no white
|
|
plate, and no pale fringe from the JPEG artifacts along the outline. The `.logo` rule lost its
|
|
`background:#fff` plate and switched to `object-fit:contain`.
|
|
|
|
Both files are tracked: `ipodderx-icon.jpg` is the untouched archival original, `ipodderx-icon.png`
|
|
the transparent version derived from it, so the provenance chain survives. The PNG is embedded as a
|
|
data URI for the sidebar mark and the favicon.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Colour scheme derived from the icon
|
|
|
|
Palette sampled from `web/ipodderx-icon.jpg` with ImageMagick rather than eyeballed. The icon gives
|
|
three families: the silver device ramp (`#1A1A1A` `#606060` `#929292` `#ADADAD` `#D6D6D6` `#F5F5F5`),
|
|
the screen blues (`#2D5391` `#5574A9` `#7595CA` `#92B2E6` `#B4CAED`), one amber from the EQ bars
|
|
(`#F49E2C`), and a blue-grey `#95A0B1`.
|
|
|
|
Dark theme: `--bg` is the screen navy taken right down, `--fg` the device highlight, `--dim` the
|
|
blue-grey verbatim, `--accent` the screen blue, `--accent2` the amber. Light theme flips to the
|
|
device body — white panels, `#D6D6D6` edges, `#1A1A1A` text — with the deeper `#2D5391` as accent,
|
|
since the pale screen blue vanishes on white. Text sitting on an accent fill moved from a hardcoded
|
|
`#0b0e14` to a `--ink` token.
|
|
|
|
Two calls: the amber doubles as the pending state, being the only warm note in the icon and exactly
|
|
where the eye should go. And since the icon has no green or red, the downloaded/error colours are
|
|
tuned to the palette's saturation and temperature rather than invented from scratch.
|
|
|
|
**Contrast was checked, not assumed, and it failed first time.** `--faint` — which carries dates,
|
|
durations and file sizes at small size — came out at 3.92 dark and 3.11 light, both under AA. Moved
|
|
along the icon's own grey ramp to `#7a8799` and `#767676`. Every pair in both themes now clears
|
|
WCAG AA; the lowest anywhere is 4.54.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — The real iPodderX icon
|
|
|
|
The sidebar mark was an invented `ix` placeholder. The original icon is not in this repo or the
|
|
legacy one: only the Python engine was open-sourced, never the Cocoa bundle whose Resources held the
|
|
`.icns` (the legacy repo's history contains nothing but .py/.so/.po/.mo/LICENSE/README).
|
|
|
|
Recovered from the Internet Archive's capture of ipodderx.com: `_graphics/iPXicon.jpg`, 173x165 --
|
|
the white iPod-style device with the antenna and EQ display. That is the largest surviving copy; the
|
|
site's 2005 `favicon.ico` is far too small to use. Saved as `web/ipodderx-icon.jpg` so the file lives
|
|
in the tree, and embedded as a base64 data URI (12.7 KB) for both the sidebar mark and the favicon,
|
|
which avoids an extra asset route.
|
|
|
|
`.logo` changed from a gradient chip to a white rounded tile, since the icon is photo-style on a
|
|
light ground.
|
|
|
|
Provenance is Ray's own: the app was Slakinski & Trometer / Thunderstone Media, and this is its
|
|
successor.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Feed URL is editable, with a copy button
|
|
|
|
The URL in a feed's Settings is now an editable field paired with Copy. Entries and download
|
|
history are keyed by feed id, not URL, so changing it keeps everything — which is the point, since
|
|
a Patreon feed URL carries an auth token that gets rotated.
|
|
|
|
Two things that would otherwise have bitten:
|
|
|
|
- **`navigator.clipboard` does not exist here.** It requires a secure context, and this is served
|
|
over plain HTTP on a LAN address, so the button would have silently done nothing. Falls back to a
|
|
hidden textarea plus `execCommand('copy')`, and reports Copied/Failed either way.
|
|
- **Changing the URL clears the stored ETag/Last-Modified.** Those validators belong to the old URL;
|
|
carrying them over could yield a bogus 304 against the new one and make a working feed look empty.
|
|
|
|
`check_url` is a tested pure function: rejects empty, unparseable and non-http(s) URLs (so
|
|
`file:///etc/passwd` is refused), rejects a URL another feed already uses, and allows a feed to keep
|
|
its own URL unchanged. Validation failures and unknown feed ids now return **400**, not 500 — bad
|
|
input is the caller's mistake, and only genuine server faults are logged as errors.
|
|
|
|
Verified live: the three rejection cases return 400 with readable messages, a no-op save returns 204,
|
|
and the feed still reports 131 entries / 11 downloaded. `cargo test` 34/34.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Fixed: pressing play made an episode vanish from the list
|
|
|
|
Reported as "where did Session Zero go, and why do parts share a number?".
|
|
|
|
Two separate things, one of them a real bug.
|
|
|
|
**The bug.** `play()` marked an episode read the moment playback started. The default view is the
|
|
Unread tab, so pressing play removed the episode from the list you were looking at — indistinguishable
|
|
from it going missing. Confirmed from the data: all four read entries had `position = 0`, meaning
|
|
playback never passed the 10-second save threshold. They had been started, never listened to.
|
|
|
|
Marking read now happens when an episode is actually consumed — on `ended`, or once playback passes
|
|
90% — never on play. Two earlier PROGRESS entries described mark-on-play as a deliberate feature;
|
|
it was a design error, and this supersedes them.
|
|
|
|
Also restored the four affected rows to unread, since their read flag was purely an artifact of the
|
|
bug (`UPDATE entries SET read=0 WHERE read=1 AND position=0` — no genuinely-played episode could
|
|
match, since playing one writes a position).
|
|
|
|
**Not a bug: repeated episode numbers.** Each story part ships as two items — the main episode and a
|
|
shorter `Junk in the Trunk` companion — with distinct GUIDs, durations and files, and the publisher
|
|
gives both the same `itunes:episode`. Zero duplicated titles across all 131 entries. `Session Zero`
|
|
additionally carries a season with no episode number and drops the "Part N" naming, so it does not
|
|
match the pattern its siblings follow.
|
|
|
|
Verified: the arc shows 11/11 items under the default Unread view again.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Titles: RSS `<title>` is the only source
|
|
|
|
Confirmed against the live feed rather than assumed: 131 feed items, 131 stored, **0 mismatches** —
|
|
entry titles are byte-for-byte what the RSS `<title>` publishes, separators included. `feed.rs` never
|
|
consults `itunes:title`, and season/episode live in their own columns, rendered as chips rather than
|
|
folded into the title.
|
|
|
|
Pinned with a test: a fixture whose `itunes:title` differs from its `<title>` must still yield the
|
|
RSS title, and an item with `itunes:season` but no `itunes:episode` keeps a null episode instead of
|
|
inventing one.
|
|
|
|
That null-episode case is real in the wild — this feed's `Music from a Darkened Room | Session Zero`
|
|
carries S8 with no episode number, while Parts 1-5 get E1-E5, and it drops the "Part N" convention
|
|
its siblings use. The arc is 11 items: Session Zero plus five parts, each part being a main episode
|
|
and its shorter `Junk in the Trunk` companion.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Phase 3: full-featured UI
|
|
|
|
Rewrite of `web/index.html` (~714 lines) plus the backend it needed.
|
|
|
|
**New stored metadata**, all of which the real feed carries on all 131 episodes and none of which
|
|
was being captured: feed and per-episode artwork (`itunes:image`), duration (`itunes:duration`,
|
|
parsed from either raw seconds or a `1:34:09` clock), season/episode numbers, and a playback
|
|
`position` so episodes resume.
|
|
|
|
**Schema migration.** `Db::open` now runs `migrate()`: `PRAGMA table_info` then `ALTER TABLE ADD
|
|
COLUMN` for anything missing, since `CREATE TABLE IF NOT EXISTS` does nothing to an existing table.
|
|
Tested on a *copy* of the live database first. Accidentally SIGPIPE'd it partway through (a `head -4`
|
|
on the output) which usefully proved it is idempotent and self-heals: the next run added the
|
|
remaining columns, 131 entries and 6 downloads intact.
|
|
|
|
**New endpoints:** entry filters (all/unread/downloaded/flagged), case-insensitive search over title
|
|
and notes, pagination totals, `POST .../position`, `read-all`, `download-latest`, and OPML
|
|
import/export over HTTP.
|
|
|
|
**UI:** persistent player bar surviving navigation (play/pause, ±15s/30s, 0.8-2.5x speed, volume,
|
|
scrubber, all remembered in localStorage), resume playback via `sendBeacon` every 10s, MediaSession
|
|
for lock-screen controls, keyboard shortcuts (space, arrows, `/`, Esc), artwork everywhere with
|
|
generated-initials fallback, filter tabs, episode search, load-more pagination, live progress bars
|
|
off SSE, real modals, toasts, light/dark toggle, and a mobile layout with a collapsing sidebar.
|
|
|
|
**Bug found and fixed in review: filtering was broken while searching worked.** `count_entries`
|
|
dropped the search clause when no term was given but still bound `?2`; rusqlite rejects a parameter
|
|
the statement never mentions -- *"Wrong number of parameters passed to query. Got 2, needed 1"*. So
|
|
`?q=...` worked and a plain Unread/Downloaded/Flagged filter 500'd. The clause is now always present
|
|
with `?2 = ''` short-circuiting it. Regression test runs all four filters with and without a search
|
|
term and asserts the page and the count agree.
|
|
|
|
**A non-bug worth recording:** artwork looked broken on the live DB (0/131) while a fresh DB filled
|
|
all 131. The parser was fine -- the live DB simply had not been rescanned yet, since backfill happens
|
|
through `record_entry`'s update path. After a forced scan: 131/131 image, duration and season.
|
|
|
|
Verified live: all four filters return sane totals (131/129/10/0), search works (`session zero` -> 2,
|
|
`trunk` -> 10), `/media/1` with a Range header returns 206, index serves 34 KB. `cargo test` 32/32.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Fixed: the UI's Download button downloaded the wrong episodes
|
|
|
|
Reported from the running instance: clicking Download on *Music from a Darkened Room | Session Zero*
|
|
showed progress, then left the episode at `pending`.
|
|
|
|
**Cause, and it was a design error in step 11, not a glitch.** `POST /api/enclosures/{id}/download`
|
|
requeued the row to `pending` and asked for an ordinary scan, justified at the time as "needed no
|
|
new machinery — the queue is the table". But a scan means *take the lowest-id pending rows, up to
|
|
`max_new_per_check`*. Against a 125-item backlog with a cap of 2, that is ids 7 and 8; the requested
|
|
row 11 was never a candidate. The progress on screen was two other episodes downloading past it.
|
|
|
|
The queue can express what is outstanding but not what was *asked for*. `Command::Download {
|
|
enclosure }` is now its own command: it fetches that one row immediately, ignoring both queue order
|
|
and the per-scan cap, still through the single worker so it cannot overlap a scan. Torrent
|
|
enclosures route to `torrent_one` the same way a scan would.
|
|
|
|
Verified live: enclosure 11 went `pending` -> `done`, 90.3 MB on disk, and nothing else was pulled
|
|
in its place. `cargo test` 30/30, with the new command's wire form pinned.
|
|
|
|
Worth remembering: **the daemon scans immediately on its first tick**, so every restart costs
|
|
`max_new_per_check` episodes on the normal path.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Phase 2: web front end (steps 9-13)
|
|
|
|
axum in the daemon process, plain HTML/JS in `web/index.html` (embedded with `include_str!`), no
|
|
WASM toolchain. One binary still.
|
|
|
|
**Config hot-reload.** `Ctx.cfg` is now `RwLock<Arc<Config>>`; `ctx.cfg()` hands out a snapshot, so
|
|
no guard is ever held across an await. The UI rewrites config.toml and calls `reload_cfg`, and a
|
|
running daemon picks the change up on its next scan. The CLI mutation commands (`add`/`rm`/`import`)
|
|
now clone a snapshot, edit, and save.
|
|
|
|
**Auth.** `[web] enabled/bind/token`; the token is minted from `/dev/urandom` on first run, written
|
|
back to config.toml, and the URL printed. `?token=` sets a year-long cookie — it has to be a cookie
|
|
because an `<audio>` element cannot send a header. Comparison is constant-time. An empty token makes
|
|
the server refuse to serve rather than serve open.
|
|
|
|
**Endpoints.** `/api/feeds` (GET/POST), `/api/feeds/{id}` (PATCH/DELETE),
|
|
`/api/feeds/{id}/entries`, `/api/entries/{feed}/{guid}/flags`, `/api/enclosures/{id}/download`,
|
|
`/api/enclosures/{id}` (DELETE), `/api/fetch`, `/api/events` (SSE off the existing broadcast bus),
|
|
`/media/{id}` (tower-http `ServeFile`, so Range works).
|
|
|
|
**`read`/`flagged` finally have a writer** — the UI sets them, and playing an episode marks it read.
|
|
Retention has ordered by these since step 5 with nothing to set them.
|
|
|
|
**Download-on-demand needed no new machinery**: requeue the row to `pending` and kick a scan, since
|
|
the queue is the table. That also un-skips an enclosure a filter rejected under older settings.
|
|
|
|
Verified against the live Glass Cannon feed, daemon on 127.0.0.1:8749:
|
|
|
|
- auth: no token 401, wrong token 401, right token 200 + cookie, API then works on the cookie alone;
|
|
`/media/1` unauthenticated is 401.
|
|
- browsing: 131 entries page with enclosures attached.
|
|
- **XSS**: injected `<script>alert(1)</script><img src=x onerror=alert(2)>` into a stored
|
|
description; it reaches the page as `<p>hi</p><img src="x">` — script tag and handler both gone.
|
|
- media: 200 with `accept-ranges: bytes`; `Range: bytes=1000000-1000999` returns 206 with the right
|
|
`content-range`, so seeking works.
|
|
- config: PATCH -> 204, written to config.toml, and the running daemon reports the new values with
|
|
no restart.
|
|
- SSE: 105 events for one download (101 progress), then `download_done`/`feed_done`/`scan_done`.
|
|
- flags: 204, unread count dropped 131 -> 130. Delete: file gone, row `reaped`/`path=NULL`.
|
|
|
|
`cargo test` 30/30.
|
|
|
|
**Caveats.** It is plain HTTP — on a LAN bind the token crosses the network in the clear, and a feed
|
|
URL may itself carry a credential (Patreon's does), which the `/api/feeds` response includes. A
|
|
reverse proxy with TLS is the answer if that matters. There is no per-user anything; the token is
|
|
all-or-nothing access.
|
|
|
|
---
|
|
|
|
## 2026-09-10 — Tested against a real feed (Patreon / Glass Cannon)
|
|
|
|
First run against a live subscriber feed: 131 items, 6.17 GB total, all `audio/mpeg`, no `<ttl>`.
|
|
Parsed clean, titled `Get in the Trunk | Anthology Series | Delta Green`, downloaded a valid MP3
|
|
(`file` confirms ID3v2.3, MPEG layer III). A second scan pulled the *next* episode, confirming
|
|
`max_new_per_check` defers rather than drops. Over the socket: 101 progress events for a 79 MB
|
|
episode -- whole-percent throttling behaving exactly as intended -- then `download_done`,
|
|
`feed_done`, `scan_done`.
|
|
|
|
**Fixed: stripped separators left doubled spaces.** The feed title separates words with `|`, a
|
|
forbidden filename character, so the folder came out `Get in the Trunk Anthology Series Delta
|
|
Green`. The Python had the same wart.
|
|
|
|
Forbidden characters are now split in two. Separators (`/ \\ | :`) become `-`; the rest
|
|
(`? * < > " '`) are simply dropped. A run of dashes and spaces then collapses to `" - "` when the
|
|
run contained whitespace and to a bare `-` when it did not, so:
|
|
|
|
| input | output |
|
|
|---|---|
|
|
| `Get in the Trunk \| Anthology Series \| Delta Green` | `Get in the Trunk - Anthology Series - Delta Green` |
|
|
| `Ep 12: The One` | `Ep 12 - The One` |
|
|
| `AC/DC` | `AC-DC` |
|
|
| `well-known.mp3` | `well-known.mp3` |
|
|
| `../../etc/passwd` | `etc-passwd` |
|
|
|
|
Leading dashes and dots are trimmed too -- a filename starting with `-` trips up CLI tools.
|
|
|
|
**Worth knowing, not a bug:**
|
|
|
|
- The channel-level `itunes:explicit` is `true`, so with the default `allow_explicit = false` all
|
|
131 episodes are skipped. Because filter verdicts are recorded once at discovery, flipping the
|
|
setting afterwards does not re-evaluate enclosures already marked `skipped` -- they would need a
|
|
`UPDATE enclosures SET state='pending'`. Worth a `ipx retry <feed>` command if this bites.
|
|
- Patreon sends neither `etag` nor `last-modified` on a GET (a `last-modified` does appear on HEAD,
|
|
which is what made it look briefly like a storage bug). So this feed can never 304: every poll
|
|
re-fetches ~160 KB and re-parses 131 entries, and `interval_mins` is the only thing limiting the
|
|
rate. Cheap, but it means conditional GET buys nothing here.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 8: OPML and polish
|
|
|
|
`ipx add <url>` fetches the feed to name it from its own title (`Accidental Tech Podcast` ->
|
|
`accidental-tech-podcast`); a feed that cannot be reached is still added, named from its URL, rather
|
|
than refused. `ipx rm` leaves downloads and history alone, so re-adding a feed does not re-pull its
|
|
back catalogue. `ipx import`/`export` walk nested OPML folder outlines and skip URLs already
|
|
subscribed. `tracing` logs to stderr, `IPX_LOG` sets the filter. `contrib/` has a systemd user unit
|
|
for the daemon, plus a timer and one-shot service for the no-daemon style (with the caveat that
|
|
without a daemon there is no socket for a UI).
|
|
|
|
Verified: `cargo test` 27/27. Round-trip — exported 3 feeds with real titles, removed one,
|
|
re-imported: exactly the missing one came back, no duplicates, config still mode 0600. Quickstart
|
|
from a genuinely empty home with no env overrides: `list` on a missing config, `add`, `fetch`
|
|
(1 downloaded, 1 explicit skipped, 1 torrent 404), `list`; every path resolved under `$HOME`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 7: torrent.rs
|
|
|
|
`src/torrent.rs`: a librqbit `Session` started lazily on first torrent (binding ports and starting a
|
|
DHT for a config that has never seen a torrent would be rude). `.torrent` URLs and magnets both go
|
|
through `AddTorrent::from_url`. Progress is polled once a second and reported through the same
|
|
`Progress` events HTTP downloads use.
|
|
|
|
Downloads go **straight into the feed folder** rather than staging and moving. The plan said move
|
|
then seed, which cannot work — seeding serves the files it downloaded, so moving them first breaks
|
|
it. In-place also removes a copy the original had to do.
|
|
|
|
Seeding stops at `seed_ratio` or `seed_time_mins`, whichever comes first, then the torrent is
|
|
released from the session (files kept).
|
|
|
|
**Bug the smoke test caught:** the stall budget only covered the download loop, but resolving a
|
|
magnet's metadata happens inside `add_torrent`, which against a dead swarm never returns — a
|
|
torrent nobody seeds wedged the scan indefinitely. `add_torrent` is now wrapped in the same budget.
|
|
|
|
Verified: `cargo test` 27/27 (ratio incl. the divide-by-zero case, stall_mins = 0 not meaning
|
|
"abort instantly"). Routing: with `enabled = false` a torrent enclosure is marked
|
|
`skipped/torrents disabled` and never attempted. Session startup works here. Stall abort measured
|
|
end to end: with `stall_mins = 1`, a dead magnet failed at 20:58:23 -> 20:59:24, exactly 61s, and
|
|
the row recorded `error / no metadata after 1 minutes, gave up`.
|
|
|
|
**Not verified: an actual successful swarm download.** This sandbox has no reachable peers, so
|
|
smoke 5's happy path — payload lands, seeding stops at the ratio — has not been run. The code paths
|
|
either side of it are tested; the swarm itself needs a real network. Worth running once against a
|
|
live torrent feed before trusting it.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 6: ipc.rs + daemon
|
|
|
|
`src/ipc.rs`: `Event` and `Command` as serde-tagged enums (`{"ev":...}` / `{"cmd":...}`), one JSON
|
|
object per line over a Unix socket. `Emitter` is the single output path — it broadcasts to socket
|
|
clients, prints the human rendering to a terminal, or both, so the scan code no longer knows how it
|
|
is being watched. That replaces `printMSG` and its `;;1;;1;;100.00;;42.31` sentinels.
|
|
|
|
`main.rs` restructured around a `Ctx` (config, db, client, emitter). `ipx daemon` binds the socket,
|
|
serves clients, and runs a 60s ticker that defers to each feed's TTL. Commands from every client
|
|
funnel through one mpsc queue into a single worker, which is what stops two scans overlapping.
|
|
Any CLI subcommand with a wire form probes the socket first and proxies to a running daemon;
|
|
`--local` forces the work to happen in-process. SIGINT/SIGTERM remove the socket on the way out.
|
|
|
|
**Bug the smoke test caught:** `fetch` runs a retention sweep first, and that sweep was emitting the
|
|
terminal `ReapDone`. A UI waiting for its fetch to finish would have stopped reading before the scan
|
|
started. Only a standalone `ipx reap` emits it now.
|
|
|
|
Verified: `cargo test` 24/24. Smoke 3 in full — daemon starts and binds; a raw socket client sending
|
|
`{"cmd":"fetch","force":true}` gets `feed_start` → 14 throttled `progress` events → `download_done`
|
|
→ `feed_done` → `scan_done`; `{"cmd":"status"}` answers `{"ev":"status","feeds":1,...}`. With the
|
|
daemon up, `ipx fetch` from the CLI logged `command from a client cmd=Fetch { .. }` in the daemon
|
|
and rendered the streamed events, so it proxied rather than downloading in parallel. Socket removed
|
|
on SIGTERM; with no daemon the same command runs locally.
|
|
|
|
Deferred: `{"cmd":"cancel","enclosure":N}` from the plan's protocol is **not implemented** —
|
|
downloads run sequentially in one worker, so there is nothing to cancel concurrently yet. It wants
|
|
a per-download cancellation token, which is worth doing when downloads go parallel. Say if you want
|
|
it sooner.
|
|
|
|
Note: the event stream is a broadcast, so a CLI client seeing a busy daemon also sees that other
|
|
work. Fine for a terminal; a UI wanting strict request/response would want per-request ids.
|
|
|
|
Also note: the daemon reads config once at startup — changing `config.toml` needs a restart.
|
|
|
|
Next: step 7 — `torrent.rs`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 5: retention.rs
|
|
|
|
`src/retention.rs`: reconcile pass (rows claiming a file that is gone become `reaped`, fixing the
|
|
step-4 wart), age sweep, quota sweep keeping the original's 50 MB headroom pad, and entry pruning.
|
|
`ipx reap [--dry-run]`; a sweep also runs before every `fetch`, as the Python did per download.
|
|
`pick()` and `aged()` are pure so the ordering rules are testable without touching a disk.
|
|
|
|
**Judgement call worth Ray's eye.** The Python meant to reap only `read = 1 AND flagged = 0` but
|
|
never managed it — a missing `plistlib` import and an `EntreiesData` typo made that filter throw on
|
|
every candidate, so with a `.ipxd` present nothing was ever deleted. Requiring `read = 1` here would
|
|
be equally dead, because nothing marks episodes read until a UI exists. So: **`flagged` is the
|
|
keep-forever marker, and `read` only decides what goes first** (`ORDER BY read DESC, downloaded_at
|
|
ASC`). Quota therefore actually reclaims space headless. Say the word if you would rather unread
|
|
episodes were never touched.
|
|
|
|
Second call: `max_age_days` deletes *files* older than the cutoff, not just fileless entries as the
|
|
plan's wording had it — "keep 30 days of episodes" is what the setting reads like on a NAS.
|
|
|
|
Verified: `cargo test` 21/21, including the two tests encoding the exact bug the Python had —
|
|
flagged files are never offered, and read sort ahead of unread. Smoke 6 with three 30 MB episodes
|
|
against a 0.1 GB quota (52.4 MB ceiling after the pad): dry run listed ep1+ep2 and deleted nothing
|
|
(3 files still on disk), the real run deleted exactly those two oldest, left ep3, flipped both rows
|
|
to `reaped` with `path = NULL`. A full re-parse with the conditional-GET headers cleared then
|
|
re-downloaded nothing.
|
|
|
|
Next: step 6 — `ipc.rs` + daemon.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 4: download.rs
|
|
|
|
`src/download.rs`: streaming download to `<download_dir>/.ipx-incomplete/` (same filesystem as the
|
|
destination, so filing it is a rename, not the original's copy-then-unlink), content sniffing,
|
|
then `place()`. Filename comes from the URL's last path segment, percent-decoded, unless
|
|
`Content-Disposition` names one (RFC 5987 `filename*=` preferred). The sanitizer keeps UTF-8 —
|
|
`latin1_to_ascii` existed because 2004 filesystems demanded ASCII — strips the same characters
|
|
`stringCleaning()` did plus control chars, and adds a real 255-byte cap the Python never had,
|
|
preserving the extension across truncation.
|
|
|
|
Sniffing replaces `detectFileType()`, which called a `typeFile` module that was already missing in
|
|
2008 and so always answered `'data'`. Two rules survive: an HTML body is a failed download (login
|
|
wall/error page), and a torrent body is a torrent whatever the MIME claimed.
|
|
|
|
Filters run once at discovery and are recorded in `enclosures.state`; the download queue is then
|
|
just "everything still `pending`", so an enclosure held back by `max_new_per_check` is picked up by
|
|
the next scan instead of being lost. Keywords are OR'd across keywords and AND'd within one — the
|
|
original's nested loop let a later keyword silently undo an earlier miss.
|
|
|
|
Verified: `cargo test` 16/16. Smoke against a local server, five enclosures, each filter path hit:
|
|
`ep1.mp3 -> done`, `ep2.mp3 -> skipped (explicit)`, `ep2.mp3?v=3 -> skipped (no keyword match)`,
|
|
`paywall.html -> error (HTML page, not media)`, `ep5.torrent -> torrent (deferred to step 7)`.
|
|
Smoke 2 and 4 pass, and because a plain rerun 304s before parsing, dedupe was proved separately by
|
|
clearing the stored etag/last-modified and re-parsing all five entries: 0 downloaded, hand-deleted
|
|
file not refetched, `.ipx-incomplete` left empty.
|
|
|
|
Known wart: `ipx list` counts `path IS NOT NULL`, so a hand-deleted file still reads as downloaded.
|
|
Reconciling rows against the filesystem belongs in step 5.
|
|
Next: step 5 — `retention.rs`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 3: feed.rs
|
|
|
|
`src/feed.rs`: conditional GET (`If-None-Match` + `If-Modified-Since`, optional basic auth) and a
|
|
RSS-first / Atom-fallback parser normalising both into `ParsedFeed`/`Entry`/`Enclosure`. Feed-level
|
|
`itunes:explicit` overrides the entry level, as the original did. `<ttl>` is captured. RSS
|
|
`content:encoded` wins over `description`. Atom enclosures come only from `rel="enclosure"` links.
|
|
|
|
GUID: the original hashed the title or description when no guid existed; here the chain is
|
|
guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5
|
|
dependency. An entry with none of them has nothing to download and is dropped.
|
|
|
|
`db.rs` gained `http_state`, `record_feed`, `touch_feed`, `set_feed_error`, `record_entry`,
|
|
`record_enclosure`. A changed title/description flips `read` back to 0 — what the original's
|
|
textDiff was ultimately for, minus the `<ins>`/`<del>` markup, which belongs in the UI.
|
|
|
|
`main.rs` gained `ipx fetch [FEED] [--force]`. A failing feed records its error and the scan
|
|
continues.
|
|
|
|
Verified: `cargo test` 10/10. Gate met against a local `python3 -m http.server` serving the
|
|
fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run
|
|
showed both skip paths, `atomcast: not modified` (304) and `testcast: not due for 45m` (the feed's
|
|
own ttl=45 beating `interval_mins = 0`).
|
|
|
|
Deferred: nothing downloads yet — enclosure rows land in state `pending`. That is step 4.
|
|
Next: step 4 — `download.rs`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 2: config.rs + db.rs
|
|
|
|
`src/config.rs`: serde structs for `[general]`, `[torrent]` and `[feeds.<id>]` with defaults, `~`
|
|
expansion, `IPX_CONFIG` / `IPX_DATA_DIR` overrides, `save()` at mode 0600, `Feed::password()`
|
|
(`password_env` beats a literal `password`), `Torrent::ports()` parsing `"6881-6889"`. A missing
|
|
config file loads as an empty one so a fresh install works. Retention defaults are 0/0
|
|
(unlimited, keep forever) — nothing gets deleted until Ray asks for it.
|
|
|
|
`src/db.rs`: schema exactly as planned, WAL + `busy_timeout`, `Db::open()` idempotent, connection
|
|
behind a `Mutex`, `feed_summary()` for `list`, `now()` helper. `enclosures.url` is UNIQUE — the
|
|
dedupe key that replaces `history.dat`.
|
|
|
|
`src/main.rs`: clap skeleton with `ipx list`. Only the subcommands that work exist; the rest arrive
|
|
with their steps.
|
|
|
|
Verified: `cargo test` 5/5 green (config defaults, port-range fallback incl. reversed range,
|
|
password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — `ipx --config
|
|
<scratch> list` printed both feeds and created `state.db` once across two runs.
|
|
|
|
Deferred: nothing. Two dead-code warnings (`Config::save`, `Feed::password`) are expected; steps 3
|
|
and 8 consume them.
|
|
Next: step 3 — `feed.rs`.
|
|
|
|
Also added `chrono` 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 1: repo skeleton
|
|
|
|
Repo created at `/src/ipodderx-rs`, default branch `main`, `cargo init --name ipx` (edition 2024,
|
|
rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this
|
|
file. Hello-world `main.rs` builds.
|
|
|
|
Dependency versions pinned today — the later steps are written against these APIs:
|
|
|
|
| crate | version | notes |
|
|
|---|---|---|
|
|
| tokio | 1.53.1 | rt-multi-thread, macros, fs, io-util, net, sync, time, signal |
|
|
| reqwest | 0.13.5 | `default-features = false`; features **`rustls`** (not `rustls-tls` — renamed in 0.13), http2, gzip, stream, json, charset, **`system-proxy`** (env-var proxy pickup is opt-in in 0.13) |
|
|
| rss | 2.1.1 | default features; the `with-syndication` feature name in my notes does not exist — Atom is handled by the separate crate |
|
|
| atom_syndication | 0.12.10 | |
|
|
| rusqlite | 0.40.2 | `bundled` |
|
|
| librqbit | 9.0.1 | `default-features = false`; features `rust-tls`, `http-api-client`. The default `default-tls` feature pulls reqwest/native-tls **and** an OpenSSL sha1 backend (`crypto-hash`), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7 |
|
|
| serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5 | | |
|
|
| clap | 4.6.6 | derive |
|
|
| infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6 | | |
|
|
| tracing 0.1.44 / tracing-subscriber 0.3.23 | | env-filter |
|
|
|
|
Deferred: nothing.
|
|
|
|
Gotcha worth keeping: three feature names in the plan were wrong against current crate versions —
|
|
`reqwest/rustls-tls` is now `rustls`, env-var proxy support moved behind `system-proxy`, and
|
|
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
|
|
is the fix. Whole tree is rustls-only now, no C TLS dependency.
|
|
Next: step 2 — `config.rs` + `db.rs`. (done)
|
|
|
|
---
|
|
|
|
## How this was built
|
|
|
|
Three planned phases, all complete. The dated entries above are the record of each.
|
|
|
|
**The engine (steps 1-8)** — repo skeleton with pinned dependencies; `config.rs` + `db.rs` (TOML
|
|
config, SQLite schema); `feed.rs` (conditional GET, RSS-then-Atom parsing); `download.rs`
|
|
(streaming downloads, filters, URL dedupe); `retention.rs` (oldest-first quota and age reaper);
|
|
`ipc.rs` plus the daemon (Unix-socket JSON lines, TTL scheduler, CLI proxying to a running daemon);
|
|
`torrent.rs` (librqbit, seeding to a ratio or a time, stall abort); OPML import/export and polish.
|
|
|
|
**The web front end (steps 9-13)** — axum serving plain HTML and JS, no WASM toolchain, running
|
|
*inside* the daemon so it reads SQLite and the event bus directly: config hot-reload and the `[web]`
|
|
section; browsing feeds and entries with `ammonia`-sanitized show notes; media actions (Range
|
|
streaming, download on demand, delete, read/starred); feed configuration written back to
|
|
config.toml; live progress over SSE from the existing broadcast bus.
|
|
|
|
**Multi-user (steps A-D)** — accounts, sessions and sign-in (Argon2id, session cookie, a trusted
|
|
proxy header for Cloudflare Zero Trust); per-user read state; per-user subscriptions; and the rules
|
|
that follow from one file serving everyone. SQLite stayed: Postgres was considered and dropped, as
|
|
nothing here contends for writes.
|
|
|
|
### The live checks each step had to pass
|
|
|
|
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
|
|
2. `ipx fetch` again → no re-download, feed skipped for TTL.
|
|
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
|
|
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
|
|
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
|
|
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
|
|
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
|
|
`reaped`.
|