rays/ipodderx-rs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: rays:v0.1.0
Branches Tags
rays:main rays:seaorm
rays:v0.8.4 rays:v0.8.3 rays:v0.8.2 rays:v0.8.1 rays:v0.8.0 rays:v0.7.0 rays:v0.6.1 rays:v0.6.0 rays:v0.5.5 rays:v0.5.4 rays:v0.5.3 rays:v0.5.2 rays:v0.5.1 rays:v0.5.0 rays:v0.4.0 rays:v0.3.0 rays:v0.2.0 rays:v0.1.0
..
compare: rays:v0.2.0
Branches Tags
rays:main rays:seaorm
rays:v0.8.4 rays:v0.8.3 rays:v0.8.2 rays:v0.8.1 rays:v0.8.0 rays:v0.7.0 rays:v0.6.1 rays:v0.6.0 rays:v0.5.5 rays:v0.5.4 rays:v0.5.3 rays:v0.5.2 rays:v0.5.1 rays:v0.5.0 rays:v0.4.0 rays:v0.3.0 rays:v0.2.0 rays:v0.1.0

37 Commits
v0.1.0 ... v0.2.0

Author SHA1 Message Date
rays
4810bb5cfb Make scanning an admin setting, and document SSO 
The per-feed schedule picker is gone and global Settings is admin-only,
enforced in the handlers with 403s rather than just hidden: polling costs
bandwidth and affects everyone reading the feed, so it belongs to the
operator. Folders, keywords and per-feed limits stay open to anyone.

docs/sso.md covers Cloudflare Zero Trust and Authentik end to end,
including why trusted_proxies names the proxy and not a subnet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 23:16:33 +00:00
rays
06f182b555 Accounts and sign-in, and fix the read toggle 
epAction's redraw closure called itself when handed a row, so Mark read
recursed until the stack blew; it now swaps that row in place. Opening an
item also marks it read, redrawn where it stands so nothing vanishes from
under the pointer on the Unread tab.

Step A of multi-user: users and sessions tables, Argon2id, a session
cookie, ipx user subcommands, and a trusted proxy header for Cloudflare
Zero Trust -- honoured only from a trusted_proxies address. The shared
token still works and is the admin. A new database starts with
admin/ipodderx.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 23:11:20 +00:00
rays
ed26fa2061 Write down the multi-user plan 
SQLite stays; sign-in is local user/pass or the Authentik already
fronting ipodderx.sdf1.net. Feeds, items and files shared; read state
and subscriptions per user.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 20:04:37 +00:00
rays
772dda5154 Call them items, not episodes 
Half the library is text feeds, so the UI no longer assumes a podcast:
counts, search, the empty detail pane, the phone back button, retention
and per-feed settings, and the download dialog all say item. S1E1 badges
and the episode column stay -- those are the itunes:episode field, which
only appears when a feed publishes it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 19:12:58 +00:00
rays
3817dbebdd Stop the UI strobing during a scan 
85 feeds meant 85 feed_done events, each rebuilding the sidebar and
reloading the episode list. Bursts collapse into one refresh, per-feed
"N new" toasts add up into one summary, and both lists keep their scroll
position across a rebuild. A full scan now costs 11 feed refreshes
instead of 85.

Settings and Log move to a footer under the feed list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 19:09:10 +00:00
rays
3ee65f3155 Default to All, move OPML into settings, dress the feed actions 
Opening a feed whose episodes are all read showed an empty list, so All
leads the tabs and is the default. OPML import/export moves under
Settings -- an occasional job, not a daily control -- freeing the
sidebar. Feed actions become icon pills, with Unsubscribe pushed to the
far end and quietened; it sat beside Settings looking identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 19:02:04 +00:00
rays
d157c88ba9 Make the UI work on a phone 
The ☰ button lived in the player bar, which is hidden until something
plays, so the feed list was unreachable on a phone. It moves to a bar
that is always present, and the sidebar gets a scrim.

The reading pane takes the whole screen over the list with a back
button, the player stacks into two rows above it, and the page no longer
scrolls sideways -- a grid column is min-content wide by default, so one
long headline dragged everything off the right edge.

Covered by a Playwright case at 390x844.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 18:58:10 +00:00
rays
7473d5b7fb Mark all read on an OPML subscription 
Marking the subscription read did nothing: its own row holds no entries.
read-all now resolves the feeds grouped under the id -- via
subscriptions(), so a child promoted to config is included -- and marks
those. Button sits before Unsubscribe, where every other feed keeps it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 18:45:46 +00:00
rays
8f5e2749ff Sum a subscription's counts from the feeds it holds 
An OPML folder has no entries of its own, so its row always showed zero
unread however much was waiting inside. Summed from every feed it holds
-- not just the ones a filter left showing, so the count doesn't move as
you type -- with the badge capped at 999+ so a four-digit number doesn't
eat the title beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 18:42:42 +00:00
rays
dfcdf47143 Line up the feed sidebar 
Every row reserves the chevron slot, so artwork and titles share one
column instead of stepping left when a feed has no children. Children
keep a single icon size and read as nested from the indent alone. Labels
stack on one line-height, and the unread count has a min-width so a
three-digit feed doesn't shove its own title.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 18:39:25 +00:00
rays
470f3e1ff1 Use an item's thumbnail as its picture 
Resolves in order of deliberateness: itunes:image, media:thumbnail, a
media:content that says it is an image, then an image enclosure -- which
is where a blog's article picture actually lives, so those entries had
artwork available all along and showed none. Audio enclosures are never
taken for pictures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 18:24:27 +00:00
rays
c77d152015 Stop showing the skip reason next to a skipped enclosure 
The chip already names the kind ("image"), so the sentence beside it added
nothing. A reason is now shown only when the state is an error. It is
still recorded and still reaches the API and the log, where it is useful.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 18:16:41 +00:00
rays
5190a29cdb Refetch when a 304 arrives with nothing stored, and soften a skip reason 
Deleting entries during a cleanup left each feed's ETag in place, so the
rescan got 304s, skipped parsing, and 57 feeds stayed empty until a
publisher happened to change something. A 304 while the feed holds zero
entries means the validator has outlived the data, so the daemon drops it
and asks again.

"not a wanted media type" was jargon, and storing it in last_error painted
an ordinary filter decision red. It reads "not audio or video" now, and a
reason is only shown as an error when the state actually is one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 18:14:57 +00:00
rays
dd9c0881ee Keep every enclosure on an item, and view without downloading 
The rss crate keeps at most one enclosure per item and, when a feed ships
several, silently keeps the last -- so a two-file item lost its first
file. enclosures_by_item reads them from the XML in document order,
unescaping attributes so a URL's &amp; survives. The row summarises the
one you would act on and counts the rest; the pane below lists them all.

Non-media enclosures gain a View link opening in a new tab: the
publisher's URL, or the local copy once downloaded. A direct link, not a
proxy, so the daemon does not become a fetch-anything relay.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 18:01:25 +00:00
rays
6d6b4dd1e4 Do not offer a player for a file that is not audio or video 
The media-type filter stopped new image enclosures being fetched, but ones
already on disk still got a play button and an <audio> element, because
the UI tested for a path rather than for a playable type. Four places did
this, including play() itself, which picked the first downloaded enclosure
whatever it was.

isPlayable() checks audio/* or video/*, falling back to the extension when
a feed declares no type. A downloaded non-media file now shows as its kind
with Save and Delete, so it stays available without posing as an episode.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 17:50:39 +00:00
rays
ddb9dbb7e1 Three-pane layout: feeds, items, item text 
Feeds beside, the feed's items above, and the selected item's text with
its enclosures below -- the shape iPodderX used. Selecting a row fills the
pane below instead of expanding inline; enclosures render there as a
player when the file is present and a labelled download when it is not.
The divider drags and its position is remembered.

The archived site kept no usable screenshot of the original window, only
marketing panels, so this follows the description rather than reference
art.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 17:45:18 +00:00
rays
a83f62ccd3 Do not auto-download enclosures that are not audio or video 
Blog feeds put each article's header image in an <enclosure>, so a text
feed read as a podcast full of episodes: 149 images, 74 MB across 11
feeds. media_types defaults to audio and video, with a per-feed override.

Such enclosures stay listed and stay downloadable by hand; the row names
what it is rather than saying "skipped". An unknown type is allowed, since
the real type is only known after downloading, and a torrent is allowed as
a container judged once unpacked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 17:13:59 +00:00
rays
8b21d19365 Log tabs with a daemon I/O view, and Playwright UI tests 
The log view splits into All / Daemon I/O / Scans / HTTP. Daemon I/O is
the control protocol itself, logged where every command funnels through
so it covers socket clients, the CLI and the web UI alike. stderr and the
in-app buffer now have separate filters, so the UI can keep debug detail
the terminal should not carry.

Playwright drives a real browser against a daemon on fixture feeds. Eight
tests, each mapping to a bug that reached a user -- the Rust tests and the
stub-DOM smoke test cannot see a wrong selector or a dead handler.

It immediately found one: OPML folders rendered expanded by default,
because the code stored closed groups, so any folder never toggled counted
as open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 17:07:19 +00:00
rays
4429f1119c Look up derived feeds everywhere, not just in config 
Moving OPML feeds into the database left several call sites still
searching config.toml only, so anything inside a subscription looked
unsubscribed: Download failed outright, status and the startup line
counted 3 feeds instead of 85, add could duplicate or collide with a
derived feed, and rm could not remove one.

The first grep for this missed the failing call because the method chain
spans lines; searching with newlines collapsed found all of them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 16:55:54 +00:00
rays
665d5b8ecb Keep OPML feeds out of config, cap downloads, log daemon work 
Writing 82 derived feeds into a hand-edited config.toml made it
unreadable. The OPML is the source of truth, so its feeds are re-derived
each scan and held in the database, inheriting the subscription's
settings; editing one promotes it to a real entry. A migration moves
existing children out -- 611 lines to 38 -- keeping all entries and files.

max_new_per_check defaulted to unlimited, so subscribing to an OPML of 82
feeds pulled whole back catalogues. It now defaults to 3 via [general],
capping every feed that does not set its own, and the pending queue orders
by publish date so a cap of 3 means the three newest.

Scans and downloads travelled as socket events only, so the log view
showed no daemon activity. They are mirrored into tracing, with routine
skips at debug -- at 82 feeds those alone would flush the buffer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 15:15:54 +00:00
rays
c86d698363 Subscribe to an OPML, not just import one 
A feed whose body sniffs as OPML is treated as a subscription list and
re-read on every scan, as iPodderX did. Listed feeds become real config
entries grouped under it, inherit its settings, land in one nested folder,
and are scanned in the same run.

When a feed leaves the OPML: removed if nothing was downloaded, kept and
flagged otherwise, so a downloaded file is never orphaned.

folder_for sanitized the whole folder string and would have flattened the
nesting; each segment is sanitized separately now, and a traversal still
cannot escape the download directory. Db::memory() also runs migrate(),
which it did not, so a migration-only column passed tests while missing in
production.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 15:01:03 +00:00
rays
5f6e2a8dc1 Add a log view, and Docker packaging 
The Log button shows the running daemon live: feed scans, downloads,
torrents and every HTTP request. It reads a ring buffer filled by a
tracing layer rather than tailing a file, so it works under Docker where
logs go to stdout. The access-log middleware skips /api/logs, or the
panel's poll would log itself forever.

Detached torrents could leave a row stuck in 'downloading' across a
restart, where nothing would ever revisit it; those are requeued at
startup.

Dockerfile, entrypoint and compose: 114 MB runtime, config bound to
0.0.0.0 on first run since container loopback is unreachable, drops to
PUID:PGID for Unraid, and a healthcheck that goes through the control
socket so a wedged worker reads as unhealthy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 12:06:54 +00:00
rays
19309d609f Run torrents off the command worker 
A torrent ran inline on the single sequential worker, so it blocked every
feed scan, HTTP download and status command behind it -- for up to
stall_mins waiting on metadata, and for up to seed_time_mins seeding after
finishing. A live daemon was wedged with 47 pending torrents and would not
answer a status command for 15s.

spawn_torrent detaches the job behind a 2-permit semaphore and marks the
row 'downloading' so a rescan cannot queue it twice. One-shot CLI runs
stay inline, or the process would exit mid-download.

Torrents themselves verified working: a 755 MB Debian netinst downloaded
to completion against a real swarm.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 02:51:21 +00:00
rays
e97f2b9c2f Schedule pickers, and target progress at one row 
"Check feeds every" becomes a number plus a unit dropdown in both global
and per-feed settings; parse_interval gained weeks to back it. The
per-feed dropdown can select the global default, clearing the override.

Fixes progress painting every pending row: the event carried no enclosure
id, so the handler had nothing to target and set the width on all of them.
Adding a feed looked like it was downloading everything. Progress,
DownloadDone and DownloadError now carry the enclosure id.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 02:38:14 +00:00
rays
9960befed5 Fix the UI dying at load, and add a page smoke test 
$('#prefs').onclick referenced a prefsModal that was never defined, and an
uncaught ReferenceError stops the whole script -- taking the theme toggle,
the feed filter, the event stream and loadFeeds() down with it, so the app
rendered an empty shell.

The scheduling patch had anchored on a function the rewrite already
deleted; str.replace matched nothing and said nothing.

tests/page-smoke.js executes the page against a stub DOM so this class of
failure is visible, since every server-side check passed while the UI was
completely dead. Handler wiring now skips a bad reference instead of
throwing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
 2026-09-10 02:31:18 +00:00
rays
9dc4c1ddfa Add feed check scheduling, global with per-feed override 
general.schedule and feeds.<id>.schedule take "every 30m", "4h", "1d" or
bare minutes. The legacy interval_mins is still read. An explicit per-feed
schedule wins over the publisher's ttl; without one, ttl still raises the
interval when they ask to be polled less often.

Fixes two bugs found while testing it:

null never cleared a field. serde maps JSON null onto the outer None of an
Option<Option<T>>, so "clear this" was indistinguishable from "not
supplied" and every clear silently no-opped with a 204.

The daemon ignored SIGTERM while working. select! races branches only at
selection time, so a signal queued behind an in-flight download and the
process had to be SIGKILLed. The stop signal now cancels work in progress:
SIGTERM mid-download exits in 1s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 02:25:38 +00:00
rays
9c5e7716c5 Name the app iPodderX and cut the icon's white background 
A blanket white-to-transparent would have holed the device, whose body is
also white, so the background is flood-filled from the corners inward and
stops at the outline. Verified composited on the dark theme background
rather than by trusting the alpha channel.

The uppercase transform on the heading had to go too, or the name renders
as IPODDERX.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 02:13:12 +00:00
rays
cbd5e68d13 Derive the colour scheme from the iPodderX icon 
Palette sampled from the icon rather than chosen to resemble it: the
silver device ramp, the screen blues, and the amber EQ bars. Dark theme
builds down from the screen navy; light theme uses the device body with
the deeper blue as accent.

Contrast was measured, not assumed: --faint, which carries dates and
sizes, failed AA in both themes and was moved along the icon's own grey
ramp until it passed. Lowest pair anywhere is now 4.54.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 02:08:59 +00:00
rays
e05ea62051 Use the original iPodderX icon instead of a placeholder 
Recovered from the Internet Archive's capture of ipodderx.com: the icon
was never in either repo, since only the Python engine was open-sourced
and the .icns lived in the Cocoa bundle. Kept as a file in web/ and
embedded as a data URI for the sidebar mark and the favicon.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 02:03:48 +00:00
rays
aecad18d22 Make the feed URL editable, with a copy button 
Entries and history are keyed by feed id, so changing a URL keeps them --
the point being that a feed URL can carry an auth token that gets rotated.
Changing it clears the stored ETag/Last-Modified, which belong to the old
URL and could otherwise produce a bogus 304.

The copy button cannot use navigator.clipboard: that needs a secure
context and this is served over plain HTTP on a LAN address. Falls back to
execCommand.

Invalid input now returns 400 rather than 500.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 02:00:20 +00:00
rays
44623098ae Mark episodes read when finished, not when started 
play() set read=1 the instant playback began. The default view is the
Unread tab, so pressing play removed the episode from the list being
looked at, which reads as the episode going missing. All four affected
rows had position=0: started, never listened to.

Read is now set on 'ended' or past 90% of the duration.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 01:55:46 +00:00
rays
72280b9ccd Pin RSS <title> as the only source of an episode title 
Confirmed byte-for-byte against a live feed, and tested against a fixture
whose itunes:title differs: the RSS title wins, and season/episode stay
metadata rather than being folded into the displayed name. An item with a
season but no episode number keeps a null episode instead of inventing one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 01:48:56 +00:00
rays
5666166769 Full-featured web UI 
Rewrites the page around a persistent player (speed, seek, resume,
MediaSession, keyboard shortcuts), artwork, filter tabs, episode search,
pagination and live progress, with modals and toasts replacing prompt()
and a status line.

Backend gains the metadata that makes that possible: feed and episode
artwork, durations, season/episode numbers and playback position, plus
filters, search, totals, mark-all-read, download-latest and OPML over
HTTP. Schema changes arrive through a real migration, since CREATE TABLE
IF NOT EXISTS does nothing to an installed database.

Fixes filtering, which returned 500 whenever no search term was given:
the search clause was dropped while its parameter was still bound.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 01:42:55 +00:00
rays
93b4815d84 Make the UI's Download button download the episode you clicked 
POST /api/enclosures/{id}/download requeued the row and asked for a
normal scan, but a scan takes the lowest-id pending rows up to
max_new_per_check. With a large backlog and a small cap the requested
row was never a candidate, so other episodes downloaded while it stayed
pending.

A queue expresses what is outstanding, not what was asked for. Download
is now its own command that fetches one specific enclosure immediately,
ignoring queue order and the per-scan cap, still via the single worker.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 01:11:12 +00:00
rays
74ec6e9281 Phase 2: web front end 
axum served from inside the daemon so it reads SQLite and the event bus
directly: browse feeds, read show notes, play with seeking, download and
delete files, mark read/flag, and edit feed settings.

Config is now hot-reloadable (Ctx.cfg behind RwLock<Arc<Config>>), so UI
edits apply without a daemon restart. Access is a shared token minted from
/dev/urandom, carried in a cookie because an <audio> element cannot send
headers. Show notes are untrusted feed HTML and are sanitized with ammonia
server-side.

read/flagged finally have a writer, which retention has needed since it
started ordering by them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 00:55:16 +00:00
rays
ed47e456d4 Turn filename separators into dashes rather than dropping them 
Splits the forbidden set: / \ | : were separating words, so they become
"-"; ? * < > " ' just go. Runs of dashes and spaces collapse to " - "
when the run held whitespace and to a bare "-" when it did not, so
"Show | Series" reads "Show - Series" while "AC/DC" stays "AC-DC".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 00:40:17 +00:00
rays
c12e8ca19c Collapse whitespace left by stripped filename separators 
A real feed titled with pipe separators produced a folder named
"Get in the Trunk  Anthology Series  Delta Green": removing a forbidden
character left the gap around it. Runs of whitespace now collapse, and
control characters map to a space rather than vanishing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
 2026-09-10 00:37:30 +00:00
39 changed files with 7255 additions and 159 deletions
Expand all files Collapse all files

4
.dockerignore Normal file
View File

@@ -0,0 +1,4 @@
target/
.git/
*.md
tests/

3
.gitignore vendored
View File

@@ -1 +1,4 @@
/target /target
/node_modules
/test-results
/playwright-report

329
Cargo.lock generated
View File

@@ -23,6 +23,18 @@ version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923" checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "ammonia"
version = "4.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc6d763210e2eb7670d1a5183a08bebefa3f97db2a738a684f2ce00bd49f681d"
dependencies = [
"cssparser",
"html5ever",
"maplit",
"url",
]
[[package]] [[package]]
name = "android_system_properties" name = "android_system_properties"
version = "0.1.6" version = "0.1.6"
@@ -97,6 +109,18 @@ dependencies = [
"rustversion", "rustversion",
] ]
[[package]]
name = "argon2"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "134c52ddac6d63c576bef8168db10c83c49c26444ecbc68060fef078925a901c"
dependencies = [
"base64ct",
"blake2",
"cpufeatures",
"password-hash",
]
[[package]] [[package]]
name = "arrayvec" name = "arrayvec"
version = "0.7.8" version = "0.7.8"
@@ -175,7 +199,7 @@ dependencies = [
"chrono", "chrono",
"derive_builder", "derive_builder",
"diligent-date-parser", "diligent-date-parser",
"quick-xml", "quick-xml 0.41.0",
] ]
[[package]] [[package]]
@@ -222,6 +246,7 @@ checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90"
dependencies = [ dependencies = [
"axum-core", "axum-core",
"bytes", "bytes",
"form_urlencoded",
"futures-util", "futures-util",
"http", "http",
"http-body", "http-body",
@@ -235,6 +260,9 @@ dependencies = [
"percent-encoding", "percent-encoding",
"pin-project-lite", "pin-project-lite",
"serde_core", "serde_core",
"serde_json",
"serde_path_to_error",
"serde_urlencoded",
"sync_wrapper", "sync_wrapper",
"tokio", "tokio",
"tower", "tower",
@@ -310,6 +338,12 @@ version = "0.23.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5" checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]]
name = "base64ct"
version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]] [[package]]
name = "bitflags" name = "bitflags"
version = "1.3.2" version = "1.3.2"
@@ -334,6 +368,24 @@ dependencies = [
"wyz", "wyz",
] ]
[[package]]
name = "blake2"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b5d4d889834ee8ecfc0f8426ad30faf7cdcb10f741a8e6d7224d95325479f6f"
dependencies = [
"digest",
]
[[package]]
name = "block-buffer"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
dependencies = [
"hybrid-array",
]
[[package]] [[package]]
name = "bs58" name = "bs58"
version = "0.5.1" version = "0.5.1"
@@ -492,6 +544,12 @@ dependencies = [
"cc", "cc",
] ]
[[package]]
name = "cmov"
version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a"
[[package]] [[package]]
name = "colorchoice" name = "colorchoice"
version = "1.0.5" version = "1.0.5"
@@ -590,6 +648,35 @@ version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6" checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
[[package]]
name = "crypto-common"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453"
dependencies = [
"hybrid-array",
]
[[package]]
name = "cssparser"
version = "0.37.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c9cdaae01d5ed7882b04d795e7f752f46ff52d2fa3b50a20d28c464510bba98"
dependencies = [
"dtoa-short",
"itoa",
"smallvec",
]
[[package]]
name = "ctutils"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e"
dependencies = [
"cmov",
]
[[package]] [[package]]
name = "darling" name = "darling"
version = "0.20.11" version = "0.20.11"
@@ -751,6 +838,17 @@ dependencies = [
"syn 2.0.119", "syn 2.0.119",
] ]
[[package]]
name = "digest"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
dependencies = [
"block-buffer",
"crypto-common",
"ctutils",
]
[[package]] [[package]]
name = "diligent-date-parser" name = "diligent-date-parser"
version = "0.1.5" version = "0.1.5"
@@ -813,6 +911,21 @@ dependencies = [
"windows-sys 0.52.0", "windows-sys 0.52.0",
] ]
[[package]]
name = "dtoa"
version = "1.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4c3cf4824e2d5f025c7b531afcb2325364084a16806f6d47fbc1f5fbd9960590"
[[package]]
name = "dtoa-short"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd1511a7b6a56299bd043a9c167a6d2bfb37bf84a6dfceaba651168adfb43c87"
dependencies = [
"dtoa",
]
[[package]] [[package]]
name = "dunce" name = "dunce"
version = "1.0.5" version = "1.0.5"
@@ -1206,6 +1319,16 @@ version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
[[package]]
name = "html5ever"
version = "0.39.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46a1761807faccc9a19e86944bbf40610014066306f96edcdedc2fb714bcb7b8"
dependencies = [
"log",
"markup5ever",
]
[[package]] [[package]]
name = "http" name = "http"
version = "1.5.0" version = "1.5.0"
@@ -1239,6 +1362,12 @@ dependencies = [
"pin-project-lite", "pin-project-lite",
] ]
[[package]]
name = "http-range-header"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9171a2ea8a68358193d15dd5d70c1c10a2afc3e7e4c5bc92bc9f025cebd7359c"
[[package]] [[package]]
name = "httparse" name = "httparse"
version = "1.10.1" version = "1.10.1"
@@ -1251,6 +1380,15 @@ version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9" checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
[[package]]
name = "hybrid-array"
version = "0.4.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17"
dependencies = [
"typenum",
]
[[package]] [[package]]
name = "hyper" name = "hyper"
version = "1.11.1" version = "1.11.1"
@@ -1498,8 +1636,11 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
name = "ipx" name = "ipx"
version = "0.1.0" version = "0.1.0"
dependencies = [ dependencies = [
"ammonia",
"anyhow", "anyhow",
"argon2",
"atom_syndication", "atom_syndication",
"axum",
"chrono", "chrono",
"clap", "clap",
"dirs", "dirs",
@@ -1508,13 +1649,17 @@ dependencies = [
"librqbit", "librqbit",
"opml", "opml",
"percent-encoding", "percent-encoding",
"quick-xml 0.42.0",
"reqwest", "reqwest",
"rss", "rss",
"rusqlite", "rusqlite",
"serde", "serde",
"serde_json", "serde_json",
"tokio", "tokio",
"tokio-stream",
"toml", "toml",
"tower",
"tower-http 0.7.1",
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
"url", "url",
@@ -1974,7 +2119,7 @@ dependencies = [
"httparse", "httparse",
"librqbit-dualstack-sockets", "librqbit-dualstack-sockets",
"network-interface", "network-interface",
"quick-xml", "quick-xml 0.41.0",
"reqwest", "reqwest",
"serde", "serde",
"serde_derive", "serde_derive",
@@ -2045,6 +2190,23 @@ version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154" checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
[[package]]
name = "maplit"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e2e65a1a2e43cfcb47a895c4c8b10d1f4a61097f9f254f183aee60cad9c651d"
[[package]]
name = "markup5ever"
version = "0.39.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7122d987ec5f704ee56f6e5b41a7d93722e9aae27ae07cafa4036c4d3f9757de"
dependencies = [
"log",
"tendril",
"web_atoms",
]
[[package]] [[package]]
name = "matchers" name = "matchers"
version = "0.2.0" version = "0.2.0"
@@ -2161,6 +2323,12 @@ dependencies = [
"winapi", "winapi",
] ]
[[package]]
name = "new_debug_unreachable"
version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
[[package]] [[package]]
name = "nix" name = "nix"
version = "0.30.1" version = "0.30.1"
@@ -2314,12 +2482,72 @@ dependencies = [
"windows-link", "windows-link",
] ]
[[package]]
name = "password-hash"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aab41826031698d6ffcd9cff78ef56ef998e39dc7e5067cdfebe373842d4723b"
dependencies = [
"getrandom 0.4.3",
"phc",
]
[[package]] [[package]]
name = "percent-encoding" name = "percent-encoding"
version = "2.3.2" version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "phc"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44dc769b75f93afdddd8c7fa12d685292ddeff1e66f7f0f3a234cf1818afe892"
dependencies = [
"base64ct",
"ctutils",
"getrandom 0.4.3",
]
[[package]]
name = "phf"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf"
dependencies = [
"phf_shared",
"serde",
]
[[package]]
name = "phf_codegen"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49aa7f9d80421bca176ca8dbfebe668cc7a2684708594ec9f3c0db0805d5d6e1"
dependencies = [
"phf_generator",
"phf_shared",
]
[[package]]
name = "phf_generator"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "135ace3a761e564ec88c03a77317a7c6b80bb7f7135ef2544dbe054243b89737"
dependencies = [
"fastrand",
"phf_shared",
]
[[package]]
name = "phf_shared"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e57fef6bc5981e38c2ce2d63bfa546861309f875b8a75f092d1d54ae2d64f266"
dependencies = [
"siphasher",
]
[[package]] [[package]]
name = "pin-project-lite" name = "pin-project-lite"
version = "0.2.17" version = "0.2.17"
@@ -2371,6 +2599,12 @@ dependencies = [
"zerocopy", "zerocopy",
] ]
[[package]]
name = "precomputed-hash"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
[[package]] [[package]]
name = "proc-macro2" name = "proc-macro2"
version = "1.0.107" version = "1.0.107"
@@ -2406,6 +2640,15 @@ dependencies = [
"serde", "serde",
] ]
[[package]]
name = "quick-xml"
version = "0.42.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41b1177fdf999d2321d3fb46ff47159d9c1fb9ad66a4879f8c50a0b504615e9b"
dependencies = [
"memchr",
]
[[package]] [[package]]
name = "quinn" name = "quinn"
version = "0.11.11" version = "0.11.11"
@@ -2666,7 +2909,7 @@ dependencies = [
"tokio-rustls", "tokio-rustls",
"tokio-util", "tokio-util",
"tower", "tower",
"tower-http", "tower-http 0.6.11",
"tower-service", "tower-service",
"url", "url",
"wasm-bindgen", "wasm-bindgen",
@@ -2727,7 +2970,7 @@ checksum = "f505d3e5e7b06b4dc0245b13294f8ef9a1a0f70284708be1e11c5b7b7441034e"
dependencies = [ dependencies = [
"atom_syndication", "atom_syndication",
"derive_builder", "derive_builder",
"quick-xml", "quick-xml 0.41.0",
] ]
[[package]] [[package]]
@@ -3092,6 +3335,12 @@ version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e" checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e"
[[package]]
name = "siphasher"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649"
[[package]] [[package]]
name = "size_format" name = "size_format"
version = "1.0.2" version = "1.0.2"
@@ -3151,6 +3400,30 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
[[package]]
name = "string_cache"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a18596f8c785a729f2819c0f6a7eae6ebeebdfffbfe4214ae6b087f690e31901"
dependencies = [
"new_debug_unreachable",
"parking_lot",
"phf_shared",
"precomputed-hash",
]
[[package]]
name = "string_cache_codegen"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "585635e46db231059f76c5849798146164652513eb9e8ab2685939dd90f29b69"
dependencies = [
"phf_generator",
"phf_shared",
"proc-macro2",
"quote",
]
[[package]] [[package]]
name = "strsim" name = "strsim"
version = "0.11.1" version = "0.11.1"
@@ -3243,6 +3516,15 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369" checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "tendril"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5fed54709c5b3a53d09bb1c113ea4f5ceafd1e772ddcb0030a82e1d56c087b08"
dependencies = [
"new_debug_unreachable",
]
[[package]] [[package]]
name = "thiserror" name = "thiserror"
version = "1.0.69" version = "1.0.69"
@@ -3474,6 +3756,7 @@ dependencies = [
"tokio", "tokio",
"tower-layer", "tower-layer",
"tower-service", "tower-service",
"tracing",
] ]
[[package]] [[package]]
@@ -3499,6 +3782,31 @@ dependencies = [
"url", "url",
] ]
[[package]]
name = "tower-http"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08a05a66a4fdd61cbbe0a1d755ffe0ca6aba159dd4820936a0ff8a8278245b9c"
dependencies = [
"bitflags 2.13.1",
"bytes",
"futures-core",
"futures-util",
"http",
"http-body",
"http-body-util",
"http-range-header",
"httpdate",
"mime",
"mime_guess",
"percent-encoding",
"pin-project-lite",
"tokio",
"tokio-util",
"tower-layer",
"tower-service",
]
[[package]] [[package]]
name = "tower-layer" name = "tower-layer"
version = "0.3.3" version = "0.3.3"
@@ -3517,6 +3825,7 @@ version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [ dependencies = [
"log",
"pin-project-lite", "pin-project-lite",
"tracing-attributes", "tracing-attributes",
"tracing-core", "tracing-core",
@@ -3790,6 +4099,18 @@ dependencies = [
"wasm-bindgen", "wasm-bindgen",
] ]
[[package]]
name = "web_atoms"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba8b815c1b593dc0baf78dd0f4fc8fdb2de53198fb1163738093e9a311c33fb3"
dependencies = [
"phf",
"phf_codegen",
"string_cache",
"string_cache_codegen",
]
[[package]] [[package]]
name = "webpki-root-certs" name = "webpki-root-certs"
version = "1.0.9" version = "1.0.9"

7
Cargo.toml
View File

@@ -4,8 +4,11 @@ version = "0.1.0"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
ammonia = "4.1.4"
anyhow = "1.0.104" anyhow = "1.0.104"
argon2 = "0.6.0"
atom_syndication = "0.12.10" atom_syndication = "0.12.10"
axum = "0.8.9"
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] } chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
clap = { version = "4.6.6", features = ["derive"] } clap = { version = "4.6.6", features = ["derive"] }
dirs = "7.0.0" dirs = "7.0.0"
@@ -14,13 +17,17 @@ infer = "0.22.0"
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] } librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
opml = "1.1.6" opml = "1.1.6"
percent-encoding = "2.3.2" percent-encoding = "2.3.2"
quick-xml = "0.42.0"
reqwest = { version = "0.13.5", default-features = false, features = ["rustls", "http2", "gzip", "stream", "json", "charset", "system-proxy"] } reqwest = { version = "0.13.5", default-features = false, features = ["rustls", "http2", "gzip", "stream", "json", "charset", "system-proxy"] }
rss = "2.1.1" rss = "2.1.1"
rusqlite = { version = "0.40.2", features = ["bundled"] } rusqlite = { version = "0.40.2", features = ["bundled"] }
serde = { version = "1.0.229", features = ["derive"] } serde = { version = "1.0.229", features = ["derive"] }
serde_json = "1.0.151" serde_json = "1.0.151"
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] } tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
tokio-stream = { version = "0.1.19", features = ["sync"] }
toml = "1.1.5" toml = "1.1.5"
tower = { version = "0.5.3", features = ["util"] }
tower-http = { version = "0.7.1", features = ["fs"] }
tracing = "0.1.44" tracing = "0.1.44"
tracing-subscriber = { version = "0.3.23", features = ["env-filter"] } tracing-subscriber = { version = "0.3.23", features = ["env-filter"] }
url = "2.5.8" url = "2.5.8"

39
Dockerfile Normal file
View File

@@ -0,0 +1,39 @@
# Build. rusqlite is bundled (compiles SQLite from source) and librqbit needs a C
# toolchain, so the builder needs cc. TLS is rustls throughout, so no OpenSSL headers.
FROM rust:1-slim-bookworm AS build
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /src
# Dependencies first, so editing the source does not rebuild librqbit every time.
COPY Cargo.toml Cargo.lock ./
RUN mkdir src && echo 'fn main(){}' > src/main.rs \
&& cargo build --release --locked \
&& rm -rf src
COPY src ./src
COPY web ./web
# cargo skips a rebuild if mtimes look untouched; make sure it does not.
RUN touch src/main.rs && cargo build --release --locked
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates gosu \
&& rm -rf /var/lib/apt/lists/*
COPY --from=build /src/target/release/ipx /usr/local/bin/ipx
COPY docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
ENV IPX_CONFIG=/config/config.toml \
IPX_DATA_DIR=/data \
IPX_LOG=ipx=info \
PUID=99 \
PGID=100
VOLUME ["/config", "/data", "/downloads"]
# Web UI, and the BitTorrent peer port (TCP and UDP -- DHT needs the UDP side).
EXPOSE 8099/tcp 6881/tcp 6881/udp
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["ipx", "daemon"]

925
PROGRESS.md
View File

@@ -17,6 +17,32 @@ The full design and step list live in the plan file at
see the step 7 entry) see the step 7 entry)
- [x] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd units, README. - [x] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd units, README.
### Phase 2 — web front end
Decided with Ray: axum serving plain HTML/JS (no WASM toolchain), running **inside the daemon**
process so it reads SQLite and the event bus directly, LAN-bindable with a shared token.
- [x] **9. Config hot-reload + web skeleton.** `Ctx.cfg` becomes `RwLock<Arc<Config>>` so the UI can
edit feeds without a daemon restart. `[web]` config section (enabled/bind/token, token
auto-generated and saved on first run). axum server started by `ipx daemon`, token checked by
middleware, `?token=` sets a cookie so `<audio>` requests authenticate too.
*Done when:* `ipx daemon` serves a page on the configured bind, and a wrong token gets 401.
- [x] **10. Browsing.** `/api/feeds`, `/api/feeds/:id/entries`, entry detail. Descriptions are
untrusted feed HTML — sanitized with `ammonia` before they reach the page.
*Done when:* the Glass Cannon feed's 131 entries browse and read correctly.
- [x] **11. Media actions.** Range-request audio streaming (`tower-http` ServeFile) so seeking
works, download-on-demand for a pending enclosure, delete a file, mark read/flagged.
*Done when:* an episode plays and seeks in a browser, and delete reaps the row.
- [x] **12. Feed configuration.** Add/remove feeds and edit folder, keywords, allow_explicit,
auto_download, max_new_per_check from the UI, written back to config.toml and hot-reloaded.
*Done when:* flipping allow_explicit in the UI takes effect on the next scan with no restart.
- [x] **13. Live progress + polish.** SSE from the existing broadcast bus so downloads show live.
README section, screenshot-free usage notes.
*Done when:* starting a fetch from the UI shows progress advancing without a reload.
Note: `read`/`flagged` finally get a writer here. Retention orders by them (see the step 5 entry),
and until now nothing set them.
## Smoke tests ## Smoke tests
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`. 1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
@@ -30,6 +56,905 @@ The full design and step list live in the plan file at
--- ---
## 2026-09-10 — Scanning is the operator's decision
The per-feed **Check schedule** picker is gone from feed settings, and the global Settings page is
hidden from anyone who is not an admin. Both are enforced in the handlers rather than merely hidden:
`PATCH /api/settings` and a `schedule` in `PATCH /api/feeds/{id}` return 403 for an ordinary user.
Polling costs bandwidth, is what a publisher notices, and one impatient setting affects everyone
reading that feed -- it belongs in config.toml.
Folders, keywords, per-feed download limits and the feed URL stay editable by anyone signed in.
`docs/sso.md` covers putting Cloudflare Zero Trust or Authentik in front of ipx: the tunnel and
Access application, the Authentik proxy provider and its forward-auth nginx block, the three lines
of ipx config each needs, and why `trusted_proxies` must name the proxy rather than a subnet --
with the command to prove the refusal works.
---
## 2026-09-10 — Marking an item read
Two bugs in one place. `epAction`'s `redraw` closure called *itself* when it had a row to
update -- `if(el) redraw()` where it meant to swap the row -- so the Mark read button in the
text pane recursed until the stack blew. It now swaps that one row in place and refreshes the
text below only when it is the item being read.
And opening an item now marks it read, which is what clicking a thing to read it means. The row
is redrawn where it stands rather than the list reloaded, so an item does not vanish from under
the pointer on the Unread tab.
---
## 2026-09-10 — Step A: accounts and sign-in
`users` and `sessions` tables, Argon2id hashing, a session cookie, and `ipx user add|list|passwd|rm`
(passwords come in on stdin, so they miss the shell history and any `ps` listing).
Three ways in, in order of how specific the claim is:
1. **A proxy header** naming the user -- `Cf-Access-Authenticated-User-Email` for the Cloudflare
Zero Trust in front of `ipodderx.sdf1.net`. Honoured **only** from an address in
`trusted_proxies` (loopback by default): a header is worth exactly as much as the hop that set
it, and the LAN port would otherwise let anyone claim to be anyone. Verified both ways -- a
spoof from an untrusted address is refused.
2. **A session cookie** from signing in at `/login`.
3. **The shared token**, which is the admin, so the healthcheck and existing links keep working.
A database with no accounts creates **admin / ipodderx** and says so loudly in the log. The UI shows
who is signed in above the sidebar footer, with a sign-out, and a 401 sends the page to `/login`.
Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B.
---
## Multi-user — the plan
Decided with Ray: **stay on SQLite** (Postgres was considered and dropped -- it is a deployment
choice, not a capability one, and nothing here contends for writes). Sign-in is either a local
username and password or the Cloudflare Zero Trust that already fronts `ipodderx.sdf1.net`, which
puts the authenticated identity in `Cf-Access-Authenticated-User-Email`. Feeds, items and files are **shared**; read state and subscriptions are **per user**.
The point of sharing: two people subscribed to the same show cost one fetch, one parse, and one file
on disk. `enclosures.url` is already globally UNIQUE, so the file half is nearly free.
- [x] **A. Users, sessions, sign-in.** `users` + `sessions` tables, Argon2 hashing, session cookie,
`ipx user add|list|passwd|rm`. A proxy header (`trusted_header` in `[web]`) signs in and
optionally creates a user -- honoured only from a `trusted_proxies` address, so a LAN client
cannot simply assert it. The existing shared token keeps working and resolves to the admin, so
the healthcheck and any scripts survive. Login page for direct access.
- [ ] **B. Per-user read state.** `entry_state(user_id, feed_id, guid, read, flagged, position)`;
the current columns on `entries` migrate into the first user's rows. Unread counts, filters and
playback position all become per user.
- [ ] **C. Per-user subscriptions.** `subscriptions(user_id, feed_id)`. config.toml stays the feed
catalogue; the UI lists only what you subscribe to. Adding a feed someone else already has costs
nothing. A feed nobody subscribes to stops being scanned but keeps its files.
- [ ] **D. One file, many users.** Auto-download when *any* subscriber wants it; retention never
deletes a file another user has starred or not yet played; deleting a download says so when
someone else still has it.
---
## 2026-09-10 — Items, not episodes
Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search
box, the empty detail pane, the phone back button, the retention and per-feed settings, and the
download dialog all say **item** now. The sidebar's `20 eps` reads `20 items`.
`S1E1` badges stay -- those come from `itunes:episode`/`itunes:season` and only appear when a feed
actually publishes them -- as does the `episode` column, which is that same field.
---
## 2026-09-10 — Scanning stopped strobing
A scan of 85 feeds fires 85 `feed_done` events, and the UI refreshed on every one: a full sidebar
rebuild plus an episode reload each time, which read as flicker. Bursts now collapse into one
refresh (500ms trailing), the per-feed "N new" toasts add up into a single summary, and both lists
keep their scroll position across a rebuild instead of jumping to the top.
Measured over a full scan: 11 feed refreshes and 2 episode reloads in 45 seconds, down from one per
event.
Settings and Log moved out of the header and toolbar into a footer under the feed list -- they are
housekeeping, not daily controls. The theme toggle stays in the header.
---
## 2026-09-10 — Defaults and chrome
* **All** is the default episode filter and the first tab; Unread, Downloaded and Flagged follow.
Opening a feed and seeing nothing because everything in it was read is a poor first impression.
* **OPML import/export moved into Settings**, under a Subscriptions heading, and the OPML button is
gone from the sidebar -- it is a thing you do once in a while, not a daily control.
* **The feed actions are pills with icons**, and Unsubscribe is pushed to the far end in a quiet
style: it sat next to Settings looking exactly like it, one slip away from losing a feed.
---
## 2026-09-10 — Phone layout
The UI was unusable on a phone, starting with the worst of it: the ☰ button lived inside the player
bar, which is hidden until something plays, so there was no way to reach the feed list at all. It
now sits in a small bar at the top of the main pane that is always there, and the sidebar slides
over the page with a scrim to tap away.
The rest:
* **One pane at a time.** The three-pane split becomes a list, with the item text taking the whole
screen over it and a `← Episodes` button back. The divider is hidden.
* **No sideways scrolling.** A grid column is min-content wide by default, so one long headline
("davewiner/hackerNewsStars") dragged the entire page off the right edge -- `min-width:0` down the
shell/main/wrap chain, `overflow-wrap:anywhere` on headings, and an explicit
`minmax(0,1fr)` column for the OPML child list.
* **The player stacks**: title row on top, transport and seek bar below, and it paints above the
reading pane so it stays reachable.
* Header artwork, buttons and log rows shrink to fit; tap targets go to 38px.
A Playwright case at 390x844 locks in the three things that actually broke: the burger is visible
with nothing playing, the page does not scroll sideways, and an item opens and closes over the list.
---
## 2026-09-10 — Marking a subscription read
An OPML subscription's page now has **Mark all read**, sitting where every other feed keeps it --
before Unsubscribe. The fix is in the handler rather than the button: `read-all` resolves the feeds
whose group is the given id (through `subscriptions()`, so a child promoted to config counts too)
and marks those, since the subscription's own row holds no entries and marking it did nothing.
---
## 2026-09-10 — A folder counts what it holds
An OPML subscription has no entries of its own, so its row always read `0 unread` no matter how much
was waiting inside it. Unread and saved are now summed from the feeds it holds -- from all of them,
not just the ones a search filter left showing, so the number doesn't move as you type. The badge
caps at `999+` (the real figure is in its tooltip); a four-digit count ate the title beside it.
---
## 2026-09-10 — Sidebar alignment
The feed list had four different left edges: a row with no children skipped the chevron entirely, so
its artwork sat a chevron-width left of a folder's; children then used a different indent *and* a
smaller icon. Nothing lined up with anything.
Every row now reserves the chevron slot whether or not it opens (an empty one is
`pointer-events:none`, so the click falls through to the row), all artwork is one size, and nesting
reads from the indent alone. Both label lines are `display:block` on a shared line-height instead of
an inline baseline, and the unread count has a `min-width` so three-digit feeds don't shove the
title. Rows came out shorter, so more feeds fit without scrolling.
---
## 2026-09-10 — An item's picture
An item's artwork now resolves in order of how deliberate the source is: `itunes:image`, then Media
RSS `media:thumbnail`, then a `media:content` that declares itself an image, and finally an image
**enclosure**. That last one matters here -- Substack puts each article's header picture in an
`<enclosure>`, which is why those blog entries had no artwork despite carrying one all along. Audio
enclosures are never mistaken for pictures.
Backfilling needed the validators cleared first: `record_entry` fills a missing image on update, but
a 304 skips parsing entirely, so the feeds would have kept their blank squares. (The self-heal added
earlier only fires when a feed has *zero* entries, which was not the case here.)
Result across the library: 325 of 3470 entries now carry their own picture, 13 feeds where every
entry has one, 69 feeds that publish no per-item image at all -- those fall back to the feed's
artwork, which is the intended behaviour rather than a gap.
---
## 2026-09-10 — Database cleanup, and the 304 trap it walked into
Cleaned up on request: removed the `CT Log Archive Torrents` folder (123 preallocated files from the
abandoned 1.6 TB tuscolo torrent, **19 GB real** on disk, no row referencing any of it, feed no
longer subscribed), the unsubscribed ct-log feed's rows, and every OPML-derived entry/enclosure that
held no file so a rescan could rebuild them with the current parser. Kept every row holding a file,
so nothing on disk was orphaned, and kept the skipped/reaped history, without which the next scan
would re-download the 149 images just deleted. Disk 22 GB -> 3.8 GB, database 22.8 MB -> 5.2 MB.
Correction worth recording: mid-download I checked that torrent folder, saw `du -sh` report 8 KB,
and told Ray it was sparse with nothing written. By the time it was abandoned it had allocated 19 GB.
The reassurance had a shelf life I did not mention.
**Then "Abort Retry Fail is empty".** The cleanup deleted entries but left each feed's
ETag/Last-Modified. The rescan sent them, servers answered 304 (120 times in the log), the daemon
skipped parsing, and 57 feeds stayed empty -- and would have until a publisher happened to change
something. Cleared the validators on the empty feeds and rescanned: entries 1524 -> 3009, feeds with
content 19 -> 64, Abort Retry Fail back to 20.
Fixed in code so it cannot recur: a 304 arriving while the feed has **zero stored entries** means the
validator has outlived the data -- a restore, a manual edit, a cleanup. The daemon now believes the
database over the validator, drops it and asks again. Proved live by deleting a feed's entries,
leaving its ETag, and rescanning: 20 entries rebuilt, self-heal logged once.
**"not a wanted media type"** was internal jargon reaching the UI, and it was stored in `last_error`
so an ordinary filter decision rendered in red as though something had failed. Reworded to "not
audio or video", 144 existing rows updated, and the UI now paints a reason red only when the state
is actually `error`.
---
## 2026-09-10 — Multiple enclosures per item, and viewing without downloading
**View without downloading.** A non-audio/video enclosure now carries a View link opening in a new
tab: the publisher's own URL when nothing is downloaded, the local copy at `/media/<id>` when it is.
Deliberately a direct link rather than a proxy — relaying arbitrary URLs through the daemon would
make it a fetch-anything service.
**Multiple enclosures.** Probed before assuming, and the result was worse than expected: the `rss`
crate models an item as having at most one enclosure (which is what RSS 2.0 says) and when a feed
carries several it silently keeps **the last**, dropping the rest. So a two-file item lost its first
file entirely.
`enclosures_by_item()` reads them straight from the XML with quick-xml, in document order,
unescaping attribute values — a feed URL's `&` arrives as `&amp;`, so skipping that would corrupt
every query string. It falls back to the parsed enclosure if the scan and the parser disagree on
item count. Atom already collected all `rel="enclosure"` links. The row now summarises the enclosure
you would act on (playable, else downloaded, else first) and says "+N more files"; the pane below
lists them all.
**A real limitation surfaced by a broken fixture.** Two browser tests failed with zero enclosures in
the detail pane. Not the new scanner — verified by running it against the fixture files directly,
which was right every time. The cause: my fixtures pointed two feeds at the *same* enclosure URL,
and `enclosures.url` is UNIQUE across the whole database, so whichever feed is scanned first claims
it and the other's entry gets nothing. That is the dedupe working as designed, but it means **two
feeds legitimately sharing a media URL will only ever show it under one of them** — worth knowing,
and worth revisiting if a network feed and a show feed ever overlap.
---
## 2026-09-10 — A file is not the same as a playable file
Reported: "Abort Retry Fail still shows downloaded and audio playback UI". The media-type filter
stopped *new* image enclosures being fetched, but those 20 JPEGs were already on disk from before
it existed — and the UI gave a play button and an `<audio>` element to anything with a `path`. An
audio element pointed at a JPEG is just a broken player.
`isPlayable()` now separates having a file from being playable: audio/* or video/*, falling back to
the file extension when a feed declares no type. Four places used the wrong test — the row's play
button, the artwork's play overlay, `encBox`, and `play()` itself, which picked the first downloaded
enclosure regardless of what it was. A downloaded non-media file now shows as
`image · downloaded · Save · Delete file`, so it is still there and still retrievable, just not
pretending to be an episode.
Covered by a new browser test with a fixture blog whose entry carries a JPEG enclosure and a feed
configured to download it: no play button on the row, no `<audio>` in the detail pane, and the Save
button still present.
---
## 2026-09-10 — Three-pane layout
Ray asked for feeds beside, items above, and the selected item's text with its enclosures below —
the layout iPodderX used. Went looking for a screenshot to work from: the archived ipodderx.com kept
only marketing panels and 130px feature icons, no full window, and the review screenshots lived on
MacMerc which the archive query did not surface. Said so rather than pretending to have reference
art; the description is a precise spec on its own.
`#split` is a grid of list / divider / detail. Selecting a row no longer expands it inline — it
highlights and fills the pane below with the title, metadata, read/keep buttons, the sanitized notes
and one block per enclosure (a player when the file is here, otherwise what it is and a Download
button). The divider drags, and the split is remembered in localStorage. The OPML group page keeps
the old scrolling layout via a `plain` class.
**Process note, because it cost a dozen calls.** Four anchor-patches in a row failed on text I had
guessed rather than read: `return el` not `return div`, `open:new Set()` mid-line rather than
starting one. Each assert aborted before writing, so the file kept reverting to a half-applied
state, and one earlier patch did land while broken and left a syntax error. What worked was
reverting to the committed copy and then reading each block verbatim before touching it, with regex
for the whitespace-sensitive parts.
The suite paid for itself again: the new pane test failed on `#detail audio` because
`max_new_per_check = 1` plus newest-first means the daemon fetches *Second* Episode, so First has no
player. My assumption, not a bug — the same mistake as the earlier `S1E1` one. The test now finds
the downloaded row by its chip instead of assuming which episode it is. 9 browser tests pass.
---
## 2026-09-10 — Image enclosures were being treated as episodes
Reported as "Abort Retry Fail shows downloads but there are none". It had 20 enclosures, all
`image/jpeg`: Substack puts each article's header image in the RSS `<enclosure>` tag, so ipx
downloaded 20 JPEGs and counted them as episodes. Across the OPML subscription: 149 images, 74 MB,
11 feeds.
`[general] media_types` defaults to `["audio", "video"]`, with a per-feed override. An enclosure
whose top-level type is not wanted is recorded and shown but not auto-downloaded.
Clarified mid-change: those enclosures should still be *visible*, just not fetched automatically.
So nothing is hidden — the row names what it is ("image", "pdf", "torrent") instead of a bare
"skipped", and the download button still works if you want that file.
Two judgement calls in `wanted_media`: an **unknown or absent** type is allowed, because the real
type is only known after downloading and refusing everything untyped would drop feeds that simply
omit the attribute; and a **torrent** is allowed, being a container rather than media, judged once
unpacked.
The enclosure-less case Ray also described was already correct — every download affordance in the
row is gated on the enclosure existing. Checked before changing anything.
---
## 2026-09-10 — Daemon I/O log tab, and Playwright
**Log tabs.** All / Daemon I/O / Scans / HTTP. "Daemon I/O" is the control protocol itself: every
command arriving (`-> {"cmd":...}`) and every event leaving (`<- {"ev":...}`), logged under
`ipx::io` at the one point every command funnels through, so it catches socket clients, the CLI
proxying and the web UI alike. Rendered in that tab as in/out rather than level+target.
The two logging destinations now have **separate filters**: stderr follows `IPX_LOG` (default
`ipx=info`), the in-app buffer follows `IPX_UI_LOG` (default `ipx=debug`) and holds 5000 lines. The
UI can therefore show protocol traffic and routine skips that would be noise on a terminal.
**Playwright.** `npm install` plus `npx playwright install --with-deps chromium`, added to
`/src/install.sh`. `playwright.config.js` starts a fixture feed server and a daemon on a scratch
config, and eight tests drive a real browser. Each maps to a bug that actually reached Ray.
Two things the harness taught, both worth keeping:
- **Playwright starts `webServer` before `globalSetup`.** Writing the scratch config in globalSetup
meant the daemon launched with no config, fell back to defaults, and tried to seize the live
daemon's socket. The config is written at config-load time instead.
- Ports 8097/8771 looked free but `ss` showed 8097 held by a process outside this container (shared
host network). Moved to 8791/8792.
**The suite found a real bug on its first green-ish run:** OPML folders rendered *expanded* by
default. The code stored the set of closed groups, so a folder never toggled — every folder in a
fresh browser — counted as open, the exact opposite of the comment above it. It stores the open ones
now. Two other failures were my tests' fault, not the code's: asserting `S1E1` on `.ep.first()` when
newest-first put a different episode there, and an unscoped `getByText` matching both the sidebar
entry and the page heading. The `SE` in that first failure was the artwork placeholder's initials,
which I nearly misread as a broken chip.
---
## 2026-09-10 — Regression: derived feeds looked "unsubscribed" to half the code
Reported as `error: enclosure 235 belongs to unsubscribed feed "abort-retry-fail"`. Moving OPML
feeds out of config.toml meant a config-only lookup no longer finds them, and `download_one` still
did exactly that — so Download on any episode from an OPML subscription failed outright.
Worse than the one bug: **my first grep for the pattern gave false confidence.** `grep -n
'cfg\.feeds\.get'` returned three hits, all legitimate, so it looked clean — but the failing call
was split across lines, `cfg` then `.feeds` then `.get`, and never matched. Re-searching with
newlines collapsed found sixteen `.feeds` accesses, several of them wrong:
- `download_one` — the reported failure.
- `Status` and the daemon's startup line counted config feeds only: 3 reported where there are 85.
- `add` deduped and slugged against config only, so adding a URL an OPML already lists would have
duplicated it, and a new feed could collide with a derived feed's id.
- `rm` on a derived feed said "no feed with id".
- The web add endpoint had the same duplicate hole.
All now go through `subscriptions()`. Verified against the exact failure: enclosure 235 went
`pending` -> `done`.
Lesson recorded because it will recur: when a lookup moves, a single-line grep is not a survey.
Collapse newlines before searching Rust method chains.
---
## 2026-09-10 — OPML feeds out of the config, a real cap, and daemon output in the log
Three reports in quick succession, all fair.
**"Config is a mess."** It was: 611 lines, 85 feeds, 82 of them machine-generated, drowning the three
Ray actually chose. Writing derived data into a hand-edited file was the wrong call. OPML children
now live in the `feeds` table (`managed = 1`, `group_id`), are re-derived every scan, and inherit the
subscription's settings — so there is nothing to store but a URL and a parent. Editing one promotes
it to a real config entry, so `config.toml` only ever holds decisions. A one-time migration moves
existing children out: **611 lines -> 38**, 85 feeds -> 3, with all 3487 entries and 216 files intact.
**"Never download more than 3."** `max_new_per_check` defaulted to *unlimited*, so subscribing to an
OPML of 82 feeds pulled every back catalogue it could reach — 216 files, 22 GB before it was caught.
There is now `[general] max_new_per_check = 3`, used whenever a feed does not set its own, so all 83
uncapped feeds were capped without touching a line of their config. `pending()` also orders by publish
date now: a cap of 3 meant "the three recorded first", not the three newest.
**"No daemon output in the log."** Scans and downloads travel as events to the socket, not through
tracing, so the log view showed only startup and HTTP lines. `Emitter::emit` mirrors them now — and
levelling matters at this scale: at 82 feeds, one INFO line per feed per tick for "not due yet"
flushed the 2000-line buffer of anything useful in minutes, so routine skips and progress are DEBUG,
real activity is INFO, failures WARN.
A splice while refactoring cut `reject` and `fetch_one` out of main.rs, and recovering them from git
over-copied three more. Both caught by the compiler, restored, and verified byte-identical against
`git show HEAD:src/main.rs` rather than eyeballed.
---
## 2026-09-10 — Subscribing to an OPML, not just importing one
Asked whether this version could do what iPodderX did: subscribe to an OPML and get a folder of the
feeds inside it. It could not — `ipx import` was one-shot from a file and nothing ever re-read it.
The original checked for a URL ending in `.opml` and re-fetched it on every scan
(`iPXClass.py:34-40`).
Now: a feed whose body sniffs as OPML is a subscription list. Sniffing the body rather than the URL
extension also catches an OPML served from a `.rss` or extension-less URL, which the original missed.
Every scan re-reads it; listed feeds become real config entries with `group = "<opml-id>"` inheriting
the parent's settings, land in one nested folder, and are scanned in the same run rather than waiting
an interval.
Removal policy, Ray's rule: **a downloaded file is never orphaned.** Gone from the OPML with nothing
downloaded -> unsubscribed and removed. Gone but with downloads -> kept and flagged `orphaned`, with
the UI saying why. Verified end to end on a local fixture: the feed with 1 download was kept and
flagged, the one with 0 was removed, the file untouched.
`folder_for` had to change: it sanitized the whole folder string, so `sub/Show` would have collapsed
into one directory. Each path segment is sanitized separately now, with a test that `../../etc/Show`
cannot climb out of the download directory.
Also: `Db::memory()` now runs `migrate()` like a real open. It did not, so a column added only in the
migration passed the tests while being absent in production — which is exactly backwards.
Live result: Ray's `lists.opml.org` subscription expanded into 82 child feeds, all grouped, 1431
entries, no feed errors, worker still responsive mid-scan.
---
## 2026-09-10 — Log view in the app, and Docker
**Log view.** A ring buffer (2000 lines) fed by a `tracing` layer, exposed at `/api/logs` with a
sequence cursor so the UI polls for "everything after N" without duplicates. Tailing a file was the
obvious approach and the wrong one: under Docker the logs go to stdout and there is no file. An
`access_log` middleware adds a line per HTTP request, so the daemon and the web side share one
stream — which is what Ray asked for. That middleware skips `/api/logs` itself, or the panel's
2-second poll would generate a line per poll forever, a log of nothing but its own requests.
UI: sidebar **Log** button, level and text filters, follow-tail toggle, copy button, colour-coded
levels.
**Restart safety.** Detached torrents introduced a `downloading` state, and a row left in it by a
restart would sit there forever — the pending queue skips it and nothing else revisits it.
`requeue_interrupted()` resets those at startup, leaving alone any row that already has a file.
**Docker.** Multi-stage build (deps cached separately from source), `debian:bookworm-slim` runtime,
114 MB. Entrypoint writes a starter config bound to `0.0.0.0` on first run, because a container's
loopback is unreachable from outside, and drops to `PUID:PGID` via gosu. Healthcheck is
`ipx status`, which goes through the control socket to the worker and so catches a *wedged* daemon,
not just a dead one.
Built and actually run, not just written: image builds, container starts, writes its config, serves
the UI (401 without a token, 200 with), `/api/logs` answers, and the process runs as
`uid=99(ipx) gid=100(users)` with files owned `99:100` and the token file at mode 600.
Worth noting for future testing here: this session's Docker socket belongs to the Unraid host, so
`-v /tmp/...:/config` binds a path on the *host*, not one visible from inside this container. The
mount looks empty from here while being perfectly correct — verify inside the container instead.
---
## 2026-09-10 — Torrents: they work, and one was freezing the whole daemon
Asked "is torrents working?". The honest answer had been "unverified since step 7" — so it got tested
properly.
**They work.** End to end against a real swarm: Debian 13.6.0 netinst, 791,674,880 bytes, completed,
`file` confirms a bootable ISO, row marked `done`. Metadata resolution, transfer, placement and
completion all good.
**But the daemon was wedged.** Ray's CT-log feed had 47 pending torrents and nothing was progressing.
The worker was blocked: a `status` command over the control socket got no reply in 15s. Torrents ran
*inline on the single command worker*, so one torrent stopped every feed scan, every HTTP download
and every status command behind it — for up to `stall_mins` (30 default) waiting on metadata, and
worse, `Torrents::fetch` blocks on seeding for up to `seed_time_mins` (60 default) **after**
finishing. One successful torrent could have frozen podcast fetching for an hour.
Torrents are now detached: `spawn_torrent` runs the job on its own task behind a 2-permit semaphore,
the row is marked `downloading` so a rescan does not queue it twice, and the worker moves straight
on. A one-shot CLI run still runs them inline, or the process would exit mid-download. Verified: the
worker now answers `status` in 0.0s.
A diagnostic misstep worth recording: `/api/settings` returning 200 was taken as proof the worker was
alive. It is not — the web server is a separate task, so HTTP stays responsive while the queue is
completely stuck. Probing the control socket is the real test.
**Those CT-log torrents are ~557 GB each, 47 of them.** The RSS advertises `length="0"`, so nothing
warns you; the size only appears once metadata resolves. `auto_download` on that feed is set to
false as a hold, since restarting would otherwise have begun a half-terabyte transfer immediately
(the scheduler's first tick fires at startup). Flip it back in the feed's Settings when wanted.
Also noted: an aborted torrent leaves librqbit's pre-created file placeholders behind in the
destination folder (zero-length). Not cleaned up yet.
---
## 2026-09-10 — Schedule pickers, and progress painting every row
**Pickers.** "Check feeds every" is a number input plus a unit dropdown (minutes/hours/days/weeks)
in both global and per-feed Settings, replacing free text. `parse_interval` gained weeks first — it
only knew m/h/d, so the backend would have rejected the new option. The per-feed dropdown's first
entry is "Use the default — every N", which disables the number and sends `null`; that rides on the
null-clearing path fixed earlier today, and would not have worked before it. `splitEvery` picks the
largest unit that divides evenly, so 120 reads "2 hours" rather than "120 min". The API now returns
`schedule_mins` so the page does not re-implement the parser.
**Bug: one download painted every pending row's progress bar.** Reported after adding the TWiT feed.
The `progress` event carried feed/url/file but no enclosure id, so the handler had nothing to target
and set the width on all of them — the code even carried a comment admitting it applied "to whatever
is downloading now". Adding a feed therefore looked like it was downloading the entire back
catalogue.
`Event::Progress`, `DownloadDone` and `DownloadError` now carry `enclosure`, threaded through both
the HTTP and torrent paths (`db::Pending` gained its row id to make that possible), and the handler
targets `.dlbar[data-bar="<id>"]` alone. A test asserts the id is on the wire. Idle bars are
transparent and only get a track while live, so a row that is not downloading shows nothing.
`tests/page-smoke.js` also drives each modal with live-shaped data now — load-time smoke never
reaches code that only runs when a dialog opens, which is exactly where these changes landed.
Verified live: weeks round-trips (`every 2w` -> 20160 min), both pickers render, clearing an
override returns to the global default. `cargo test` 37/37.
---
## 2026-09-10 — The whole UI was dead, and server-side tests could not see it
Reported as "my feeds seem to have disappeared", then "settings and dark/light mode don't do
anything either". One cause for all three.
`$('#prefs').onclick = prefsModal` referenced a function that did not exist. An uncaught
`ReferenceError` stops the entire script, and that line sits above the theme toggle, the feed
filter, the SSE connection and the `loadFeeds()` call that fills the sidebar — so everything below
it silently never ran.
**Cause: a patch anchored on something already deleted.** The scheduling UI was inserted with
`str.replace` anchored on `function toggleSettings(){`, which belonged to the *old* basic UI that
the full rewrite had already removed. Python's replace matches nothing and says nothing, and there
was no assert — so `prefsModal`, `everyText`, `due` and `globalEvery` were never added, while the
line *calling* `prefsModal` went in fine via a different anchor that did match.
**Why it got through.** Every check was server-side: curl for status codes, JSON shape, config
contents. All passed, because the server was fine. `node --check` also passed — it parses, and a
ReferenceError is a runtime failure. The page was never executed.
`tests/page-smoke.js` now runs the real page script against a stub DOM, fails on anything thrown,
and flags handlers wired to elements that do not exist. Confirmed non-vacuous by reintroducing the
exact bug: exit 1 pointing at the offending line, clean once restored. Run it with
`node tests/page-smoke.js`.
Wiring is also defensive now — `on(sel, ev, fn)` logs and skips rather than throwing, so one dead
reference cannot blank the app again.
---
## 2026-09-10 — Scheduling, and two bugs it uncovered
**Scheduling.** The original engine had none — it only skipped feeds on `<ttl>`; the schedule lived
in the Cocoa GUI that was never open-sourced, so there was no original behaviour to match. Scope
agreed with Ray: interval only, global default plus per-feed override, no quiet hours.
`general.schedule` takes "every 30m", "every 4h", "1d", or a bare number of minutes;
`feeds.<id>.schedule` overrides it. The legacy `interval_mins` is still read so existing configs
work, and saving from the UI migrates it. A malformed value in the file warns and falls back rather
than stopping the daemon; a malformed value over the API is a 400.
Precedence, which is a judgement call: an explicit per-feed schedule wins outright, including over
the publisher's `<ttl>`. With no per-feed schedule, `<ttl>` still raises the interval when the
publisher asks to be polled less often. Visible in practice — TWiT resolves to 720 min from its own
ttl against a 60 min global.
UI: a gear opens global Settings (schedule, quota, max age); each feed's Settings gains a schedule
field; the feed header shows "every 4 hours · next in 37m".
**Bug 1: `null` never cleared anything.** For `Option<Option<T>>`, serde maps JSON `null` onto the
*outer* `None`, which the handler reads as "field absent, leave alone" — so `Some(None)` was
unreachable and every clear was a no-op returning 204. That silently affected `folder`, `schedule`
and `max_new_per_check`; unsetting any of them from the UI did nothing and looked like it worked.
Fixed with a `double_option` deserializer, and pinned by a test covering absent / null / value.
**Bug 2: the daemon ignored SIGTERM while working.** `tokio::select!` races its branches only at
selection time; once inside `run(...).await` the shutdown future was not polled at all, so a signal
queued behind a 90 MB download. Under systemd every restart would have hit `TimeoutStopSec` and been
SIGKILLed mid-transfer.
This one cost real time: `pkill` appeared to work but didn't, a stale daemon kept port 8099, and
because the restart also `rm -f`'d the socket it defeated the "already listening" guard and a second
daemon started. Every PATCH "verified" after that went to the *old* binary — so bug 1 was reported
fixed while completely untested. Lesson: confirm the process actually died and the socket guard is
doing its job, rather than trusting the restart.
The stop signal now lives in a `watch` channel raced *inside* each job. Measured: SIGTERM mid-
download exits in 1s (was: blocked until the transfer finished). An abandoned download leaves a
partial in `.ipx-incomplete`, discarded on the next attempt.
`cargo test` 37/37.
---
## 2026-09-10 — App name, and a transparent icon
Title is `iPodderX` in the browser tab and the sidebar. The heading's `text-transform: uppercase`
had to go with it, or the camel case would have rendered as `IPODDERX`.
The icon's white background is gone. A blanket `-transparent white` would have punched holes
straight through the device, whose body is also white — so the background is flood-filled inward
from all four corners at 12% fuzz, which only clears the *connected* region and stops at the dark
outline. Then trimmed and scaled to 128px.
Checked by compositing onto the real `#0e131b` rather than trusting the alpha channel: no white
plate, and no pale fringe from the JPEG artifacts along the outline. The `.logo` rule lost its
`background:#fff` plate and switched to `object-fit:contain`.
Both files are tracked: `ipodderx-icon.jpg` is the untouched archival original, `ipodderx-icon.png`
the transparent version derived from it, so the provenance chain survives. The PNG is embedded as a
data URI for the sidebar mark and the favicon.
---
## 2026-09-10 — Colour scheme derived from the icon
Palette sampled from `web/ipodderx-icon.jpg` with ImageMagick rather than eyeballed. The icon gives
three families: the silver device ramp (`#1A1A1A` `#606060` `#929292` `#ADADAD` `#D6D6D6` `#F5F5F5`),
the screen blues (`#2D5391` `#5574A9` `#7595CA` `#92B2E6` `#B4CAED`), one amber from the EQ bars
(`#F49E2C`), and a blue-grey `#95A0B1`.
Dark theme: `--bg` is the screen navy taken right down, `--fg` the device highlight, `--dim` the
blue-grey verbatim, `--accent` the screen blue, `--accent2` the amber. Light theme flips to the
device body — white panels, `#D6D6D6` edges, `#1A1A1A` text — with the deeper `#2D5391` as accent,
since the pale screen blue vanishes on white. Text sitting on an accent fill moved from a hardcoded
`#0b0e14` to a `--ink` token.
Two calls: the amber doubles as the pending state, being the only warm note in the icon and exactly
where the eye should go. And since the icon has no green or red, the downloaded/error colours are
tuned to the palette's saturation and temperature rather than invented from scratch.
**Contrast was checked, not assumed, and it failed first time.** `--faint` — which carries dates,
durations and file sizes at small size — came out at 3.92 dark and 3.11 light, both under AA. Moved
along the icon's own grey ramp to `#7a8799` and `#767676`. Every pair in both themes now clears
WCAG AA; the lowest anywhere is 4.54.
---
## 2026-09-10 — The real iPodderX icon
The sidebar mark was an invented `ix` placeholder. The original icon is not in this repo or the
legacy one: only the Python engine was open-sourced, never the Cocoa bundle whose Resources held the
`.icns` (the legacy repo's history contains nothing but .py/.so/.po/.mo/LICENSE/README).
Recovered from the Internet Archive's capture of ipodderx.com: `_graphics/iPXicon.jpg`, 173x165 --
the white iPod-style device with the antenna and EQ display. That is the largest surviving copy; the
site's 2005 `favicon.ico` is far too small to use. Saved as `web/ipodderx-icon.jpg` so the file lives
in the tree, and embedded as a base64 data URI (12.7 KB) for both the sidebar mark and the favicon,
which avoids an extra asset route.
`.logo` changed from a gradient chip to a white rounded tile, since the icon is photo-style on a
light ground.
Provenance is Ray's own: the app was Slakinski & Trometer / Thunderstone Media, and this is its
successor.
---
## 2026-09-10 — Feed URL is editable, with a copy button
The URL in a feed's Settings is now an editable field paired with Copy. Entries and download
history are keyed by feed id, not URL, so changing it keeps everything — which is the point, since
a Patreon feed URL carries an auth token that gets rotated.
Two things that would otherwise have bitten:
- **`navigator.clipboard` does not exist here.** It requires a secure context, and this is served
over plain HTTP on a LAN address, so the button would have silently done nothing. Falls back to a
hidden textarea plus `execCommand('copy')`, and reports Copied/Failed either way.
- **Changing the URL clears the stored ETag/Last-Modified.** Those validators belong to the old URL;
carrying them over could yield a bogus 304 against the new one and make a working feed look empty.
`check_url` is a tested pure function: rejects empty, unparseable and non-http(s) URLs (so
`file:///etc/passwd` is refused), rejects a URL another feed already uses, and allows a feed to keep
its own URL unchanged. Validation failures and unknown feed ids now return **400**, not 500 — bad
input is the caller's mistake, and only genuine server faults are logged as errors.
Verified live: the three rejection cases return 400 with readable messages, a no-op save returns 204,
and the feed still reports 131 entries / 11 downloaded. `cargo test` 34/34.
---
## 2026-09-10 — Fixed: pressing play made an episode vanish from the list
Reported as "where did Session Zero go, and why do parts share a number?".
Two separate things, one of them a real bug.
**The bug.** `play()` marked an episode read the moment playback started. The default view is the
Unread tab, so pressing play removed the episode from the list you were looking at — indistinguishable
from it going missing. Confirmed from the data: all four read entries had `position = 0`, meaning
playback never passed the 10-second save threshold. They had been started, never listened to.
Marking read now happens when an episode is actually consumed — on `ended`, or once playback passes
90% — never on play. Two earlier PROGRESS entries described mark-on-play as a deliberate feature;
it was a design error, and this supersedes them.
Also restored the four affected rows to unread, since their read flag was purely an artifact of the
bug (`UPDATE entries SET read=0 WHERE read=1 AND position=0` — no genuinely-played episode could
match, since playing one writes a position).
**Not a bug: repeated episode numbers.** Each story part ships as two items — the main episode and a
shorter `Junk in the Trunk` companion — with distinct GUIDs, durations and files, and the publisher
gives both the same `itunes:episode`. Zero duplicated titles across all 131 entries. `Session Zero`
additionally carries a season with no episode number and drops the "Part N" naming, so it does not
match the pattern its siblings follow.
Verified: the arc shows 11/11 items under the default Unread view again.
---
## 2026-09-10 — Titles: RSS `<title>` is the only source
Confirmed against the live feed rather than assumed: 131 feed items, 131 stored, **0 mismatches** —
entry titles are byte-for-byte what the RSS `<title>` publishes, separators included. `feed.rs` never
consults `itunes:title`, and season/episode live in their own columns, rendered as chips rather than
folded into the title.
Pinned with a test: a fixture whose `itunes:title` differs from its `<title>` must still yield the
RSS title, and an item with `itunes:season` but no `itunes:episode` keeps a null episode instead of
inventing one.
That null-episode case is real in the wild — this feed's `Music from a Darkened Room | Session Zero`
carries S8 with no episode number, while Parts 1-5 get E1-E5, and it drops the "Part N" convention
its siblings use. The arc is 11 items: Session Zero plus five parts, each part being a main episode
and its shorter `Junk in the Trunk` companion.
---
## 2026-09-10 — Phase 3: full-featured UI
Rewrite of `web/index.html` (~714 lines) plus the backend it needed.
**New stored metadata**, all of which the real feed carries on all 131 episodes and none of which
was being captured: feed and per-episode artwork (`itunes:image`), duration (`itunes:duration`,
parsed from either raw seconds or a `1:34:09` clock), season/episode numbers, and a playback
`position` so episodes resume.
**Schema migration.** `Db::open` now runs `migrate()`: `PRAGMA table_info` then `ALTER TABLE ADD
COLUMN` for anything missing, since `CREATE TABLE IF NOT EXISTS` does nothing to an existing table.
Tested on a *copy* of the live database first. Accidentally SIGPIPE'd it partway through (a `head -4`
on the output) which usefully proved it is idempotent and self-heals: the next run added the
remaining columns, 131 entries and 6 downloads intact.
**New endpoints:** entry filters (all/unread/downloaded/flagged), case-insensitive search over title
and notes, pagination totals, `POST .../position`, `read-all`, `download-latest`, and OPML
import/export over HTTP.
**UI:** persistent player bar surviving navigation (play/pause, ±15s/30s, 0.8-2.5x speed, volume,
scrubber, all remembered in localStorage), resume playback via `sendBeacon` every 10s, MediaSession
for lock-screen controls, keyboard shortcuts (space, arrows, `/`, Esc), artwork everywhere with
generated-initials fallback, filter tabs, episode search, load-more pagination, live progress bars
off SSE, real modals, toasts, light/dark toggle, and a mobile layout with a collapsing sidebar.
**Bug found and fixed in review: filtering was broken while searching worked.** `count_entries`
dropped the search clause when no term was given but still bound `?2`; rusqlite rejects a parameter
the statement never mentions -- *"Wrong number of parameters passed to query. Got 2, needed 1"*. So
`?q=...` worked and a plain Unread/Downloaded/Flagged filter 500'd. The clause is now always present
with `?2 = ''` short-circuiting it. Regression test runs all four filters with and without a search
term and asserts the page and the count agree.
**A non-bug worth recording:** artwork looked broken on the live DB (0/131) while a fresh DB filled
all 131. The parser was fine -- the live DB simply had not been rescanned yet, since backfill happens
through `record_entry`'s update path. After a forced scan: 131/131 image, duration and season.
Verified live: all four filters return sane totals (131/129/10/0), search works (`session zero` -> 2,
`trunk` -> 10), `/media/1` with a Range header returns 206, index serves 34 KB. `cargo test` 32/32.
---
## 2026-09-10 — Fixed: the UI's Download button downloaded the wrong episodes
Reported from the running instance: clicking Download on *Music from a Darkened Room | Session Zero*
showed progress, then left the episode at `pending`.
**Cause, and it was a design error in step 11, not a glitch.** `POST /api/enclosures/{id}/download`
requeued the row to `pending` and asked for an ordinary scan, justified at the time as "needed no
new machinery — the queue is the table". But a scan means *take the lowest-id pending rows, up to
`max_new_per_check`*. Against a 125-item backlog with a cap of 2, that is ids 7 and 8; the requested
row 11 was never a candidate. The progress on screen was two other episodes downloading past it.
The queue can express what is outstanding but not what was *asked for*. `Command::Download {
enclosure }` is now its own command: it fetches that one row immediately, ignoring both queue order
and the per-scan cap, still through the single worker so it cannot overlap a scan. Torrent
enclosures route to `torrent_one` the same way a scan would.
Verified live: enclosure 11 went `pending` -> `done`, 90.3 MB on disk, and nothing else was pulled
in its place. `cargo test` 30/30, with the new command's wire form pinned.
Worth remembering: **the daemon scans immediately on its first tick**, so every restart costs
`max_new_per_check` episodes on the normal path.
---
## 2026-09-10 — Phase 2: web front end (steps 9-13)
axum in the daemon process, plain HTML/JS in `web/index.html` (embedded with `include_str!`), no
WASM toolchain. One binary still.
**Config hot-reload.** `Ctx.cfg` is now `RwLock<Arc<Config>>`; `ctx.cfg()` hands out a snapshot, so
no guard is ever held across an await. The UI rewrites config.toml and calls `reload_cfg`, and a
running daemon picks the change up on its next scan. The CLI mutation commands (`add`/`rm`/`import`)
now clone a snapshot, edit, and save.
**Auth.** `[web] enabled/bind/token`; the token is minted from `/dev/urandom` on first run, written
back to config.toml, and the URL printed. `?token=` sets a year-long cookie — it has to be a cookie
because an `<audio>` element cannot send a header. Comparison is constant-time. An empty token makes
the server refuse to serve rather than serve open.
**Endpoints.** `/api/feeds` (GET/POST), `/api/feeds/{id}` (PATCH/DELETE),
`/api/feeds/{id}/entries`, `/api/entries/{feed}/{guid}/flags`, `/api/enclosures/{id}/download`,
`/api/enclosures/{id}` (DELETE), `/api/fetch`, `/api/events` (SSE off the existing broadcast bus),
`/media/{id}` (tower-http `ServeFile`, so Range works).
**`read`/`flagged` finally have a writer** — the UI sets them, and playing an episode marks it read.
Retention has ordered by these since step 5 with nothing to set them.
**Download-on-demand needed no new machinery**: requeue the row to `pending` and kick a scan, since
the queue is the table. That also un-skips an enclosure a filter rejected under older settings.
Verified against the live Glass Cannon feed, daemon on 127.0.0.1:8749:
- auth: no token 401, wrong token 401, right token 200 + cookie, API then works on the cookie alone;
`/media/1` unauthenticated is 401.
- browsing: 131 entries page with enclosures attached.
- **XSS**: injected `<script>alert(1)</script><img src=x onerror=alert(2)>` into a stored
description; it reaches the page as `<p>hi</p><img src="x">` — script tag and handler both gone.
- media: 200 with `accept-ranges: bytes`; `Range: bytes=1000000-1000999` returns 206 with the right
`content-range`, so seeking works.
- config: PATCH -> 204, written to config.toml, and the running daemon reports the new values with
no restart.
- SSE: 105 events for one download (101 progress), then `download_done`/`feed_done`/`scan_done`.
- flags: 204, unread count dropped 131 -> 130. Delete: file gone, row `reaped`/`path=NULL`.
`cargo test` 30/30.
**Caveats.** It is plain HTTP — on a LAN bind the token crosses the network in the clear, and a feed
URL may itself carry a credential (Patreon's does), which the `/api/feeds` response includes. A
reverse proxy with TLS is the answer if that matters. There is no per-user anything; the token is
all-or-nothing access.
---
## 2026-09-10 — Tested against a real feed (Patreon / Glass Cannon)
First run against a live subscriber feed: 131 items, 6.17 GB total, all `audio/mpeg`, no `<ttl>`.
Parsed clean, titled `Get in the Trunk | Anthology Series | Delta Green`, downloaded a valid MP3
(`file` confirms ID3v2.3, MPEG layer III). A second scan pulled the *next* episode, confirming
`max_new_per_check` defers rather than drops. Over the socket: 101 progress events for a 79 MB
episode -- whole-percent throttling behaving exactly as intended -- then `download_done`,
`feed_done`, `scan_done`.
**Fixed: stripped separators left doubled spaces.** The feed title separates words with `|`, a
forbidden filename character, so the folder came out `Get in the Trunk Anthology Series Delta
Green`. The Python had the same wart.
Forbidden characters are now split in two. Separators (`/ \\ | :`) become `-`; the rest
(`? * < > " '`) are simply dropped. A run of dashes and spaces then collapses to `" - "` when the
run contained whitespace and to a bare `-` when it did not, so:
| input | output |
|---|---|
| `Get in the Trunk \| Anthology Series \| Delta Green` | `Get in the Trunk - Anthology Series - Delta Green` |
| `Ep 12: The One` | `Ep 12 - The One` |
| `AC/DC` | `AC-DC` |
| `well-known.mp3` | `well-known.mp3` |
| `../../etc/passwd` | `etc-passwd` |
Leading dashes and dots are trimmed too -- a filename starting with `-` trips up CLI tools.
**Worth knowing, not a bug:**
- The channel-level `itunes:explicit` is `true`, so with the default `allow_explicit = false` all
131 episodes are skipped. Because filter verdicts are recorded once at discovery, flipping the
setting afterwards does not re-evaluate enclosures already marked `skipped` -- they would need a
`UPDATE enclosures SET state='pending'`. Worth a `ipx retry <feed>` command if this bites.
- Patreon sends neither `etag` nor `last-modified` on a GET (a `last-modified` does appear on HEAD,
which is what made it look briefly like a storage bug). So this feed can never 304: every poll
re-fetches ~160 KB and re-parses 131 entries, and `interval_mins` is the only thing limiting the
rate. Cheap, but it means conditional GET buys nothing here.
---
## 2026-09-09 — Step 8: OPML and polish ## 2026-09-09 — Step 8: OPML and polish
`ipx add <url>` fetches the feed to name it from its own title (`Accidental Tech Podcast` -> `ipx add <url>` fetches the feed to name it from its own title (`Accidental Tech Podcast` ->

127
README.md
View File

@@ -58,6 +58,10 @@ interval_mins = 60 # default poll; a feed's own <ttl> wins when longer
organize = "feed" # "feed" | "date" organize = "feed" # "feed" | "date"
max_total_gb = 50 # 0 = unlimited max_total_gb = 50 # 0 = unlimited
max_age_days = 30 # 0 = keep forever max_age_days = 30 # 0 = keep forever
max_new_per_check = 3 # per feed, per scan. 0 = unlimited (pulls whole back catalogues)
media_types = ["audio", "video"] # what downloads automatically. Anything else is still
# listed and can be fetched by hand -- blog feeds put article
# images in enclosures. Empty takes everything.
[torrent] [torrent]
enabled = true enabled = true
@@ -101,6 +105,121 @@ Events: `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `downl
Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy
daemon also sees that daemon's other work. daemon also sees that daemon's other work.
## OPML
Two different things, both supported:
**Importing and exporting** a file copies subscriptions in or out once — `ipx import subs.opml`,
`ipx export subs.opml`, or the OPML button in the UI.
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
other feed; every scan re-reads it and keeps your feed list in step. An OPML is recognised by its
content, so a URL without a `.opml` extension still works.
The feeds inside it are **not written to `config.toml`** — the OPML is the source of truth, so they
are re-derived each scan and held in the database. Your config keeps only what you chose. They show
as a collapsible folder in the sidebar, download into one nested folder, and newly listed ones are
scanned in the same run rather than waiting for the next interval. They inherit the subscription's
settings; change anything on one and it gets its own config entry from then on.
When a feed drops out of the OPML upstream:
| it has downloads | what happens |
|---|---|
| no | unsubscribed and removed from the config |
| yes | kept, flagged in the UI as no longer listed |
A downloaded file is never left behind with nothing explaining where it came from.
## Web UI
```toml
[web]
enabled = true
bind = "0.0.0.0:8080" # 127.0.0.1:8080 by default
token = "" # generated and written back on first run
```
`ipx daemon` then serves it in the same process (`ipx daemon --web ADDR` overrides the bind for one
run). On first start it mints a token, saves it to config.toml, and prints the URL to open:
```
web ui token generated. Open:
http://0.0.0.0:8080/?token=1f4c…
```
`?token=` sets a year-long cookie, so you only paste it once per browser. Everything is behind that
token, including `/media/...` — a cookie rather than a header precisely because an `<audio>` element
cannot send headers.
Browse feeds, read show notes, play episodes in the browser (Range requests are served, so seeking
works), download or delete individual files, mark episodes read or flag them to keep, and edit a
feed's folder/keywords/explicit/auto-download/limit settings. Config edits are written to
config.toml and hot-reloaded — no daemon restart.
Show notes are feed-supplied HTML from an untrusted source; they are sanitized with `ammonia`
server-side before they reach the page.
**It is plain HTTP.** On a LAN bind, the token and everything else crosses the network in the
clear — and a feed URL can itself contain a credential (Patreon's, for one, carries an auth token).
Put it behind a reverse proxy with TLS if that matters to you.
## Layout
Feeds down the side; the selected feed's episodes across the top; the selected episode's text and
its enclosures in the pane below, which is where you play, download or delete them. The divider
between the two panes drags and the position is remembered. An OPML subscription's page instead
lists the feeds inside it.
An item may carry several enclosures. All of them appear in the pane below; the row summarises the
one you would act on and notes how many others there are. Anything that is not audio or video gets a
View link — opening the publisher's copy, or the local one once downloaded — rather than a player.
## Log view
The **Log** button in the sidebar shows the running daemon's output live, in four tabs:
**Daemon I/O** is the control protocol itself — every command in and every event out, as JSON;
**Scans** is feed and download activity; **HTTP** is web requests; **All** is everything: feed scans, downloads,
torrent activity and every HTTP request, with level and text filters and a copy button. It reads a
2000-line ring buffer held inside the process (`/api/logs`), not a file — so it works the same under
Docker, where logs go to stdout and there is no file to tail. The buffer keeps `debug` even when the terminal does not, so protocol traffic and routine
skips are there without making stderr unreadable — `IPX_UI_LOG` changes what it captures and
`IPX_LOG` what reaches stderr.
## Tests
```sh
cargo test # the server: parsing, filters, retention, schedules, SQL
node tests/page-smoke.js # the page script loads without throwing
npx playwright test # a real browser against a real daemon
```
The Rust tests cover the server and the smoke test catches a script that fails to load, but neither
can see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which
is what has actually slipped through. The Playwright suite drives a headless browser against a
daemon started on fixture feeds, and each test maps to a bug that reached a user: the page rendering
empty, a dead theme toggle, settings not persisting, episode metadata, filter tabs, the feed URL
field and its copy button, the log tabs, and OPML folders.
`npm install` gets the test runner; the browser itself comes from
`npx playwright install --with-deps chromium` (in `install.sh`).
## Docker
```sh
docker compose up -d # builds the image and starts it
docker compose logs -f ipx # the token is printed on first start
```
`docker-compose.yml` mounts `./config`, `./data` and a downloads directory, publishes 8099 for the
UI and 6881 (TCP **and** UDP — DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
land owned the way Unraid shares expect. On first start the entrypoint writes a config bound to
`0.0.0.0`, since a container's loopback is not reachable from outside it, and prints the URL with
its generated token.
The healthcheck runs `ipx status`, which goes through the control socket to the command worker — so
it catches a daemon that is alive but wedged, not merely one that has died.
## Running it as a service ## Running it as a service
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would `contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
@@ -109,3 +228,11 @@ rather run periodic scans with no daemon (in which case there is no socket for a
## License ## License
MIT. See [LICENSE](LICENSE). MIT. See [LICENSE](LICENSE).
## Signing in
A new database starts with **admin / ipodderx** at `/login`; change it with
`echo -n <password> | ipx user passwd admin`. Accounts are managed with `ipx user add|list|passwd|rm`.
To let Cloudflare Zero Trust or Authentik do the authenticating instead, see
[docs/sso.md](docs/sso.md).

24
docker-compose.yml Normal file
View File

@@ -0,0 +1,24 @@
services:
ipodderx:
image: 192.168.1.130:5000/ipodderx:latest
container_name: iPodderX
restart: unless-stopped
environment:
PUID: "99"
PGID: "100"
TZ: "America/Toronto"
IPX_LOG: "ipx=info"
ports:
- "8099:8099" # web UI
- "6881:6881/tcp" # BitTorrent peers
- "6881:6881/udp" # DHT
volumes:
- /mnt/fast/appdata/ipodderx:/config # config.toml, and the web token
- /mnt/user/ipodderx/:/data # state.db
- /mnt/user/ipodderx/downloads:/downloads
healthcheck:
test: ["CMD", "ipx", "status"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s

40
docker-entrypoint.sh Executable file
View File

@@ -0,0 +1,40 @@
#!/bin/sh
set -e
# A container's loopback is not reachable from outside it, so the default bind of
# 127.0.0.1 would leave the UI unreachable. Write a starter config that binds 0.0.0.0
# on first run; after that the file is yours and is never rewritten.
if [ ! -f "$IPX_CONFIG" ]; then
mkdir -p "$(dirname "$IPX_CONFIG")"
cat > "$IPX_CONFIG" <<TOML
[general]
download_dir = "/downloads"
schedule = "every 60m"
max_total_gb = 0
max_age_days = 0
[torrent]
enabled = true
port_range = "6881-6889"
[web]
enabled = true
bind = "0.0.0.0:8099"
token = ""
TOML
echo "ipx: wrote a starter config to $IPX_CONFIG"
fi
mkdir -p "$IPX_DATA_DIR" /downloads
# Unraid shares expect 99:100. Running as root would leave root-owned downloads.
if [ "$(id -u)" = "0" ] && [ -n "$PUID" ] && [ -n "$PGID" ]; then
if ! getent group ipx >/dev/null 2>&1; then addgroup --gid "$PGID" ipx 2>/dev/null || true; fi
if ! getent passwd ipx >/dev/null 2>&1; then
adduser --uid "$PUID" --gid "$PGID" --disabled-password --gecos "" ipx 2>/dev/null || true
fi
chown -R "$PUID:$PGID" "$IPX_DATA_DIR" "$(dirname "$IPX_CONFIG")" 2>/dev/null || true
exec gosu "$PUID:$PGID" "$@"
fi
exec "$@"

212
docs/sso.md Normal file
View File

@@ -0,0 +1,212 @@
# Signing in through Cloudflare Zero Trust or Authentik
ipx can take the signed-in identity from whatever sits in front of it, instead of asking for a
password itself. Both products below do the same thing in the end: they authenticate the person and
pass the result to the origin in a **header**. ipx reads that header, finds (or creates) the
matching account, and gets on with it.
Read [How this is secured](#how-this-is-secured) before exposing anything. The short version: a
header is worth exactly as much as the hop that set it, so ipx only believes one from an address you
list.
---
## The ipx side (both setups)
```toml
[web]
enabled = true
bind = "0.0.0.0:8099"
token = "…" # keep it: it is the admin, used by the healthcheck
# The header your proxy sets. Empty (the default) disables this whole path.
trusted_header = "Cf-Access-Authenticated-User-Email" # Authentik: "X-authentik-username"
# Addresses allowed to assert that header -- the proxy, and nothing else.
trusted_proxies = ["127.0.0.1", "::1"]
# Create an account the first time the proxy vouches for a name ipx has not seen.
auto_create_users = true
session_days = 30
```
Restart the daemon after editing. Accounts made this way have **no password**: they can only ever
arrive through the proxy. `ipx user list` marks them `proxy only`.
The first account created is an admin. Every later one is an ordinary user, and an ordinary user
cannot change global settings or how often feeds are scanned. Promote someone with:
```sh
ipx user list
echo -n 'a good password' | ipx user passwd <name> # optional: also lets them sign in directly
```
Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN
when the tunnel is down. A brand new database starts with **admin / ipodderx** — change it.
---
## Cloudflare Zero Trust
This is what runs `ipodderx.sdf1.net`: a `cloudflared` tunnel to the origin, with an Access
application in front of it. Cloudflare authenticates the visitor and adds
`Cf-Access-Authenticated-User-Email` to every request it forwards.
### 1. The tunnel
In **Zero Trust → Networks → Tunnels**, either use the existing tunnel or create one, then add a
public hostname:
| Field | Value |
|---|---|
| Subdomain / domain | `ipodderx` / `sdf1.net` |
| Type | HTTP |
| URL | `localhost:8099` (or the LAN address of the box) |
Use `localhost` when `cloudflared` runs on the same machine as ipx — that keeps the origin request
coming from `127.0.0.1`, which is already in `trusted_proxies`. If `cloudflared` runs elsewhere (its
own container, another host), put **its** address in `trusted_proxies` instead, and make sure
nothing else can reach port 8099.
### 2. The Access application
**Zero Trust → Access → Applications → Add an application → Self-hosted**:
- Application domain: `ipodderx.sdf1.net`
- Session duration: whatever suits; ipx keeps its own 30-day session on top.
- Add a policy — *Allow*, with a rule such as `Emails` → your address, or `Emails ending in` →
your domain. Anyone this policy admits gets an ipx account when `auto_create_users` is on, so keep
the policy as narrow as the people you actually want reading your feeds.
### 3. Point ipx at the header
```toml
trusted_header = "Cf-Access-Authenticated-User-Email"
trusted_proxies = ["127.0.0.1", "::1"]
```
The username becomes the email address, lower-cased (`ray@example.com`). That is what shows in the
sidebar and what `ipx user list` prints.
### 4. Check it
```sh
# From the box itself: no header, no session -> the sign-in page.
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' -H 'Accept: text/html' http://127.0.0.1:8099/
# Pretending to be the tunnel (only works because 127.0.0.1 is trusted):
curl -s -H 'Cf-Access-Authenticated-User-Email: you@example.com' http://127.0.0.1:8099/api/me
```
Then load `https://ipodderx.sdf1.net` in a browser: Cloudflare should ask who you are, and ipx
should show your address in the sidebar footer without ever asking for a password.
---
## Authentik
Authentik does this with a **Proxy Provider** plus an **outpost**, which sits in the request path and
adds `X-authentik-username` (also `X-authentik-email`, `X-authentik-name`, `X-authentik-groups`).
### 1. Provider
**Applications → Providers → Create → Proxy Provider**:
- Name: `ipx`
- Authorization flow: your usual (`default-provider-authorization-implicit-consent`)
- Mode: **Forward auth (single application)** if an existing reverse proxy fronts ipx, or
**Proxy** to let the outpost talk to ipx directly.
- External host: `https://ipodderx.example.net`
- Internal host (Proxy mode): `http://<ip of the ipx box>:8099`
### 2. Application and outpost
**Applications → Create**, bind it to that provider, and give it a policy so only the people you
mean are let through. Then add the provider to an outpost (**Applications → Outposts**, the embedded
one is fine).
### 3. Forward auth, if you use nginx/SWAG in front
In the server block for ipx:
```nginx
location /outpost.goauthentik.io {
proxy_pass http://authentik-server:9000/outpost.goauthentik.io;
proxy_set_header Host $host;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
add_header Set-Cookie $auth_cookie;
auth_request_set $auth_cookie $upstream_http_set_cookie;
}
location / {
auth_request /outpost.goauthentik.io/auth/nginx;
error_page 401 = @goauthentik_proxy_signin;
auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;
# This is the line that matters to ipx.
auth_request_set $authentik_username $upstream_http_x_authentik_username;
proxy_set_header X-authentik-username $authentik_username;
proxy_pass http://ipx:8099;
}
```
### 4. Point ipx at the header
```toml
trusted_header = "X-authentik-username"
trusted_proxies = ["172.18.0.5"] # the outpost or nginx container, NOT a whole subnet
```
Usernames arrive as Authentik knows them (`ray`), lower-cased.
---
## How this is secured
**The header is only believed from `trusted_proxies`.** Every other source is ignored, and the
request falls through to a session cookie or the shared token. This is the whole security boundary,
so:
- List the **proxy's own address**, not a range. `["127.0.0.1"]` when the tunnel runs beside ipx;
the container's IP when it does not.
- Never list a LAN subnet. Anyone on your network could then send
`Cf-Access-Authenticated-User-Email: admin@…` and be your admin.
- Make sure the origin port is not reachable *around* the proxy by anyone you would not admit
through it. If it is, bind ipx to `127.0.0.1` and let only the proxy reach it.
Verify the refusal, don't assume it — set `trusted_proxies = ["10.9.9.9"]` briefly and confirm a
header from your machine gets a `401`:
```sh
curl -s -o /dev/null -w '%{http_code}\n' \
-H 'Cf-Access-Authenticated-User-Email: someone@example.com' http://127.0.0.1:8099/api/me
```
**What ipx does not do:** it does not verify Cloudflare's `Cf-Access-Jwt-Assertion` signature or
Authentik's session. It trusts the hop. That is a deliberate trade — it keeps the configuration to
three lines — and it is sound exactly as long as the point above holds.
**Turning it off:** clear `trusted_header`. Existing proxy-only accounts stay, but nobody can sign
in with them until they are given a password (`ipx user passwd <name>`).
---
## Everyday administration
```sh
ipx user list # who exists, and how each one signs in
echo -n 'secret123' | ipx user add sam # local account, password on stdin
ipx user add sam --no-password # proxy-only account, created ahead of time
echo -n 'newsecret' | ipx user passwd sam # change a password
ipx user rm sam # remove the account
```
Set `auto_create_users = false` once everyone who should have an account has one. After that the
proxy vouching for an unknown name is logged and refused, rather than quietly making an account.
Scanning intervals, the disk quota, retention and the download folder are **admin-only** — the
Settings button is hidden for everyone else, and the API refuses the change even if the request is
made by hand. Ordinary users still control their own folders, keywords and downloads per feed.

58
package-lock.json generated Normal file
View File

@@ -0,0 +1,58 @@
{
"name": "ipx-ui-tests",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "ipx-ui-tests",
"devDependencies": {
"@playwright/test": "^1.56.0"
}
},
"node_modules/@playwright/test": {
"version": "1.63.0",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.63.0.tgz",
"integrity": "sha512-oxMK4vllB9RK5NQ2l1pq1IfOf2AvnEuj/vYGDj0H2nMtmtZpKtCwt/l00GEO6xjGfpBNAvjovvYdCm50dRQkpQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.63.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/playwright": {
"version": "1.63.0",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.63.0.tgz",
"integrity": "sha512-+7ziBLidS4NaNCdt57SUDT+wYmmd5fmiQejUic/kb+YsYSCPyOOE9sebzMjNmQrsnNpDJqd4WHvV/8lfKfUDUg==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.63.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/playwright-core": {
"version": "1.63.0",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.63.0.tgz",
"integrity": "sha512-rYCsBF/M5HjUch52bbtVONEFjv6Xu8sm8h72dNlR5bzIE1fvC/bxgspzkjSfU+MweEMmPM8KJebG6nnyxo5mCg==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=20"
}
}
}
}

13
package.json Normal file
View File

@@ -0,0 +1,13 @@
{
"name": "ipx-ui-tests",
"private": true,
"description": "Browser tests for the ipx web UI. The Rust tests cover the server; these cover the page.",
"scripts": {
"test": "playwright test",
"test:headed": "playwright test --headed",
"smoke": "node tests/page-smoke.js"
},
"devDependencies": {
"@playwright/test": "^1.56.0"
}
}

43
playwright.config.js Normal file
View File

@@ -0,0 +1,43 @@
const { defineConfig } = require('@playwright/test');
const setup = require('./tests/ui/global-setup');
// Before anything else, including the servers below.
setup.prepare();
// Real browser against a real daemon. The stub-DOM smoke test catches a script that
// fails to load; it cannot catch a wrong selector, a handler that runs but does nothing,
// or a page that renders empty -- which is exactly what has slipped through before.
module.exports = defineConfig({
testDir: './tests/ui',
timeout: 30_000,
expect: { timeout: 10_000 },
fullyParallel: false, // one daemon, one database
workers: 1,
reporter: process.env.CI ? 'line' : [['list']],
use: {
baseURL: 'http://127.0.0.1:8791',
trace: 'retain-on-failure',
screenshot: 'only-on-failure',
},
webServer: [
{
command: 'node tests/ui/fixtures/serve.js',
port: 8792,
reuseExistingServer: false,
stdout: 'ignore',
},
{
// Build first so the tests always run against current source.
command: 'cargo build -q && exec ./target/debug/ipx daemon',
port: 8791,
reuseExistingServer: false,
timeout: 180_000,
stdout: 'pipe',
env: {
IPX_CONFIG: `${setup.root}/config/config.toml`,
IPX_DATA_DIR: `${setup.root}/data`,
IPX_LOG: 'ipx=info',
},
},
],
});

92
src/auth.rs Normal file
View File

@@ -0,0 +1,92 @@
//! Who is asking. Sign-in is either a local password or a header set by whatever fronts
//! this -- Cloudflare Zero Trust on `ipodderx.sdf1.net`, which puts the authenticated
//! address in `Cf-Access-Authenticated-User-Email`.
use anyhow::{Result, bail};
use argon2::Argon2;
use argon2::password_hash::{PasswordHasher, PasswordVerifier, phc::PasswordHash};
/// Argon2id with the crate's defaults, which are the OWASP-recommended parameters. The
/// salt is generated per password by the hasher itself.
pub fn hash_password(password: &str) -> Result<String> {
if password.len() < 8 {
bail!("password must be at least 8 characters");
}
Argon2::default()
.hash_password(password.as_bytes())
.map(|h| h.to_string())
.map_err(|e| anyhow::anyhow!("could not hash the password: {e}"))
}
/// False for a wrong password *and* for a stored hash this build cannot parse; either way
/// the answer is no.
pub fn verify_password(password: &str, stored: &str) -> bool {
let Ok(parsed) = PasswordHash::new(stored) else {
tracing::warn!("stored password hash is unreadable; refusing the sign-in");
return false;
};
Argon2::default()
.verify_password(password.as_bytes(), &parsed)
.is_ok()
}
/// A session id: 256 bits of urandom, hex. Long enough that guessing is not a strategy.
pub fn new_session_token() -> String {
let mut bytes = [0u8; 32];
if getrandom(&mut bytes).is_err() {
// Falling back to the clock would be a predictable session id. Better to fail.
panic!("no source of randomness for a session token");
}
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
fn getrandom(buf: &mut [u8]) -> std::io::Result<()> {
use std::io::Read;
std::fs::File::open("/dev/urandom")?.read_exact(buf)
}
/// A username taken from a proxy header. Cloudflare sends an email address; the local part
/// is what a person recognises, and the whole thing stays unique enough for one household.
pub fn name_from_header(raw: &str) -> Option<String> {
let name = raw.trim();
if name.is_empty() || name.len() > 190 {
return None;
}
// Anything that could confuse a lookup or a log line is not a name.
if name.chars().any(|c| c.is_control() || c == ',' || c == ';') {
return None;
}
Some(name.to_ascii_lowercase())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_password_verifies_only_against_itself() {
let h = hash_password("correct horse battery").unwrap();
assert!(verify_password("correct horse battery", &h));
assert!(!verify_password("Correct horse battery", &h));
assert!(!verify_password("", &h));
// A hash from a different scheme, or a truncated one, must not authenticate.
assert!(!verify_password("correct horse battery", "not-a-hash"));
assert!(hash_password("short").is_err());
}
#[test]
fn session_tokens_are_long_and_distinct() {
let a = new_session_token();
let b = new_session_token();
assert_eq!(a.len(), 64);
assert_ne!(a, b);
}
#[test]
fn a_header_name_is_cleaned_or_refused() {
assert_eq!(name_from_header(" Ray@Example.COM "), Some("ray@example.com".into()));
assert_eq!(name_from_header(""), None);
assert_eq!(name_from_header("ray\nadmin"), None);
assert_eq!(name_from_header("ray;admin"), None);
}
}

207
src/config.rs
View File

@@ -5,29 +5,43 @@ use serde::{Deserialize, Serialize};
use std::collections::BTreeMap; use std::collections::BTreeMap;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
#[derive(Debug, Default, Deserialize, Serialize)] #[derive(Debug, Default, Clone, Deserialize, Serialize)]
pub struct Config { pub struct Config {
#[serde(default)] #[serde(default)]
pub general: General, pub general: General,
#[serde(default)] #[serde(default)]
pub torrent: Torrent, pub torrent: Torrent,
#[serde(default)]
pub web: Web,
/// Keyed by feed id: the TOML table name, which replaces the old genHash(feedURL). /// Keyed by feed id: the TOML table name, which replaces the old genHash(feedURL).
#[serde(default)] #[serde(default)]
pub feeds: BTreeMap<String, Feed>, pub feeds: BTreeMap<String, Feed>,
} }
#[derive(Debug, Deserialize, Serialize)] #[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)] #[serde(default)]
pub struct General { pub struct General {
pub download_dir: PathBuf, pub download_dir: PathBuf,
pub socket: PathBuf, pub socket: PathBuf,
/// Default poll interval; a feed's own <ttl> wins when it is longer. /// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d".
pub interval_mins: u64, /// A feed's own `schedule` overrides this.
pub schedule: String,
/// Superseded by `schedule`. Still read so existing configs keep working.
#[serde(skip_serializing_if = "Option::is_none")]
pub interval_mins: Option<u64>,
pub organize: Organize, pub organize: Organize,
/// 0 = unlimited. /// 0 = unlimited.
pub max_total_gb: f64, pub max_total_gb: f64,
/// 0 = keep forever. /// 0 = keep forever.
pub max_age_days: u64, pub max_age_days: u64,
/// How many new enclosures a single scan may take, when a feed does not say.
/// Unlimited by default was a trap: subscribing to an OPML of 80 feeds then pulled
/// every back-catalogue episode at once. 0 means unlimited, deliberately chosen.
pub max_new_per_check: usize,
/// Top-level media types worth downloading. Blog feeds put each article's header
/// image in an <enclosure>, so taking everything filled the disk with artwork and
/// counted it as episodes. Empty means take anything.
pub media_types: Vec<String>,
} }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
@@ -39,7 +53,7 @@ pub enum Organize {
Date, Date,
} }
#[derive(Debug, Deserialize, Serialize)] #[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)] #[serde(default)]
pub struct Torrent { pub struct Torrent {
pub enabled: bool, pub enabled: bool,
@@ -51,6 +65,48 @@ pub struct Torrent {
pub stall_mins: u64, pub stall_mins: u64,
} }
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct Web {
pub enabled: bool,
/// Use 0.0.0.0 to reach it from the LAN. Anything but loopback needs the token.
pub bind: String,
/// Shared secret. Generated and written back on first run when left empty. It signs
/// in as the admin, which is what keeps the healthcheck and any scripts working.
pub token: String,
/// A header naming the signed-in user, set by whatever fronts this -- Cloudflare Zero
/// Trust sends `Cf-Access-Authenticated-User-Email`. Empty disables the whole path.
pub trusted_header: String,
/// Addresses allowed to assert that header. A header is only as trustworthy as the
/// hop that set it, so an empty list means nobody: on a LAN-bound port anyone could
/// otherwise claim to be anyone. Loopback covers a tunnel running beside the daemon.
pub trusted_proxies: Vec<String>,
/// Create an account the first time the proxy vouches for a name it has not seen.
pub auto_create_users: bool,
/// Sign a session out after this long without a request.
pub session_days: i64,
}
impl Default for Web {
fn default() -> Self {
Self {
enabled: false,
bind: "127.0.0.1:8080".into(),
token: String::new(),
trusted_header: String::new(),
trusted_proxies: vec!["127.0.0.1".into(), "::1".into()],
auto_create_users: true,
session_days: 30,
}
}
}
impl Web {
pub fn binds_publicly(&self) -> bool {
!self.bind.starts_with("127.") && !self.bind.starts_with("localhost")
}
}
#[derive(Debug, Clone, Deserialize, Serialize)] #[derive(Debug, Clone, Deserialize, Serialize)]
pub struct Feed { pub struct Feed {
pub url: String, pub url: String,
@@ -65,7 +121,17 @@ pub struct Feed {
pub allow_explicit: bool, pub allow_explicit: bool,
#[serde(default = "yes")] #[serde(default = "yes")]
pub auto_download: bool, pub auto_download: bool,
/// Cap on new downloads per scan. None = unlimited. /// Set on feeds that came from a subscribed OPML: the id of the OPML feed they
/// belong to. The OPML is re-read on every scan and this list kept in step.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub group: Option<String>,
/// Overrides the global schedule for this feed. Same forms: "every 6h", "2d".
#[serde(default, skip_serializing_if = "Option::is_none")]
pub schedule: Option<String>,
/// Media types for this feed. None follows `[general]`; an empty list takes anything.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub media_types: Option<Vec<String>>,
/// Cap on new downloads per scan for this feed. None follows `[general]`.
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub max_new_per_check: Option<usize>, pub max_new_per_check: Option<usize>,
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
@@ -86,10 +152,13 @@ impl Default for General {
Self { Self {
download_dir: home().join("Podcasts"), download_dir: home().join("Podcasts"),
socket: default_socket(), socket: default_socket(),
interval_mins: 60, schedule: "every 60m".into(),
interval_mins: None,
organize: Organize::Feed, organize: Organize::Feed,
max_total_gb: 0.0, max_total_gb: 0.0,
max_age_days: 0, max_age_days: 0,
max_new_per_check: 3,
media_types: vec!["audio".into(), "video".into()],
} }
} }
} }
@@ -106,6 +175,49 @@ impl Default for Torrent {
} }
} }
impl General {
/// Minutes between checks. Falls back to the legacy `interval_mins`, then to an hour.
/// A malformed value warns rather than stopping the daemon.
pub fn interval(&self) -> u64 {
if let Some(n) = parse_interval(&self.schedule) {
return n;
}
if !self.schedule.trim().is_empty() {
tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default");
}
self.interval_mins.filter(|n| *n > 0).unwrap_or(60)
}
}
/// Parses a check interval into minutes.
///
/// Accepts "every 30m", "30m", "4h", "1d", "2w", "every 4 hours", or a bare number of
/// minutes.
/// Returns None for anything it cannot read, or for zero.
pub fn parse_interval(s: &str) -> Option<u64> {
let s = s.trim().to_lowercase();
let s = s.strip_prefix("every").unwrap_or(&s).trim();
if s.is_empty() {
return None;
}
let digits: String = s.chars().take_while(|c| c.is_ascii_digit()).collect();
if digits.is_empty() {
return None;
}
let n: u64 = digits.parse().ok()?;
let unit = s[digits.len()..].trim();
let mins = match unit {
"" | "m" | "min" | "mins" | "minute" | "minutes" => n,
"h" | "hr" | "hrs" | "hour" | "hours" => n.checked_mul(60)?,
"d" | "day" | "days" => n.checked_mul(1440)?,
"w" | "week" | "weeks" => n.checked_mul(10080)?,
_ => return None,
};
(mins > 0).then_some(mins)
}
impl Torrent { impl Torrent {
/// Inclusive listen port range. Falls back to the BitTorrent default on garbage input. /// Inclusive listen port range. Falls back to the BitTorrent default on garbage input.
pub fn ports(&self) -> (u16, u16) { pub fn ports(&self) -> (u16, u16) {
@@ -188,6 +300,25 @@ fn default_socket() -> PathBuf {
} }
} }
/// Whether an enclosure's type is one we want.
///
/// An unknown type is allowed: the real type is only known after downloading, and
/// refusing everything untyped would drop feeds that simply omit the attribute.
pub fn wanted_media(mime: Option<&str>, wanted: &[String]) -> bool {
if wanted.is_empty() {
return true;
}
let Some(mime) = mime.map(str::trim).filter(|m| !m.is_empty()) else {
return true;
};
let top = mime.split('/').next().unwrap_or(mime).to_ascii_lowercase();
// A .torrent is a container for media, not media itself; judge it once unpacked.
if mime.to_ascii_lowercase().contains("torrent") {
return true;
}
wanted.iter().any(|w| w.trim().eq_ignore_ascii_case(&top) || w.trim().eq_ignore_ascii_case(mime))
}
/// Feed ids are the TOML table key, so they must be readable and punctuation-free. /// Feed ids are the TOML table key, so they must be readable and punctuation-free.
pub fn slug(text: &str) -> String { pub fn slug(text: &str) -> String {
let mut out = String::new(); let mut out = String::new();
@@ -245,7 +376,7 @@ mod tests {
.unwrap(); .unwrap();
assert_eq!(cfg.general.download_dir, PathBuf::from("/tmp/pods")); assert_eq!(cfg.general.download_dir, PathBuf::from("/tmp/pods"));
assert_eq!(cfg.general.interval_mins, 60); assert_eq!(cfg.general.interval(), 60);
assert_eq!(cfg.general.organize, Organize::Feed); assert_eq!(cfg.general.organize, Organize::Feed);
assert!(cfg.torrent.enabled); assert!(cfg.torrent.enabled);
@@ -256,6 +387,41 @@ mod tests {
assert_eq!(feed.keywords, vec!["deep dive"]); assert_eq!(feed.keywords, vec!["deep dive"]);
} }
#[test]
fn intervals_parse_from_the_forms_people_actually_type() {
for (input, want) in [
("every 30m", 30), ("30m", 30), ("30", 30), ("every 30 minutes", 30),
("every 4h", 240), ("4h", 240), ("4 hours", 240), ("EVERY 4H", 240),
("1d", 1440), ("every 2 days", 2880), (" every 90m ", 90),
("1w", 10080), ("every 2 weeks", 20160), ("2 w", 20160),
] {
assert_eq!(parse_interval(input), Some(want), "{input:?}");
}
for bad in ["", " ", "every", "soon", "-5m", "0", "0h", "every 0 minutes", "5 fortnights"] {
assert_eq!(parse_interval(bad), None, "{bad:?} should not parse");
}
}
#[test]
fn interval_falls_back_through_legacy_then_default() {
let mut g = General::default();
assert_eq!(g.interval(), 60, "the default schedule");
g.schedule = "every 15m".into();
assert_eq!(g.interval(), 15);
// A config written before `schedule` existed still works.
g.schedule = String::new();
g.interval_mins = Some(45);
assert_eq!(g.interval(), 45);
// Garbage must not stop the daemon.
g.schedule = "whenever".into();
assert_eq!(g.interval(), 45);
g.interval_mins = None;
assert_eq!(g.interval(), 60);
}
#[test] #[test]
fn port_range_falls_back_when_malformed() { fn port_range_falls_back_when_malformed() {
let mut t = Torrent::default(); let mut t = Torrent::default();
@@ -268,6 +434,26 @@ mod tests {
assert_eq!(t.ports(), (6881, 6889), "reversed range is not a range"); assert_eq!(t.ports(), (6881, 6889), "reversed range is not a range");
} }
#[test]
fn media_types_keep_article_artwork_out() {
let want = vec!["audio".to_string(), "video".to_string()];
assert!(wanted_media(Some("audio/mpeg"), &want));
assert!(wanted_media(Some("audio/mp4"), &want));
assert!(wanted_media(Some("video/quicktime"), &want));
assert!(!wanted_media(Some("image/jpeg"), &want), "a blog header image is not an episode");
assert!(!wanted_media(Some("text/html"), &want));
// A torrent is a container; what is inside is judged after unpacking.
assert!(wanted_media(Some("application/x-bittorrent"), &want));
// Unknown type: only discoverable by downloading, so do not refuse it outright.
assert!(wanted_media(None, &want));
assert!(wanted_media(Some(""), &want));
// An empty list means take anything, which is how it behaved before.
assert!(wanted_media(Some("image/jpeg"), &[]));
// A full type can be named exactly.
assert!(wanted_media(Some("image/jpeg"), &["image/jpeg".to_string()]));
}
#[test] #[test]
fn slugs_are_readable_and_unique() { fn slugs_are_readable_and_unique() {
assert_eq!(slug("Accidental Tech Podcast"), "accidental-tech-podcast"); assert_eq!(slug("Accidental Tech Podcast"), "accidental-tech-podcast");
@@ -279,7 +465,7 @@ mod tests {
let mut taken = BTreeMap::new(); let mut taken = BTreeMap::new();
taken.insert("the-daily".to_string(), Feed { taken.insert("the-daily".to_string(), Feed {
url: "u".into(), folder: None, keywords: vec![], allow_explicit: false, url: "u".into(), folder: None, group: None, media_types: None, schedule: None, keywords: vec![], allow_explicit: false,
auto_download: true, max_new_per_check: None, username: None, auto_download: true, max_new_per_check: None, username: None,
password: None, password_env: None, password: None, password_env: None,
}); });
@@ -291,6 +477,9 @@ mod tests {
let mut f = Feed { let mut f = Feed {
url: "https://x/y".into(), url: "https://x/y".into(),
folder: None, folder: None,
group: None,
media_types: None,
schedule: None,
keywords: vec![], keywords: vec![],
allow_explicit: false, allow_explicit: false,
auto_download: true, auto_download: true,

717
src/db.rs
View File

@@ -1,7 +1,7 @@
//! SQLite state. Replaces the per-feed .ipxd plists, history.dat and qmcache.dat. //! SQLite state. Replaces the per-feed .ipxd plists, history.dat and qmcache.dat.
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use rusqlite::{Connection, OptionalExtension}; use rusqlite::{Connection, OptionalExtension, params};
use std::path::Path; use std::path::Path;
use std::sync::Mutex; use std::sync::Mutex;
@@ -16,11 +16,20 @@ CREATE TABLE IF NOT EXISTS feeds (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
url TEXT NOT NULL, url TEXT NOT NULL,
title TEXT, title TEXT,
image TEXT,
etag TEXT, etag TEXT,
last_modified TEXT, last_modified TEXT,
last_checked INTEGER, last_checked INTEGER,
ttl_mins INTEGER, ttl_mins INTEGER,
last_error TEXT last_error TEXT,
-- Came from a subscribed OPML that no longer lists it, but has downloads, so kept.
orphaned INTEGER NOT NULL DEFAULT 0,
-- The OPML subscription this feed came from.
group_id TEXT,
-- 1 = derived from an OPML and not written to config.toml. Writing 80-odd generated
-- entries into a hand-edited file made it unreadable; the OPML is the source of
-- truth, so they are re-derived instead. Customising one promotes it to config.
managed INTEGER NOT NULL DEFAULT 0
); );
CREATE TABLE IF NOT EXISTS entries ( CREATE TABLE IF NOT EXISTS entries (
@@ -33,6 +42,12 @@ CREATE TABLE IF NOT EXISTS entries (
first_seen INTEGER NOT NULL, first_seen INTEGER NOT NULL,
read INTEGER NOT NULL DEFAULT 0, read INTEGER NOT NULL DEFAULT 0,
flagged INTEGER NOT NULL DEFAULT 0, flagged INTEGER NOT NULL DEFAULT 0,
image TEXT,
duration INTEGER,
episode INTEGER,
season INTEGER,
-- Seconds into the audio, so playback resumes where it was left.
position INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (feed_id, guid) PRIMARY KEY (feed_id, guid)
); );
@@ -54,12 +69,80 @@ CREATE TABLE IF NOT EXISTS enclosures (
); );
CREATE INDEX IF NOT EXISTS enclosures_entry ON enclosures (feed_id, guid); CREATE INDEX IF NOT EXISTS enclosures_entry ON enclosures (feed_id, guid);
-- pass_hash is NULL for someone who only ever arrives through the proxy: there is no
-- password to check, and leaving it empty is not the same as leaving it unset.
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE COLLATE NOCASE,
pass_hash TEXT,
is_admin INTEGER NOT NULL DEFAULT 0,
created INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS sessions (
token TEXT PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created INTEGER NOT NULL,
seen INTEGER NOT NULL
);
"; ";
/// Someone who can sign in. `pass_hash` is None for an account that only ever arrives
/// through the proxy.
#[derive(Debug, Clone)]
pub struct User {
pub id: i64,
pub name: String,
pub pass_hash: Option<String>,
pub is_admin: bool,
}
/// A feed derived from an OPML subscription rather than written into the config.
#[derive(Debug, Clone)]
pub struct Managed {
pub id: String,
pub url: String,
pub title: Option<String>,
pub group_id: String,
pub orphaned: bool,
}
/// Adds columns that later versions introduced. CREATE TABLE IF NOT EXISTS does nothing to
/// a table that already exists, so an installed database needs them added explicitly.
fn migrate(conn: &Connection) -> Result<()> {
let wanted: &[(&str, &str, &str)] = &[
("feeds", "image", "TEXT"),
("feeds", "orphaned", "INTEGER NOT NULL DEFAULT 0"),
("feeds", "group_id", "TEXT"),
("feeds", "managed", "INTEGER NOT NULL DEFAULT 0"),
("entries", "image", "TEXT"),
("entries", "duration", "INTEGER"),
("entries", "episode", "INTEGER"),
("entries", "season", "INTEGER"),
("entries", "position", "INTEGER NOT NULL DEFAULT 0"),
];
for (table, column, ty) in wanted {
let mut stmt = conn.prepare(&format!("PRAGMA table_info({table})"))?;
let existing: Vec<String> = stmt
.query_map([], |r| r.get::<_, String>(1))?
.collect::<rusqlite::Result<Vec<_>>>()?;
if !existing.iter().any(|c| c == column) {
tracing::info!(table, column, "adding column");
conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?;
}
}
Ok(())
}
/// What `ipx list` shows next to each configured feed. /// What `ipx list` shows next to each configured feed.
#[derive(Debug, Default)] #[derive(Debug, Default)]
pub struct FeedSummary { pub struct FeedSummary {
pub title: Option<String>, pub title: Option<String>,
pub image: Option<String>,
/// Came from a subscribed OPML that no longer lists it, but it has downloads, so it
/// was kept rather than removed.
pub orphaned: bool,
pub last_checked: Option<i64>, pub last_checked: Option<i64>,
pub last_error: Option<String>, pub last_error: Option<String>,
pub entries: i64, pub entries: i64,
@@ -78,6 +161,7 @@ impl Db {
conn.pragma_update(None, "foreign_keys", "ON")?; conn.pragma_update(None, "foreign_keys", "ON")?;
conn.pragma_update(None, "busy_timeout", 5000)?; conn.pragma_update(None, "busy_timeout", 5000)?;
conn.execute_batch(SCHEMA).context("creating schema")?; conn.execute_batch(SCHEMA).context("creating schema")?;
migrate(&conn).context("migrating schema")?;
Ok(Self { conn: Mutex::new(conn) }) Ok(Self { conn: Mutex::new(conn) })
} }
@@ -86,6 +170,9 @@ impl Db {
pub fn memory() -> Result<Self> { pub fn memory() -> Result<Self> {
let conn = Connection::open_in_memory()?; let conn = Connection::open_in_memory()?;
conn.execute_batch(SCHEMA)?; conn.execute_batch(SCHEMA)?;
// Same path as a real open, so a column added only in migrate() cannot pass the
// tests while being missing in production (or the reverse).
migrate(&conn)?;
Ok(Self { conn: Mutex::new(conn) }) Ok(Self { conn: Mutex::new(conn) })
} }
@@ -99,13 +186,16 @@ impl Db {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let mut sum: FeedSummary = conn let mut sum: FeedSummary = conn
.query_row( .query_row(
"SELECT title, last_checked, last_error FROM feeds WHERE id = ?1", "SELECT title, image, last_checked, last_error, coalesce(orphaned, 0)
FROM feeds WHERE id = ?1",
[feed_id], [feed_id],
|r| { |r| {
Ok(FeedSummary { Ok(FeedSummary {
title: r.get(0)?, title: r.get(0)?,
last_checked: r.get(1)?, image: r.get(1)?,
last_error: r.get(2)?, last_checked: r.get(2)?,
last_error: r.get(3)?,
orphaned: r.get::<_, i64>(4)? != 0,
..Default::default() ..Default::default()
}) })
}, },
@@ -166,11 +256,12 @@ impl Db {
etag: Option<&str>, etag: Option<&str>,
last_modified: Option<&str>, last_modified: Option<&str>,
ttl_mins: Option<u64>, ttl_mins: Option<u64>,
image: Option<&str>,
) -> Result<()> { ) -> Result<()> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
conn.execute( conn.execute(
"INSERT INTO feeds (id, url, title, etag, last_modified, last_checked, ttl_mins, last_error) "INSERT INTO feeds (id, url, title, etag, last_modified, last_checked, ttl_mins, last_error, image)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, NULL) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, NULL, ?8)
ON CONFLICT(id) DO UPDATE SET ON CONFLICT(id) DO UPDATE SET
url = excluded.url, url = excluded.url,
title = coalesce(excluded.title, feeds.title), title = coalesce(excluded.title, feeds.title),
@@ -178,8 +269,9 @@ impl Db {
last_modified = excluded.last_modified, last_modified = excluded.last_modified,
last_checked = excluded.last_checked, last_checked = excluded.last_checked,
ttl_mins = excluded.ttl_mins, ttl_mins = excluded.ttl_mins,
image = coalesce(excluded.image, feeds.image),
last_error = NULL", last_error = NULL",
rusqlite::params![feed_id, url, title, etag, last_modified, now(), ttl_mins.map(|t| t as i64)], rusqlite::params![feed_id, url, title, etag, last_modified, now(), ttl_mins.map(|t| t as i64), image],
)?; )?;
Ok(()) Ok(())
} }
@@ -195,6 +287,17 @@ impl Db {
Ok(()) Ok(())
} }
/// Forgets the cached ETag/Last-Modified. Those validators belong to the old URL, so
/// keeping them across a URL change could produce a bogus 304 against the new one.
pub fn clear_validators(&self, feed_id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"UPDATE feeds SET etag = NULL, last_modified = NULL, last_checked = NULL WHERE id = ?1",
[feed_id],
)?;
Ok(())
}
pub fn set_feed_error(&self, feed_id: &str, url: &str, msg: &str) -> Result<()> { pub fn set_feed_error(&self, feed_id: &str, url: &str, msg: &str) -> Result<()> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
conn.execute( conn.execute(
@@ -213,9 +316,13 @@ impl Db {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let inserted = conn.execute( let inserted = conn.execute(
"INSERT OR IGNORE INTO entries "INSERT OR IGNORE INTO entries
(feed_id, guid, title, link, published, description, first_seen, read, flagged) (feed_id, guid, title, link, published, description, first_seen, read, flagged,
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, 0, 0)", image, duration, episode, season)
rusqlite::params![feed_id, e.guid, e.title, e.link, e.published, e.description, now()], VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, 0, 0, ?8, ?9, ?10, ?11)",
rusqlite::params![
feed_id, e.guid, e.title, e.link, e.published, e.description, now(),
e.image, e.duration, e.episode, e.season
],
)?; )?;
if inserted == 0 { if inserted == 0 {
// The SET expressions see the pre-update row, so this compares old vs new. // The SET expressions see the pre-update row, so this compares old vs new.
@@ -223,9 +330,16 @@ impl Db {
"UPDATE entries SET "UPDATE entries SET
title = coalesce(?3, title), title = coalesce(?3, title),
description = coalesce(?4, description), description = coalesce(?4, description),
image = coalesce(?5, image),
duration = coalesce(?6, duration),
episode = coalesce(?7, episode),
season = coalesce(?8, season),
read = CASE WHEN description IS NOT ?4 OR title IS NOT ?3 THEN 0 ELSE read END read = CASE WHEN description IS NOT ?4 OR title IS NOT ?3 THEN 0 ELSE read END
WHERE feed_id = ?1 AND guid = ?2", WHERE feed_id = ?1 AND guid = ?2",
rusqlite::params![feed_id, e.guid, e.title, e.description], rusqlite::params![
feed_id, e.guid, e.title, e.description,
e.image, e.duration, e.episode, e.season
],
)?; )?;
} }
Ok(inserted == 1) Ok(inserted == 1)
@@ -275,6 +389,7 @@ impl Db {
/// An enclosure waiting to be downloaded. /// An enclosure waiting to be downloaded.
#[derive(Debug)] #[derive(Debug)]
pub struct Pending { pub struct Pending {
pub id: i64,
pub url: String, pub url: String,
pub mime: Option<String>, pub mime: Option<String>,
} }
@@ -285,12 +400,17 @@ impl Db {
pub fn pending(&self, feed_id: &str, limit: usize) -> Result<Vec<Pending>> { pub fn pending(&self, feed_id: &str, limit: usize) -> Result<Vec<Pending>> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare( let mut stmt = conn.prepare(
"SELECT url, mime FROM enclosures // Newest first: a cap of 3 should mean the three latest episodes, not the
WHERE feed_id = ?1 AND state = 'pending' ORDER BY id LIMIT ?2", // three that happen to have been recorded first.
"SELECT x.id, x.url, x.mime FROM enclosures x
JOIN entries e ON e.feed_id = x.feed_id AND e.guid = x.guid
WHERE x.feed_id = ?1 AND x.state = 'pending'
ORDER BY coalesce(e.published, e.first_seen) DESC, x.id DESC
LIMIT ?2",
)?; )?;
let rows = stmt let rows = stmt
.query_map(rusqlite::params![feed_id, limit as i64], |r| { .query_map(rusqlite::params![feed_id, limit as i64], |r| {
Ok(Pending { url: r.get(0)?, mime: r.get(1)? }) Ok(Pending { id: r.get(0)?, url: r.get(1)?, mime: r.get(2)? })
})? })?
.collect::<rusqlite::Result<Vec<_>>>()?; .collect::<rusqlite::Result<Vec<_>>>()?;
Ok(rows) Ok(rows)
@@ -386,6 +506,15 @@ impl Db {
} }
impl Db { impl Db {
pub fn unread_count(&self, feed_id: &str) -> Result<i64> {
let conn = self.conn.lock().unwrap();
Ok(conn.query_row(
"SELECT count(*) FROM entries WHERE feed_id = ?1 AND read = 0",
[feed_id],
|r| r.get(0),
)?)
}
/// (pending, downloaded) across all feeds, for the status command. /// (pending, downloaded) across all feeds, for the status command.
pub fn counts(&self) -> Result<(i64, i64)> { pub fn counts(&self) -> Result<(i64, i64)> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
@@ -396,6 +525,490 @@ impl Db {
} }
} }
/// An entry plus its enclosures, for the web UI.
#[derive(Debug, serde::Serialize)]
pub struct EntryRow {
pub guid: String,
pub feed_id: String,
pub title: Option<String>,
pub link: Option<String>,
pub published: Option<i64>,
pub description: Option<String>,
pub read: bool,
pub flagged: bool,
pub image: Option<String>,
pub duration: Option<i64>,
pub episode: Option<i64>,
pub season: Option<i64>,
pub position: i64,
pub enclosures: Vec<EncRow>,
}
/// The search clause. `?2` is referenced unconditionally -- binding a parameter the
/// statement does not mention is an error, so an empty needle short-circuits instead.
const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2
OR lower(coalesce(e.description, '')) LIKE ?2)";
/// Which slice of a feed the UI is asking for.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Filter {
All,
Unread,
Downloaded,
Flagged,
}
impl Filter {
pub fn parse(s: &str) -> Self {
match s {
"unread" => Self::Unread,
"downloaded" => Self::Downloaded,
"flagged" => Self::Flagged,
_ => Self::All,
}
}
/// The WHERE fragment for this filter. `e` is entries, and the EXISTS subquery is
/// correlated against it.
fn sql(self) -> &'static str {
match self {
Self::All => "1=1",
Self::Unread => "e.read = 0",
Self::Flagged => "e.flagged = 1",
Self::Downloaded => {
"EXISTS (SELECT 1 FROM enclosures x
WHERE x.feed_id = e.feed_id AND x.guid = e.guid AND x.path IS NOT NULL)"
}
}
}
}
#[derive(Debug, Clone, serde::Serialize)]
pub struct EncRow {
pub id: i64,
pub feed_id: String,
pub guid: String,
pub url: String,
pub mime: Option<String>,
pub length: Option<i64>,
pub path: Option<String>,
pub state: String,
pub last_error: Option<String>,
}
impl Db {
/// One page of a feed's entries, newest first, each with its enclosures attached.
/// `search` matches title and description, case-insensitively.
pub fn entries(
&self,
feed_id: &str,
filter: Filter,
search: Option<&str>,
offset: i64,
limit: i64,
) -> Result<Vec<EntryRow>> {
let conn = self.conn.lock().unwrap();
let like = search
.map(|q| format!("%{}%", q.trim().to_lowercase()))
.unwrap_or_default();
let sql = format!(
"SELECT e.guid, e.feed_id, e.title, e.link, e.published, e.description, e.read,
e.flagged, e.image, e.duration, e.episode, e.season, e.position
FROM entries e
WHERE e.feed_id = ?1 AND {} AND {SEARCH}
ORDER BY coalesce(e.published, e.first_seen) DESC, e.rowid DESC
LIMIT ?4 OFFSET ?3",
filter.sql()
);
let mut stmt = conn.prepare(&sql)?;
let map = |r: &rusqlite::Row| -> rusqlite::Result<EntryRow> {
Ok(EntryRow {
guid: r.get(0)?,
feed_id: r.get(1)?,
title: r.get(2)?,
link: r.get(3)?,
published: r.get(4)?,
description: r.get(5)?,
read: r.get::<_, i64>(6)? != 0,
flagged: r.get::<_, i64>(7)? != 0,
image: r.get(8)?,
duration: r.get(9)?,
episode: r.get(10)?,
season: r.get(11)?,
position: r.get(12)?,
enclosures: vec![],
})
};
let mut rows: Vec<EntryRow> = stmt
.query_map(rusqlite::params![feed_id, like, offset, limit], map)?
.collect::<rusqlite::Result<Vec<_>>>()?;
if rows.is_empty() {
return Ok(rows);
}
// Only the guids on this page, so a feed with thousands of entries stays cheap.
let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(",");
let sql = format!(
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error
FROM enclosures WHERE feed_id = ? AND guid IN ({placeholders}) ORDER BY id"
);
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len() + 1);
params.push(&feed_id);
for row in &rows {
params.push(&row.guid);
}
let mut stmt = conn.prepare(&sql)?;
let encs = stmt
.query_map(params.as_slice(), |r| {
Ok(EncRow {
id: r.get(0)?,
feed_id: r.get(1)?,
guid: r.get(2)?,
url: r.get(3)?,
mime: r.get(4)?,
length: r.get(5)?,
path: r.get(6)?,
state: r.get(7)?,
last_error: r.get(8)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
for enc in encs {
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid) {
row.enclosures.push(enc);
}
}
Ok(rows)
}
/// How many entries match, so the UI knows whether there is another page.
pub fn count_entries(&self, feed_id: &str, filter: Filter, search: Option<&str>) -> Result<i64> {
let conn = self.conn.lock().unwrap();
let like = search
.map(|q| format!("%{}%", q.trim().to_lowercase()))
.unwrap_or_default();
let sql = format!(
"SELECT count(*) FROM entries e WHERE e.feed_id = ?1 AND {} AND {SEARCH}",
filter.sql()
);
Ok(conn.query_row(&sql, rusqlite::params![feed_id, like], |r| r.get(0))?)
}
/// Where playback got to, so it resumes there next time.
pub fn set_position(&self, feed_id: &str, guid: &str, secs: i64) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"UPDATE entries SET position = ?3 WHERE feed_id = ?1 AND guid = ?2",
rusqlite::params![feed_id, guid, secs.max(0)],
)?;
Ok(())
}
/// Marks every entry in a feed read, for the "mark all read" button.
// ---- users and sessions ----
pub fn create_user(&self, name: &str, pass_hash: Option<&str>, admin: bool) -> Result<i64> {
let conn = self.conn.lock().unwrap();
conn.execute(
"INSERT INTO users (name, pass_hash, is_admin, created) VALUES (?1, ?2, ?3, ?4)",
params![name, pass_hash, admin as i64, now()],
)?;
Ok(conn.last_insert_rowid())
}
pub fn user_by_name(&self, name: &str) -> Result<Option<User>> {
self.one_user("SELECT id, name, pass_hash, is_admin FROM users WHERE name = ?1", name)
}
pub fn user_by_id(&self, id: i64) -> Result<Option<User>> {
self.one_user("SELECT id, name, pass_hash, is_admin FROM users WHERE id = ?1", id)
}
fn one_user<P: rusqlite::ToSql>(&self, sql: &str, key: P) -> Result<Option<User>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(sql)?;
let mut rows = stmt.query(params![key])?;
Ok(match rows.next()? {
Some(r) => Some(User {
id: r.get(0)?,
name: r.get(1)?,
pass_hash: r.get(2)?,
is_admin: r.get::<_, i64>(3)? != 0,
}),
None => None,
})
}
pub fn users(&self) -> Result<Vec<User>> {
let conn = self.conn.lock().unwrap();
let mut stmt =
conn.prepare("SELECT id, name, pass_hash, is_admin FROM users ORDER BY name")?;
let out = stmt
.query_map([], |r| {
Ok(User {
id: r.get(0)?,
name: r.get(1)?,
pass_hash: r.get(2)?,
is_admin: r.get::<_, i64>(3)? != 0,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(out)
}
pub fn set_password(&self, id: i64, hash: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("UPDATE users SET pass_hash = ?2 WHERE id = ?1", params![id, hash])?;
Ok(())
}
pub fn set_admin(&self, id: i64, admin: bool) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("UPDATE users SET is_admin = ?2 WHERE id = ?1", params![id, admin as i64])?;
Ok(())
}
/// Sessions go with the user: a deleted account must not leave a usable cookie behind.
pub fn delete_user(&self, id: i64) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("DELETE FROM sessions WHERE user_id = ?1", [id])?;
conn.execute("DELETE FROM users WHERE id = ?1", [id])?;
Ok(())
}
pub fn create_session(&self, user_id: i64, token: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"INSERT INTO sessions (token, user_id, created, seen) VALUES (?1, ?2, ?3, ?3)",
params![token, user_id, now()],
)?;
Ok(())
}
/// The user behind a session cookie, if it is still live. Idle sessions expire after
/// `max_idle_secs`; touching `seen` is what keeps a session in daily use alive.
pub fn session_user(&self, token: &str, max_idle_secs: i64) -> Result<Option<User>> {
let conn = self.conn.lock().unwrap();
let cutoff = now() - max_idle_secs;
let mut stmt = conn.prepare(
"SELECT u.id, u.name, u.pass_hash, u.is_admin
FROM sessions s JOIN users u ON u.id = s.user_id
WHERE s.token = ?1 AND s.seen >= ?2",
)?;
let mut rows = stmt.query(params![token, cutoff])?;
let found = match rows.next()? {
Some(r) => Some(User {
id: r.get(0)?,
name: r.get(1)?,
pass_hash: r.get(2)?,
is_admin: r.get::<_, i64>(3)? != 0,
}),
None => None,
};
drop(rows);
drop(stmt);
if found.is_some() {
conn.execute("UPDATE sessions SET seen = ?2 WHERE token = ?1", params![token, now()])?;
} else {
// Either unknown or timed out; either way it is dead weight.
conn.execute("DELETE FROM sessions WHERE token = ?1 OR seen < ?2", params![token, cutoff])?;
}
Ok(found)
}
pub fn delete_session(&self, token: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("DELETE FROM sessions WHERE token = ?1", [token])?;
Ok(())
}
/// Marks every entry of the given feeds read. Takes a list because an OPML subscription
/// holds no entries itself -- marking it read means the feeds inside it.
pub fn mark_all_read(&self, feed_ids: &[String]) -> Result<usize> {
let conn = self.conn.lock().unwrap();
let mut n = 0;
for id in feed_ids {
n += conn.execute("UPDATE entries SET read = 1 WHERE feed_id = ?1 AND read = 0", [id])?;
}
Ok(n)
}
/// The next N enclosures with no file, newest entry first -- what "download latest"
/// queues up.
pub fn undownloaded(&self, feed_id: &str, limit: i64) -> Result<Vec<i64>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT x.id FROM enclosures x
JOIN entries e ON e.feed_id = x.feed_id AND e.guid = x.guid
WHERE x.feed_id = ?1 AND x.path IS NULL AND x.state != 'reaped'
ORDER BY coalesce(e.published, e.first_seen) DESC
LIMIT ?2",
)?;
Ok(stmt
.query_map(rusqlite::params![feed_id, limit], |r| r.get(0))?
.collect::<rusqlite::Result<Vec<_>>>()?)
}
pub fn enclosure(&self, id: i64) -> Result<Option<EncRow>> {
let conn = self.conn.lock().unwrap();
Ok(conn
.query_row(
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error
FROM enclosures WHERE id = ?1",
[id],
|r| {
Ok(EncRow {
id: r.get(0)?,
feed_id: r.get(1)?,
guid: r.get(2)?,
url: r.get(3)?,
mime: r.get(4)?,
length: r.get(5)?,
path: r.get(6)?,
state: r.get(7)?,
last_error: r.get(8)?,
})
},
)
.optional()?)
}
/// `read` and `flagged` finally get a writer: retention orders by them.
pub fn set_entry_flag(&self, feed_id: &str, guid: &str, field: EntryFlag, on: bool) -> Result<()> {
let conn = self.conn.lock().unwrap();
let sql = match field {
EntryFlag::Read => "UPDATE entries SET read = ?3 WHERE feed_id = ?1 AND guid = ?2",
EntryFlag::Flagged => "UPDATE entries SET flagged = ?3 WHERE feed_id = ?1 AND guid = ?2",
};
conn.execute(sql, rusqlite::params![feed_id, guid, on as i64])?;
Ok(())
}
/// How many files this feed has on disk. Decides whether a feed dropped from an OPML
/// can be removed or must be kept.
pub fn downloaded_count(&self, feed_id: &str) -> Result<i64> {
let conn = self.conn.lock().unwrap();
Ok(conn.query_row(
"SELECT count(*) FROM enclosures WHERE feed_id = ?1 AND path IS NOT NULL",
[feed_id],
|r| r.get(0),
)?)
}
/// Names a feed without touching its conditional-GET validators. An OPML subscription
/// takes its name from the document's own <head><title>.
pub fn set_title(&self, feed_id: &str, title: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"UPDATE feeds SET title = ?2 WHERE id = ?1 AND coalesce(title, '') != ?2",
rusqlite::params![feed_id, title],
)?;
Ok(())
}
/// Records a feed that came from an OPML. Its settings are the parent's; only what
/// identifies it is stored.
pub fn upsert_managed(
&self,
id: &str,
url: &str,
title: &str,
group_id: &str,
) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"INSERT INTO feeds (id, url, title, group_id, managed, orphaned)
VALUES (?1, ?2, ?3, ?4, 1, 0)
ON CONFLICT(id) DO UPDATE SET
url = excluded.url,
title = coalesce(feeds.title, excluded.title),
group_id = excluded.group_id,
managed = 1,
orphaned = 0",
rusqlite::params![id, url, title, group_id],
)?;
Ok(())
}
/// Every feed derived from an OPML, whichever group.
pub fn managed_feeds(&self) -> Result<Vec<Managed>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT id, url, title, group_id, orphaned FROM feeds
WHERE managed = 1 AND group_id IS NOT NULL ORDER BY coalesce(title, id)",
)?;
Ok(stmt
.query_map([], |r| {
Ok(Managed {
id: r.get(0)?,
url: r.get(1)?,
title: r.get(2)?,
group_id: r.get(3)?,
orphaned: r.get::<_, i64>(4)? != 0,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?)
}
/// Forgets a derived feed entirely. Only for one with nothing downloaded.
pub fn drop_managed(&self, id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("DELETE FROM feeds WHERE id = ?1 AND managed = 1", [id])?;
conn.execute("DELETE FROM entries WHERE feed_id = ?1", [id])?;
conn.execute("DELETE FROM enclosures WHERE feed_id = ?1 AND path IS NULL", [id])?;
Ok(())
}
/// Stops treating a feed as derived, because it now has its own config entry.
pub fn unmanage(&self, id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("UPDATE feeds SET managed = 0 WHERE id = ?1", [id])?;
Ok(())
}
pub fn set_orphaned(&self, feed_id: &str, on: bool) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"INSERT INTO feeds (id, url, orphaned) VALUES (?1, '', ?2)
ON CONFLICT(id) DO UPDATE SET orphaned = excluded.orphaned",
rusqlite::params![feed_id, on as i64],
)?;
Ok(())
}
/// Nothing can be in flight the moment the daemon starts, so any row still marked
/// `downloading` is a leftover from a restart or a crash. Left alone it would sit
/// there forever: the pending queue skips it and nothing else ever revisits it.
pub fn requeue_interrupted(&self) -> Result<usize> {
let conn = self.conn.lock().unwrap();
Ok(conn.execute(
"UPDATE enclosures SET state = 'pending' WHERE state = 'downloading' AND path IS NULL",
[],
)?)
}
/// Puts an enclosure back in the queue so the next scan picks it up. This is how a
/// `skipped` verdict (from a filter that has since been changed) gets revisited.
pub fn requeue(&self, id: i64) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"UPDATE enclosures SET state = 'pending', last_error = NULL
WHERE id = ?1 AND path IS NULL",
[id],
)?;
Ok(())
}
}
#[derive(Debug, Clone, Copy)]
pub enum EntryFlag {
Read,
Flagged,
}
/// Unix seconds. Everything time-shaped in the DB is stored this way. /// Unix seconds. Everything time-shaped in the DB is stored this way.
pub fn now() -> i64 { pub fn now() -> i64 {
std::time::SystemTime::now() std::time::SystemTime::now()
@@ -420,6 +1033,80 @@ mod tests {
assert_eq!(sum.downloaded, 0); assert_eq!(sum.downloaded, 0);
} }
#[test]
fn every_filter_works_with_and_without_a_search_term() {
// Regression: the search clause used to be omitted when no term was given, while
// ?2 was still bound -- rusqlite rejects a parameter the statement never mentions,
// so plain filtering failed with "Wrong number of parameters passed to query".
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO entries (feed_id, guid, title, description, first_seen, read, flagged) VALUES
('f','a','Alpha dive','notes one',100,0,0),
('f','b','Beta', 'notes two',200,1,0),
('f','c','Gamma dive','notes three',300,1,1);
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
(1,'f','b','u1','/tmp/b','done');",
)
.unwrap();
for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] {
// Both paths must run without erroring, and agree with each other.
let rows = db.entries("f", f, None, 0, 50).unwrap();
let n = db.count_entries("f", f, None).unwrap();
assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page");
let rows = db.entries("f", f, Some("dive"), 0, 50).unwrap();
let n = db.count_entries("f", f, Some("dive")).unwrap();
assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees");
}
assert_eq!(db.count_entries("f", Filter::All, None).unwrap(), 3);
assert_eq!(db.count_entries("f", Filter::Unread, None).unwrap(), 1);
assert_eq!(db.count_entries("f", Filter::Downloaded, None).unwrap(), 1);
assert_eq!(db.count_entries("f", Filter::Flagged, None).unwrap(), 1);
assert_eq!(db.count_entries("f", Filter::All, Some("dive")).unwrap(), 2);
assert_eq!(db.count_entries("f", Filter::All, Some("NOTES two")).unwrap(), 1,
"search is case-insensitive and covers the description");
}
#[test]
fn pending_takes_the_latest_episodes_first() {
// A cap of 3 must mean the three newest, not the three recorded first.
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO entries (feed_id, guid, published, first_seen) VALUES
('f','old',100,100), ('f','mid',200,200), ('f','new',300,300);
INSERT INTO enclosures (id, feed_id, guid, url, state) VALUES
(1,'f','old','u-old','pending'),
(2,'f','mid','u-mid','pending'),
(3,'f','new','u-new','pending');",
)
.unwrap();
let got: Vec<String> = db.pending("f", 2).unwrap().into_iter().map(|p| p.url).collect();
assert_eq!(got, vec!["u-new", "u-mid"], "newest first, oldest left for later");
}
#[test]
fn a_restart_requeues_interrupted_downloads() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO enclosures (id, feed_id, guid, url, state, path) VALUES
(1,'f','a','u1','downloading',NULL),
(2,'f','b','u2','pending',NULL),
(3,'f','c','u3','downloading','/tmp/already-here'),
(4,'f','d','u4','done','/tmp/x');",
)
.unwrap();
assert_eq!(db.requeue_interrupted().unwrap(), 1, "only the in-flight, fileless one");
let conn = db.conn.lock().unwrap();
let state = |id: i64| -> String {
conn.query_row("SELECT state FROM enclosures WHERE id = ?1", [id], |r| r.get(0)).unwrap()
};
assert_eq!(state(1), "pending");
assert_eq!(state(3), "downloading", "it has a file; leave it alone");
assert_eq!(state(4), "done");
}
#[test] #[test]
fn enclosure_url_is_the_dedupe_key() { fn enclosure_url_is_the_dedupe_key() {
let db = Db::memory().unwrap(); let db = Db::memory().unwrap();

103
src/download.rs
View File

@@ -7,19 +7,56 @@ use tokio::io::AsyncWriteExt;
use crate::config::{Config, Feed as FeedCfg, Organize}; use crate::config::{Config, Feed as FeedCfg, Organize};
/// Characters the original's stringCleaning() stripped, plus the control range and the /// Forbidden characters that were separating words: they become "-" so the words stay
/// trailing dots/spaces it left in. A real length cap is new -- the Python had none. /// apart. The original's stringCleaning() deleted them, turning "Show | Series" into
const FORBIDDEN: &[char] = &['/', '\\', '?', '*', ':', '<', '>', '|', '"', '\'']; /// "Show Series".
const SEPARATORS: &[char] = &['/', '\\', '|', ':'];
/// Forbidden characters that were never separators: they just go.
const STRIPPED: &[char] = &['?', '*', '<', '>', '"', '\''];
/// A real length cap is new -- the Python had none.
const MAX_NAME_BYTES: usize = 255; const MAX_NAME_BYTES: usize = 255;
/// Keeps UTF-8: the original transliterated to ASCII via latin1_to_ascii because 2004 /// Keeps UTF-8: the original transliterated to ASCII via latin1_to_ascii because 2004
/// filesystems demanded it. Ours do not. /// filesystems demanded it. Ours do not.
pub fn sanitize(name: &str) -> String { pub fn sanitize(name: &str) -> String {
let mut out: String = name let mapped: String = name
.chars() .chars()
.filter(|c| !c.is_control() && !FORBIDDEN.contains(c)) .map(|c| if c.is_control() { ' ' } else { c })
.filter(|c| !STRIPPED.contains(c))
.map(|c| if SEPARATORS.contains(&c) { '-' } else { c })
.collect(); .collect();
out = out.trim().trim_matches('.').trim().to_owned();
// Collapse each run of dashes and spaces into one thing. A run containing a dash
// becomes " - " when it also had whitespace ("Show | Series" -> "Show - Series",
// "Ep 12: One" -> "Ep 12 - One") and a bare "-" when it did not ("AC/DC" -> "AC-DC").
// A run of plain whitespace collapses to a single space.
let mut out = String::with_capacity(mapped.len());
let mut chars = mapped.chars().peekable();
while let Some(c) = chars.next() {
if !(c == '-' || c.is_whitespace()) {
out.push(c);
continue;
}
let mut has_dash = c == '-';
let mut has_space = c.is_whitespace();
while let Some(&next) = chars.peek() {
if next == '-' {
has_dash = true;
} else if next.is_whitespace() {
has_space = true;
} else {
break;
}
chars.next();
}
match (has_dash, has_space) {
(true, true) => out.push_str(" - "),
(true, false) => out.push('-'),
_ => out.push(' '),
}
}
// Leading/trailing separators and dots are noise, and a leading "-" trips up CLI tools.
out = out.trim().trim_matches(|c| c == '.' || c == '-').trim().to_owned();
if out.len() > MAX_NAME_BYTES { if out.len() > MAX_NAME_BYTES {
// Truncate on a char boundary, keeping the extension if there is a plausible one. // Truncate on a char boundary, keeping the extension if there is a plausible one.
@@ -231,17 +268,27 @@ fn unique_path(dir: &Path, name: &str) -> PathBuf {
} }
/// Download folder for a feed: per-feed name, or per-day when organize = "date". /// Download folder for a feed: per-feed name, or per-day when organize = "date".
///
/// A folder may name more than one level ("Subscriptions/Some Show") -- feeds from a
/// subscribed OPML nest under it -- so each segment is sanitized separately rather than
/// letting the sanitizer eat the separator.
pub fn folder_for(cfg: &Config, id: &str, feed_cfg: &FeedCfg, title: Option<&str>) -> String { pub fn folder_for(cfg: &Config, id: &str, feed_cfg: &FeedCfg, title: Option<&str>) -> String {
match cfg.general.organize { match cfg.general.organize {
Organize::Date => chrono::Local::now().format("%m-%d-%Y").to_string(), Organize::Date => chrono::Local::now().format("%m-%d-%Y").to_string(),
Organize::Feed => sanitize( Organize::Feed => {
feed_cfg let raw = feed_cfg
.folder .folder
.as_deref() .as_deref()
.or(title) .or(title)
.filter(|s| !s.trim().is_empty()) .filter(|s| !s.trim().is_empty())
.unwrap_or(id), .unwrap_or(id);
), raw.split('/')
.map(str::trim)
.filter(|seg| !seg.is_empty() && *seg != "." && *seg != "..")
.map(sanitize)
.collect::<Vec<_>>()
.join("/")
}
} }
} }
@@ -264,12 +311,26 @@ mod tests {
#[test] #[test]
fn sanitize_strips_path_and_control_characters() { fn sanitize_strips_path_and_control_characters() {
assert_eq!(sanitize("../../etc/passwd"), "etcpasswd"); assert_eq!(sanitize("../../etc/passwd"), "etc-passwd");
assert_eq!(sanitize("Ep 12: The \"Best\" One?"), "Ep 12 The Best One"); assert_eq!(sanitize("Ep 12: The \"Best\" One?"), "Ep 12 - The Best One");
assert_eq!(sanitize("bad\u{0}name\u{7}.mp3"), "badname.mp3"); assert_eq!(sanitize("bad\u{0}name\u{7}.mp3"), "bad name .mp3");
assert_eq!(sanitize(" spaced.mp3 "), "spaced.mp3"); assert_eq!(sanitize(" spaced.mp3 "), "spaced.mp3");
} }
#[test]
fn sanitize_turns_separators_into_dashes() {
// A real Patreon feed title; the pipes are forbidden characters.
assert_eq!(
sanitize("Get in the Trunk | Anthology Series | Delta Green"),
"Get in the Trunk - Anthology Series - Delta Green"
);
assert_eq!(sanitize("Ep 12: The One"), "Ep 12 - The One");
assert_eq!(sanitize("a b"), "a b", "plain whitespace stays whitespace");
assert_eq!(sanitize("AC/DC"), "AC-DC", "no spaces around it, so no spaces added");
assert_eq!(sanitize("well-known.mp3"), "well-known.mp3", "existing dashes survive");
assert_eq!(sanitize("Show -- Thing"), "Show - Thing");
}
#[test] #[test]
fn sanitize_never_yields_an_empty_or_dot_name() { fn sanitize_never_yields_an_empty_or_dot_name() {
assert_eq!(sanitize(""), "download"); assert_eq!(sanitize(""), "download");
@@ -319,6 +380,22 @@ mod tests {
); );
} }
#[test]
fn a_folder_can_nest_without_the_sanitizer_eating_the_separator() {
let mut cfg = Config::default();
cfg.general.download_dir = "/tmp".into();
let mut f = crate::config::Feed {
url: "u".into(), folder: Some("Subscriptions/Some | Show".into()), group: None, media_types: None,
schedule: None, keywords: vec![], allow_explicit: false, auto_download: true,
max_new_per_check: None, username: None, password: None, password_env: None,
};
assert_eq!(folder_for(&cfg, "id", &f, None), "Subscriptions/Some - Show");
// A traversal in a folder name must not climb out of the download directory.
f.folder = Some("../../etc/Show".into());
assert_eq!(folder_for(&cfg, "id", &f, None), "etc/Show");
}
#[test] #[test]
fn keyword_matching_is_or_across_keywords_and_and_within_one() { fn keyword_matching_is_or_across_keywords_and_and_within_one() {
let kws = vec!["deep dive".to_string(), "interview".to_string()]; let kws = vec!["deep dive".to_string(), "interview".to_string()];

337
src/feed.rs
View File

@@ -10,6 +10,7 @@ use crate::config::Feed as FeedCfg;
pub struct ParsedFeed { pub struct ParsedFeed {
pub title: Option<String>, pub title: Option<String>,
pub ttl_mins: Option<u64>, pub ttl_mins: Option<u64>,
pub image: Option<String>,
pub entries: Vec<Entry>, pub entries: Vec<Entry>,
} }
@@ -22,10 +23,16 @@ pub struct Entry {
pub description: Option<String>, pub description: Option<String>,
pub categories: Vec<String>, pub categories: Vec<String>,
pub explicit: bool, pub explicit: bool,
/// Episode artwork; falls back to the feed's in the UI.
pub image: Option<String>,
/// Seconds.
pub duration: Option<i64>,
pub episode: Option<i64>,
pub season: Option<i64>,
pub enclosures: Vec<Enclosure>, pub enclosures: Vec<Enclosure>,
} }
#[derive(Debug, Default, PartialEq)] #[derive(Debug, Default, Clone, PartialEq)]
pub struct Enclosure { pub struct Enclosure {
pub url: String, pub url: String,
pub mime: Option<String>, pub mime: Option<String>,
@@ -80,10 +87,40 @@ pub async fn fetch(
Ok(Fetched::Body { bytes, etag, last_modified }) Ok(Fetched::Body { bytes, etag, last_modified })
} }
/// True when a body is an OPML document rather than a feed.
///
/// The original matched on the URL ending in ".opml" (iPXClass.py:34), which misses an
/// OPML served from a URL without that extension. Sniffing the body catches both.
pub fn is_opml(bytes: &[u8]) -> bool {
let head = &bytes[..bytes.len().min(1024)];
let text = String::from_utf8_lossy(head).to_lowercase();
text.contains("<opml")
}
/// The feeds listed in an OPML document, as (title, xml_url), walking nested folders.
pub fn parse_opml(bytes: &[u8]) -> Result<Vec<(String, String)>> {
let text = String::from_utf8_lossy(bytes);
let doc = opml::OPML::from_str(&text)
.map_err(|e| anyhow!("that does not parse as OPML: {e}"))?;
let mut out = vec![];
crate::collect_outlines(&doc.body.outlines, &mut out);
Ok(out)
}
/// The <head><title> of an OPML document.
pub fn opml_title(bytes: &[u8]) -> Option<String> {
let text = String::from_utf8_lossy(bytes);
let doc = opml::OPML::from_str(&text).ok()?;
doc.head
.and_then(|h| h.title)
.map(|t| t.trim().to_owned())
.filter(|t| !t.is_empty())
}
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`. /// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> { pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
match rss::Channel::read_from(bytes) { match rss::Channel::read_from(bytes) {
Ok(ch) => Ok(from_rss(ch)), Ok(ch) => Ok(from_rss(ch, bytes)),
Err(rss_err) => match atom_syndication::Feed::read_from(bytes) { Err(rss_err) => match atom_syndication::Feed::read_from(bytes) {
Ok(feed) => Ok(from_atom(feed)), Ok(feed) => Ok(from_atom(feed)),
Err(atom_err) => Err(anyhow!("not RSS ({rss_err}) and not Atom ({atom_err})")), Err(atom_err) => Err(anyhow!("not RSS ({rss_err}) and not Atom ({atom_err})")),
@@ -91,7 +128,78 @@ pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
} }
} }
fn from_rss(ch: rss::Channel) -> ParsedFeed { /// Every `<enclosure>` of every `<item>`, in document order.
///
/// The `rss` crate models an item as having at most one enclosure -- which is what RSS 2.0
/// says -- and when a feed carries several it keeps only the *last*, silently losing the
/// rest. Feeds do ship several, so read them from the XML directly.
fn enclosures_by_item(bytes: &[u8]) -> Vec<Vec<Enclosure>> {
use quick_xml::events::Event;
let mut reader = quick_xml::Reader::from_reader(bytes);
reader.config_mut().trim_text(true);
let mut buf = Vec::new();
let mut out: Vec<Vec<Enclosure>> = Vec::new();
let mut current: Option<Vec<Enclosure>> = None;
let read_enclosure = |e: &quick_xml::events::BytesStart| -> Option<Enclosure> {
let (mut url, mut mime, mut length) = (String::new(), None, None);
for attr in e.attributes().flatten() {
// Values arrive escaped: a feed URL's "&" is "&amp;" in the document.
let val = quick_xml::escape::unescape(&attr.value)
.map(|v| v.trim().to_string())
.unwrap_or_default();
match attr.key.local_name().as_ref() {
"url" => url = val,
"type" => mime = Some(val).filter(|v| !v.is_empty()),
"length" => length = val.parse().ok(),
_ => {}
}
}
(!url.is_empty()).then_some(Enclosure { url, mime, length })
};
loop {
match reader.read_event_into(&mut buf) {
Ok(Event::Start(e)) => match e.name().local_name().as_ref() {
"item" => current = Some(Vec::new()),
"enclosure" => {
if let (Some(list), Some(enc)) = (current.as_mut(), read_enclosure(&e)) {
list.push(enc);
}
}
_ => {}
},
Ok(Event::Empty(e)) => match e.name().local_name().as_ref() {
// <item/> with no children still counts, so the indexes stay aligned.
"item" => out.push(Vec::new()),
"enclosure" => {
if let (Some(list), Some(enc)) = (current.as_mut(), read_enclosure(&e)) {
list.push(enc);
}
}
_ => {}
},
Ok(Event::End(e)) => {
if e.name().local_name().as_ref() == "item"
&& let Some(list) = current.take()
{
out.push(list);
}
}
Ok(Event::Eof) | Err(_) => break,
_ => {}
}
buf.clear();
}
if let Some(list) = current.take() {
out.push(list);
}
out
}
fn from_rss(ch: rss::Channel, bytes: &[u8]) -> ParsedFeed {
let per_item = enclosures_by_item(bytes);
let explicit = ch let explicit = ch
.itunes_ext() .itunes_ext()
.and_then(|it| it.explicit()) .and_then(|it| it.explicit())
@@ -100,17 +208,21 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
let entries = ch let entries = ch
.items() .items()
.iter() .iter()
.filter_map(|item| { .enumerate()
let enclosures: Vec<Enclosure> = item .filter_map(|(idx, item)| {
.enclosure() // Straight from the XML, so an item with several keeps all of them. Falls
.into_iter() // back to the parsed one if the scan and the parser disagree on item count.
.map(|e| Enclosure { let enclosures: Vec<Enclosure> = per_item.get(idx).cloned().unwrap_or_else(|| {
url: e.url().trim().to_owned(), item.enclosure()
mime: non_empty(Some(e.mime_type())), .into_iter()
length: e.length().parse().ok(), .map(|e| Enclosure {
}) url: e.url().trim().to_owned(),
.filter(|e| !e.url.is_empty()) mime: non_empty(Some(e.mime_type())),
.collect(); length: e.length().parse().ok(),
})
.filter(|e| !e.url.is_empty())
.collect()
});
let guid = pick_guid( let guid = pick_guid(
item.guid().map(|g| g.value()), item.guid().map(|g| g.value()),
@@ -123,6 +235,7 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
.itunes_ext() .itunes_ext()
.and_then(|it| it.explicit()) .and_then(|it| it.explicit())
.is_some_and(is_yes); .is_some_and(is_yes);
let it = item.itunes_ext();
Some(Entry { Some(Entry {
guid, guid,
@@ -138,6 +251,10 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
.filter(|c| !c.is_empty() && !c.starts_with("http")) .filter(|c| !c.is_empty() && !c.starts_with("http"))
.collect(), .collect(),
explicit: explicit || entry_explicit, explicit: explicit || entry_explicit,
image: item_image(item, &enclosures),
duration: it.and_then(|i| i.duration()).and_then(parse_duration),
episode: it.and_then(|i| i.episode()).and_then(|e| e.trim().parse().ok()),
season: it.and_then(|i| i.season()).and_then(|e| e.trim().parse().ok()),
enclosures, enclosures,
}) })
}) })
@@ -146,6 +263,12 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
ParsedFeed { ParsedFeed {
title: non_empty(Some(ch.title())), title: non_empty(Some(ch.title())),
ttl_mins: ch.ttl().and_then(|t| t.trim().parse().ok()), ttl_mins: ch.ttl().and_then(|t| t.trim().parse().ok()),
// itunes:image is the square artwork; <image><url> is the older, often smaller one.
image: ch
.itunes_ext()
.and_then(|i| i.image())
.map(str::to_owned)
.or_else(|| ch.image().map(|i| i.url().to_owned())),
entries, entries,
} }
} }
@@ -193,6 +316,10 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
.map(str::to_owned), .map(str::to_owned),
categories: e.categories().iter().map(|c| c.term().to_owned()).collect(), categories: e.categories().iter().map(|c| c.term().to_owned()).collect(),
explicit: false, explicit: false,
image: None,
duration: None,
episode: None,
season: None,
enclosures, enclosures,
}) })
}) })
@@ -201,6 +328,7 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
ParsedFeed { ParsedFeed {
title: non_empty(Some(feed.title().as_str())), title: non_empty(Some(feed.title().as_str())),
ttl_mins: None, ttl_mins: None,
image: feed.logo().or_else(|| feed.icon()).map(str::to_owned),
entries, entries,
} }
} }
@@ -230,6 +358,55 @@ fn non_empty(s: Option<&str>) -> Option<String> {
s.map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned) s.map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned)
} }
/// The picture to show beside an item, in order of how deliberate it is:
/// `itunes:image`, then Media RSS `media:thumbnail`, then a `media:content` that is an
/// image, and finally an image enclosure -- which is how a blog's article picture arrives
/// (Substack puts it there), so those entries get artwork rather than a blank square.
fn item_image(item: &rss::Item, enclosures: &[Enclosure]) -> Option<String> {
if let Some(url) = item.itunes_ext().and_then(|i| i.image()) {
return non_empty(Some(url));
}
let media = item.extensions().get("media");
let attr = |name: &str, want_image: bool| -> Option<String> {
media?.get(name)?.iter().find_map(|e| {
if want_image {
// media:content carries anything; only take it when it says it is a picture.
let is_image = e.attrs.get("type").is_some_and(|t| t.starts_with("image/"))
|| e.attrs.get("medium").is_some_and(|m| m == "image");
if !is_image {
return None;
}
}
non_empty(e.attrs.get("url").map(String::as_str))
})
};
attr("thumbnail", false)
.or_else(|| attr("content", true))
.or_else(|| {
enclosures
.iter()
.find(|e| e.mime.as_deref().is_some_and(|m| m.starts_with("image/")))
.map(|e| e.url.clone())
})
}
/// itunes:duration is either plain seconds ("5649") or a clock ("1:34:09", "23:45").
fn parse_duration(s: &str) -> Option<i64> {
let s = s.trim();
if s.is_empty() {
return None;
}
if !s.contains(':') {
return s.parse().ok().filter(|n| *n > 0);
}
let mut total: i64 = 0;
for part in s.split(':') {
total = total * 60 + part.trim().parse::<i64>().ok()?;
}
Some(total).filter(|n| *n > 0)
}
/// RSS pubDate is RFC 2822; some feeds ship RFC 3339 instead. /// RSS pubDate is RFC 2822; some feeds ship RFC 3339 instead.
fn parse_date(s: &str) -> Option<i64> { fn parse_date(s: &str) -> Option<i64> {
let s = s.trim(); let s = s.trim();
@@ -312,6 +489,137 @@ mod tests {
); );
} }
#[test]
fn the_rss_title_always_wins_and_episode_numbers_stay_metadata() {
// Some feeds set a different itunes:title. The displayed title is always the RSS
// <title>, verbatim -- separators and all -- and season/episode are stored
// alongside it rather than folded into it.
let xml = br#"<?xml version="1.0"?>
<rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
<channel><title>Show</title><link>https://x</link><description>d</description>
<item>
<title>Music from a Darkened Room | Session Zero</title>
<itunes:title>Session Zero</itunes:title>
<guid>sz</guid>
<itunes:season>8</itunes:season>
<itunes:duration>6720</itunes:duration>
<enclosure url="https://x/sz.mp3" length="1" type="audio/mpeg"/>
</item>
<item>
<title>Music from a Darkened Room Part 1 | Murphy's Drawer</title>
<guid>p1</guid>
<itunes:season>8</itunes:season><itunes:episode>1</itunes:episode>
<enclosure url="https://x/p1.mp3" length="1" type="audio/mpeg"/>
</item>
</channel></rss>"#;
let feed = parse(xml).unwrap();
let sz = &feed.entries[0];
assert_eq!(
sz.title.as_deref(),
Some("Music from a Darkened Room | Session Zero"),
"itunes:title must not override the RSS title"
);
assert_eq!(sz.season, Some(8));
assert_eq!(sz.episode, None, "a missing episode number stays missing");
assert_eq!(sz.duration, Some(6720));
let p1 = &feed.entries[1];
assert_eq!(p1.title.as_deref(), Some("Music from a Darkened Room Part 1 | Murphy's Drawer"));
assert_eq!((p1.season, p1.episode), (Some(8), Some(1)));
}
#[test]
fn opml_is_recognised_and_its_feeds_listed() {
let xml = br#"<opml version="2.0"><head><title>My Subscriptions</title></head><body>
<outline text="Folder">
<outline type="rss" text="Alpha" xmlUrl="https://a.example/rss"/>
<outline type="rss" text="Beta" xmlUrl="https://b.example/rss"/>
</outline>
<outline text="Not a feed"/>
</body></opml>"#;
assert!(is_opml(xml));
assert_eq!(opml_title(xml).as_deref(), Some("My Subscriptions"));
let feeds = parse_opml(xml).unwrap();
assert_eq!(feeds.len(), 2, "nested folders are walked, non-feed outlines skipped");
assert_eq!(feeds[0], ("Alpha".into(), "https://a.example/rss".into()));
// A feed must never be mistaken for a subscription list.
assert!(!is_opml(include_bytes!("../tests/data/rss2.xml")));
assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
}
#[test]
fn an_item_may_carry_several_enclosures() {
// The rss crate keeps only one per item -- the last -- so these come from the XML.
let xml = br#"<?xml version="1.0"?>
<rss version="2.0"><channel><title>M</title><link>https://x</link><description>d</description>
<item><title>Two files</title><guid>m1</guid>
<enclosure url="https://x/a.mp3?v=1&amp;t=2" length="111" type="audio/mpeg"/>
<enclosure url="https://x/b.mp4" length="222" type="video/mp4"/>
</item>
<item><title>One file</title><guid>m2</guid>
<enclosure url="https://x/c.mp3" length="333" type="audio/mpeg"/></item>
<item><title>None</title><guid>m3</guid></item>
</channel></rss>"#;
let f = parse(xml).unwrap();
assert_eq!(f.entries.len(), 3);
let two = &f.entries[0].enclosures;
assert_eq!(two.len(), 2, "both enclosures survive");
assert_eq!(
two[0].url, "https://x/a.mp3?v=1&t=2",
"document order, and the escaped ampersand is decoded"
);
assert_eq!(two[0].length, Some(111));
assert_eq!(two[1].url, "https://x/b.mp4");
assert_eq!(two[1].mime.as_deref(), Some("video/mp4"));
assert_eq!(f.entries[1].enclosures.len(), 1);
assert_eq!(f.entries[2].enclosures.len(), 0, "an item may have none");
}
#[test]
fn an_items_picture_comes_from_the_most_deliberate_source() {
let xml = br#"<?xml version="1.0"?>
<rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
xmlns:media="http://search.yahoo.com/mrss/">
<channel><title>P</title><link>https://x</link><description>d</description>
<item><title>Has itunes</title><guid>a</guid>
<itunes:image href="https://x/itunes.jpg"/>
<media:thumbnail url="https://x/thumb.jpg"/>
<enclosure url="https://x/a.jpg" length="1" type="image/jpeg"/></item>
<item><title>Has thumbnail</title><guid>b</guid>
<media:thumbnail url="https://x/thumb.jpg"/>
<enclosure url="https://x/b.jpg" length="1" type="image/jpeg"/></item>
<item><title>Has media content</title><guid>c</guid>
<media:content url="https://x/pic.jpg" type="image/jpeg"/>
<media:content url="https://x/clip.mp4" type="video/mp4"/></item>
<item><title>Only an image enclosure</title><guid>d</guid>
<enclosure url="https://x/d.jpg" length="1" type="image/jpeg"/></item>
<item><title>Audio only</title><guid>e</guid>
<enclosure url="https://x/e.mp3" length="1" type="audio/mpeg"/></item>
</channel></rss>"#;
let f = parse(xml).unwrap();
let img = |i: usize| f.entries[i].image.as_deref();
assert_eq!(img(0), Some("https://x/itunes.jpg"), "itunes:image wins");
assert_eq!(img(1), Some("https://x/thumb.jpg"), "then media:thumbnail");
assert_eq!(img(2), Some("https://x/pic.jpg"), "media:content, and only the image one");
assert_eq!(img(3), Some("https://x/d.jpg"), "a blog's article picture arrives as an enclosure");
assert_eq!(img(4), None, "audio is not a picture");
}
#[test]
fn durations_parse_from_seconds_or_a_clock() {
assert_eq!(parse_duration("5649"), Some(5649));
assert_eq!(parse_duration("23:45"), Some(1425));
assert_eq!(parse_duration("1:34:09"), Some(5649));
assert_eq!(parse_duration("0"), None, "zero is not a duration");
assert_eq!(parse_duration(""), None);
assert_eq!(parse_duration("garbage"), None);
}
#[test] #[test]
fn rejects_html_masquerading_as_a_feed() { fn rejects_html_masquerading_as_a_feed() {
assert!(parse(b"<html><body>nope</body></html>").is_err()); assert!(parse(b"<html><body>nope</body></html>").is_err());
@@ -324,3 +632,4 @@ mod tests {
assert_eq!(pick_guid(None, None, None, None), None); assert_eq!(pick_guid(None, None, None, None), None);
} }
} }

65
src/ipc.rs
View File

@@ -17,14 +17,17 @@ pub enum Event {
FeedError { feed: String, msg: String }, FeedError { feed: String, msg: String },
Progress { Progress {
feed: String, feed: String,
/// Which enclosure this is about. Without it a UI cannot tell one download's
/// progress from another's and ends up animating every pending row.
enclosure: i64,
url: String, url: String,
file: String, file: String,
done: u64, done: u64,
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
total: Option<u64>, total: Option<u64>,
}, },
DownloadDone { feed: String, url: String, path: String, bytes: u64 }, DownloadDone { feed: String, enclosure: i64, url: String, path: String, bytes: u64 },
DownloadError { feed: String, url: String, msg: String }, DownloadError { feed: String, enclosure: i64, url: String, msg: String },
TorrentDeferred { feed: String, url: String }, TorrentDeferred { feed: String, url: String },
Reaped { path: String, bytes: u64 }, Reaped { path: String, bytes: u64 },
/// Terminal: a client that asked for work stops reading here. /// Terminal: a client that asked for work stops reading here.
@@ -70,9 +73,9 @@ impl Event {
} }
Event::Error { msg } => format!("error: {msg}"), Event::Error { msg } => format!("error: {msg}"),
// Noise in a terminal; a UI still gets them on the socket. // Noise in a terminal; a UI still gets them on the socket.
Event::FeedStart { .. } | Event::TorrentDeferred { .. } | Event::ScanDone { .. } => { Event::FeedStart { feed } => format!("{feed}: checking"),
return None; Event::TorrentDeferred { feed, .. } => format!("{feed}: torrent deferred"),
} Event::ScanDone { feeds } => format!("scan complete, {feeds} feed(s)"),
}) })
} }
} }
@@ -90,6 +93,12 @@ pub enum Command {
#[serde(default)] #[serde(default)]
dry_run: bool, dry_run: bool,
}, },
/// Fetch one specific enclosure now, ignoring max_new_per_check and the queue order.
/// A scan cannot express "this one, now": it takes the lowest-id pending rows up to
/// the per-scan cap, so an explicit request has to bypass both.
Download {
enclosure: i64,
},
Status, Status,
} }
@@ -110,7 +119,45 @@ impl Emitter {
} }
pub fn emit(&self, e: Event) { pub fn emit(&self, e: Event) {
// Also log it. Scans and downloads travel as events, not tracing calls, so
// without this the log view shows only startup and HTTP lines and none of the
// work the daemon is actually doing. Progress goes to debug: it fires on every
// whole percent and would otherwise crowd everything else out of the buffer.
// Level by how much it matters. With 80-odd feeds in an OPML subscription, one
// line per feed per tick for "not due yet" would push everything worth reading
// out of the buffer within a few minutes.
let routine = match &e {
Event::Progress { .. } | Event::FeedSkip { .. } | Event::FeedStart { .. } => true,
Event::FeedDone { new, downloaded, failed, torrents, .. } => {
*new == 0 && *downloaded == 0 && *failed == 0 && *torrents == 0
}
_ => false,
};
let bad = matches!(
&e,
Event::FeedError { .. } | Event::DownloadError { .. } | Event::Error { .. }
);
if let Some(line) = e.human() {
let line = line.trim();
if bad {
tracing::warn!(target: "ipx::scan", "{line}");
} else if routine {
tracing::debug!(target: "ipx::scan", "{line}");
} else {
tracing::info!(target: "ipx::scan", "{line}");
}
}
if let Some(tx) = &self.tx { if let Some(tx) = &self.tx {
// The outbound half of the protocol, as it goes on the wire. Progress is the
// high-volume one, so it sits at debug.
if let Ok(json) = serde_json::to_string(&e) {
if matches!(e, Event::Progress { .. }) {
tracing::debug!(target: "ipx::io", "<- {json}");
} else {
tracing::info!(target: "ipx::io", "<- {json}");
}
}
// An error here only means nobody is listening yet. // An error here only means nobody is listening yet.
let _ = tx.send(e.clone()); let _ = tx.send(e.clone());
} }
@@ -234,6 +281,11 @@ mod tests {
let got: Command = serde_json::from_str(r#"{"cmd":"reap","dry_run":true}"#).unwrap(); let got: Command = serde_json::from_str(r#"{"cmd":"reap","dry_run":true}"#).unwrap();
assert!(matches!(got, Command::Reap { dry_run: true })); assert!(matches!(got, Command::Reap { dry_run: true }));
// "Download this one now" is its own command precisely because a scan cannot
// express it: a scan takes the lowest-id pending rows up to max_new_per_check.
let got: Command = serde_json::from_str(r#"{"cmd":"download","enclosure":11}"#).unwrap();
assert!(matches!(got, Command::Download { enclosure: 11 }));
assert!(serde_json::from_str::<Command>(r#"{"cmd":"nope"}"#).is_err()); assert!(serde_json::from_str::<Command>(r#"{"cmd":"nope"}"#).is_err());
} }
@@ -241,6 +293,7 @@ mod tests {
fn events_serialise_to_the_documented_shape() { fn events_serialise_to_the_documented_shape() {
let ev = Event::Progress { let ev = Event::Progress {
feed: "atp".into(), feed: "atp".into(),
enclosure: 42,
url: "https://x/ep.mp3".into(), url: "https://x/ep.mp3".into(),
file: "ep.mp3".into(), file: "ep.mp3".into(),
done: 10_485_760, done: 10_485_760,
@@ -249,10 +302,12 @@ mod tests {
let json = serde_json::to_string(&ev).unwrap(); let json = serde_json::to_string(&ev).unwrap();
assert!(json.starts_with(r#"{"ev":"progress""#), "got {json}"); assert!(json.starts_with(r#"{"ev":"progress""#), "got {json}");
assert!(json.contains(r#""done":10485760"#)); assert!(json.contains(r#""done":10485760"#));
assert!(json.contains(r#""enclosure":42"#), "a UI needs this to target one row");
// total is omitted rather than null when the server sent no length. // total is omitted rather than null when the server sent no length.
let ev = Event::Progress { let ev = Event::Progress {
feed: "a".into(), feed: "a".into(),
enclosure: 1,
url: "u".into(), url: "u".into(),
file: "f".into(), file: "f".into(),
done: 1, done: 1,

154
src/logbuf.rs Normal file
View File

@@ -0,0 +1,154 @@
//! In-process ring buffer of log lines, so the UI can show what the daemon is doing.
//!
//! Tailing a file would not survive Docker, where logs go to stdout and there is no file
//! to read. Capturing inside the tracing pipeline works the same either way.
use std::collections::VecDeque;
use std::sync::{LazyLock, Mutex};
use tracing::field::{Field, Visit};
use tracing_subscriber::Layer;
use tracing_subscriber::layer::Context;
/// Kept small enough to be cheap to hold and to serialise in one response.
const CAPACITY: usize = 5000;
#[derive(Clone, Debug, serde::Serialize)]
pub struct LogLine {
/// Monotonic, so a client can ask for "everything after N" without duplicates.
pub seq: u64,
pub ts: i64,
pub level: String,
pub target: String,
pub msg: String,
}
struct Ring {
lines: VecDeque<LogLine>,
next_seq: u64,
}
static BUF: LazyLock<Mutex<Ring>> = LazyLock::new(|| {
Mutex::new(Ring { lines: VecDeque::with_capacity(CAPACITY), next_seq: 1 })
});
pub fn push(level: &str, target: &str, msg: String) {
let mut ring = match BUF.lock() {
Ok(r) => r,
Err(p) => p.into_inner(), // a poisoned log buffer must not take the process down
};
let seq = ring.next_seq;
ring.next_seq += 1;
if ring.lines.len() == CAPACITY {
ring.lines.pop_front();
}
ring.lines.push_back(LogLine {
seq,
ts: crate::db::now(),
level: level.to_owned(),
target: target.to_owned(),
msg,
});
}
/// Lines newer than `after`, oldest first, plus the highest seq now held.
pub fn since(after: u64, limit: usize) -> (Vec<LogLine>, u64) {
let ring = match BUF.lock() {
Ok(r) => r,
Err(p) => p.into_inner(),
};
let latest = ring.next_seq.saturating_sub(1);
let mut out: Vec<LogLine> = ring
.lines
.iter()
.filter(|l| l.seq > after)
.cloned()
.collect();
// On a first load (after = 0) the tail is what matters, not the head.
if out.len() > limit {
out.drain(..out.len() - limit);
}
(out, latest)
}
/// A tracing layer that mirrors every event into the ring.
pub struct RingLayer;
impl<S: tracing::Subscriber> Layer<S> for RingLayer {
fn on_event(&self, event: &tracing::Event<'_>, _ctx: Context<'_, S>) {
let mut v = Collect::default();
event.record(&mut v);
let meta = event.metadata();
push(meta.level().as_str(), meta.target(), v.finish());
}
}
#[derive(Default)]
struct Collect {
message: String,
fields: Vec<String>,
}
impl Collect {
fn finish(self) -> String {
if self.fields.is_empty() {
self.message
} else if self.message.is_empty() {
self.fields.join(" ")
} else {
format!("{} {}", self.message, self.fields.join(" "))
}
}
fn add(&mut self, field: &Field, value: String) {
if field.name() == "message" {
self.message = value;
} else {
self.fields.push(format!("{}={}", field.name(), value));
}
}
}
impl Visit for Collect {
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
self.add(field, format!("{value:?}"));
}
fn record_str(&mut self, field: &Field, value: &str) {
self.add(field, value.to_owned());
}
fn record_i64(&mut self, field: &Field, value: i64) {
self.add(field, value.to_string());
}
fn record_u64(&mut self, field: &Field, value: u64) {
self.add(field, value.to_string());
}
fn record_bool(&mut self, field: &Field, value: bool) {
self.add(field, value.to_string());
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_ring_drops_oldest_and_keeps_sequence_stable() {
for i in 0..(CAPACITY + 50) {
push("INFO", "t", format!("line {i}"));
}
let (all, latest) = since(0, CAPACITY * 2);
assert_eq!(all.len(), CAPACITY, "bounded");
assert!(latest >= (CAPACITY + 50) as u64);
assert!(
all.first().unwrap().seq < all.last().unwrap().seq,
"oldest first"
);
// "everything after the last one I saw" must return nothing new.
let (none, _) = since(latest, 100);
assert!(none.is_empty());
// A first load takes the tail, not the head.
let (tail, _) = since(0, 5);
assert_eq!(tail.len(), 5);
assert_eq!(tail.last().unwrap().seq, latest);
}
}

871
src/main.rs
View File

File diff suppressed because it is too large Load Diff

1089
src/web.rs Normal file
View File

File diff suppressed because it is too large Load Diff

105
tests/page-smoke.js Normal file
View File

@@ -0,0 +1,105 @@
// Executes web/index.html's script against a stub DOM and fails on anything thrown.
//
// This exists because a ReferenceError at load once blanked the whole UI: a patch
// anchored on a function that no longer existed, so `prefsModal` was referenced but
// never defined. `node --check` passes that happily -- it is a parse, not a run --
// and every server-side test passed too, because the server was fine.
//
// node tests/page-smoke.js
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const html = fs.readFileSync(path.join(__dirname, '..', 'web', 'index.html'), 'utf8');
const script = html.split('<script>')[1].split('</script>')[0];
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
const missing = [];
const el = (name) => new Proxy({ style: {}, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
replaceWith(){}, querySelector(){ return el('nested'); }, querySelectorAll(){ return []; },
play(){ return Promise.resolve(); }, pause(){}, closest(){ return null; } },
{ get: (t, k) => k in t ? t[k] : undefined, set: (t, k, v) => (t[k] = v, true) });
const document = {
querySelector(sel) {
if (sel.startsWith('#') && !ids.has(sel.slice(1))) { missing.push(sel); return null; }
return el(sel);
},
querySelectorAll: () => [],
createElement: () => el('created'),
addEventListener(){}, body: el('body'),
documentElement: { dataset: {} },
};
const ctx = {
document, console,
window: { isSecureContext: false, addEventListener(){} },
localStorage: { getItem: () => null, setItem(){}, removeItem(){} },
navigator: { clipboard: undefined, sendBeacon(){}, mediaSession: undefined },
fetch: (url) => Promise.resolve({
ok: true, status: 200, text: () => Promise.resolve(''),
json: () => Promise.resolve(
String(url).includes('/api/settings')
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
: []),
}),
EventSource: function () { this.close = () => {}; },
MediaMetadata: function () {},
Blob: function () {},
setTimeout, clearTimeout, setInterval, clearInterval,
confirm: () => false, prompt: () => null, alert(){},
Date, Math, JSON, Object, Array, String, Number, Promise, Error, FormData: function(){},
URLSearchParams, encodeURIComponent, decodeURIComponent, parseInt, parseFloat, isNaN,
};
ctx.globalThis = ctx;
ctx.window.location = { href: '' };
try {
vm.createContext(ctx);
vm.runInContext(script, ctx, { filename: 'index.html<script>', timeout: 5000 });
} catch (e) {
console.error('FAIL: the page script threw while loading\n ' + e.stack.split('\n').slice(0, 3).join('\n '));
process.exit(1);
}
// The modals are built on demand, so a load-time check never reaches them. Drive the
// ones that construct markup from live data, which is where a bad field reference hides.
const feed = {
id: 'f', url: 'https://x/rss', title: 'A Feed', image: null, folder: null,
keywords: ['a'], allow_explicit: false, auto_download: true, max_new_per_check: 3,
schedule: 'every 6h', schedule_mins: 360, every_mins: 360,
last_checked: 1, next_check: 2, entries: 1, downloaded: 0, unread: 1, last_error: null,
};
const drive = [
['settingsModal', () => ctx.settingsModal(feed)],
['settingsModal (no override)', () => ctx.settingsModal({ ...feed, schedule: null, schedule_mins: null })],
['downloadLatestModal', () => ctx.downloadLatestModal(feed)],
['removeFeed', () => ctx.removeFeed(feed)],
['prefsModal', () => ctx.prefsModal()],
['logsModal', () => ctx.logsModal()],
// `const S` is not reachable from here: top-level const/let do not become properties
// of a vm context the way var and function declarations do.
['renderGroup', () => ctx.renderGroup(feed, [{ ...feed, id: 'child', group: 'f', orphaned: true }])],
];
for (const [name, fn] of drive) {
try {
const r = fn();
if (r && typeof r.catch === 'function') r.catch(e => {
console.error(`FAIL: ${name} rejected: ${e.message}`); process.exit(1);
});
} catch (e) {
console.error(`FAIL: ${name} threw: ${e.message}`);
process.exit(1);
}
}
if (missing.length) {
console.error('FAIL: handlers wired to elements that do not exist: ' + [...new Set(missing)].join(', '));
process.exit(1);
}
console.log('OK: page script loads clean, every selector it wires at load exists');
// logsModal arms a poll timer; without this the pending interval keeps node alive.
process.exit(0);

258
tests/ui/app.spec.js Normal file
View File

@@ -0,0 +1,258 @@
const { test, expect } = require('@playwright/test');
const { TOKEN } = require('./global-setup');
// The token sets a cookie, so every test starts by presenting it once.
test.beforeEach(async ({ page }) => {
await page.goto(`/?token=${TOKEN}`);
await expect(page.locator('#feedlist')).toBeVisible();
});
test('the page loads and lists the configured feeds', async ({ page }) => {
// Regression: a ReferenceError in the script left the shell rendered and the sidebar
// empty, with every handler below the error dead. Server-side checks all passed.
// Three top-level feeds in the fixture config; the OPML's child is inside a closed folder.
await expect(page.locator('.feed')).toHaveCount(4, { timeout: 15_000 });
await expect(page.getByText('Test Show')).toBeVisible();
const errors = [];
page.on('pageerror', e => errors.push(e.message));
await page.reload();
await expect(page.locator('.feed').first()).toBeVisible();
expect(errors, 'the page script must not throw at load').toEqual([]);
});
test('the theme toggle actually changes the theme', async ({ page }) => {
// Regression: this button was wired after a line that threw, so it did nothing.
const before = await page.evaluate(() => document.documentElement.dataset.theme || 'system');
await page.locator('#theme').click();
await expect
.poll(() => page.evaluate(() => document.documentElement.dataset.theme))
.not.toBe(before);
});
test('settings opens and saves the global schedule', async ({ page }) => {
await page.locator('#prefs').click();
await expect(page.locator('#modal.on')).toBeVisible();
await expect(page.locator('#gnum')).toBeVisible();
await page.locator('#gnum').fill('4');
await page.locator('#gunit').selectOption('h');
await page.locator('#gsave').click();
await expect(page.locator('#modal.on')).toBeHidden();
// It must survive a reload, i.e. actually reach the config.
await page.locator('#prefs').click();
await expect(page.locator('#gnum')).toHaveValue('4');
await expect(page.locator('#gunit')).toHaveValue('h');
});
test('episodes show with their metadata, and the text opens below', async ({ page }) => {
await page.getByText('Test Show').click();
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
await expect(page.getByText('First Episode')).toBeVisible();
// Newest first, so target the episode by name rather than by position.
const first = page.locator('.ep', { hasText: 'First Episode' });
await expect(first).toContainText('S1E1');
await expect(first).toContainText('30:30'); // itunes:duration 1830
await expect(page.locator('.ep', { hasText: 'Second Episode' })).toContainText('15:00');
// Selecting an item shows its text in the pane below, not inline in the row.
await first.click();
await expect(first).toHaveClass(/sel/);
await expect(page.locator('#detail')).toContainText('Show notes for the first one');
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
});
test('the three panes are there and the item text lands in the bottom one', async ({ page }) => {
await page.getByText('Test Show').click();
await expect(page.locator('#list')).toBeVisible();
await expect(page.locator('#grab')).toBeVisible(); // the draggable divider
await expect(page.locator('#detail')).toContainText('Pick an item');
await page.locator('.ep', { hasText: 'First Episode' }).click();
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
// The enclosure travels with the item, into the same pane.
await expect(page.locator('#detail .encbox')).toHaveCount(1);
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
// its chip rather than assuming which episode the daemon happened to fetch.
const downloaded = page.locator('.ep', { hasText: 'downloaded' }).first();
await downloaded.click();
await expect(page.locator('#detail audio')).toBeVisible();
await expect(page.locator('#detail .encbox .btn', { hasText: 'Save' })).toBeVisible();
// Selecting another item replaces the pane rather than stacking.
await page.locator('.ep', { hasText: 'First Episode' }).click();
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
await expect(page.locator('#detail audio')).toHaveCount(0);
});
test('a downloaded file that is not audio gets no player', async ({ page }) => {
// Regression: anything with a file got an <audio> element and a play button, so a blog's
// header image rendered as a broken player.
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
const row = page.locator('.ep', { hasText: 'An Article' });
await expect(row).toBeVisible({ timeout: 20_000 });
await expect(row.locator('[data-a="play"]')).toHaveCount(0);
await row.click();
await expect(page.locator('#detail .dt')).toHaveText('An Article');
await expect(page.locator('#detail audio')).toHaveCount(0);
await expect(page.locator('#detail .encbox')).toContainText('image');
await expect(page.locator('#detail .encbox')).toContainText('downloaded');
// Still offered as a file, just not as an episode: viewable and keepable.
await expect(page.locator('#detail .btn', { hasText: 'Save' })).toBeVisible();
const view = page.locator('#detail a', { hasText: 'View' });
await expect(view).toHaveAttribute('target', '_blank');
await expect(view).toHaveAttribute('rel', /noopener/);
await expect(view).toHaveAttribute('href', /\/media\/\d+/);
});
test('an item with several enclosures lists them all', async ({ page }) => {
await page.locator('.feed', { hasText: 'Multi Show' }).click();
const row = page.locator('.ep', { hasText: 'Two Files' });
await expect(row).toBeVisible({ timeout: 20_000 });
// The row says there is more than one without listing them.
await expect(row).toContainText('+1 more file');
await row.click();
// The pane below lists every one: the audio and the image.
await expect(page.locator('#detail .encbox')).toHaveCount(2);
await expect(page.locator('#detail .encbox').nth(1)).toContainText('image');
});
test('the filter tabs change what is listed', async ({ page }) => {
await page.getByText('Test Show').click();
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
const all = await page.locator('.ep').count(); // All is the default tab
await expect(page.locator('#count')).toContainText('item');
await page.locator('.tabs button', { hasText: 'Unread' }).first().click();
expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all);
await page.locator('.tabs button', { hasText: 'Flagged' }).first().click();
await expect(page.locator('#count')).toContainText('0 items');
});
test('a feed URL is editable and has a copy button', async ({ page }) => {
await page.getByText('Test Show').click();
await page.locator('.btn', { hasText: 'Settings' }).first().click();
await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
await expect(page.locator('#scopy')).toBeVisible();
// navigator.clipboard is absent over plain http, so the button must not throw.
const errors = [];
page.on('pageerror', e => errors.push(e.message));
await page.locator('#scopy').click();
await expect(page.locator('#scopy')).toHaveText(/Copied|Failed/);
expect(errors).toEqual([]);
});
test('the log view has tabs and shows daemon traffic', async ({ page }) => {
await page.locator('#logs').click();
await expect(page.locator('#logbox')).toBeVisible();
await expect(page.locator('#logtabs button')).toHaveCount(4);
// Generate traffic, then check the Daemon I/O tab shows both directions.
await page.locator('#logtabs button', { hasText: 'Daemon I/O' }).click();
await page.evaluate(() =>
fetch('/api/fetch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ force: true }),
}));
await expect(page.locator('#logbox .l').first()).toBeVisible({ timeout: 15_000 });
await expect(page.locator('#logbox')).toContainText('"cmd":"fetch"', { timeout: 15_000 });
await expect(page.locator('#logbox')).toContainText('"ev":', { timeout: 15_000 });
});
test('an OPML subscription is a collapsible folder', async ({ page }) => {
// Read the subscription so its feeds exist.
await page.evaluate(() =>
fetch('/api/fetch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
}));
// Every row reserves the chevron slot for alignment; only a folder's is clickable.
const chev = page.locator('.feed.group .chev');
await expect(chev).toBeVisible({ timeout: 20_000 });
// Closed by default: the child is not listed until the folder is opened.
const before = await page.locator('.feed').count();
await chev.click();
await expect(page.locator('.feed')).toHaveCount(before + 1);
// Scoped to the sidebar: the name also appears as the page heading once selected.
await expect(page.locator('#feedlist').getByText('Grouped Show')).toBeVisible();
// The subscription's own page lists what is inside it.
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
await expect(page.locator('.childrow')).toHaveCount(1);
});
test('marking an OPML subscription read covers the feeds inside it', async ({ page }) => {
await page.evaluate(() =>
fetch('/api/fetch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
}));
// The folder's own row has no entries, so anything it marks read came from its child.
const folder = page.locator('.feed', { hasText: 'Test Subscriptions' }).first();
await expect(folder).toBeVisible({ timeout: 20_000 });
await expect(folder.locator('.badge')).not.toHaveText('0');
await folder.click();
await page.locator('#content .acts button', { hasText: 'Mark all read' }).click();
await expect(folder.locator('.badge')).toHaveText('0');
});
test.describe('on a phone', () => {
test.use({ viewport: { width: 390, height: 844 } });
test('the feed list is reachable and an item reads full screen', async ({ page }) => {
// The burger used to live in the player bar, which is hidden until something plays --
// leaving no way to reach the feeds at all.
await expect(page.locator('#burger')).toBeVisible();
await expect(page.locator('#player')).not.toBeVisible();
await page.locator('#burger').click();
await page.locator('.feed', { hasText: 'Test Show' }).first().click();
await expect(page.locator('#sidebar')).not.toHaveClass(/open/);
// Nothing may push the page sideways at this width.
const over = await page.evaluate(() =>
document.documentElement.scrollWidth - window.innerWidth);
expect(over).toBeLessThanOrEqual(0);
// One pane at a time: the list, then the item over it, then back.
await expect(page.locator('#detail')).not.toBeVisible();
await page.locator('.ep').first().click();
await expect(page.locator('#detail')).toBeVisible();
await page.locator('#dback').click();
await expect(page.locator('#detail')).not.toBeVisible();
});
});
test('opening an item marks it read, and the toggle flips it back', async ({ page }) => {
const errors = [];
page.on('pageerror', e => errors.push(e.message));
await page.getByText('Test Show').click();
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
await expect(row()).toBeVisible({ timeout: 20_000 });
// Another test may have opened this item already, so start from a known state: the
// toggle in the text below flips it back -- which used to recurse until the stack blew.
await row().click();
await page.locator('#detail button', { hasText: 'Mark unread' }).click();
await expect(row()).not.toHaveClass(/read/);
await expect(page.locator('#detail button', { hasText: 'Mark read' })).toBeVisible();
// Opening it is reading it.
await row().click();
await expect(row()).toHaveClass(/read/);
expect(errors).toEqual([]);
});

BIN
tests/ui/fixtures/art.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.0 KiB

BIN
tests/ui/fixtures/art2.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.0 KiB

BIN
tests/ui/fixtures/ep1.mp3 Normal file
View File

Binary file not shown.

BIN
tests/ui/fixtures/ep2.mp3 Normal file
View File

Binary file not shown.

9
tests/ui/fixtures/multi.xml Normal file
View File

@@ -0,0 +1,9 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Multi Show</title><link>http://127.0.0.1:8792/</link>
<description>An item with more than one file.</description>
<item><title>Two Files</title><guid>mu-1</guid>
<pubDate>Mon, 01 Sep 2026 10:00:00 +0000</pubDate>
<description>Audio and a picture.</description>
<enclosure url="http://127.0.0.1:8792/ep2.mp3" length="40000" type="audio/mpeg"/>
<enclosure url="http://127.0.0.1:8792/art2.jpg" length="3020" type="image/jpeg"/>
</item></channel></rss>

5
tests/ui/fixtures/other.xml Normal file
View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Grouped Show</title><link>http://127.0.0.1:8792/</link>
<description>Inside the OPML.</description>
<item><title>Grouped Ep</title><guid>g-1</guid><description>x</description></item>
</channel></rss>

8
tests/ui/fixtures/pics.xml Normal file
View File

@@ -0,0 +1,8 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Picture Blog</title><link>http://127.0.0.1:8792/</link>
<description>A text blog whose entries carry a header image, as Substack does.</description>
<item><title>An Article</title><guid>pic-1</guid>
<pubDate>Mon, 01 Sep 2026 10:00:00 +0000</pubDate>
<description>&lt;p&gt;Words, not audio.&lt;/p&gt;</description>
<enclosure url="http://127.0.0.1:8792/art.jpg" length="3020" type="image/jpeg"/></item>
</channel></rss>

19
tests/ui/fixtures/serve.js Normal file
View File

@@ -0,0 +1,19 @@
// Serves the fixture feeds so the daemon under test has something real to scan.
const http = require('http');
const fs = require('fs');
const path = require('path');
const dir = __dirname;
const port = Number(process.env.FIXTURE_PORT || 8792);
http.createServer((req, res) => {
const name = decodeURIComponent(req.url.split('?')[0].replace(/^\//, '')) || 'index';
const file = path.join(dir, path.basename(name));
fs.readFile(file, (err, body) => {
if (err) { res.writeHead(404).end('no'); return; }
const type = file.endsWith('.mp3') ? 'audio/mpeg'
: file.endsWith('.opml') ? 'text/x-opml' : 'application/xml';
res.writeHead(200, { 'content-type': type, 'content-length': body.length });
res.end(body);
});
}).listen(port, '127.0.0.1', () => console.log(`fixtures on ${port}`));

15
tests/ui/fixtures/show.xml Normal file
View File

@@ -0,0 +1,15 @@
<?xml version="1.0"?>
<rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
<channel><title>Test Show</title><link>http://127.0.0.1:8792/</link><description>A fixture feed.</description>
<itunes:image href="http://127.0.0.1:8792/art.png"/>
<item><title>First Episode</title><guid>ui-1</guid>
<pubDate>Mon, 01 Sep 2026 10:00:00 +0000</pubDate>
<description>&lt;p&gt;Show notes for the first one.&lt;/p&gt;</description>
<itunes:duration>1830</itunes:duration><itunes:season>1</itunes:season><itunes:episode>1</itunes:episode>
<enclosure url="http://127.0.0.1:8792/ep1.mp3" length="40000" type="audio/mpeg"/></item>
<item><title>Second Episode</title><guid>ui-2</guid>
<pubDate>Mon, 08 Sep 2026 10:00:00 +0000</pubDate>
<description>Notes for the second.</description>
<itunes:duration>900</itunes:duration>
<enclosure url="http://127.0.0.1:8792/ep1.mp3?2" length="40000" type="audio/mpeg"/></item>
</channel></rss>

4
tests/ui/fixtures/subs.opml Normal file
View File

@@ -0,0 +1,4 @@
<opml version="2.0"><head><title>Test Subscriptions</title></head>
<body><outline text="Folder">
<outline type="rss" text="Grouped Show" xmlUrl="http://127.0.0.1:8792/other.xml"/>
</outline></body></opml>

53
tests/ui/global-setup.js Normal file
View File

@@ -0,0 +1,53 @@
// Builds a scratch config and data dir so the browser tests drive a real daemon with
// known feeds, rather than whatever happens to be on the machine.
const fs = require('fs');
const path = require('path');
const os = require('os');
const root = path.join(os.tmpdir(), 'ipx-ui-test');
const TOKEN = 'testtokentesttokentesttoken12345'; // fixed, so tests need not scrape a log
// Called from playwright.config.js at load time, NOT as globalSetup: Playwright starts
// webServer *before* globalSetup, so a config written there does not exist yet when the
// daemon launches -- it would fall back to the real config and fight the live daemon.
function prepare() {
fs.rmSync(root, { recursive: true, force: true });
for (const d of ['config', 'data', 'downloads']) {
fs.mkdirSync(path.join(root, d), { recursive: true });
}
fs.writeFileSync(path.join(root, 'config', 'config.toml'), `
[general]
download_dir = "${path.join(root, 'downloads')}"
socket = "${path.join(root, 'ipx.sock')}"
schedule = "every 60m"
max_new_per_check = 1
[torrent]
enabled = false
[web]
enabled = true
bind = "127.0.0.1:8791"
token = "${TOKEN}"
[feeds.test-show]
url = "http://127.0.0.1:8792/show.xml"
auto_download = true
# Downloads its image, so the UI has a file that is not playable to deal with.
[feeds.picture-blog]
url = "http://127.0.0.1:8792/pics.xml"
auto_download = true
media_types = ["image"]
[feeds.multi-show]
url = "http://127.0.0.1:8792/multi.xml"
auto_download = true
[feeds.test-subscriptions]
url = "http://127.0.0.1:8792/subs.opml"
auto_download = false
`);
}
module.exports = { prepare, root, TOKEN };

1388
web/index.html Normal file
View File

File diff suppressed because one or more lines are too long

BIN
web/ipodderx-icon.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.3 KiB

BIN
web/ipodderx-icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

91
web/login.html Normal file
View File

File diff suppressed because one or more lines are too long