48 Commits

Author SHA1 Message Date
9aae3097e7 Release 0.5.2
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 14:27:42 +00:00
2ff2074755 Answer status to the client that asked, not everyone
status is a terminal event. Broadcast, the healthcheck's answer ended any
ipx fetch that was watching a scan, which stopped reading at the next probe
while the scan carried on. It could not happen while status waited behind
the scan; answering it at once made it happen every 30 seconds. Each
connection's writer now takes private replies beside the broadcast, and
the test checks another client hears nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 14:27:42 +00:00
1698cf8d1e Answer status on the socket instead of queuing it behind the worker
The worker runs one job at a time, and status was one of its jobs, so the
Docker healthcheck waited behind the startup scan (54 seconds of it after
the last deploy) and timed out at 5. Any scan or download longer than
three probes would have had a working daemon marked unhealthy. The socket
now answers status straight away; everything else still queues. A test
fills the queue and checks status comes back anyway.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 14:19:00 +00:00
b94a74ef15 Sign out through the proxy when the proxy signed you in
Sign out cleared ipx's cookies and showed its password page, while
Cloudflare Access still vouched for the person: nothing was signed out,
and the page looked like the wrong login. /api/me now says, for someone
the proxy signed in, where to go instead ([web] sign_out_url, which is
/cdn-cgi/access/logout behind Access), and /login sends anyone the proxy
vouches for on to their feeds. The header check both use is one function.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 14:10:30 +00:00
9a8a3c696f docs: the ipodderx tile in Authentik's library
A bookmark application with no provider, so ipodderx shows in the library
beside Outline. Recorded with its id and how to delete it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 14:03:06 +00:00
bedf64e645 docs/sso.md: the sign-in setup ipodderx.sdf1.net really runs
Authentik is Cloudflare Access's OpenID Connect identity provider, not
something in the request path, and the tunnel's requests reach ipx from
the content_default gateway, 192.168.16.1, not 127.0.0.1. The page is
rewritten from what was measured, with checks for both the trusted and
the refused path, and docs/history.md records every change made to get
there with how to undo it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 13:58:18 +00:00
586d2c07a1 ipx user rename: give an account the name the proxy signs it in as
An account made by hand before the proxy was set up is called what it was
given ('rays'), while Cloudflare Access vouches for an email address. With
auto_create_users on, the first visit through the tunnel would make a
second, empty account. Renaming keeps the id, so feeds, read state and
admin rights go with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 13:54:29 +00:00
2ba83c3aed Keep when each account was added and when it last signed in
users.created comes back, beside a new last_login, for whoever maintains
the server. A password sign-in, the token link and a request through the
proxy all count, recorded to the hour so the proxy's per-request vouching
is not a write each time. Settings -> Users and ipx user list show both.
The three user queries now share one row mapping.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 13:37:09 +00:00
1352f0d54d Show notes cut off mid-tag give way to the item's description
libsyn served Daily Meditation Podcast's content:encoded cut at the '>'
inside a Tailwind class pasted from a web app, so 57 items began halfway
through a tag and the page showed the rest of it as text. Their
description was whole. A body that closes an attribute list before any
tag opens now falls back to the description, for RSS and Atom alike.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 13:29:00 +00:00
a958f7cb37 Trim the state database; Popular lists feeds the way Directory does
Drops the created columns on users, subscriptions and sessions, which were
written by every insert and read by nothing, and migrate()'s add list, whose
columns all predate 0.3.0. Removes Db::subscribed_feed_ids (no callers),
Db::subscriber_count (one caller wanting > 0) and Managed.orphaned (never
read). The old-database test now builds the tables with foreign keys on.

Popular now lists the feeds inside an OPML or a Patreon creator, never the
collection, as Directory does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 13:15:55 +00:00
457a58dcc5 Directory lists the feeds inside an OPML, not the OPML
Popular still counts an OPML as one feed, since everyone subscribed to it
counts for every feed inside and they would bury the rest. The directory is
for finding a show, so it lists them one by one and never the OPML. A feed
inside an OPML that looks private is hidden with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:26:52 +00:00
2af57065c6 Release 0.5.1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:15:53 +00:00
564b011c7a Give the folder triangle a wider gutter
The feed list's left padding grows from 8 to 16 px and the triangle's
button from 16 to 24 px wide, so it is no longer cramped against the
folder's art. Everything in the list shifts together, so feeds still line
up with the places above.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:13:32 +00:00
9eb7aadced Release 0.5.0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:06:23 +00:00
c6bceaef37 Docs: a slow migration and a CLI run at the same time
Every ipx command migrates when it opens the database, so the healthcheck
collided with the daemon while it dropped the old entries columns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:59:05 +00:00
dc63d6acaf Cut what the audit found: dead columns, one-time upgrades, three deps
Works through TODO.md from the 2026-09-12 over-engineering audit. Drops the
entries.read/flagged/position columns (migrate() removes them from older
databases), migrate_opml_children, the legacy interval_mins key, the
contrib/ systemd units, test-only Db wrappers, a duplicate token generator,
redundant logbuf visitors, unused page state and CSS, and the infer, dirs
and tokio-stream dependencies. The icon is served once as /icon.png instead
of inlined four times, taking about 94 KB off the two pages.

The adoption's subscription half was not dead: it gives a fresh install's
first admin the config's feeds. It stays as adopt_catalogue, now tested.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:55:44 +00:00
8937f35f00 TODO.md: cleared, the design pass is done
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:31:08 +00:00
0990f2a90d Design pass on the web UI: amber for new, EQ bars, keyboard feed list
Works through TODO.md from the 2026-09-11 review. Unread badges, dots and
download bars take the icon's amber; the playing item is marked by EQ bars
that move only while it plays. The feed list is usable from the keyboard,
focus rings show everywhere, and folders get a mosaic of their shows' art
with the triangle hung in the margin. Sentence-case labels, fewer bold
weights, tinted initials tiles, shorter header lines, "Kept" everywhere,
and the list gets the room the empty panes had. Reduced motion is honoured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-11 22:24:41 +00:00
0cc002cdfa todo.md: folders in the sidebar; drop the system-theme item
The disclosure triangle, looked at closely: the feed list cannot be
reached from the keyboard (span and div, no tabindex), every feed is
pushed 28 px right for a slot only folders use, the target is 18 px,
shows barely nest under their folder, a folder looks like a feed, and
a selected feed's placeholder tile vanishes in Dark and Light.

Following the system light or dark setting is off the list, by
choice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 19:39:49 +00:00
2c9e899762 todo.md: the design review's list, to do later
A review of the web UI against screenshots of every view in all three
themes. The palette and Classic carry the iPodderX identity; Dark and
Light do not. First three: amber for new (unread badges, dots,
download progress), EQ bars as the playing marker, and toolbar focus
rings that overflow:hidden currently clips. The rest is weights,
sentence-case labels, pane sizes, header lines, wording, system theme,
reduced motion, art, sign-in and the Settings export/import icons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 18:53:42 +00:00
2e416f96cf Patreon creators split into their shows; filters follow settings
A Patreon token pasted into Add feed, or a creator link without
&show=, becomes a folder of that creator's shows, found through
Patreon's web API and kept in step like a subscribed OPML (sync_group,
split out of sync_opml). A creator already read as one feed is split
too: each show takes over the files and read state it held
(Db::adopt). A creator with one show stays a plain feed.

Filter verdicts are judged again every scan, so turning on Allow
explicit brings skipped items back. Add feed has an explicit box.
Feeds in a group follow your settings on the group, as its dialog
said. A new feed no longer takes the id of a removed one at a
different URL and shows its old items. See CHANGELOG.md [Unreleased]
and docs/history.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 18:24:47 +00:00
9269aa99f7 README: a short overview that points into docs/
It described the layout from before 0.4.0 (items across the top, a
player below), and carried long sections on OPML, the log view and
this server's own deploy steps, all of which docs/ and CLAUDE.md cover.
It now says what ipx does, how to run it with Docker or from source,
the first sign-in, the TLS caveat, where the docs are, and the tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:36:34 +00:00
ae8123250b Release 0.4.0
The original iPodderX layout (toolbar, places, item table, Files pane),
the Classic theme, Directory and Popular, All Subscriptions with mark
everything read, sortable columns and a Size column, one meaning per
icon across the UI, and fixes for the double play, Escape in dialogs,
the dark-theme password box and paid feeds listed in Popular. See
CHANGELOG.md [0.4.0] and docs/history.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:35:08 +00:00
2c34a144ba History: the icon pass, sorting and the double play; fix a stale changelog line
docs/history.md gets the long-form entry CLAUDE.md asks for: what the
UI pass found, the three bugs it turned up (Escape inside a dialog's
text box, the white password box, the stray dot), why sorting is done
on the server, and how the Files pane came to play a file twice.
CHANGELOG's Added line for the item table named the old columns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:32:41 +00:00
669e8b5124 Files pane plays through the player bar, once
The pane drew its own <audio controls> for a downloaded file, and its
onplay also started the player bar, so one click played the same file
twice at once. The pane now has a play button (with the file's type
icon, like the other rows) that hands that exact file to the player
bar, the only player. play() takes the file, so another of an item's
files starts from its top instead of resuming the first. Dead CSS for
the pane's <audio> removed.

Test: play in the Files pane leaves one <audio> on the page, the bar's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:30:03 +00:00
5362436766 Sortable item table, Size in its own column, Subscribed as an icon
- Every column heading sorts (kept, title, feed, file type, size,
  published); a second click reverses it. The server sorts through a
  fixed whitelist (order_sql), so it covers the whole list, not the
  fifty loaded; the choice is remembered in the browser.
- Size is its own column and shows KB for small files instead of
  "0 MB". The Item heading is Title.
- Popular/Directory/Add feed: Subscribed is a green circle-check.
- Tests: every sort column runs and orders both ways (db); the table
  sorts by title both ways and remembers across a reload (browser).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:17:16 +00:00
0efc49519c One icon per meaning across the UI; mark everything read in All Subscriptions
- All Subscriptions' header checks every feed and marks everything read
  (POST /api/read-all, the same feeds the view lists); it asks first.
- Minus unsubscribes everywhere (the feed header's x read as "close"),
  x only closes or cancels, plus adds/subscribes/imports, and a dialog's
  confirm carries its action's icon. Remaining word buttons, the player
  and the folder arrow are Font Awesome 7.3.1 icons.
- Toolbar grouped by what it acts on (add, unsubscribe, scan | play,
  read, keep); read and keep show the selected item's state.
- The OPML subscription page uses the same header as a feed.
- Fixed: Escape ignored inside a dialog's text box (Add feed could not
  be closed with it), white password box in the dark theme, stray dot
  in an undated item's details.
- Tests: one action one icon across toolbar, page and all 8 dialogs;
  All Subscriptions mark everything read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:05:47 +00:00
57dcba2d1a One file icon, green when downloaded; mark unread is an envelope
- The separate check (and warning) beside a file's type icon is gone:
  the type icon itself is green once the file is downloaded and red when
  the download failed, with the details in its tooltip. One icon per row
  keeps the column lined up. On Classic's blue selection they are a
  lighter green and red rather than white.
- Mark unread under an item's title was a solid circle, which read as a
  record button. It is Font Awesome's closed envelope now.
- Drops the unused circle-check, circle-exclamation and circle icons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:20:32 +00:00
e95cccc66f Icons are Font Awesome Free, embedded as SVG
The ICON set is now Font Awesome Free 7.3.1: the 29 icons the page uses,
taken from svgs/solid and svgs/regular at that tag and embedded as SVG
paths, 12.8 KB in all. No webfont to download and nothing fetched from a
CDN. The CC BY 4.0 attribution is above the set, and in the README.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:14:51 +00:00
7295be8b25 Drawn icons throughout; file state and type as icons, no PENDING
- Every button's icon is drawn from one small SVG set (ICON), in bold
  strokes of the button's own colour, instead of font characters: ⟳ ⤓ ↗
  and the like came out thin and tiny and differed from font to font.
  Static buttons name theirs with data-icon. Keep is a flag everywhere.
- A file's state is an icon: a check when downloaded, a warning with the
  error in its tooltip when it failed, nothing while it waits. Its type
  (audio, video, image, pdf, torrent, other) is an icon with the word in
  its tooltip. The DOWNLOADED and PENDING chips are gone, which also
  fixes them being hard to read on Classic's blue selection.
- Tests find state and type by their tooltips.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:11:15 +00:00
f1b0d97b81 Classic: white text across a selected row, white lists
A selected row's file size stayed grey on the Aqua blue; the whole row
is white now. The directory's rows took the source list's pale
blue-grey; lists were white in the original.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:03:04 +00:00
ae47e31a97 Classic theme after the 2004 Mac app; header and dialog buttons as icons
- A Classic theme beside Dark and Light: brushed-metal toolbar and
  status bar, a pale blue-grey source list, Aqua blue for whatever is
  selected, red unread badges, a striped table with blue titles, a blue
  bar behind the item's title, and Lucida Grande. The theme button steps
  through all three and remembers the choice.
- The feed and OPML headers' buttons (scan, download latest, mark all
  read, settings, unsubscribe) and the Settings, feed settings and
  Download latest dialogs' buttons (save, download, cancel) are icons,
  with the words in title and aria-label.
- Tests: the theme button reaches Classic and it survives a reload; the
  header buttons are found by action.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:00:30 +00:00
8a309eb652 Files pane and item buttons are icons, with the words in tooltips
Save, delete, view and download in the Files pane, and mark read, keep
and open the original under an item's title, are icons now. The word is
in each one's title and aria-label, so it is still there on hover and
for screen readers. Tests find them by title or action, not text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:57:03 +00:00
1a2b0d87c6 CLAUDE.md: what to do when Docker Hub rate-limits the image build
The build asks Docker Hub about its two base images every time unless
they are stored locally, and this host has no Docker Hub login, so a busy
day ends in 429 Too Many Requests. Pulling them from mirror.gcr.io and
tagging them locally lets the build go ahead without asking.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:49:58 +00:00
d7fac2d0e7 Never list paid-feed services; scan on add; no "reaped"; slimmer header
- Security: feeds from Patreon, Supercast, Supporting Cast, Glow and
  Memberful are never listed in Popular or the Directory. A Supercast
  feed keeps its key in the URL's path, which the query check missed, so
  it was being listed.
- Adding a feed queues a scan of it, and an OPML import that added feeds
  scans what is due, so items show without pressing Scan.
- A file deleted to save space, or by hand, looks as if it was never
  downloaded: no "reaped" chip, just the Download button. The retention
  summary says "deleted".
- The feed header keeps its title and stats to one line each and wraps
  its buttons; a single feed's table drops the Feed column.
- Tests: adding a feed shows its item without Scan; a deleted file shows
  no "reaped"; paid-feed hosts and acast public ids in the unit test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:47:03 +00:00
df9b7645d6 The original iPodderX layout: toolbar, places, item table, Files pane
- A toolbar across the window with the original's groups: add and
  unsubscribe, play, mark read and keep for the selected item, scan, a
  search box for what is showing, and Settings and Log (admins only).
- Directory, Popular and All Subscriptions sit at the top of the feed
  list and open in the main pane; the Popular and Directory buttons and
  their dialogs are gone.
- All Subscriptions lists every item from every feed you subscribe to:
  GET /api/entries, the per-feed query with its scope widened. The
  enclosure lookup after it matches files to rows by feed and guid, since
  a page can now span feeds.
- Items are a table (unread, kept, item, feed, file, published) with a
  Files pane beside it, the text below, and a status bar with totals. On
  a phone the files follow the text and the table is title and date.
- Tests: enclosures are checked in #files; the toolbar's read, keep and
  play act on the selected item; All Subscriptions holds only your feeds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:39:05 +00:00
f3825cfc57 Popular is a top 10; a Directory lists every listed feed A to Z
- GET /api/popular returns the ten most subscribed feeds. The new GET
  /api/directory returns every feed that may be listed, sorted by name,
  from the same list: everyone counted, you included, never a URL, never
  a private feed or a feed inside an OPML. POST /api/popular/{id} still
  subscribes to anything on it.
- A Directory button sits beside Popular; both open the same list
  screen. The sidebar toolbar wraps rather than squeezing four buttons.
- Tests: the directory is A to Z, Popular is its top ten, Paid Show is
  in neither, and subscribing works from the directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:13:16 +00:00
c0f4b0bcb2 Popular button in the sidebar; the popular list counts everyone
- A Popular button beside + Feed opens the popular list directly; the
  Add feed dialog keeps it too.
- The list counts every subscriber, you included. Your own feeds stay on
  it, marked Subscribed, and clicking one opens it. Private feeds and
  feeds inside an OPML are still never listed, for anyone.
- GET /api/popular rows carry `subscribed`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:05:24 +00:00
8b8b48302b Popular on this server; changelog follows Keep a Changelog; 0.3.0
- Add feed lists what other accounts subscribe to, most subscribers
  first, and subscribes you by id (GET /api/popular, POST
  /api/popular/{id}). Rows never carry a URL. Feeds from an OPML and
  anything that looks private (a login, credentials in the URL, a key
  such as auth= or token=) are never listed, and the subscribe route
  checks the id against the same list.
- CHANGELOG.md follows Keep a Changelog 1.1.0: 0.1.0 (2026-09-09, the
  CLI), 0.2.0 (2026-09-10, the web UI), 0.3.0 (2026-09-11, accounts and
  sharing). The long-form entries moved unchanged to docs/history.md.
- Cargo.toml is 0.3.0. CLAUDE.md says how to add an entry and cut a
  release.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 13:56:57 +00:00
5d3fdde4da Upload an OPML file to import; tests for every way in and out
- The import screen has a file picker beside the paste box. The page
  reads the file, checks it looks like OPML before sending, and clears
  the picker when it is refused and after it is imported. The file is
  sent as text and never written to disk on the server.
- The server parses the OPML before touching anything and answers 400
  "that is not an OPML file" (was a 500). subscribe_opml takes a parsed
  document, so ipx import also refuses a non-OPML file by name.
- Tests: Settings' Export OPML download and paste import; uploading an
  RSS file (refused) and a real OPML; the server's 400; the admin's
  export round-tripped into a second account; ipx import/export in a
  scratch config.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 13:42:38 +00:00
8784d0a3fd OPML import subscribes you; export lists only your feeds
Import predated accounts: it only added URLs missing from config.toml
and subscribed nobody. Importing another account's export did nothing
("Imported 0 feed(s)"), and a genuinely new feed had no subscriber, so
it was never scanned. Web and CLI import now share subscribe_opml,
which subscribes the caller (the CLI: the first admin) to every feed in
the file and reports new vs already-subscribed.

Export wrote the whole catalogue to anyone signed in, including other
people's private feed URLs. It now lists only your own subscriptions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 12:53:29 +00:00
5e95557cbb User admin in the web UI, admin-only log, unread-first OPML feeds
- Settings > Manage users: add an account (password, or none for proxy
  sign-in), toggle admin, remove. Backed by GET/POST /api/users and
  PATCH/DELETE /api/users/{id}, 403 for non-admins. The only admin
  cannot be demoted or removed.
- GET /api/logs is admin-only and the Log button is hidden for others;
  the log names every account, feed and failed sign-in.
- Feeds inside an OPML list those with unread items first, in the
  sidebar folder and on the subscription's page.
- Deploying is now buildx --push to 192.168.1.130:5000 and recreating
  the ipodderx service of the Arcane project content; CLAUDE.md and the
  README's Docker section say so.
- Tests: Playwright for user admin, the last-admin guard, 403s for a
  non-admin and the unread ordering (new Aardvark Radio fixture); a unit
  test for last_admin; the smoke test drives usersModal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 12:43:28 +00:00
6114add4a6 Documentation: docs/, a changelog, and CLAUDE.md
PROGRESS.md becomes CHANGELOG.md with the finished step lists moved to an
appendix. The README is an overview pointing at docs/: configuration,
cli, users, sso (refreshed for accounts and admin-only settings), and
architecture. CLAUDE.md collects what working on this code actually
requires -- pkill -x not -f, the page being compiled in, the dead columns
on entries, the Playwright worker that deleted its own database.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 03:00:25 +00:00
c47c224372 Pruning respects a star from anyone
prune_entries still guarded on entries.flagged, which nothing writes
since read state moved to entry_state -- so starring a text item with no
file would not have saved it from the age sweep. It follows the reaper's
rule now, and takes orphaned read state with whatever it deletes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 02:41:12 +00:00
686851b448 Warn before deleting a file other people share
A feed with other subscribers labels the button Delete for everyone and
names them in the confirmation. The server decides: if anyone else has
starred the item or not played it, DELETE returns 409 with the reason and
only ?force=true proceeds. A feed's header says when it is shared, which
answers why a file nobody here asked for exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 02:37:07 +00:00
f0d03c79c8 first commit 2026-09-11 02:31:09 +00:00
7df4ee7dde Retention follows per-user read and starred
reap_candidates still read entries.read/flagged, which nothing writes
since read state moved to entry_state -- so starring no longer protected
a file and the read-first ordering was dead. One file serves every
subscriber, so anyone starring it keeps it, and it counts as read only
once everyone subscribed has read it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 02:25:24 +00:00
d46ec73261 Per-user read state and subscriptions
Read, starred and position move to entry_state; subscriptions carry each
person's keywords, auto-download, explicit and per-scan limit. The feed
list and unread counts are per person, and the existing library is
adopted by the admin on first start.

The feed URL, folder and schedule stay shared and admin-only: one file
serves everyone, so they describe the file rather than a preference.
Scanning merges subscribers' wants -- anyone wanting an item is enough --
via merge_policy, which is pure and tested.

Also: the test fixture wiped its data directory from every Playwright
worker, deleting the database out from under the running daemon.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 02:17:16 +00:00
34 changed files with 5263 additions and 1278 deletions

305
CHANGELOG.md Normal file
View File

@@ -0,0 +1,305 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
The long form, with what was wrong before and how it was found, is in
[docs/history.md](docs/history.md).
## [Unreleased]
## [0.5.2] - 2026-09-12
### Added
- Settings → Users and `ipx user list` show when each account was added and when it last signed
in, to the hour.
- `ipx user rename <name> <new name>` renames an account and keeps its feeds, read state and admin
rights. An account made before the proxy was set up can take the name the proxy signs it in as.
### Changed
- Directory and Popular list the feeds inside an OPML one by one, and no longer the OPML itself,
so you can subscribe to just the shows you want.
- The database no longer records when subscriptions and sign-in sessions were created. Nothing
ever read it, and an existing database drops the columns on its next start.
### Fixed
- Show notes that the podcast's host cut off in the middle of a tag no longer open with a scrap of
HTML: the item's other copy of its notes is used instead, from the next time the feed changes.
Daily Meditation Podcast had 57.
- Docker no longer shows ipodderx as starting, or calls it unhealthy, while it scans or downloads:
`ipx status` answers at once instead of waiting for the job in progress to finish.
- Signing out after signing in through Cloudflare Access no longer lands on ipodderx's own password
page. With the new `sign_out_url` set, Sign out ends the Access session, and the password page
sends anyone the proxy signs in straight to their feeds.
- The sign-in guide, `docs/sso.md`, describes the setup ipodderx.sdf1.net really runs: Authentik as
Cloudflare Access's identity provider, and how to find the address ipx has to trust. It had never
been checked against a real setup, and pointed at the wrong address.
## [0.5.1] - 2026-09-12
### Fixed
- The triangle that opens an OPML or Patreon folder was cramped against the folder's art. It has
more room now, and a wider target to click.
## [0.5.0] - 2026-09-12
### Added
- A Patreon token pasted into Add feed, or a creator's RSS link without `&show=`, becomes a folder
of that creator's shows, kept in step on every scan like a subscribed OPML. A creator with only
one show stays a plain feed. One already added as a single long feed is split into its shows on
its next scan, keeping its files and what you had read.
- Add feed has an "Allow items marked explicit" box, so a new feed's first scan no longer skips
every explicit item.
### Changed
- Unread counts, unread dots and download progress are amber, the colour of the icon's EQ bars.
Blue is kept for the primary action and links, so a count no longer looks like a button.
- What is playing is marked by small EQ bars, in its row and in the player. They move only while it
plays.
- A folder in the sidebar shows its first four shows' art as a mosaic, and its shows sit under its
title. Only folders have a triangle, so every feed lines up with Directory and Popular above.
- Feeds without art get initials in a colour of their own, instead of all the same grey.
- The Flagged tab is called Kept, as the Keep button and Settings already said.
- A feed's header is one short line; when it checks next is in its tooltip.
- The item list takes more of the window, and the Files pane shows only when the item has files.
- Column headings and tags are in sentence case, and fewer things are bold.
- Unsubscribe is a round button beside the feed's other actions.
- Export and Import in Settings say what they do.
- The sign-in page shows the original icon large.
- Nothing animates when your system asks for reduced motion.
- The pages are about 90 KB smaller: the icon is served once instead of written into each.
- A web token generated for a new install is 64 characters instead of 32.
- The README is a short overview of what ipx does and how to run it, and points into `docs/` for
the rest. It still described the layout from before 0.4.0.
### Removed
- The systemd units in `contrib/`. Run ipx with Docker, or point a unit of your own at
`ipx daemon`.
- Upgrading from before 0.3.0 directly: what was read, kept or part-played before accounts is no
longer carried over to the admin, and OPML feeds that old versions wrote into `config.toml` are
no longer moved out of it. Upgrade through 0.4.0 first.
- `interval_mins` in `config.toml` is ignored; use `schedule`.
### Fixed
- The feed list works from the keyboard: Tab reaches every feed, Enter opens it, and Right and Left
open and close a folder. Every button shows where the focus is, including in the toolbar, which
used to clip the ring.
- "1 items" reads "1 item".
- A selected feed without art no longer loses its initials tile in Dark and Light.
- The player shows the feed's initials when there is no art, not the episode's.
- Turning on Allow explicit, or changing keywords or auto-download, brings back what those settings
had skipped on the feed's next scan. Before, an item was judged once, when first seen, and a
skipped one stayed skipped whatever you changed.
- Feeds inside an OPML or a Patreon creator follow your settings on the folder unless you set their
own, as the folder's settings dialog said they did. Before, the folder's settings reached nothing
inside it.
- A new feed no longer takes the name of one you removed earlier and shows that feed's old items.
Re-adding the same feed still gets its old name, and its history, back.
## [0.4.0] - 2026-09-11
### Added
- A Classic theme after the 2004 Mac app, beside Dark and Light: brushed-metal toolbar, Aqua
blue selection, red unread badges, a striped table and Lucida Grande. The theme button steps
through all three and remembers the choice.
- A toolbar across the top, after the original iPodderX: add and unsubscribe, play, mark read
and keep for the selected item, scan, a search box for what is showing, and Settings and Log.
- Directory, Popular and All Subscriptions at the top of the feed list, opening in the main pane.
Directory lists every feed anyone here subscribes to, A to Z (`GET /api/directory`). All
Subscriptions lists every item from every feed you subscribe to (`GET /api/entries`).
- Items show as a table (unread, kept, title, feed, file, size, published) with a Files pane beside
it, and a status bar with the totals.
- Mark everything read from All Subscriptions, across every feed you subscribe to
(`POST /api/read-all`). It asks first. All Subscriptions can also check every feed from its header.
- Click a column heading in the item table to sort by it (kept, title, feed, file type, size,
published); click again to reverse. The server sorts, so it covers the whole list, not just the
fifty shown, and the choice is remembered.
### Changed
- Popular shows the top 10, not 20, and counts everyone, you included. Your own feeds stay on it,
marked Subscribed, and clicking one opens it.
- Adding a feed scans it straight away, and an OPML import that added feeds scans them, so their
items show without pressing Scan.
- A file deleted to save space, or by hand, looks as if it was never downloaded: no "reaped"
label, just the Download button. The retention summary says "deleted", not "reaped".
- Buttons are icons, with the words in their tooltips: the Files pane (save, delete, view,
download), an item's own buttons (mark read, keep, open the original), the feed header (scan,
download latest, mark all read, settings, unsubscribe), and the Settings, feed settings and
Download latest dialogs (save, download, cancel). The icons are Font Awesome Free, embedded as
SVG: only the ones used, no font to download, and nothing fetched from anyone else. They
replace font characters such as ⟳ ⤓ ↗, which came out thin and tiny and differed from font to
font. Keep is a flag everywhere, as it was in the original, and mark unread is an envelope.
- One meaning per icon. Minus unsubscribes, x closes or cancels, plus adds or subscribes, and a
dialog's confirm button carries the icon of what it does. The feed header's unsubscribe was an x
and read as closing the page. The remaining word buttons are icons too:
- Log, Add feed, Users, Unsubscribe and OPML.
- Popular's Subscribe, Copy and Sign out.
- The Subscribed label in Popular, the Directory and Add feed, which is now a green check.
- The player's back, play, forward and close, which were font characters, and the folder arrow.
- The toolbar's read and keep buttons show the selected item's state, with the same icons as the
item's own buttons. Play, read and keep sit together, and Scan sits with add and unsubscribe.
- An OPML subscription's page has the same header as a feed's, with its buttons in the same places.
- The item table's size has its own column, apart from the file's type, and shows KB for small
files instead of "0 MB". The Item heading is now Title.
- A file's type is an icon (audio, video, image, PDF, torrent, other), green once it is
downloaded and red when the download failed, with the details in its tooltip. One icon per
row keeps the column lined up. The DOWNLOADED and PENDING labels are gone.
### Fixed
- Playing a file from the Files pane played it twice at once, in the pane and in the player bar.
The pane has a play button now, and the player bar is the only player.
- The password box in Manage users was white in the dark theme.
- An item with no date showed a stray dot in its details.
- Escape did not close a dialog while the cursor was in one of its boxes, so Add feed, which opens
in its URL box, could not be closed with Escape.
### Security
- Feeds from paid-feed services (Patreon, Supercast, Supporting Cast, Glow, Memberful) are never
listed in Popular or the Directory. A Supercast feed, which keeps its key in the URL's path
rather than the query, was being listed.
## [0.3.0] - 2026-09-11
### Added
- Add feed lists what other people on this server subscribe to, most subscribers first, and
subscribes you by id (`GET /api/popular`, `POST /api/popular/{id}`). Feeds from an OPML, and
feeds with a login or a key in their URL, are never listed.
- Upload an OPML file to import, beside the paste box. The page checks it looks like OPML before
sending it and clears the picker afterwards.
- Settings → Manage users: add and remove accounts, and choose who is an admin
(`GET`/`POST /api/users`, `PATCH`/`DELETE /api/users/{id}`).
- Per-user subscriptions, and per-user read, starred and playback state. The existing library is
adopted by the admin on first start.
- Subscribing to a feed someone else already has costs no second fetch and no second copy. Scanning
merges every subscriber's wants.
- Delete on a shared feed reads **Delete for everyone**, and the server answers `409` while anyone
else has starred the item or not played it (`?force=true` overrides).
- A shared feed's header says how many other people read it.
- `docs/` for configuration, the CLI, users, SSO and architecture, and `CLAUDE.md` for anyone
working on the code.
- Browser tests for OPML import and export by every route, user admin, unread ordering, and
`ipx import`/`ipx export`.
### Changed
- Feeds inside an OPML subscription list the ones with unread items first.
- OPML import subscribes you to every feed in the file. `ipx import` subscribes the first admin.
- OPML export lists only your own subscriptions.
- Production runs as a Docker image pushed to `192.168.1.130:5000` and recreated with
`docker compose`.
- This changelog follows Keep a Changelog. The long-form entries moved to `docs/history.md`.
### Fixed
- Importing another account's OPML export subscribed nobody and reported "Imported 0 feed(s)".
Feeds it added had no subscriber, so they were never scanned.
- Importing something that is not OPML answered `500`. It is now `400` "that is not an OPML file",
refused before anything changes.
- Starring stopped protecting a file from the quota and age sweeps once read state became per-user.
### Security
- The log is admin-only (`GET /api/logs` answers `403`, and the Log button is hidden). It names
every account, every feed and every failed sign-in.
- OPML export no longer hands anyone signed in the whole catalogue, including other people's
private feed URLs.
## [0.2.0] - 2026-09-10
### Added
- Web UI served by the daemon: plain HTML and JS compiled into the binary, with feeds, items,
filters, search, sanitised show notes and live progress over SSE.
- Player bar with resume, speed, keyboard shortcuts and lock-screen controls.
- Three-pane layout: feeds beside, items above, and the selected item's text and files below.
- Phone layout.
- Accounts and sign-in: Argon2id passwords, session cookies, `ipx user add|list|passwd|rm`, and a
trusted proxy header for Cloudflare Zero Trust or Authentik (`docs/sso.md`).
- Subscribing to an OPML: it is re-read every scan and its feeds show as a folder. A feed dropped
from it is removed unless it has downloads.
- Scheduling: a global interval with per-feed overrides (`every 30m`, `4h`, `1d`, `2w`).
- `[general] media_types`, default audio and video, and `max_new_per_check`, default 3.
- Every enclosure of an item, a View link for files that are not audio or video, and per-item
artwork.
- In-app log view with Daemon I/O, Scans and HTTP tabs.
- Mark all read on an OPML subscription.
- Editable feed URL with a copy button.
- Docker image whose healthcheck goes through the control socket.
- The iPodderX name, icon, and a colour scheme taken from the icon.
- `tests/page-smoke.js` and a Playwright browser suite.
### Changed
- Global settings and scan schedules are admin-only. The per-feed schedule picker is gone.
- Feeds from an OPML live in the database, not `config.toml`.
- Opening an item marks it read. Playing it marks it read only at the end or past 90%.
- "Episodes" became "items", since half the library is text.
- A burst of scan events causes one refresh, not one per feed.
- All is the default filter. OPML import and export, Settings and Log moved out of the header.
- Torrents run detached, two at a time.
### Fixed
- Download fetched the next queued episodes instead of the one clicked.
- Pressing play made an item vanish from the Unread list.
- Clearing a folder, schedule or cap from the UI did nothing.
- The daemon ignored SIGTERM until the current download finished.
- A missing function stopped the page script and left the whole UI dead.
- One download painted progress on every pending row.
- A torrent could freeze scanning for up to an hour.
- Downloading from an OPML feed failed with "belongs to unsubscribed feed".
- Image enclosures were downloaded, counted as episodes and given a play button.
- A feed whose entries had been deleted stayed empty, because the server kept answering `304`.
- An item with several enclosures kept only the last.
- OPML folders rendered open by default.
- Mark read in the text pane recursed until the stack overflowed.
- The Unread, Downloaded and Flagged filters answered `500` without a search term.
- An OPML subscription always showed 0 unread.
- Folder names kept doubled spaces where separators were stripped.
- Sidebar rows had four different left edges.
### Security
- The web UI needs a token or a sign-in. The token is compared in constant time, and an empty
token refuses to serve.
- Show notes are sanitised with `ammonia`.
- A proxy's user header is honoured only from an address in `trusted_proxies`.
- A feed URL must be http(s), so `file:///etc/passwd` is refused.
- Download folders are sanitised per path segment, so `../../etc/Show` cannot climb out.
## [0.1.0] - 2026-09-09
### Added
- `ipx`, a Rust rewrite of the iPodderX engine: TOML config, SQLite state, and `ipx list`, `add`,
`rm` and `fetch`.
- RSS and Atom parsing with conditional GET, `<ttl>` and basic auth.
- Streaming downloads with explicit, keyword and per-scan filters, deduplicated by enclosure URL.
- Quota and age retention that never touches a starred file, and `ipx reap [--dry-run]`.
- `ipx daemon` with a JSON-lines Unix socket. CLI commands proxy to a running daemon.
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.2...main
[0.5.2]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.1...v0.5.2
[0.5.1]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.0...v0.5.1
[0.5.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.4.0...v0.5.0
[0.4.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...v0.4.0
[0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0
[0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0
[0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0

153
CLAUDE.md Normal file
View File

@@ -0,0 +1,153 @@
# Working on ipodderx-rs
Notes for whoever picks this up next. Read [docs/architecture.md](docs/architecture.md) for how the
thing is built; this file is about working on it without repeating mistakes that have already been
made here.
## Where things are
Production is the `iPodderX` container on Tower (192.168.1.130), the `ipodderx` service of the
Arcane project `content`: `/mnt/fast/arcane/projects/content/compose.yaml`. That file is what runs;
`docker-compose.yml` in this repo is a copy, and editing it changes nothing in production.
| | Host | In the container |
|---|---|---|
| Image | `192.168.1.130:5000/ipodderx:latest` | |
| Config | `/mnt/fast/appdata/ipodderx/config.toml` | `/config/config.toml` |
| Database | `/mnt/user/ipodderx/state.db` | `/data/state.db` |
| Downloads | `/mnt/user/ipodderx/downloads` | `/downloads` |
| Web UI | `192.168.1.130:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel | `0.0.0.0:8099` |
| Sign-in via the tunnel | Cloudflare Access app `ipodderx`, with Authentik as its identity provider; see [docs/sso.md](docs/sso.md) | trusts `Cf-Access-Authenticated-User-Email` from `192.168.16.1`, the `content_default` gateway |
Deploying a change is: build and push the image, then pull it and recreate the container.
```sh
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml pull ipodderx
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml up -d ipodderx
docker logs --tail 20 iPodderX
```
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
**A build that fails with `429 Too Many Requests` on a base image** is Docker Hub rate-limiting
this host. There is no Docker Hub login here, and the build asks about `debian:bookworm-slim` and
`rust:1-slim-bookworm` every time unless they are already stored locally. Pull them from Google's
mirror and tag them; the build then uses the local copies without asking Docker Hub:
```sh
docker pull mirror.gcr.io/library/debian:bookworm-slim
docker tag mirror.gcr.io/library/debian:bookworm-slim debian:bookworm-slim
docker pull mirror.gcr.io/library/rust:1-slim-bookworm
docker tag mirror.gcr.io/library/rust:1-slim-bookworm rust:1-slim-bookworm
```
Run those again now and then, or the local copies go stale.
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
the daemon answers there and can read its database, not just that the web port is up. The socket
answers `status` itself instead of queuing it behind the worker's current job, so a long scan or
download does not fail the check; it also means a worker stuck on one job would still pass. The
container restarts on its own after a reboot.
Before the container, ipx ran by hand in code-server, with its files in `/config/.config/ipx/` and
`/config/.local/share/ipx/`. Those are still there and the container does not read them. If you run
a daemon by hand for testing, stop it with **`pkill -x ipx`, never `pkill -f ipx`**. `-f` matches
the shell running the command and kills the session (exit 144). This has happened more than once.
## Before you touch the page
`web/index.html` is `include_str!`d into the binary, so **every page change needs a rebuild** before
it is visible. It is one file: markup, CSS and script.
After any edit to it:
```sh
node tests/page-smoke.js
```
That loads the script against a stub DOM and checks every selector it wires at load actually
exists. It exists because a patch once anchored on a deleted function, `String.replace` silently
matched nothing, and the whole UI died with a `ReferenceError` while every server-side test passed.
Patching that file by guessing an anchor string has failed repeatedly. Read the exact block first
(`sed -n 'START,ENDp'`), match it verbatim, and assert the replacement happened rather than hoping.
## Tests
```sh
cargo test # ~51 tests: parsing, filters, retention, schedules, SQL, per-user state
node tests/page-smoke.js
npx playwright test # 16 browser tests against a real daemon on fixture feeds
```
Things about the browser suite that have cost time:
* It starts **its own daemon and database** under `/tmp/ipx-ui-test`, wiped once per run. Playwright
re-imports the config in every worker, so `prepare()` guards on `TEST_WORKER_INDEX` — without
that guard a worker deleted the database out from under the running daemon, which then kept
serving from the unlinked inode while everything else saw an empty file.
* Tests **share that daemon and run in order**. A test that opens an item marks it read and changes
what later tests see. Write assertions that do not depend on what ran before, or normalise the
state first.
* Fixture feeds must not share an enclosure URL, because `enclosures.url` is globally unique and
whichever feed is scanned first claims it.
* `webServer` starts **before** `globalSetup`, which is why the fixture config is written at
config-load time instead.
Non-trivial logic leaves one runnable check behind. Pure functions (`merge_policy`, `pick`,
`matches_keywords`, `parse_interval`) are the easiest place to put it.
## Things that are true and easy to get wrong
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
* **Read state lives in `entry_state`, per user, and nowhere else.** `entries` had `read`, `flagged`
and `position` columns from before accounts; two bugs came from queries still reading them
(retention, and the entry pruner), and `migrate()` now drops them.
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
derived and never written to config.
* **One fetch serves everyone**, so scan policy is a union of subscribers' wants (`merge_policy`).
Anyone wanting an item is enough to fetch it.
* **The UI hiding a control is not enforcement.** Admin-only actions check `user.is_admin` in the
handler and return `403`.
* **A `tokio::select!` only races its branches at the point of selection.** A long download has to
watch the shutdown channel itself; the daemon ignored SIGTERM for exactly this reason.
* Only one daemon per socket. Removing the socket file defeats the guard and you get two daemons
fighting over the database, with the stale one still holding the port.
* `/api/settings` answering `200` does **not** mean the daemon is well — the web server is a
different task. `ipx status` checks the control socket and the database; to see the worker
getting through its jobs, watch for `scan complete` in the log.
* **Every `ipx` command runs `migrate()` when it opens the database**, the healthcheck's
`ipx status` included. A migration that rewrites a big table (`DROP COLUMN`) takes seconds on
production, and a command run meanwhile fails with `migrating schema`. It changes nothing; wait
for `daemon started` in the log. Copy `state.db` aside before deploying one.
## House style
Comments explain **why**, not what. If a line looks odd, the comment says what went wrong without
it. No emoji, no exclamation marks, no "obviously". Prose in the UI and docs is plain English and
addressed to the person using it.
Every change gets one line under `## [Unreleased]` in [CHANGELOG.md](CHANGELOG.md), in its
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) group: Added, Changed, Deprecated,
Removed, Fixed or Security. Say it the way someone using ipx would notice it. When there is more to
say, such as what was wrong before or what it cost to find out, write it up at the top of
[docs/history.md](docs/history.md), dated. That record has been more useful than the git log more
than once.
Cutting a release: rename `[Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD` and open a new empty
`[Unreleased]` above it, bump `version` in `Cargo.toml`, tag the commit `vX.Y.Z`, and update the
compare links at the bottom of the changelog.
Deliberate simplifications get a `ponytail:` comment naming the ceiling and the upgrade path, e.g.
`// ponytail: global connection mutex, move to a pool if feed count makes it contend`.
## Known gaps
* Cloudflare's `Cf-Access-Jwt-Assertion` is not verified — ipx trusts the hop plus `trusted_proxies`
(documented in [docs/sso.md](docs/sso.md)).
* A feed's `<description>` subtitle is dropped whenever `content:encoded` exists, which loses
Substack-style subtitles.

34
Cargo.lock generated
View File

@@ -436,17 +436,6 @@ dependencies = [
"shlex",
]
[[package]]
name = "cfb"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a347dcabdae9c31b0825fd6a8bed285ec9c2acb89c47827126d52fa4f59cece3"
dependencies = [
"fnv",
"uuid",
"web-time",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
@@ -867,15 +856,6 @@ dependencies = [
"dirs-sys",
]
[[package]]
name = "dirs"
version = "7.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d57d423b3c82e89b9a24ca3091fee61f456a26edbd28d26c65906f4bc1dcd8f"
dependencies = [
"dirs-sys",
]
[[package]]
name = "dirs-sys"
version = "0.5.0"
@@ -1608,15 +1588,6 @@ dependencies = [
"serde_core",
]
[[package]]
name = "infer"
version = "0.22.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f4200d433cbd5178df7797c9c2e75b348b728e39631cf14520d1e2fc424201f4"
dependencies = [
"cfb",
]
[[package]]
name = "intervaltree"
version = "0.2.7"
@@ -1634,7 +1605,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
[[package]]
name = "ipx"
version = "0.1.0"
version = "0.5.2"
dependencies = [
"ammonia",
"anyhow",
@@ -1643,9 +1614,7 @@ dependencies = [
"axum",
"chrono",
"clap",
"dirs",
"futures-util",
"infer",
"librqbit",
"opml",
"percent-encoding",
@@ -1656,7 +1625,6 @@ dependencies = [
"serde",
"serde_json",
"tokio",
"tokio-stream",
"toml",
"tower",
"tower-http 0.7.1",

View File

@@ -1,6 +1,6 @@
[package]
name = "ipx"
version = "0.1.0"
version = "0.5.2"
edition = "2024"
[dependencies]
@@ -11,9 +11,7 @@ atom_syndication = "0.12.10"
axum = "0.8.9"
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
clap = { version = "4.6.6", features = ["derive"] }
dirs = "7.0.0"
futures-util = { version = "0.3.34", default-features = false, features = ["std"] }
infer = "0.22.0"
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
opml = "1.1.6"
percent-encoding = "2.3.2"
@@ -24,7 +22,6 @@ rusqlite = { version = "0.40.2", features = ["bundled"] }
serde = { version = "1.0.229", features = ["derive"] }
serde_json = "1.0.151"
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
tokio-stream = { version = "0.1.19", features = ["sync"] }
toml = "1.1.5"
tower = { version = "0.5.3", features = ["util"] }
tower-http = { version = "0.7.1", features = ["fs"] }

259
README.md
View File

@@ -1,238 +1,83 @@
# ipodderx-rs
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent),
files them into per-feed folders, and reaps old episodes to stay under a disk quota.
Runs as a one-shot CLI or as a daemon with a Unix-socket JSON event stream for a UI to attach to.
A self-hosted podcatcher for a household. It checks your feeds, downloads the episodes, and serves
a web UI modelled on the 2004 Mac app **iPodderX**, for any number of people sharing one copy of
the files. One Rust binary, `ipx`, is both the daemon and the command line.
## Lineage
It is a rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python engine
behind iPodderX (2004-2008, Ray Slakinski & August Trometer).
This is a modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the
Python 2 engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), which was
open-sourced under the MIT License in 2010.
## What it does
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
oldest-first disk quota reaper.
- **The web UI.** It has a toolbar, and a feed list that opens with Directory, Popular and All
Subscriptions. Items sit in a sortable table with a Files pane, and there is a player bar. It
comes in Dark, Light and Classic themes, and works on a phone.
- **Several people, one copy.** Each person has their own subscriptions and their own read, kept
and playback state. There is one file on disk per episode, however many people want it. People
sign in with a password or through a proxy (Cloudflare Zero Trust or Authentik), and admins
manage accounts and settings.
- **Scanning.** Feeds are checked on a schedule, globally or per feed, and a feed's own TTL is
honoured. Keyword, explicit-content and media-type filters decide what is downloaded, with a cap
on new downloads per scan.
- **Downloads.** Files come over HTTP or BitTorrent and are filed into a folder per feed.
Retention deletes the oldest files to stay under a disk quota or an age limit, and never touches
an item someone has kept.
- **OPML.** You can import and export your own subscriptions. You can also subscribe to an OPML
URL, which keeps a whole list in step as a folder.
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
3DES-encrypted preferences, and the `printMSG` stdout protocol (replaced by a JSON-lines socket).
## Run it
## Quickstart
With Docker:
```sh
cargo install --path .
ipx add https://atp.fm/rss # names the feed from its own title
ipx list
ipx fetch # scan now
ipx daemon # or run continuously, honouring each feed's <ttl>
docker build -t ipodderx .
docker compose up -d
```
Config lives at `~/.config/ipx/config.toml` (mode 0600, since it may hold feed passwords);
state at `~/.local/share/ipx/state.db`. Override with `IPX_CONFIG` and `IPX_DATA_DIR`.
Set `IPX_LOG=ipx=debug` for verbose logging on stderr.
`docker-compose.yml` is set up for the author's own server. Point its `image` and its three volumes
(`/config`, `/data` and `/downloads`) at yours first. The UI is on port 8099. BitTorrent uses 6881
over TCP and UDP. Files are written as `PUID`/`PGID`, 99:100 by default.
## Commands
| command | what it does |
|---|---|
| `ipx add <url> [--folder X] [--keywords a,b]` | subscribe; the id comes from the feed title |
| `ipx rm <feed>` | unsubscribe; downloads and history are kept |
| `ipx list` / `ipx status` | subscriptions and their state |
| `ipx fetch [FEED] [--force]` | scan; `--force` ignores the TTL |
| `ipx reap [--dry-run]` | run retention now |
| `ipx import/export <file.opml>` | move subscriptions in or out |
| `ipx daemon` | scheduler plus the control socket |
Any command with a wire form probes the socket first: if a daemon is running it does the work,
and the CLI just renders the events it streams back. `--local` forces in-process execution.
## Configuration
```toml
[general]
download_dir = "~/Podcasts"
socket = "/run/user/1000/ipx.sock" # default: $XDG_RUNTIME_DIR/ipx.sock
interval_mins = 60 # default poll; a feed's own <ttl> wins when longer
organize = "feed" # "feed" | "date"
max_total_gb = 50 # 0 = unlimited
max_age_days = 30 # 0 = keep forever
max_new_per_check = 3 # per feed, per scan. 0 = unlimited (pulls whole back catalogues)
media_types = ["audio", "video"] # what downloads automatically. Anything else is still
# listed and can be fetched by hand -- blog feeds put article
# images in enclosures. Empty takes everything.
[torrent]
enabled = true
seed_ratio = 1.0 # stop seeding at this ratio ...
seed_time_mins = 60 # ... or after this long, whichever comes first
port_range = "6881-6889"
stall_mins = 30 # give up on a torrent making no progress
[feeds.atp]
url = "https://atp.fm/rss"
folder = "Accidental Tech Podcast" # default: the feed title
keywords = ["deep dive"] # OR across keywords, AND within one
allow_explicit = false
auto_download = true
max_new_per_check = 3 # the rest wait for the next scan
username = "ray" # optional HTTP basic auth
password_env = "IPX_ATP_PASS" # or a literal `password`
```
Retention keeps files that are `flagged` in the database, and deletes read episodes before unread
ones, oldest first.
## Socket protocol
Newline-delimited JSON over a Unix socket, both directions.
From source:
```sh
$ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
{"ev":"feed_start","feed":"atp"}
{"ev":"progress","feed":"atp","url":"...","file":"ep1.mp3","done":8192,"total":3000000}
{"ev":"download_done","feed":"atp","url":"...","path":"...","bytes":3000000}
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
{"ev":"scan_done","feeds":1}
cargo build --release
./target/release/ipx daemon
```
Commands: `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `status`.
Events: `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
`scan_done`, `reap_done` and `status` are terminal -- a client that asked for work stops there.
The first start creates **admin / ipodderx**. Sign in at `/login`, then change it:
Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy
daemon also sees that daemon's other work.
```sh
echo -n 'a good password' | ipx user passwd admin
```
## OPML
The UI is plain HTTP, so put TLS in front of it if it is reachable from outside your network.
Two different things, both supported:
## Documentation
**Importing and exporting** a file copies subscriptions in or out once — `ipx import subs.opml`,
`ipx export subs.opml`, or the OPML button in the UI.
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
other feed; every scan re-reads it and keeps your feed list in step. An OPML is recognised by its
content, so a URL without a `.opml` extension still works.
The feeds inside it are **not written to `config.toml`** — the OPML is the source of truth, so they
are re-derived each scan and held in the database. Your config keeps only what you chose. They show
as a collapsible folder in the sidebar, download into one nested folder, and newly listed ones are
scanned in the same run rather than waiting for the next interval. They inherit the subscription's
settings; change anything on one and it gets its own config entry from then on.
When a feed drops out of the OPML upstream:
| it has downloads | what happens |
| | |
|---|---|
| no | unsubscribed and removed from the config |
| yes | kept, flagged in the UI as no longer listed |
A downloaded file is never left behind with nothing explaining where it came from.
## Web UI
```toml
[web]
enabled = true
bind = "0.0.0.0:8080" # 127.0.0.1:8080 by default
token = "" # generated and written back on first run
```
`ipx daemon` then serves it in the same process (`ipx daemon --web ADDR` overrides the bind for one
run). On first start it mints a token, saves it to config.toml, and prints the URL to open:
```
web ui token generated. Open:
http://0.0.0.0:8080/?token=1f4c…
```
`?token=` sets a year-long cookie, so you only paste it once per browser. Everything is behind that
token, including `/media/...` — a cookie rather than a header precisely because an `<audio>` element
cannot send headers.
Browse feeds, read show notes, play episodes in the browser (Range requests are served, so seeking
works), download or delete individual files, mark episodes read or flag them to keep, and edit a
feed's folder/keywords/explicit/auto-download/limit settings. Config edits are written to
config.toml and hot-reloaded — no daemon restart.
Show notes are feed-supplied HTML from an untrusted source; they are sanitized with `ammonia`
server-side before they reach the page.
**It is plain HTTP.** On a LAN bind, the token and everything else crosses the network in the
clear — and a feed URL can itself contain a credential (Patreon's, for one, carries an auth token).
Put it behind a reverse proxy with TLS if that matters to you.
## Layout
Feeds down the side; the selected feed's episodes across the top; the selected episode's text and
its enclosures in the pane below, which is where you play, download or delete them. The divider
between the two panes drags and the position is remembered. An OPML subscription's page instead
lists the feeds inside it.
An item may carry several enclosures. All of them appear in the pane below; the row summarises the
one you would act on and notes how many others there are. Anything that is not audio or video gets a
View link — opening the publisher's copy, or the local one once downloaded — rather than a player.
## Log view
The **Log** button in the sidebar shows the running daemon's output live, in four tabs:
**Daemon I/O** is the control protocol itself — every command in and every event out, as JSON;
**Scans** is feed and download activity; **HTTP** is web requests; **All** is everything: feed scans, downloads,
torrent activity and every HTTP request, with level and text filters and a copy button. It reads a
2000-line ring buffer held inside the process (`/api/logs`), not a file — so it works the same under
Docker, where logs go to stdout and there is no file to tail. The buffer keeps `debug` even when the terminal does not, so protocol traffic and routine
skips are there without making stderr unreadable — `IPX_UI_LOG` changes what it captures and
`IPX_LOG` what reaches stderr.
| [docs/configuration.md](docs/configuration.md) | Every config key, path and environment variable |
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
| [docs/sso.md](docs/sso.md) | Signing in through Cloudflare Zero Trust or Authentik |
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, control socket, HTTP API |
| [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
| [docs/history.md](docs/history.md) | How it was built, with what was wrong and why |
| [CLAUDE.md](CLAUDE.md) | Notes for working on the code, including how production is deployed |
## Tests
```sh
cargo test # the server: parsing, filters, retention, schedules, SQL
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
node tests/page-smoke.js # the page script loads without throwing
npx playwright test # a real browser against a real daemon
npx playwright test # a real browser against a real daemon on fixture feeds
```
The Rust tests cover the server and the smoke test catches a script that fails to load, but neither
can see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which
is what has actually slipped through. The Playwright suite drives a headless browser against a
daemon started on fixture feeds, and each test maps to a bug that reached a user: the page rendering
empty, a dead theme toggle, settings not persisting, episode metadata, filter tabs, the feed URL
field and its copy button, the log tabs, and OPML folders.
`npm install` gets the test runner; the browser itself comes from
`npx playwright install --with-deps chromium` (in `install.sh`).
## Docker
```sh
docker compose up -d # builds the image and starts it
docker compose logs -f ipx # the token is printed on first start
```
`docker-compose.yml` mounts `./config`, `./data` and a downloads directory, publishes 8099 for the
UI and 6881 (TCP **and** UDP — DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
land owned the way Unraid shares expect. On first start the entrypoint writes a config bound to
`0.0.0.0`, since a container's loopback is not reachable from outside it, and prints the URL with
its generated token.
The healthcheck runs `ipx status`, which goes through the control socket to the command worker — so
it catches a daemon that is alive but wedged, not merely one that has died.
## Running it as a service
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
rather run periodic scans with no daemon (in which case there is no socket for a UI to attach to).
`npm install` gets the test runner, and `npx playwright install --with-deps chromium` gets the
browser.
## License
MIT. See [LICENSE](LICENSE).
## Signing in
A new database starts with **admin / ipodderx** at `/login`; change it with
`echo -n <password> | ipx user passwd admin`. Accounts are managed with `ipx user add|list|passwd|rm`.
To let Cloudflare Zero Trust or Authentik do the authenticating instead, see
[docs/sso.md](docs/sso.md).
MIT, see [LICENSE](LICENSE). The icons are [Font Awesome Free](https://fontawesome.com) 7.3.1 by
@fontawesome, under [CC BY 4.0](https://fontawesome.com/license/free), embedded as SVG.

27
TODO.md Normal file
View File

@@ -0,0 +1,27 @@
# To do
## Trim the state database
From an audit of the database layer and a read-only copy of production on 2026-09-12. The data
itself was clean: no leftover tables or indexes, 47 free pages, one stray `entry_state` row.
Check each against the code before cutting it.
- [x] **`migrate()`'s add list.** All eight columns arrived in 0.2.0, and 0.5.0 only supports
upgrades from 0.3.0 on. Drop the list and its loop; keep the `retired` drop loop, which a
database coming from 0.4.0 still needs. (`src/db.rs`)
- [x] **`Db::subscribed_feed_ids`.** No callers; its doc says the scanner walks it, and it does not.
(`src/db.rs`)
- [x] **`Db::subscriber_count`.** One caller, which only asks whether it is above zero:
`subscriber_counts()?.contains_key(&id)`. (`src/db.rs`, `src/web.rs`)
- [x] **The `created` columns** on `users`, `subscriptions` and `sessions`. Written on every insert,
never read. Add them to `retired` and drop them from the inserts. (`src/db.rs`)
- [x] **`Managed.orphaned`.** Selected by `managed_feeds()` on every call and never read;
`FeedSummary.orphaned` is what the UI uses. (`src/db.rs`)
## Popular
- [x] **Popular lists feeds the way Directory does**: the feeds inside an OPML or a Patreon
creator, never the collection itself. (`src/web.rs`, `web/index.html`)
After these: `cargo test`, `node tests/page-smoke.js`, `npx playwright test`. Copy `state.db`
aside before deploying: `migrate()` drops columns on the first start.

View File

@@ -1,7 +0,0 @@
[Unit]
Description=ipx feed scan (one shot)
[Service]
Type=oneshot
ExecStart=%h/.cargo/bin/ipx fetch
Environment=IPX_LOG=ipx=info

View File

@@ -1,18 +0,0 @@
# User unit: install to ~/.config/systemd/user/ipx.service, then
# systemctl --user enable --now ipx
# The socket lands in $XDG_RUNTIME_DIR/ipx.sock by default, so a UI running as the
# same user can attach without extra configuration.
[Unit]
Description=ipx podcatcher
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=%h/.cargo/bin/ipx daemon
Restart=on-failure
RestartSec=30
Environment=IPX_LOG=ipx=info
[Install]
WantedBy=default.target

View File

@@ -1,16 +0,0 @@
# Alternative to the daemon: a periodic one-shot scan, closer to how the original
# iPodderX agent was driven. Use this OR ipx.service, not both -- with no daemon
# running there is no socket, so a UI cannot attach.
#
# Install ipx-scan.service and ipx.timer to ~/.config/systemd/user/, then
# systemctl --user enable --now ipx.timer
[Unit]
Description=Periodic ipx feed scan
[Timer]
OnBootSec=5min
OnUnitActiveSec=1h
Persistent=true
[Install]
WantedBy=timers.target

146
docs/architecture.md Normal file
View File

@@ -0,0 +1,146 @@
# How it works
One binary, `ipx`. `ipx daemon` runs three things in one process: a scheduler, a Unix-socket
control server, and the web UI. Everything else is a CLI that either does the work itself or hands
it to a running daemon.
## Modules
| File | Responsibility | What it replaced in the Python |
|---|---|---|
| `src/main.rs` | CLI, dispatch, scan loop, download policy | `iPXAgent.py` |
| `src/config.rs` | TOML load/save, `General`/`Feed`/`Web`, intervals, slugs | `iPXSettings.py`, `feeds.plist` |
| `src/db.rs` | SQLite schema, migrations, every query | `.ipxd` plists, `history.dat`, `qmcache.dat` |
| `src/feed.rs` | Conditional GET, RSS/Atom/OPML parsing | `FeedData.__getFeed/__getEntries` |
| `src/download.rs` | Streaming download, naming, type sniffing, placement | `iPXDownloader.getFile` |
| `src/torrent.rs` | librqbit session, seeding limits, stall abort | vendored BitTorrent 4.2.1 |
| `src/retention.rs` | Quota and age sweeps | `iPXQuotaManager.py` |
| `src/ipc.rs` | Event and command types, the socket server | `printMSG` on stdout |
| `src/auth.rs` | Argon2id hashing, session tokens, header names | — |
| `src/web.rs` | axum: HTTP API, auth, SSE, media streaming | — |
| `src/logbuf.rs` | Ring buffer behind the UI's Log view | — |
| `web/index.html` | The whole front end, `include_str!`d into the binary | — |
The page is compiled in, so **editing `web/index.html` needs a rebuild**.
## A scan
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
re-derived into the database each scan, never written to config.toml. A Patreon creator link
(a token, no `show=`) with more than one show is treated the same way, before any fetch: its
shows come from Patreon's web API and each becomes a derived feed.
4. Record entries. A changed title or description flips the item back to unread.
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
a reason. What a filter skipped is judged again every scan, so a change of settings brings it
back. A feed in a group takes your settings on the group for anything you have not set on it.
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
or an error page — and is deleted.
## Data model
```
feeds id, url, title, image, etag, last_modified, last_checked, ttl_mins,
last_error, orphaned, group_id, managed
entries feed_id, guid, title, link, published, description, first_seen,
image, duration, episode, season PK (feed_id, guid)
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
bytes_done, downloaded_at, last_error
users id, name, pass_hash, is_admin, created, last_login
sessions token, user_id, seen
subscriptions user_id, feed_id, keywords, auto_download, allow_explicit,
max_new_per_check PK (user_id, feed_id)
entry_state user_id, feed_id, guid, read, flagged, position
PK (user_id, feed_id, guid)
```
Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position` columns from
before accounts; two bugs came from queries still reading them, and `migrate()` drops them from an
older database.
Schema changes: add the table or column to `SCHEMA`. `CREATE TABLE IF NOT EXISTS` leaves a table
that already exists alone, so a new column on one also goes in `migrate()`'s `wanted` list, and a
retired one in its `retired` list; both are checked with `PRAGMA table_info`. Columns from before
0.3.0, the oldest version an upgrade may start from, need no entry. `Db::memory()` runs the same
path as `Db::open`, so a migration cannot pass the tests while missing in production.
## Control socket
Newline-delimited JSON, both directions, over `$XDG_RUNTIME_DIR/ipx.sock`.
```sh
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
{"ev":"feed_start","feed":"atp"}
{"ev":"progress","feed":"atp","enclosure":42,"file":"ep1.mp3","done":8192,"total":3000000}
{"ev":"download_done","feed":"atp","enclosure":42,"path":"…","bytes":3000000}
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
{"ev":"scan_done","feeds":1}
```
**Commands**`fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `download`
(`enclosure`), `status`.
**Events**`feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
`scan_done`, `reap_done` and `status` are terminal: a client that asked for work stops reading
there. Commands run one at a time, in the order they arrive, except `status`: the socket answers it
straight away, so the Docker healthcheck is never left waiting behind a scan or a download, and
answers only the client that asked, since `status` would end any other client's session.
Progress carries the enclosure id, without which a UI cannot tell one download from another and
ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast,
so a client attached to a busy daemon also sees that daemon's other work.
Inside the process the same events go over a `tokio::broadcast`; commands arrive on an `mpsc` and
are handled by a single worker, so nothing races over the same download. Shutdown is a `watch`
channel raced *inside* each job — `tokio::select!` only races branches at the point of selection,
so a long download had to be able to notice the signal itself.
## HTTP API
Everything below `/api` needs a signed-in user; the browser gets a redirect to `/login`, anything
else a `401`.
| Route | |
|---|---|
| `GET /` | the app |
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable, sortable (`sort` = kept, title, feed, type, size or published; `dir` = asc or desc) |
| `GET /api/entries` | the same, across every feed you subscribe to (All Subscriptions) |
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
| `POST /api/read-all` | everything read in every feed you subscribe to (All Subscriptions) |
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, kept, position |
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
| `POST /api/fetch` | |
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with an OPML's feeds in place of the OPML and everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
| `GET /api/events` | SSE, the same broadcast the socket carries |
| `GET /api/logs` | admin-only; the ring buffer, with a sequence cursor |
| `GET /media/{id}` | the file, with Range support so seeking works |
Show notes are feed-supplied HTML from an untrusted source, sanitized with `ammonia` server-side
before they reach the page.
## Testing
```sh
cargo test # parsing, filters, retention, schedules, SQL, per-user isolation
node tests/page-smoke.js # the page script loads and every selector it wires at load exists
npx playwright test # a real browser against a real daemon on fixture feeds
```
The Rust tests cannot see a wrong selector, a handler that runs and does nothing, or a page that
renders empty — which is what has actually reached users. Each Playwright case maps to a bug that
did.
The suite starts its own daemon and database under `/tmp/ipx-ui-test`, wiped once per run. Tests
share that daemon and run in order, so a test that marks something read changes what later tests
see — make assertions that do not depend on earlier ones.

91
docs/cli.md Normal file
View File

@@ -0,0 +1,91 @@
# Command line
```
ipx [--config PATH] [--local] <command>
```
Every command that has a wire form probes the control socket first: if a daemon is running, the
daemon does the work and the CLI just renders the events it streams back. That is deliberate — two
processes must never download the same thing. `--local` forces the work to happen in-process.
| Command | What it does |
|---|---|
| `ipx list` | Subscriptions and their state |
| `ipx status` | Counts: feeds, pending, downloaded |
| `ipx fetch [FEED] [--force]` | Scan everything, or one feed. `--force` ignores the TTL |
| `ipx add <url> [--folder X] [--keywords a,b]` | Subscribe; the id comes from the feed title |
| `ipx rm <feed>` | Unsubscribe; downloads and history are kept |
| `ipx import <file.opml>` / `ipx export <file.opml>` | Move subscriptions in or out. Import subscribes the first admin, as the shared web token does; in the web UI it subscribes whoever is signed in |
| `ipx reap [--dry-run]` | Run retention now |
| `ipx user <add\|list\|passwd\|rm>` | Accounts for the web UI |
| `ipx daemon [--web ADDR]` | Scheduler, control socket and web UI |
## Accounts
Passwords are read from **stdin**, so they miss the shell history and any `ps` listing.
```sh
echo -n 'a good password' | ipx user add ray # local account
ipx user add ray@example.com --no-password # signs in through the proxy only
echo -n 'a good password' | ipx user passwd admin # change a password
ipx user list # who exists, and how each signs in
ipx user rm sam # account, subscriptions and read state
```
The first account created is an admin; later ones are ordinary users. A database with no accounts
at all gets **admin / ipodderx** on the next daemon start, announced in the log — change it.
To avoid even the command line, read it interactively:
```sh
read -s PW && echo -n "$PW" | ipx user passwd admin
```
## Scanning
```sh
ipx fetch # everything due
ipx fetch atp --force # one feed, ignoring its TTL and schedule
```
A scan: conditional GET (`If-None-Match` / `If-Modified-Since`), parse, record new entries, apply
the filters, then download up to the per-scan cap, newest first. A feed nothing has changed in
answers `304` and costs one request.
## Retention
```sh
ipx reap --dry-run # what would go, oldest first
ipx reap # actually delete
```
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
with no file are pruned from the database. **An item anyone kept keeps its file**, and one only counts
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
stops the next scan fetching it again.
## The daemon
```sh
ipx daemon # scheduler + socket + web UI
ipx daemon --web 0.0.0.0:8099 # override the configured bind for one run
```
One daemon per socket; a second refuses to start rather than fight over the database. It shuts down
cleanly on SIGTERM, including mid-download.
To kill it, match the binary exactly:
```sh
pkill -x ipx
```
`pkill -f ipx` matches the shell running the command too, and kills your own session.
## Talking to it directly
```sh
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
```
See [architecture.md](architecture.md#control-socket) for the protocol.

120
docs/configuration.md Normal file
View File

@@ -0,0 +1,120 @@
# Configuration
One TOML file, read at startup and re-read whenever the web UI writes to it — most changes take
effect without a restart. Default location `$XDG_CONFIG_HOME/ipx/config.toml`
(`~/.config/ipx/config.toml`), overridden with `--config` or `$IPX_CONFIG`.
| What | Where | Override |
|---|---|---|
| Config | `~/.config/ipx/config.toml` | `--config`, `$IPX_CONFIG` |
| Database | `~/.local/share/ipx/state.db` | `$IPX_DATA_DIR` |
| Control socket | `$XDG_RUNTIME_DIR/ipx.sock` | `[general] socket` |
| Downloads | `[general] download_dir` | — |
`~` is expanded in paths. The database is SQLite in WAL mode; back it up by copying `state.db`
while the daemon is stopped, or with `sqlite3 state.db .backup`.
## `[general]`
```toml
[general]
download_dir = "~/Podcasts"
socket = "/run/user/1000/ipx.sock"
schedule = "every 1h" # "every 30m", "every 4h", "2d", "90" (minutes)
organize = "feed" # "feed" | "date"
max_total_gb = 50 # 0 = unlimited
max_age_days = 30 # 0 = keep forever
max_new_per_check = 3 # per feed, per scan. 0 = unlimited
media_types = ["audio", "video"]
```
* **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
pad. Kept items are never deleted, and a file only counts as read once every subscriber has
read it. `0` disables it entirely.
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
Kept ones stay. `0` disables it.
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
fetched 216 files and 22 GB in one scan.
* **`media_types`** — top-level MIME types taken automatically. Anything else is still listed and
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
without this the disk fills with artwork. Empty takes everything.
## `[torrent]`
```toml
[torrent]
enabled = true
seed_ratio = 1.0 # stop seeding at this ratio ...
seed_time_mins = 60 # ... or after this long, whichever comes first
port_range = "6881-6889"
stall_mins = 30 # give up on a torrent making no progress
```
A `.torrent` body is handed to the torrent path whatever MIME type it was advertised as. Torrents
run on their own tasks (two at a time) so a slow swarm never blocks a scan.
## `[web]`
```toml
[web]
enabled = true
bind = "0.0.0.0:8099" # 127.0.0.1:8080 by default
token = "" # generated and saved on first run
trusted_header = "" # e.g. "Cf-Access-Authenticated-User-Email"
trusted_proxies = ["127.0.0.1", "::1"]
auto_create_users = true
sign_out_url = "" # e.g. "/cdn-cgi/access/logout"
session_days = 30
```
* **`token`** — the shared secret, which signs in as the **admin**. `?token=…` sets a cookie, so
you paste it once per browser. It is what the Docker healthcheck and any scripts use.
* **`trusted_header`** — a header naming the signed-in user, set by whatever fronts ipx. Empty
disables that path. See [sso.md](sso.md).
* **`trusted_proxies`** — addresses allowed to assert that header, and the entire security boundary
for it. Name the proxy, never a subnet.
* **`auto_create_users`** — create an account the first time the proxy vouches for a new name.
* **`sign_out_url`** — where Sign out sends someone the proxy signed in: the proxy's own sign-out,
`/cdn-cgi/access/logout` behind Cloudflare Access. Empty sends them to the sign-in page, where
the proxy signs them straight back in.
* **`session_days`** — sign a session out after this long without a request.
It is plain HTTP. On a LAN bind everything crosses the network in the clear — and a feed URL can
itself carry a credential. Put TLS in front of it if that matters.
## `[feeds.<id>]`
The table key is the feed id: stable, human-readable, and used in paths and the API. `ipx add`
derives it from the feed title.
```toml
[feeds.atp]
url = "https://atp.fm/rss"
folder = "Accidental Tech Podcast" # default: the feed title
schedule = "every 6h" # overrides [general] for this feed
media_types = ["audio"] # overrides [general] for this feed
username = "ray" # HTTP basic auth
password_env = "IPX_ATP_PASS" # preferred over a literal `password`
```
With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
`max_new_per_check` live on each person's subscription in the database**, not here — the values in
config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
and are the same for everyone. See [users.md](users.md).
Feeds derived from a subscribed OPML are **not** written here: the OPML is the source of truth and
they are re-derived on every scan. Editing one in the UI promotes it to a real config entry.
## Environment
| Variable | Effect |
|---|---|
| `IPX_CONFIG` | Config file path |
| `IPX_DATA_DIR` | Directory holding `state.db` |
| `IPX_LOG` | What reaches stderr (`ipx=debug`, `ipx::scan=debug`, …) |
| `IPX_UI_LOG` | What the in-process log buffer captures for the UI's Log view |
| `http_proxy` / `https_proxy` | Honoured for feed and enclosure fetches |

View File

@@ -1,58 +1,513 @@
# Progress
# How ipx was built
Running record of what has actually landed. Newest entry first.
The full design and step list live in the plan file at
`/config/.claude/plans/i-want-to-create-playful-quiche.md`.
The long-form record, newest first: what landed each day, what was wrong before, and what it cost
to find out. [CHANGELOG.md](../CHANGELOG.md) has the short version, by release; this is where the
reasoning lives. New write-ups go at the top.
## Build order
See [README.md](../README.md) for what the thing is.
- [x] **1. Repo skeleton** — git init (`main`), `cargo init --name ipx`, deps pinned, LICENSE,
README, this file.
- [x] **2. `config.rs` + `db.rs`** — TOML config structs + SQLite schema.
- [x] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries.
- [x] **4. `download.rs`** — downloads, filters, dedupe.
- [x] **5. `retention.rs`** — oldest-first quota + age reaper.
- [x] **6. `ipc.rs` + daemon** — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon.
- [x] **7. `torrent.rs`** — librqbit, seed to ratio/time, stall abort. (swarm download unverified —
see the step 7 entry)
- [x] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd units, README.
## 2026-09-12 — Healthy while busy
### Phase 2 — web front end
After a deploy the container sat at "starting" for a minute, and Docker's health log showed two
`ipx status` probes exceeding their 5-second timeout. The daemon's own log explained it. The first
scan after the start fetched 23 feeds, from 14:10:41 to 14:11:35, and both probes' `status`
commands waited in the job queue behind it; they were answered together at 14:11:35, straight after
`scan_done`. The worker runs one job at a time and `status` was one of its jobs, so any scan or
download longer than about a minute and a half, three 30-second probes, would have had Docker call
a working daemon unhealthy.
Decided with Ray: axum serving plain HTML/JS (no WASM toolchain), running **inside the daemon**
process so it reads SQLite and the event bus directly, LAN-bindable with a shared token.
The socket now answers `status` itself, from two short queries, and only real work goes through the
queue. The trade is that healthy now means the daemon answers on its socket and can read its
database; a worker stuck on one job would still pass. Asking a daemon that downloads hour-long
podcasts to be idle within five seconds was never a fair test of whether it was alive. A test holds
the queue full and checks `status` still comes back.
- [x] **9. Config hot-reload + web skeleton.** `Ctx.cfg` becomes `RwLock<Arc<Config>>` so the UI can
edit feeds without a daemon restart. `[web]` config section (enabled/bind/token, token
auto-generated and saved on first run). axum server started by `ipx daemon`, token checked by
middleware, `?token=` sets a cookie so `<audio>` requests authenticate too.
*Done when:* `ipx daemon` serves a page on the configured bind, and a wrong token gets 401.
- [x] **10. Browsing.** `/api/feeds`, `/api/feeds/:id/entries`, entry detail. Descriptions are
untrusted feed HTML — sanitized with `ammonia` before they reach the page.
*Done when:* the Glass Cannon feed's 131 entries browse and read correctly.
- [x] **11. Media actions.** Range-request audio streaming (`tower-http` ServeFile) so seeking
works, download-on-demand for a pending enclosure, delete a file, mark read/flagged.
*Done when:* an episode plays and seeks in a browser, and delete reaps the row.
- [x] **12. Feed configuration.** Add/remove feeds and edit folder, keywords, allow_explicit,
auto_download, max_new_per_check from the UI, written back to config.toml and hot-reloaded.
*Done when:* flipping allow_explicit in the UI takes effect on the next scan with no restart.
- [x] **13. Live progress + polish.** SSE from the existing broadcast bus so downloads show live.
README section, screenshot-free usage notes.
*Done when:* starting a fetch from the UI shows progress advancing without a reload.
The first version broadcast the answer, as the queued one had been. Timing `status` during a forced
scan in production showed the catch: `status` is a terminal event, so the `ipx fetch` watching that
scan stopped reading at the first probe and printed the status line as its last, while the scan
carried on. When `status` waited behind the scan it could never arrive first, so this had never
shown. The answer now goes only to the client that asked, and the test checks that another client
hears nothing.
Note: `read`/`flagged` finally get a writer here. Retention orders by them (see the step 5 entry),
and until now nothing set them.
## 2026-09-12 — Signing in through Authentik, for real
## Smoke tests
Ray could not get Authentik's sign-in to reach ipx, following `docs/sso.md`, which had been written
without ever being tried. Looking at the Cloudflare account through its API showed that side was
already complete. Authentik is Zero Trust's OpenID Connect identity provider; the Access application
`ipodderx` allows only it and a list of five addresses; the tunnel `rays-unraid` routes
`ipodderx.sdf1.net` to `192.168.1.130:8099`; DNS is a proxied CNAME to the tunnel. Access's log
showed `rays@sdf1.net` signing in through it. Nothing on Cloudflare was changed, so no other site
was touched.
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
2. `ipx fetch` again → no re-download, feed skipped for TTL.
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
`reaped`.
The gaps were all at ipx's end: `trusted_header` was empty, `trusted_proxies` held only loopback,
and the account was called `rays` while the header carries `rays@sdf1.net`.
Finding the address to trust took the most time. The page said `127.0.0.1`, but `cloudflared` runs in
its own container and reaches ipx through the host's published port. ipx logs no peer addresses, so
the address was read from `/proc/net/tcp` inside the ipx container: `192.168.16.1`, the gateway of
`content_default`, where Docker's masquerade puts traffic crossing from another bridge. A request
from Tower's own shell arrived as `192.168.1.130` instead, and a throwaway `busybox` on the default
bridge as `192.168.16.1`: the first was refused with the header, the second believed. LAN machines
keep their own addresses, since Docker forwards published ports with iptables (the userland proxy
only handles loopback).
Every change, in order, with how to undo it:
1. **Code**, commit `586d2c0`: `ipx user rename`, deployed. Revert the commit and redeploy to
remove it; nothing depends on it once used.
2. **Account**: `docker exec iPodderX ipx user rename rays rays@sdf1.net`. Same id, so its feeds,
read state, password and admin rights stayed. Undo: `docker exec iPodderX ipx user rename
rays@sdf1.net rays`. Signing in at `/login` now takes the new name.
3. **Config**, `/mnt/fast/appdata/ipodderx/config.toml`, `[web]`: `trusted_header` from `""` to
`"Cf-Access-Authenticated-User-Email"`, and `"192.168.16.1"` added to `trusted_proxies`. The
file as it was is `config.toml.2026-09-12-sso.bak` beside it. Undo: copy the backup back and
`docker compose -f /mnt/fast/arcane/projects/content/compose.yaml restart ipodderx`.
4. **Cloudflare, Docker networks and other containers**: unchanged. The `busybox` test container
was removed when it exited, and its image afterwards.
5. **Authentik**, later the same day, because ipodderx had no tile in its library while Outline
did: a bookmark application `ipodderx` (pk `5854a98e-816a-4c4f-9f27-63e69dc29d1d`), made
through the API with a token of Ray's. No provider and no policy bindings, like Outline's, the
iPodderX icon, and a link to `https://ipodderx.sdf1.net`. It changes nothing about who can sign
in. Undo: delete it under Applications → Applications, or
`DELETE /api/v3/core/applications/ipodderx/`.
6. **Signing out**, later again. Sign out landed on ipx's password page while Access still vouched
for Ray, so it signed nothing out, and the page looked like the wrong login. Cloudflare's
`/cdn-cgi/access/logout` ends the Access session for every Access application at once (there is
no per-application sign-out, and it takes no redirect), and Authentik's end-session only ends
one application's session unless single logout is set up there. Ray chose Access's sign-out. New
`[web] sign_out_url`, set to `/cdn-cgi/access/logout` in production (the file as it was is
`config.toml.2026-09-12-signout.bak`), and `/login` now sends anyone the proxy vouches for on to
`/`. Undo: take the key out and restart; the code does nothing without it.
What the address trusts is any container on Tower that connects through the host's port, not only
`cloudflared`. Verifying Cloudflare's signed `Cf-Access-Jwt-Assertion` would remove that, and is
the upgrade if it matters.
## 2026-09-12 — Trimming the state database
An audit of the database layer, with a read-only copy of production to check it against. The
data was already clean: no tables or indexes left from older versions, 47 free pages after the
column drops earlier the same day, and one stray `entry_state` row. The code had five things:
- `migrate()` still added eight columns to any table missing them. All eight shipped in 0.2.0 and
upgrades now start from 0.3.0 at the oldest, so the list and its loop went; the `retired` drop
list stays, since a database coming from 0.4.0 still has the old read columns.
- `created` on `users`, `subscriptions` and `sessions` was written by every insert and read by
nothing. They joined `retired`. The old-database test now builds all three tables, foreign keys
included, since `DROP COLUMN` on a table that references another was the part worth proving.
- `Db::subscribed_feed_ids` had no callers, though its doc said the scanner walked it.
`Db::subscriber_count` had one caller asking whether it was above zero, which
`subscriber_counts().contains_key` answers. `Managed.orphaned` was selected and never read.
- `users.created` came back the same afternoon, with `last_login` beside it. Nothing read it, but
when an account was made and when it last signed in is what you want to know when tidying
accounts, and it cannot be recovered later. Both existing accounts got their creation times back
from the backup taken before the drop, and a last sign-in from their newest session in it.
`last_login` is kept to the hour, because the proxy vouches for every request and that would
otherwise be a write each time.
## 2026-09-12 — Cutting what had outlived its reason
A whole-repo audit for over-engineering listed twelve things to cut, and all of them went.
- **Upgrades from before accounts.** `migrate_opml_children` moved OPML feeds that old versions
wrote into `config.toml` out to the database, and ran at every daemon start to do nothing after
the first. Production ran it in 0.3.0; anything older has to pass through 0.4.0.
- **Half of the adoption, and not the other half.** The audit called `adopt_existing_library` a
one-time migration and it was cut whole. It did two jobs: copy the old read state into
`entry_state`, which was dead, and subscribe the first admin to the whole catalogue while nobody
subscribed to anything, which is how a fresh install's first account gets `config.toml`'s feeds.
The browser suite caught it at once, signing in to an empty sidebar; `cargo test` had no idea.
The second job is back as `adopt_catalogue`, with a unit test of its own.
- **The dead `entries` columns.** `read`, `flagged` and `position` moved to `entry_state` with
accounts. The adoption's copy was their last reader, but `record_entry` still wrote them, and
still reset `read` when a title changed, which nothing looked at. Two bugs came from queries
reading them. `migrate()` now drops them from an existing database (SQLite has had `DROP COLUMN`
since 3.35), and a test builds an old table to prove it. On production each drop rewrote the
66 MB `entries` table, about four seconds apiece, so the first start took thirteen. An
`ipx status` run in that window failed with `migrating schema`: every `ipx` command migrates when
it opens the database, and it collided with the daemon doing the same. A failed `ALTER TABLE`
changes nothing, and the database had been copied to `backup/` first anyway.
- **`interval_mins`**, which `schedule` replaced. An old config that still has the key loads; the
key is ignored, and the config test carries it to keep that true.
- **Three dependencies.** `infer` was only asked whether a file is a torrent, and the check after
it already looked for `d8:announce`, which is what `infer` looks for. `dirs` was three lookups of
`XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME`. `tokio-stream` wrapped the broadcast receiver for
the event stream; `futures_util::stream::unfold` does the same, lagging clients included.
- **Two token generators.** The web token came from a copy of the session-token code, with a
clock fallback on top. It uses `auth::new_session_token` now, and is 64 characters.
- **The icon inlined four times**, 23 KB of base64 each, into both pages. It is `/icon.png` now,
outside the sign-in wall with `/login`, since the sign-in page shows it.
- Also: the `contrib/` systemd units from before the container, `Db::entries` and
`Db::count_entries` that only the tests called, three `logbuf` visitors that repeated the trait's
defaults, and unused state, a helper and dead CSS in the page.
## 2026-09-11 — A design pass on the web UI
A review against screenshots of every view in all three themes found that Dark and Light read as a
generic dark dashboard: one pale blue did every job, most labels were bold, and nothing led. The
list it produced, in `TODO.md`, was worked through in one go. What was worth knowing:
- **Amber means new.** Badges, unread dots and download bars take the icon's EQ amber; blue is left
for the primary action and links. Light's amber was `#b06f10`, which gives white text 4.1:1,
short of AA for 11 px bold. It is `#9a5f0a` now, 5.2:1.
- **EQ bars mark what is playing.** Three `<i>` bars stand at 60, 100 and 40 % and animate only
while `body.playing` is set. The first version left the animation on but paused, expecting each
bar to hold a different frame. The frames it held were within a pixel of each other, and on
screen the bars read as three dots. Under reduced motion one rule drops every animation and
transition, which leaves the bars standing.
- **The focus ring was clipped.** `.tgroup` and `#topbar` both set `overflow:hidden`, so a ring
drawn outside a toolbar button was cut off. Rings inside clipping parents are inset instead.
- **The feed list could not be used from the keyboard at all.** Rows were `<div>`s and the triangle
a `<span>`, so Tab went from the feed filter to Sign out. Rows now take focus, the triangle is a
`<button aria-expanded>`, and `renderFeeds` puts focus back on the same row after redrawing,
since every live update replaces every row. The list's own key handler stops Space and the
arrows from reaching the player's shortcuts on the document.
- **The triangle hangs in the margin.** Every row used to reserve an 18 px slot for it, pushing a
hundred feeds 28 px right of the places above for the sake of two folders. It is now absolutely
placed in the row's left padding, the full height of the row, so a near miss no longer opens
the folder's page.
- **A selected tile vanished** because the initials tile and the selected row were both `--raise`.
Tiles now mix their tint into `--bg`, which no row uses.
- **The Files pane hides itself** with `#split:has(>#files[hidden])`, which collapses its column.
The phone layout already hides the pane, and a zero-width extra track there is harmless.
- **Flagged became Kept** in the tab, and in the server's refusal to delete a file someone else
kept. The filter value and the column stay `flagged`; renaming those buys nothing.
## 2026-09-11 — A Patreon creator is a list of shows
Ray asked whether ipx could sync with Patreon. Not in full. The documented API (v2, the
`identity.memberships` scope) lists the creators you back and whether each has a feed (`has_rss`),
but no resource carries the `auth` token that makes a feed URL work. That token only comes from the
creator's page. It is also one per membership, not one per account: techpod's differs from Glass
Cannon's, so no single token finds everything you back.
What does work is one creator at a time, which is what Ray wanted for Glass Cannon and its 33 shows:
- `patreon.com/rss?auth=<token>`, with no creator named, returns that token's creator. Its self link,
about 660 bytes in, gives the campaign by number (`/rss/369921`). Patreon ignores `Range` here, so
ipx reads the stream until the number appears and hangs up, instead of taking all 2.8 MB.
- A show's `show=` number is a Patreon collection. Asked anonymously, the collection listing
(`/api/collection?filter[campaign_id]=`) and a post's `collections` both hide patron-only ones: you
get "FAQ". `/api/campaigns/<id>?include=shows` lists every show, anonymously, in one response.
- Every spelling works: `rss/glasscannon?auth=…&show=N`, `rss/369921?…` and `rss?auth=…&show=N` all
serve the same 131 items. Enclosure URLs are the same in the creator feed and the show feed, and
stable between fetches.
That last point shaped the design. `enclosures.url` is unique, so whichever feed is scanned first owns
the file. The first cut only asked a creator for its shows while it had no entries of its own, so that a
creator already read as a plain feed, holding every show's episodes, would never be split into shows
that came up empty. Within the hour that was the wrong call: Glass Cannon had gone into production on
the build before this one, been read as one feed of 2,385 items, and the rule kept it that way. Finding
anything in that heap was the problem Ray wanted solved.
So a creator with more than one show is always a group, run through the same sync as an OPML
(`sync_group`, split out of `sync_opml`). When it becomes one, its items are cleared and each show
takes over the enclosures the creator holds as the show lists them (`Db::adopt`), downloaded files and
everyone's read state included. One show leaves it a plain feed, which is what techpod already was. If
the shows cannot be listed, a creator already split fails the scan rather than being read as one heap;
one that never was is read as one feed until they can be. An answer without a `shows` list is an error,
not "no shows".
**Filter verdicts follow the settings.** Ray also reported that turning on Allow explicit and
rescanning brought nothing back. An item was judged once, when first seen, and `skipped` was final. The
2026-09-10 entry below saw it coming ("worth a `ipx retry <feed>` command if this bites"). It bit:
2,166 Glass Cannon items and all 88 of Shadowdark's sat at `skipped: explicit` with the setting on.
Every scan now runs the filters again over what they skipped (not over `torrents disabled`, which is not
a filter's call) and requeues what they now let through. Only that direction: a queued item is never
pulled back, because Download latest and a manual download both work by queueing.
**Two gaps beside it.** Add feed had no explicit box, so every new feed's first scan skipped all its
explicit items; it has one now, stored on your subscription like the feed dialog's. And a feed inside a
group ignored your settings on the group, though the group's dialog said they were inherited: settings
live on each person's subscription, and nothing read the group's. `Db::subscribers` now fills what you
have not set on the feed from your subscription to the group, and the feed list shows the same.
**A name that was already used.** Replaying the split on a copy of the production database left one
show with a Supercast episode in it. "Glass Cannon Live! Ascension | Pathfinder 2E" slugs to
`glass-cannon-live-ascension-pathfinder-2`, the id of a Supercast feed of the same show that had been
removed. Removing a feed keeps its rows on purpose, so that re-adding it does not fetch the back
catalogue again, but choosing a new id only checked config.toml and derived feeds. The Patreon show took
the old id and everything still filed under it. An id is now also taken when the database has a feed by
that id at a different URL; the same URL may still have it back, which is the re-add case.
Shows already added by hand are matched by token and show number, not by exact URL (`same_feed`), so a
bare token does not add Get in the Trunk and Shadowdark a second time under another spelling.
The show listing is Patreon's own undocumented web API. If it changes, only finding new shows stops.
## 2026-09-11 — One meaning per icon, sortable columns, and one player
Ray asked for a pass over the whole UI: consistent icons, and buttons placed next to what they act
on. The pass started from screenshots of every view and dialog, taken against the browser suite's
own daemon and fixtures. Several things did not agree:
- Unsubscribe was a minus in the toolbar but an x in the feed header. There it read as "close this
page", and x also meant Cancel in every dialog.
- Half the dialogs still had word buttons: Log, Add feed, Users, Unsubscribe, OPML, Popular's
Subscribe, the Copy buttons and Sign out.
- The player bar and the folder arrow were font characters, and so was the row's "playing" marker.
Those came out thin and varied from font to font.
- The OPML subscription page laid out its header differently from a feed's, and All Subscriptions
had no buttons at all.
The rule now is one meaning per icon. Minus unsubscribes, x closes or cancels, plus adds, subscribes
or imports, and a dialog's confirm button carries the icon of what it does. The words go in the
tooltip. The toolbar is grouped by what it acts on: add, unsubscribe and scan act on feeds; play,
read and keep act on the selected item. Its read and keep buttons now show that item's state, as
the item's own buttons already did. A browser test opens all eight dialogs and fails on any button
without an icon and a tooltip. It also checks that the toolbar and the page use the same icon for
the same action.
The pass turned up three bugs:
- **Escape ignored inside a dialog's text box.** The keydown handler returned early for any key
typed in a text box, and Escape sat below that check. So Add feed, which opens with the cursor in
its URL box, could not be closed with Escape. It showed up because the screenshot run stalled:
the dialog stayed open, and every later click landed on its overlay until the run timed out.
- **A white password box in the dark theme.** The input rule listed text, search and number
fields, but not password ones.
- **A stray "• •" under an item with no date.** Each detail carried its own trailing dot, whether
or not the next one was there.
All Subscriptions gained Mark everything read, as `POST /api/read-all`. It marks the feeds from the
person's subscriptions, which is the same set the view's query is scoped by. It asks first, since
one click covers every feed.
**Sorting** happens on the server. A list arrives fifty at a time, so sorting in the page would
only have reordered what was loaded. The column name picks one of a fixed set of SQL expressions
(`order_sql`), so nothing the browser sends reaches the query, and an unknown name means newest
first. Ties also fall back to newest first, so "Load more" does not skip or repeat items at a page
boundary. A database test runs every column both ways. Size came out of the File column into its
own, and small files show KB, since whole megabytes made every article image "0 MB".
**The double play.** Ray found that pressing play in the Files pane played the file twice at once.
The pane drew its own `<audio controls>`, and its `onplay` handler also started the player bar on
the same file. That meant two audio elements from one click. The pane now has a play button that
hands that exact file to the player bar, the only player. `play()` now takes the file. Its check
for resuming compares the file as well as the item, so another of an item's files starts from its
beginning instead of carrying on with the first. A test plays from the pane and checks the page
has one `<audio>` element.
---
## 2026-09-11 — The original's layout
Ray pointed at a screenshot of the Mac app (techpp.com, 2012) and asked for its panes, its
toolbar, and a Directory that lives in the feed list rather than behind a button. What it had, and
what ipx now does:
- A toolbar across the window: subscribe and unsubscribe, play, flag, refresh, and a search box
scoped to the feed on show. ipx's has the same groups, acting on the selected feed and item, with
Settings and Log at the right end, for admins only.
- A source list opening with Directory, Playlist Builder and All Subscriptions above the feeds.
ipx has Directory, Popular and All Subscriptions there, opening in the main pane. Playlist
Builder is left out, since nothing here builds playlists.
- The entries as a table, with a Files pane beside it and the entry below. The columns are unread,
kept, the item, its feed, its file and when. Sorting by column is not done yet.
- A status bar with the totals for what is on show.
All Subscriptions needed one new endpoint, `GET /api/entries`. It is the per-feed query with
`feed_id = ?` swapped for the person's subscriptions. The enclosure lookup that follows it used to
filter by feed as well; a page can now span feeds, so each file is matched to its row by feed and
guid instead.
A phone has no room for a pane beside the table, so there the files follow the item's text in the
full-screen reader, and the table drops to title and date.
---
## 2026-09-11 — Popular on this server
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
the list was fed. After every successful download, `iPXClass.py:1249` called `doPing`, which sent
the enclosure URL and the feed URL, unencoded and over plain HTTP, to
`directory.iPodderX.com/feedData/survey/files`. It was on by default, and only the Mac app's
preferences could switch it off (`anonFeedback`). The directory itself lived on that server and in
the Cocoa app, and both are gone.
ipx keeps the idea and drops the ping. Add feed lists what other accounts on this server subscribe
to, ranked by subscriber count. The old ping would have sent a Patreon feed's `auth=` key along
with everything else, so this is built around not doing that:
- A row carries an id, a title, artwork and a count. No URL reaches the page, so no key can.
- Subscribing goes by id through `POST /api/popular/{id}`, which checks the id against the same
list. Guessing a private feed's id gets a 400.
- Anything that looks private is never listed: a login configured for the feed, credentials in
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
`unlisted` flag is the upgrade if that happens.
It happened the same day. The first screenshot of the new Directory listed "Glass Cannon Live!
Ascension (for Ray Slakinski)", a Supercast feed at `feeds.supercast.com/feeds/<key>`. Treating
any long path segment as a key would have hidden public feeds too: acast's show ids look the
same. So paid-feed services are named instead (Patreon, Supercast, Supporting Cast, Glow,
Memberful), and any feed from one of them is private whatever its URL looks like. The per-feed
flag is still the upgrade for a service not on that list.
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
they would bury everything anyone chose on purpose.
---
## 2026-09-11 — Import an OPML file by uploading it
The import screen now has a file picker as well as the paste box. The page reads the file and
checks it looks like OPML before sending anything. If it doesn't, the page says so and clears the
picker, and it also clears the picker after an import. The file travels as text and is never
written to disk on the server, so there is nothing to clean up there.
The server now parses the OPML before touching anything, and returns `400` "that is not an OPML
file" instead of a `500`. `subscribe_opml` takes a parsed document, so `ipx import` also refuses a
non-OPML file by name before changing anything.
---
## 2026-09-11 — OPML import and export are per person
Importing an OPML now subscribes you to every feed in it. Feeds already in the catalogue cost
nothing, and unknown ones are added under the OPML's title. Before this, import only added URLs
missing from `config.toml` and subscribed nobody. So importing an export from another account did
nothing at all, and a genuinely new feed had no subscriber and was never scanned. The page said
"Imported 0 feed(s)". `ipx import` had the same gap; it now subscribes the first admin. Both go
through `subscribe_opml`.
Export now lists only your own subscriptions. It used to write out the whole catalogue to anyone
signed in, including other people's feeds and any private URLs in them.
Tests now cover every way in and out:
- Settings' Export OPML download and paste-to-import screen.
- A round trip that imports the admin's actual export into a second account and checks both
exports match. That round trip is exactly what failed.
- `ipx import` and `ipx export`, run in their own scratch config and database.
---
## 2026-09-11 — The log is admin-only
`GET /api/logs` now returns `403` to anyone who is not an admin, and the page hides the Log button
from them. Before this, every signed-in person could read the whole log. That includes every
account's name, every feed anyone subscribes to, and every failed sign-in. `/api/events` stays open
to everyone, because it carries the scan progress each person's page shows.
---
## 2026-09-11 — Managing users from the web
Settings has a **Manage users…** screen for an admin. From it you can add someone, with a password
or with none for someone the proxy signs in, tick or untick Admin, and remove an account. It is
backed by `GET/POST /api/users` and `PATCH/DELETE /api/users/{id}`, which return `403` for anyone
who is not an admin. The only admin cannot be demoted or removed, because nobody would then be
able to manage accounts except from the CLI on the box. Before this, accounts could only be managed
with `ipx user`.
---
## 2026-09-11 — Unread feeds first inside an OPML
An OPML subscription's feeds, both in the sidebar folder and on its own page, now list the ones with
unread items first. They were listed alphabetically, so with dozens of feeds the few with anything
new were scattered through the list. Within each half the order is still alphabetical. The browser
suite's fixture OPML gained a second feed, Aardvark Radio, which sorts first by name and by position,
so the new test only passes if unread wins.
---
## 2026-09-11 — Deploying is a Docker image
Production moved from a hand-started daemon in code-server to the `iPodderX` container in the Arcane
project `content`. `CLAUDE.md` now deploys by pushing to the registry at `192.168.1.130:5000` and
recreating that one service with `docker compose`. The old instructions copied a binary over a
process nobody supervised, so it did not come back after a reboot. Two known gaps are gone: the
image does support accounts, and the entrypoint drops to `99:100`, so downloads are no longer
owned by root. The README's Docker section said `docker compose up -d` builds the image and named the
service `ipx`; the compose file pulls from the registry and the service is `ipodderx`.
---
## 2026-09-11 — Documentation
`PROGRESS.md` became this changelog; the finished step lists moved to an appendix. The README is now
an overview that points at `docs/`: [configuration](configuration.md) (every key, path and
environment variable), [cli](cli.md), [users](users.md) (what several people share),
[sso](sso.md) (refreshed for accounts and the admin-only settings), and
[architecture](architecture.md) (modules, the scan, the schema, the socket protocol, the HTTP
API). `CLAUDE.md` collects what someone working on the code needs to know and has already been
learned the hard way here -- `pkill -x` not `-f`, the page being compiled in, the dead columns on
`entries`, the Playwright worker that deleted its own database.
---
## 2026-09-11 — Pruning respects a star from anyone
Same defect as the reaper, one function along: `prune_entries` guarded on `entries.flagged`, which
nothing has written since read state moved to `entry_state`. Starring a text item with no file would
not have saved it from the age sweep. It follows the reaper's rule now, and takes orphaned read
state with whatever it deletes.
---
## 2026-09-11 — Step D: one file, and everyone who wants it
The last of it, which is all about telling the truth before acting:
* **Delete says whose file it is.** A feed with other subscribers labels the button *Delete for
everyone*, and the confirmation names how many people share it. The server then has the last
word: if anyone else has starred the item or not played it yet, `DELETE /api/enclosures/{id}`
returns **409** with the reason in plain words, and only `?force=true` goes through. So the
rule is enforced where it matters rather than in the page that happens to be asking.
* **A feed's header says it is shared** -- "shared with 1 other person" -- which is the answer to
"why is there a file here I never asked for": someone else's subscription fetched it, and one
copy serves you both.
* Retention already respects the same rule from the entry before this one: starred by anyone keeps
it, read by everyone releases it.
`others_wanting` is tested with three subscribers disagreeing, and a browser test walks the whole
delete flow: the label, both prompts, declining the second, and the file still being there.
---
## 2026-09-11 — Retention caught up with per-user state
Moving read and starred into `entry_state` left `reap_candidates` reading `entries.read` and
`entries.flagged`, which nothing writes any more: **starring stopped protecting a file**, and the
"delete the ones already read first" ordering was dead. Harmless while the quota and age limits are
0 -- the reaper deletes nothing at all then -- but it would have bitten the moment one was set.
One file serves every subscriber, so both tests are now about all of them: **anyone** starring it
keeps it, and it only counts as read once **everyone** subscribed has read it. A file whose feed
nobody subscribes to has nobody left to keep it, so it sorts with the read ones. Tested with two
subscribers disagreeing.
---
## 2026-09-11 — Steps B and C: what is yours, what is everyone's
Read, starred and playback position moved out of `entries` into `entry_state (user_id, feed_id,
guid, ...)`; subscriptions became rows in `subscriptions (user_id, feed_id, ...)` carrying **your**
keywords, auto-download, explicit and per-scan limit. The feed list, unread counts, filters and
mark-all-read are all per person now. On first start the existing library is adopted by the admin:
2438 read/starred items and all 86 feeds, so nothing was lost.
The split follows from the file being shared:
* **Yours**: read state, starred, position, keywords, auto-download, explicit, per-scan limit,
and which feeds you see at all.
* **Everyone's**: the feed URL, its download folder, and when it is scanned -- there is one copy of
a file however many people subscribe, so those describe the file, not a preference. Admin-only,
refused with a 403 for anyone else rather than merely hidden.
Scanning merges the subscribers' wants, because one fetch and one file serve them all: an item is
downloaded if **anyone** wants it (any one person's keyword set matching is enough, and one person
taking everything removes the filter), auto-download is on if anyone has it on, and the per-scan cap
is the largest anyone asked for. `merge_policy` is a pure function with a test covering each of
those. Subscribing to a feed someone already has costs no second fetch and no second copy on disk;
unsubscribing takes it off your list alone, and only when the last subscriber leaves does the feed
stop being scanned.
**A test-harness bug worth naming**: Playwright imports the config in every worker, so the fixture's
`prepare()` ran again mid-run and deleted the data directory out from under the daemon. The daemon
kept serving from the unlinked inode while the CLI and any query opened a fresh empty database at
the same path -- which looked exactly like sign-in being broken. Only the launching process wipes
now (a worker has `TEST_WORKER_INDEX`).
---
@@ -107,34 +562,6 @@ who is signed in above the sidebar footer, with a sign-out, and a 401 sends the
Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B.
---
## Multi-user — the plan
Decided with Ray: **stay on SQLite** (Postgres was considered and dropped -- it is a deployment
choice, not a capability one, and nothing here contends for writes). Sign-in is either a local
username and password or the Cloudflare Zero Trust that already fronts `ipodderx.sdf1.net`, which
puts the authenticated identity in `Cf-Access-Authenticated-User-Email`. Feeds, items and files are **shared**; read state and subscriptions are **per user**.
The point of sharing: two people subscribed to the same show cost one fetch, one parse, and one file
on disk. `enclosures.url` is already globally UNIQUE, so the file half is nearly free.
- [x] **A. Users, sessions, sign-in.** `users` + `sessions` tables, Argon2 hashing, session cookie,
`ipx user add|list|passwd|rm`. A proxy header (`trusted_header` in `[web]`) signs in and
optionally creates a user -- honoured only from a `trusted_proxies` address, so a LAN client
cannot simply assert it. The existing shared token keeps working and resolves to the admin, so
the healthcheck and any scripts survive. Login page for direct access.
- [ ] **B. Per-user read state.** `entry_state(user_id, feed_id, guid, read, flagged, position)`;
the current columns on `entries` migrate into the first user's rows. Unread counts, filters and
playback position all become per user.
- [ ] **C. Per-user subscriptions.** `subscriptions(user_id, feed_id)`. config.toml stays the feed
catalogue; the UI lists only what you subscribe to. Adding a feed someone else already has costs
nothing. A feed nobody subscribes to stops being scanned but keeps its files.
- [ ] **D. One file, many users.** Auto-download when *any* subscriber wants it; retention never
deletes a file another user has starred or not yet played; deleting a download says so when
someone else still has it.
---
## 2026-09-10 — Items, not episodes
Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search
@@ -1185,3 +1612,37 @@ Gotcha worth keeping: three feature names in the plan were wrong against current
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
is the fix. Whole tree is rustls-only now, no C TLS dependency.
Next: step 2 — `config.rs` + `db.rs`. (done)
---
## How this was built
Three planned phases, all complete. The dated entries above are the record of each.
**The engine (steps 1-8)** — repo skeleton with pinned dependencies; `config.rs` + `db.rs` (TOML
config, SQLite schema); `feed.rs` (conditional GET, RSS-then-Atom parsing); `download.rs`
(streaming downloads, filters, URL dedupe); `retention.rs` (oldest-first quota and age reaper);
`ipc.rs` plus the daemon (Unix-socket JSON lines, TTL scheduler, CLI proxying to a running daemon);
`torrent.rs` (librqbit, seeding to a ratio or a time, stall abort); OPML import/export and polish.
**The web front end (steps 9-13)** — axum serving plain HTML and JS, no WASM toolchain, running
*inside* the daemon so it reads SQLite and the event bus directly: config hot-reload and the `[web]`
section; browsing feeds and entries with `ammonia`-sanitized show notes; media actions (Range
streaming, download on demand, delete, read/starred); feed configuration written back to
config.toml; live progress over SSE from the existing broadcast bus.
**Multi-user (steps A-D)** — accounts, sessions and sign-in (Argon2id, session cookie, a trusted
proxy header for Cloudflare Zero Trust); per-user read state; per-user subscriptions; and the rules
that follow from one file serving everyone. SQLite stayed: Postgres was considered and dropped, as
nothing here contends for writes.
### The live checks each step had to pass
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
2. `ipx fetch` again → no re-download, feed skipped for TTL.
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
`reaped`.

View File

@@ -1,9 +1,8 @@
# Signing in through Cloudflare Zero Trust or Authentik
# Signing in through Cloudflare Access and Authentik
ipx can take the signed-in identity from whatever sits in front of it, instead of asking for a
password itself. Both products below do the same thing in the end: they authenticate the person and
pass the result to the origin in a **header**. ipx reads that header, finds (or creates) the
matching account, and gets on with it.
password itself. The proxy authenticates the person and passes the result to ipx in a **header**;
ipx reads it, finds (or creates) the matching account, and gets on with it.
Read [How this is secured](#how-this-is-secured) before exposing anything. The short version: a
header is worth exactly as much as the hop that set it, so ipx only believes one from an address you
@@ -11,185 +10,173 @@ list.
---
## The ipx side (both setups)
## How ipodderx.sdf1.net does it
Checked end to end on 2026-09-12. An earlier version of this page had never been tried against a
real setup and pointed at the wrong address.
```
browser ─► Cloudflare Access, app "ipodderx" ─── sign in ───► Authentik (OpenID Connect)
─► tunnel "rays-unraid" (the cloudflared container on Tower)
─► http://192.168.1.130:8099 ─► ipx
```
Authentik is not in the request path. It is the identity provider Cloudflare Access asks. Access
then adds `Cf-Access-Authenticated-User-Email`, the email address Authentik gave it, to every
request it forwards through the tunnel, and ipx signs that person in.
| Piece | Where | Setting |
|---|---|---|
| Identity provider | Zero Trust → Settings → Authentication | `Authentik`, OpenID Connect; scopes `openid email profile` |
| Access application | Zero Trust → Access → Applications → `ipodderx` | Domain `ipodderx.sdf1.net`; identity providers: Authentik only, with instant auth; session 730h; policy *Require Login* allows a list of email addresses |
| Tunnel route | Zero Trust → Networks → Tunnels → `rays-unraid` → Public hostnames | `ipodderx.sdf1.net` → HTTP `192.168.1.130:8099` |
| DNS | `sdf1.net` | `ipodderx` CNAME to the tunnel, proxied |
| ipx | `/mnt/fast/appdata/ipodderx/config.toml`, `[web]` | below |
```toml
[web]
enabled = true
bind = "0.0.0.0:8099"
token = "…" # keep it: it is the admin, used by the healthcheck
# The header your proxy sets. Empty (the default) disables this whole path.
trusted_header = "Cf-Access-Authenticated-User-Email" # Authentik: "X-authentik-username"
# Addresses allowed to assert that header -- the proxy, and nothing else.
trusted_proxies = ["127.0.0.1", "::1"]
# Create an account the first time the proxy vouches for a name ipx has not seen.
bind = "0.0.0.0:8099"
trusted_header = "Cf-Access-Authenticated-User-Email"
trusted_proxies = ["127.0.0.1", "::1", "192.168.16.1"]
auto_create_users = true
sign_out_url = "/cdn-cgi/access/logout"
session_days = 30
```
Restart the daemon after editing. Accounts made this way have **no password**: they can only ever
arrive through the proxy. `ipx user list` marks them `proxy only`.
Restart ipx after editing it: `docker compose -f /mnt/fast/arcane/projects/content/compose.yaml
restart ipodderx`.
The first account created is an admin. Every later one is an ordinary user, and an ordinary user
cannot change global settings or how often feeds are scanned. Promote someone with:
### What was missing
Cloudflare and Authentik were already right. Three things on the ipx side were not:
1. **`trusted_header` was empty**, which switches the whole proxy path off. ipx ignored the header
and asked for a password.
2. **`trusted_proxies` listed only `127.0.0.1`.** The tunnel's requests do not come from there;
see the next section.
3. **The account had the wrong name.** It was made by hand as `rays`, but the header carries
`rays@sdf1.net`. With `auto_create_users` on, the first visit would have made a second, empty
account. `ipx user rename rays rays@sdf1.net` fixed that without losing anything.
### The address to trust, and why it is 192.168.16.1
`cloudflared` runs in its own container and reaches ipx through the host's published port. Docker
(iptables firewall backend) masquerades traffic between its bridge networks, so the tunnel's
requests arrive from the **gateway of ipx's own network**, `content_default`:
```sh
ipx user list
echo -n 'a good password' | ipx user passwd <name> # optional: also lets them sign in directly
docker network inspect content_default -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}'
```
Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN
when the tunnel is down. A brand new database starts with **admin / ipodderx** — change it.
That was measured, not assumed. ipx does not log where a request came from, so the addresses were
read from the kernel's connection table inside the container while the site was open. (`/proc/net/tcp`
lists them in hex.)
If the `content` project's network is ever recreated, its gateway can change. Check it again, and
update `trusted_proxies` to match.
### Names
The username is the email address, lower-cased: `rays@sdf1.net`. To sign in at `/login` with a
password from the LAN, use that name too.
To let someone else in, add their address to the Access policy; they need an Authentik account with
that email. With `auto_create_users = true` they get an ipx account on their first visit, as an
ordinary user with no feeds. An account made before the proxy can be given the name the proxy will
send:
```sh
docker exec iPodderX ipx user rename <old name> <email address>
```
### Signing out
**Sign out** sends someone the proxy signed in to `sign_out_url`, here Cloudflare's
`/cdn-cgi/access/logout`. That ends your Access session for **every** Access application,
`code.sdf1.net` included: Cloudflare has no way to end just one, and its sign-out page does not send
you anywhere afterwards. The next visit goes back through Authentik, which lets you straight in if
you are still signed in there. Signing out of Authentik itself is Authentik's own sign-out.
ipx never shows its password page to someone the proxy vouches for: `/login` sends them on to their
feeds.
### The tile in Authentik's library
Authentik's library lists Authentik's own applications, and ipodderx signs in through the one
called `Cloudflare Access`, so ipodderx needs a bookmark of its own to show up there. It is
Applications → Applications → `ipodderx`: no provider, launch URL `https://ipodderx.sdf1.net`, and
the iPodderX icon. Like Outline's, it has no policy bindings, so everyone in Authentik sees the
tile. Who actually gets in is still up to the Access policy.
### Check it
```sh
# From Tower itself: not a trusted address, so the header is ignored.
curl -s -H 'Accept: application/json' -H 'Cf-Access-Authenticated-User-Email: rays@sdf1.net' \
http://192.168.1.130:8099/api/me # -> sign in
# From a container on a Docker bridge, as cloudflared is: believed.
docker run --rm --network bridge mirror.gcr.io/library/busybox wget -qO- \
--header 'Accept: application/json' --header 'Cf-Access-Authenticated-User-Email: rays@sdf1.net' \
http://192.168.1.130:8099/api/me # -> {"admin":true,"name":"rays@sdf1.net"}
```
Then open `https://ipodderx.sdf1.net` in a private window. Authentik should ask who you are, and
ipx should show `rays@sdf1.net` in the sidebar footer without asking for a password.
---
## Cloudflare Zero Trust
## The ipx settings
This is what runs `ipodderx.sdf1.net`: a `cloudflared` tunnel to the origin, with an Access
application in front of it. Cloudflare authenticates the visitor and adds
`Cf-Access-Authenticated-User-Email` to every request it forwards.
### 1. The tunnel
In **Zero Trust → Networks → Tunnels**, either use the existing tunnel or create one, then add a
public hostname:
| Field | Value |
| Key | What it does |
|---|---|
| Subdomain / domain | `ipodderx` / `sdf1.net` |
| Type | HTTP |
| URL | `localhost:8099` (or the LAN address of the box) |
| `trusted_header` | The header the proxy sets. Empty, the default, turns the proxy path off. |
| `trusted_proxies` | The addresses allowed to set it. Nothing else is believed. |
| `auto_create_users` | Make an account the first time the proxy vouches for a name ipx has not seen. |
| `sign_out_url` | Where Sign out sends someone the proxy signed in: the proxy's own sign-out. Empty sends them to the sign-in page, where the proxy signs them straight back in. |
| `session_days` | How long a password sign-in lasts without use. |
Use `localhost` when `cloudflared` runs on the same machine as ipx — that keeps the origin request
coming from `127.0.0.1`, which is already in `trusted_proxies`. If `cloudflared` runs elsewhere (its
own container, another host), put **its** address in `trusted_proxies` instead, and make sure
nothing else can reach port 8099.
The first account ever created is an admin. Every later one is an ordinary user, who cannot change
global settings, a feed's URL or folder, or how often feeds are scanned: the API refuses those with
a `403`, not just the UI. Everything else about a feed is theirs alone; see [users.md](users.md).
### 2. The Access application
**Zero Trust → Access → Applications → Add an application → Self-hosted**:
- Application domain: `ipodderx.sdf1.net`
- Session duration: whatever suits; ipx keeps its own 30-day session on top.
- Add a policy — *Allow*, with a rule such as `Emails` → your address, or `Emails ending in`
your domain. Anyone this policy admits gets an ipx account when `auto_create_users` is on, so keep
the policy as narrow as the people you actually want reading your feeds.
### 3. Point ipx at the header
```toml
trusted_header = "Cf-Access-Authenticated-User-Email"
trusted_proxies = ["127.0.0.1", "::1"]
```
The username becomes the email address, lower-cased (`ray@example.com`). That is what shows in the
sidebar and what `ipx user list` prints.
### 4. Check it
```sh
# From the box itself: no header, no session -> the sign-in page.
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' -H 'Accept: text/html' http://127.0.0.1:8099/
# Pretending to be the tunnel (only works because 127.0.0.1 is trusted):
curl -s -H 'Cf-Access-Authenticated-User-Email: you@example.com' http://127.0.0.1:8099/api/me
```
Then load `https://ipodderx.sdf1.net` in a browser: Cloudflare should ask who you are, and ipx
should show your address in the sidebar footer without ever asking for a password.
Local sign-in at `/login` keeps working alongside the proxy, which is how you get in from the LAN
when the tunnel is down. So does the shared `[web] token`, which signs in as the admin and is the
way back in if you lock yourself out. A brand new database starts with **admin / ipodderx**;
change it.
---
## Authentik
## Authentik in the request path instead
Authentik does this with a **Proxy Provider** plus an **outpost**, which sits in the request path and
adds `X-authentik-username` (also `X-authentik-email`, `X-authentik-name`, `X-authentik-groups`).
Not what ipodderx.sdf1.net uses, and **not verified**. Authentik can also sit in front of ipx
itself, with a **Proxy Provider** and an **outpost** that adds `X-authentik-username`:
### 1. Provider
**Applications → Providers → Create → Proxy Provider**:
- Name: `ipx`
- Authorization flow: your usual (`default-provider-authorization-implicit-consent`)
- Mode: **Forward auth (single application)** if an existing reverse proxy fronts ipx, or
**Proxy** to let the outpost talk to ipx directly.
- External host: `https://ipodderx.example.net`
- Internal host (Proxy mode): `http://<ip of the ipx box>:8099`
### 2. Application and outpost
**Applications → Create**, bind it to that provider, and give it a policy so only the people you
mean are let through. Then add the provider to an outpost (**Applications → Outposts**, the embedded
one is fine).
### 3. Forward auth, if you use nginx/SWAG in front
In the server block for ipx:
```nginx
location /outpost.goauthentik.io {
proxy_pass http://authentik-server:9000/outpost.goauthentik.io;
proxy_set_header Host $host;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
add_header Set-Cookie $auth_cookie;
auth_request_set $auth_cookie $upstream_http_set_cookie;
}
location / {
auth_request /outpost.goauthentik.io/auth/nginx;
error_page 401 = @goauthentik_proxy_signin;
auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;
# This is the line that matters to ipx.
auth_request_set $authentik_username $upstream_http_x_authentik_username;
proxy_set_header X-authentik-username $authentik_username;
proxy_pass http://ipx:8099;
}
```
### 4. Point ipx at the header
```toml
trusted_header = "X-authentik-username"
trusted_proxies = ["172.18.0.5"] # the outpost or nginx container, NOT a whole subnet
```
Usernames arrive as Authentik knows them (`ray`), lower-cased.
- Applications → Providers → Create → Proxy Provider; mode **Proxy** (the outpost talks to ipx) or
**Forward auth** (an existing reverse proxy asks the outpost).
- Applications → Create, bound to that provider, with a policy; add the provider to an outpost.
- In ipx: `trusted_header = "X-authentik-username"`, and the outpost's or reverse proxy's address
in `trusted_proxies`. Measure that address as above rather than guessing it.
---
## How this is secured
**The header is only believed from `trusted_proxies`.** Every other source is ignored, and the
request falls through to a session cookie or the shared token. This is the whole security boundary,
so:
request falls through to a session cookie or the shared token. That is the whole security boundary.
- List the **proxy's own address**, not a range. `["127.0.0.1"]` when the tunnel runs beside ipx;
the container's IP when it does not.
- Never list a LAN subnet. Anyone on your network could then send
`Cf-Access-Authenticated-User-Email: admin@…` and be your admin.
- Make sure the origin port is not reachable *around* the proxy by anyone you would not admit
through it. If it is, bind ipx to `127.0.0.1` and let only the proxy reach it.
With the tunnel reaching ipx through the host's port, `192.168.16.1` means **any container on Tower
that connects to `192.168.1.130:8099`**, not only `cloudflared`. Machines on the LAN, and Tower
itself, arrive under their own addresses and cannot set the header; the checks above show both
sides. Never list a LAN address or range: anyone there could then send
`Cf-Access-Authenticated-User-Email: rays@sdf1.net` and be you.
Verify the refusal, don't assume it — set `trusted_proxies = ["10.9.9.9"]` briefly and confirm a
header from your machine gets a `401`:
**What ipx does not do:** it does not verify Cloudflare's signed `Cf-Access-Jwt-Assertion`. It
trusts the hop. Verifying the signature would make the containers on Tower irrelevant to the
boundary, and is the upgrade if that ever matters.
```sh
curl -s -o /dev/null -w '%{http_code}\n' \
-H 'Cf-Access-Authenticated-User-Email: someone@example.com' http://127.0.0.1:8099/api/me
```
**What ipx does not do:** it does not verify Cloudflare's `Cf-Access-Jwt-Assertion` signature or
Authentik's session. It trusts the hop. That is a deliberate trade — it keeps the configuration to
three lines — and it is sound exactly as long as the point above holds.
**Turning it off:** clear `trusted_header`. Existing proxy-only accounts stay, but nobody can sign
**Turning it off:** clear `trusted_header` and restart. Proxy-made accounts stay, but nobody can sign
in with them until they are given a password (`ipx user passwd <name>`).
---
@@ -197,16 +184,20 @@ in with them until they are given a password (`ipx user passwd <name>`).
## Everyday administration
```sh
ipx user list # who exists, and how each one signs in
ipx user list # who exists, how each signs in, and when
echo -n 'secret123' | ipx user add sam # local account, password on stdin
ipx user add sam --no-password # proxy-only account, created ahead of time
ipx user add sam@example.com --no-password # proxy-only account, made ahead of time
ipx user rename sam sam@example.com # give an account the name the proxy sends
echo -n 'newsecret' | ipx user passwd sam # change a password
ipx user rm sam # remove the account
```
Set `auto_create_users = false` once everyone who should have an account has one. After that the
proxy vouching for an unknown name is logged and refused, rather than quietly making an account.
In the container, put `docker exec iPodderX` in front, and `docker exec -i iPodderX` for the ones
that read a password.
Scanning intervals, the disk quota, retention and the download folder are **admin-only** the
Settings button is hidden for everyone else, and the API refuses the change even if the request is
made by hand. Ordinary users still control their own folders, keywords and downloads per feed.
Set `auto_create_users = false` once everyone who should have an account has one. After that the
proxy vouching for an unknown name is logged and refused. Make people ahead of time instead, with
the exact name the header will carry.
See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.

98
docs/users.md Normal file
View File

@@ -0,0 +1,98 @@
# Accounts, and what several people share
ipx serves any number of people from one copy of the data. The rule that decides everything else:
**there is one file on disk per enclosure URL.** Two people subscribed to the same show cost one
fetch, one parse and one file.
## What is yours, what is everyone's
| Yours alone | The same for everyone |
|---|---|
| Read, kept, playback position | The feed's URL |
| Which feeds you see at all | Its download folder |
| Keywords, auto-download, explicit, per-scan cap | When it is scanned |
| | The file on disk |
The right-hand column describes the feed and the file rather than a preference — two people wanting
different folders would mean two copies. Those three are **admin-only**, and the API returns `403`
for anyone else rather than merely hiding the controls.
## How the scanner merges everyone's wants
One fetch serves every subscriber, so the policy is a union:
* an item is downloaded if **anyone** wants it — one person's keyword set matching is enough, and
one person with no keywords removes the filter for that feed entirely
* auto-download is on if **anyone** has it on
* the per-scan cap is the **largest** anyone asked for
So "auto-download off" means *I don't cause downloads*, not *I never see them*. If someone else's
subscription pulls an item, you see it listed as downloaded and can play it, because the enclosure
is shared.
## Deleting
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
for everyone** and names them in the confirmation, and the server has the last word: if anyone else
has kept the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
reason, and only `?force=true` goes through.
Retention follows the same rule: an item anyone kept keeps its file, and it counts as read only once
every subscriber has read it.
## Signing in
Three ways, tried in order of how specific the claim is:
1. **A proxy header** naming the user — Cloudflare Zero Trust or Authentik. Honoured only from an
address in `trusted_proxies`. See [sso.md](sso.md).
2. **A session cookie** from signing in at `/login`. Argon2id hashes, sessions in the database,
idle timeout `session_days`.
3. **The shared `[web] token`**, which signs in as the admin — this is what the Docker healthcheck
and any scripts use.
A database with no accounts creates **admin / ipodderx** on the next daemon start and says so in
the log. Change it:
```sh
echo -n 'a good password' | ipx user passwd admin
```
## Adding someone
```sh
echo -n 'their password' | ipx user add sam
```
They sign in at `/login` and start with **no feeds**: subscriptions are per person. Adding a feed
someone else already has costs nothing — no second fetch, no second copy — it just appears on their
list with their own read state. Unsubscribing removes it from their list alone; only when the last
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
re-subscribing does not pull the back catalogue again.
**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
included. Directory lists every one of them A to Z. Your own feeds are marked Subscribed.
It shows a title, artwork and a count, never a URL or who reads it. An OPML subscription is listed
as the feeds inside it, one by one, and never the OPML itself, so you can take just the shows you
want. Anything that looks private is left out: a login configured for the feed, credentials in its URL,
or a key such as `auth=` or `token=` in the query, or a feed from a paid-feed service such as
Patreon or Supercast, which put the key in the path, and any feed inside an OPML that looks private
itself. Those are someone's paid subscriptions, and listing them would let anyone here read what
they pay for.
An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none
for someone the proxy signs in), tick or untick Admin, or remove an account. Removing one takes its
subscriptions and read state with it; downloaded files stay. The only admin cannot be demoted or
removed there, so there is always someone who can manage the rest.
## Admin
The first account is an admin. An admin can change global settings (scanning interval, quota,
retention, media types, download folder), a feed's URL, folder and schedule, and who has an account
and who else is an admin, and read the log, which names everyone's feeds and sign-ins. Everyone else
gets the Settings and Log buttons hidden and a `403` if they ask anyway.
```sh
ipx user list # the admin column says who
```

View File

@@ -26,9 +26,6 @@ pub struct General {
/// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d".
/// A feed's own `schedule` overrides this.
pub schedule: String,
/// Superseded by `schedule`. Still read so existing configs keep working.
#[serde(skip_serializing_if = "Option::is_none")]
pub interval_mins: Option<u64>,
pub organize: Organize,
/// 0 = unlimited.
pub max_total_gb: f64,
@@ -83,6 +80,10 @@ pub struct Web {
pub trusted_proxies: Vec<String>,
/// Create an account the first time the proxy vouches for a name it has not seen.
pub auto_create_users: bool,
/// Where Sign out sends someone the proxy signed in. Signing out of ipx alone cannot stick
/// while the proxy still vouches for them, so this is the proxy's own sign-out:
/// `/cdn-cgi/access/logout` behind Cloudflare Access. Empty sends them to /login.
pub sign_out_url: String,
/// Sign a session out after this long without a request.
pub session_days: i64,
}
@@ -96,6 +97,7 @@ impl Default for Web {
trusted_header: String::new(),
trusted_proxies: vec!["127.0.0.1".into(), "::1".into()],
auto_create_users: true,
sign_out_url: String::new(),
session_days: 30,
}
}
@@ -153,7 +155,6 @@ impl Default for General {
download_dir: home().join("Podcasts"),
socket: default_socket(),
schedule: "every 60m".into(),
interval_mins: None,
organize: Organize::Feed,
max_total_gb: 0.0,
max_age_days: 0,
@@ -176,8 +177,8 @@ impl Default for Torrent {
}
impl General {
/// Minutes between checks. Falls back to the legacy `interval_mins`, then to an hour.
/// A malformed value warns rather than stopping the daemon.
/// Minutes between checks, or an hour when `schedule` is empty or unreadable. A malformed
/// value warns rather than stopping the daemon.
pub fn interval(&self) -> u64 {
if let Some(n) = parse_interval(&self.schedule) {
return n;
@@ -185,7 +186,7 @@ impl General {
if !self.schedule.trim().is_empty() {
tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default");
}
self.interval_mins.filter(|n| *n > 0).unwrap_or(60)
60
}
}
@@ -278,9 +279,7 @@ pub fn config_path() -> PathBuf {
if let Ok(p) = std::env::var("IPX_CONFIG") {
return PathBuf::from(p);
}
dirs::config_dir()
.unwrap_or_else(|| home().join(".config"))
.join("ipx/config.toml")
xdg("XDG_CONFIG_HOME", ".config").join("ipx/config.toml")
}
/// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`.
@@ -288,9 +287,7 @@ pub fn data_dir() -> PathBuf {
if let Ok(p) = std::env::var("IPX_DATA_DIR") {
return PathBuf::from(p);
}
dirs::data_dir()
.unwrap_or_else(|| home().join(".local/share"))
.join("ipx")
xdg("XDG_DATA_HOME", ".local/share").join("ipx")
}
fn default_socket() -> PathBuf {
@@ -346,8 +343,16 @@ pub fn unique_slug(text: &str, taken: &BTreeMap<String, Feed>) -> String {
(2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap()
}
/// `$var`, or `~/fallback` when it is unset or empty, as the XDG base directory spec says.
fn xdg(var: &str, fallback: &str) -> PathBuf {
std::env::var_os(var)
.filter(|v| !v.is_empty())
.map(PathBuf::from)
.unwrap_or_else(|| home().join(fallback))
}
fn home() -> PathBuf {
dirs::home_dir().unwrap_or_else(|| PathBuf::from("."))
std::env::var_os("HOME").map(PathBuf::from).unwrap_or_else(|| PathBuf::from("."))
}
fn expand_tilde(p: &Path) -> PathBuf {
@@ -367,6 +372,8 @@ mod tests {
r#"
[general]
download_dir = "/tmp/pods"
# A key older versions read. An old config that still has it has to load.
interval_mins = 45
[feeds.example]
url = "https://example.com/feed.xml"
@@ -403,22 +410,17 @@ mod tests {
}
#[test]
fn interval_falls_back_through_legacy_then_default() {
fn interval_falls_back_to_an_hour() {
let mut g = General::default();
assert_eq!(g.interval(), 60, "the default schedule");
g.schedule = "every 15m".into();
assert_eq!(g.interval(), 15);
// A config written before `schedule` existed still works.
// Empty or garbage must not stop the daemon.
g.schedule = String::new();
g.interval_mins = Some(45);
assert_eq!(g.interval(), 45);
// Garbage must not stop the daemon.
assert_eq!(g.interval(), 60);
g.schedule = "whenever".into();
assert_eq!(g.interval(), 45);
g.interval_mins = None;
assert_eq!(g.interval(), 60);
}

911
src/db.rs

File diff suppressed because it is too large Load Diff

View File

@@ -223,7 +223,7 @@ enum Sniffed {
/// 2008 and so always answered 'data'.
async fn sniff(path: &Path) -> Result<Sniffed> {
let head = read_head(path, 512).await?;
if infer::is(&head, "torrent") || head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
if head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
return Ok(Sniffed::Torrent);
}
let text = String::from_utf8_lossy(&head);

View File

@@ -117,6 +117,103 @@ pub fn opml_title(bytes: &[u8]) -> Option<String> {
.filter(|t| !t.is_empty())
}
/// The token and show of a Patreon feed link, or None for any other URL.
///
/// Patreon gives each patron one token per creator. With no show it stands for the creator,
/// whose feed carries every show at once.
fn patreon_parts(url: &str) -> Option<(String, Option<String>)> {
let u = url::Url::parse(url).ok()?;
if !matches!(u.host_str()?, "patreon.com" | "www.patreon.com") || !u.path().starts_with("/rss") {
return None;
}
let param = |name: &str| u.query_pairs().find(|(k, _)| k == name).map(|(_, v)| v.into_owned());
Some((param("auth")?, param("show")))
}
/// A Patreon link naming a creator but no show.
pub fn is_patreon_creator(url: &str) -> bool {
matches!(patreon_parts(url), Some((_, None)))
}
/// What was typed into Add feed, as a URL. A bare Patreon token is taken as its creator's
/// feed, since the token alone says whose it is.
pub fn expand_input(input: &str) -> String {
let s = input.trim();
let token = s.len() >= 20 && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_');
if token { format!("https://www.patreon.com/rss?auth={s}") } else { s.to_owned() }
}
/// Whether two URLs are the same feed. One Patreon show has several spellings -- by the
/// creator's name, by number, or with no creator at all -- and the token and show are what
/// identify it.
pub fn same_feed(a: &str, b: &str) -> bool {
a == b || patreon_parts(a).is_some_and(|p| Some(p) == patreon_parts(b))
}
/// A Patreon creator's name and shows, each show as (title, feed URL).
///
/// ponytail: Patreon's own web API, undocumented, asked without signing in. If it changes,
/// finding shows stops and the show feeds already found keep working. The documented API
/// needs an OAuth client per install and does not list shows.
pub async fn patreon_shows(
client: &reqwest::Client,
url: &str,
) -> Result<(Option<String>, Vec<(String, String)>)> {
// The creator feed names its campaign by number in its self link, a few hundred bytes in.
// The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
let mut resp = client.get(url).send().await.context("connecting")?;
if !resp.status().is_success() {
return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
}
let mut head = Vec::new();
while patreon_campaign(&head).is_none() && head.len() < 64 * 1024 {
let Some(chunk) = resp.chunk().await.context("reading the feed")? else { break };
head.extend_from_slice(&chunk);
}
let campaign = patreon_campaign(&head)
.ok_or_else(|| anyhow!("the Patreon feed does not say whose it is"))?;
let api = format!(
"https://www.patreon.com/api/campaigns/{campaign}\
?include=shows&fields%5Bcampaign%5D=name&fields%5Bcollection%5D=title"
);
let resp = client.get(api).send().await.context("asking Patreon for the shows")?;
if !resp.status().is_success() {
return Err(anyhow!("Patreon would not list the shows: HTTP {}", resp.status()));
}
let (name, shows) = parse_patreon_shows(&resp.bytes().await.context("reading the shows")?)?;
Ok((name, shows.into_iter().map(|(id, title)| (title, format!("{url}&show={id}"))).collect()))
}
/// The campaign number in the start of a Patreon feed.
fn patreon_campaign(head: &[u8]) -> Option<String> {
let text = String::from_utf8_lossy(head);
text.match_indices("patreon.com/rss/").find_map(|(i, m)| {
let id: String = text[i + m.len()..].chars().take_while(char::is_ascii_digit).collect();
(!id.is_empty()).then_some(id)
})
}
/// A campaign's name and its shows as (id, title), from Patreon's JSON:API answer.
fn parse_patreon_shows(json: &[u8]) -> Result<(Option<String>, Vec<(String, String)>)> {
let v: serde_json::Value = serde_json::from_slice(json).context("Patreon's answer is not JSON")?;
// Missing is not the same as none. Read as no shows, the creator feed would be scanned as
// a plain feed, claim every show's files, and leave the shows empty once the list returned.
let ids = v["data"]["relationships"]["shows"]["data"]
.as_array()
.ok_or_else(|| anyhow!("Patreon's answer does not list the shows"))?;
let title = |id: &str| -> Option<String> {
let show = v["included"].as_array()?.iter().find(|x| x["type"] == "collection" && x["id"] == id)?;
show["attributes"]["title"].as_str().map(|t| t.trim().to_owned())
};
let shows = ids
.iter()
.filter_map(|s| s["id"].as_str())
.map(|id| (id.to_owned(), title(id).unwrap_or_else(|| format!("Show {id}"))))
.collect();
Ok((v["data"]["attributes"]["name"].as_str().map(str::to_owned), shows))
}
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
match rss::Channel::read_from(bytes) {
@@ -243,7 +340,7 @@ fn from_rss(ch: rss::Channel, bytes: &[u8]) -> ParsedFeed {
link: non_empty(item.link()),
published: item.pub_date().and_then(parse_date),
// Content wins over description, as __getEntries preferred entry.content.
description: non_empty(item.content()).or_else(|| non_empty(item.description())),
description: body(item.content(), item.description()),
categories: item
.categories()
.iter()
@@ -309,11 +406,7 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
title: non_empty(Some(e.title().as_str())),
link: alt.map(str::to_owned),
published: e.published().or(Some(e.updated())).map(|d| d.timestamp()),
description: e
.content()
.and_then(|c| c.value())
.or_else(|| e.summary().map(|s| s.as_str()))
.map(str::to_owned),
description: body(e.content().and_then(|c| c.value()), e.summary().map(|s| s.as_str())),
categories: e.categories().iter().map(|c| c.term().to_owned()).collect(),
explicit: false,
image: None,
@@ -358,6 +451,25 @@ fn non_empty(s: Option<&str>) -> Option<String> {
s.map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned)
}
/// An item's show notes: its full body when that is whole, else its description.
///
/// libsyn served Daily Meditation Podcast's `content:encoded` cut at the `>` inside a class name
/// pasted from a web app (`[&:has([data-writing-block])>*]:pointer-events-auto`), so the body
/// began halfway through a tag and the page showed the rest of the tag as text. The same item's
/// `description` was whole. With no description to fall back on, a damaged body beats none.
fn body(content: Option<&str>, description: Option<&str>) -> Option<String> {
non_empty(content)
.filter(|c| !starts_mid_tag(c))
.or_else(|| non_empty(description))
.or_else(|| non_empty(content))
}
/// Text that closes an attribute list (`">`) before any tag has opened is the tail of a tag whose
/// start was cut off.
fn starts_mid_tag(html: &str) -> bool {
html[..html.find('<').unwrap_or(html.len())].contains("\">")
}
/// The picture to show beside an item, in order of how deliberate it is:
/// `itunes:image`, then Media RSS `media:thumbnail`, then a `media:content` that is an
/// image, and finally an image enclosure -- which is how a blog's article picture arrives
@@ -451,6 +563,18 @@ mod tests {
);
}
#[test]
fn a_body_cut_off_mid_tag_gives_way_to_the_description() {
// How libsyn served Daily Meditation Podcast #3477: content:encoded began inside a tag.
let cut = r#"*]:pointer-events-auto R6Vx5W_threadScrollVars" dir="auto" data-turn="assistant"> <p>What if</p>"#;
let whole = r#"<div class="[&:has([data-writing-block])>*]:pointer-events-auto"><p>What if</p></div>"#;
assert_eq!(body(Some(cut), Some(whole)).as_deref(), Some(whole));
assert_eq!(body(Some("<p>Notes</p>"), Some("Summary")).as_deref(), Some("<p>Notes</p>"), "a whole body wins");
assert_eq!(body(Some("Plain notes, no tags."), Some("Summary")).as_deref(), Some("Plain notes, no tags."));
assert_eq!(body(Some(cut), None).as_deref(), Some(cut), "a damaged body beats none");
assert_eq!(body(None, Some("Summary")).as_deref(), Some("Summary"));
}
#[test]
fn feed_level_explicit_overrides_entries() {
let xml = br#"<?xml version="1.0"?>
@@ -550,6 +674,45 @@ mod tests {
assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
}
#[test]
fn a_patreon_creator_is_a_list_of_its_shows() {
let tok = "AbCdEfGhIjKlMnOpQrStUvWxYz012_-9";
assert_eq!(expand_input(&format!(" {tok} ")), format!("https://www.patreon.com/rss?auth={tok}"));
assert_eq!(expand_input("https://example.com/rss"), "https://example.com/rss");
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss/glasscannon?auth={tok}")));
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss?auth={tok}")));
assert!(!is_patreon_creator(&format!("https://www.patreon.com/rss/x?auth={tok}&show=1")), "one show is a feed");
assert!(!is_patreon_creator(&format!("https://example.com/rss?auth={tok}")));
// The show you already have by name is the one a bare token would add by number.
assert!(same_feed(
&format!("https://www.patreon.com/rss/glasscannon?auth={tok}&show=2073588"),
&format!("https://www.patreon.com/rss?auth={tok}&show=2073588"),
));
assert!(!same_feed(
&format!("https://www.patreon.com/rss?auth={tok}&show=1"),
&format!("https://www.patreon.com/rss?auth={tok}&show=2"),
));
// The self link carries the campaign by number, whichever spelling was asked for.
let head = br#"<rss><channel><link>https://www.patreon.com/glasscannon</link>
<atom:link href="https://www.patreon.com/rss/369921?auth=t" rel="self"/>"#;
assert_eq!(patreon_campaign(head).as_deref(), Some("369921"));
assert_eq!(patreon_campaign(b"<rss><channel><title>T"), None);
let json = br#"{"data":{"id":"369921","type":"campaign","attributes":{"name":"The Glass Cannon Network"},
"relationships":{"shows":{"data":[{"id":"2073588","type":"collection"},{"id":"2073636","type":"collection"}]}}},
"included":[{"id":"2073588","type":"collection","attributes":{"title":"Get in the Trunk "}},
{"id":"2073636","type":"collection","attributes":{"title":"Shadowdark"}}]}"#;
let (name, shows) = parse_patreon_shows(json).unwrap();
assert_eq!(name.as_deref(), Some("The Glass Cannon Network"));
assert_eq!(shows, [("2073588".into(), "Get in the Trunk".into()), ("2073636".into(), "Shadowdark".into())]);
// An answer that stops naming the shows is an error, never "this creator has none".
assert!(parse_patreon_shows(br#"{"data":{"attributes":{"name":"X"}}}"#).is_err());
}
#[test]
fn an_item_may_carry_several_enclosures() {
// The rss crate keeps only one per item -- the last -- so these come from the XML.

View File

@@ -62,10 +62,10 @@ impl Event {
Event::DownloadDone { path, .. } => format!(" saved {path}"),
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
Event::Reaped { path, bytes } => {
format!("reap {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
format!("deleted {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
}
Event::ReapDone { files, bytes } => format!(
"reaped {files} file(s), {:.1} MB",
"deleted {files} old file(s), {:.1} MB",
*bytes as f64 / 1_048_576.0
),
Event::Status { feeds, pending, downloaded } => {
@@ -172,11 +172,18 @@ pub async fn daemon_is_live(path: &Path) -> bool {
UnixStream::connect(path).await.is_ok()
}
/// Answers `status` for the socket, without the worker. The worker runs one job at a time, and a
/// healthcheck left waiting behind a scan or a long download timed out and called a busy daemon
/// dead. The answer goes to the client that asked and no one else: broadcast, it ended any
/// `ipx fetch` that was watching a scan, since `status` is a terminal event.
pub type StatusFn = std::sync::Arc<dyn Fn() -> Event + Send + Sync>;
/// Accepts connections, feeding commands to `cmds` and events from `events` back out.
pub async fn serve(
path: PathBuf,
events: broadcast::Sender<Event>,
cmds: mpsc::Sender<Command>,
status: StatusFn,
) -> Result<()> {
// A socket file left by a crashed daemon would block the bind; a live one was already
// rejected by the caller's daemon_is_live() check.
@@ -195,8 +202,9 @@ pub async fn serve(
let (stream, _) = listener.accept().await?;
let rx = events.subscribe();
let cmds = cmds.clone();
let status = status.clone();
tokio::spawn(async move {
if let Err(e) = handle(stream, rx, cmds).await {
if let Err(e) = handle(stream, rx, cmds, status).await {
tracing::debug!(error = %e, "client gone");
}
});
@@ -207,12 +215,21 @@ async fn handle(
stream: UnixStream,
mut rx: broadcast::Receiver<Event>,
cmds: mpsc::Sender<Command>,
status: StatusFn,
) -> Result<()> {
let (read, mut write) = stream.into_split();
// Events out.
// Events out: everything broadcast, and the answers meant for this client alone.
let (reply, mut replies) = mpsc::channel::<Event>(4);
let writer = tokio::spawn(async move {
while let Ok(ev) = rx.recv().await {
loop {
let ev = tokio::select! {
Some(ev) = replies.recv() => ev,
got = rx.recv() => match got {
Ok(ev) => ev,
Err(_) => break,
},
};
let mut line = serde_json::to_string(&ev).unwrap_or_default();
line.push('\n');
if write.write_all(line.as_bytes()).await.is_err() {
@@ -229,6 +246,15 @@ async fn handle(
continue;
}
match serde_json::from_str::<Command>(line) {
// Answered here, not queued behind whatever the worker is on: see StatusFn.
Ok(Command::Status) => {
tracing::info!(target: "ipx::io", "-> {line}");
let ev = status();
if let Ok(json) = serde_json::to_string(&ev) {
tracing::info!(target: "ipx::io", "<- {json}");
}
let _ = reply.send(ev).await;
}
Ok(cmd) => {
if cmds.send(cmd).await.is_err() {
break; // Worker is gone; so are we.
@@ -329,4 +355,29 @@ mod tests {
}
.is_terminal());
}
#[tokio::test]
async fn status_is_answered_while_the_worker_is_busy() {
// The queue is full and nobody drains it, as when the worker is deep in a long download:
// anything sent to it would wait for ever.
let (cmds, _worker) = mpsc::channel::<Command>(1);
cmds.send(Command::Reap { dry_run: true }).await.unwrap();
let (events, _) = broadcast::channel::<Event>(8);
// Another client, watching a scan: it must not be handed someone else's answer, which
// would end its session.
let mut watcher = events.subscribe();
let status: StatusFn = std::sync::Arc::new(|| Event::Status { feeds: 1, pending: 2, downloaded: 3 });
let (client, server) = UnixStream::pair().unwrap();
tokio::spawn(handle(server, events.subscribe(), cmds, status));
let (read, mut write) = client.into_split();
write.write_all(b"{\"cmd\":\"status\"}\n").await.unwrap();
let line = tokio::time::timeout(std::time::Duration::from_secs(2), BufReader::new(read).lines().next_line())
.await
.expect("status waited behind the worker")
.unwrap()
.unwrap();
assert!(line.contains(r#""ev":"status""#), "{line}");
assert!(watcher.try_recv().is_err(), "the answer went to every client, not just the one asking");
}
}

View File

@@ -111,18 +111,11 @@ impl Visit for Collect {
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
self.add(field, format!("{value:?}"));
}
// Numbers and bools reach record_debug through the trait's defaults, which prints them the
// same way. A string would print quoted there, hence its own method.
fn record_str(&mut self, field: &Field, value: &str) {
self.add(field, value.to_owned());
}
fn record_i64(&mut self, field: &Field, value: i64) {
self.add(field, value.to_string());
}
fn record_u64(&mut self, field: &Field, value: u64) {
self.add(field, value.to_string());
}
fn record_bool(&mut self, field: &Field, value: bool) {
self.add(field, value.to_string());
}
}
#[cfg(test)]

View File

@@ -99,6 +99,9 @@ enum UserCmd {
Passwd { name: String },
/// Delete an account and everything it knows: its subscriptions and read state
Rm { name: String },
/// Rename an account, keeping its feeds, read state and admin rights. This is how an
/// account made before the proxy takes the name the proxy signs it in as
Rename { name: String, new_name: String },
}
/// What a brand new database starts with, so there is always a way in. Announced loudly
@@ -273,11 +276,16 @@ fn user_cmd(ctx: &Arc<Ctx>, cmd: UserCmd) -> Result<()> {
println!("no accounts yet: ipx user add <name>");
}
for u in users {
let added = u
.created
.and_then(|t| chrono::DateTime::from_timestamp(t, 0))
.map_or("?".into(), |d| d.format("%Y-%m-%d").to_string());
let seen = u.last_login.map_or("never signed in".into(), |t| format!("signed in {}", ago(Some(t))));
println!(
"{:<20} {:<8} {}",
"{:<20} {:<6} {:<11} added {added} {seen}",
u.name,
if u.is_admin { "admin" } else { "" },
if u.pass_hash.is_some() { "password" } else { "proxy only" }
if u.pass_hash.is_some() { "password" } else { "proxy only" },
);
}
Ok(())
@@ -292,6 +300,22 @@ fn user_cmd(ctx: &Arc<Ctx>, cmd: UserCmd) -> Result<()> {
println!("password changed for {name}");
Ok(())
}
UserCmd::Rename { name, new_name } => {
let name = name.trim().to_ascii_lowercase();
// The same rules as a name the proxy vouches for, or the proxy would never find it.
let new_name = crate::auth::name_from_header(&new_name)
.ok_or_else(|| anyhow::anyhow!("not a usable name: no commas, semicolons or line breaks"))?;
let user = ctx
.db
.user_by_name(&name)?
.ok_or_else(|| anyhow::anyhow!("no such account: {name}"))?;
if ctx.db.user_by_name(&new_name)?.is_some() {
anyhow::bail!("{new_name} already exists");
}
ctx.db.rename_user(user.id, &new_name)?;
println!("renamed {name} to {new_name}");
Ok(())
}
UserCmd::Rm { name } => {
let name = name.trim().to_ascii_lowercase();
let user = ctx
@@ -315,14 +339,24 @@ async fn run(ctx: &Arc<Ctx>, cmd: Cmd) -> Result<()> {
Cmd::Reap { dry_run } => reap(ctx, dry_run, true),
Cmd::Download { enclosure } => download_one(ctx, enclosure).await,
Cmd::Status => {
let (pending, downloaded) = ctx.db.counts()?;
let feeds = subscriptions(ctx).map(|s| s.len()).unwrap_or(0);
ctx.out.emit(Event::Status { feeds, pending, downloaded });
ctx.out.emit(status(ctx));
Ok(())
}
}
}
/// The counts `ipx status` prints. A running daemon's socket answers with this directly rather
/// than through the job queue.
fn status(ctx: &Ctx) -> Event {
match ctx.db.counts() {
Ok((pending, downloaded)) => {
let feeds = subscriptions(ctx).map(|s| s.len()).unwrap_or(0);
Event::Status { feeds, pending, downloaded }
}
Err(e) => Event::Error { msg: format!("{e:#}") },
}
}
async fn daemon(
ctx: Arc<Ctx>,
config_path: PathBuf,
@@ -334,8 +368,7 @@ async fn daemon(
anyhow::bail!("a daemon is already listening on {}", socket.display());
}
// A database with nobody in it cannot be signed into, and an install that predates
// accounts still has to serve its owner. Both get the same starting point.
// A database with nobody in it cannot be signed into.
if ctx.db.users()?.is_empty() {
ctx.db.create_user("admin", Some(&crate::auth::hash_password(DEFAULT_PASSWORD)?), true)?;
tracing::warn!(
@@ -344,10 +377,13 @@ async fn daemon(
);
}
match migrate_opml_children(&ctx) {
Ok(n) if n > 0 => tracing::info!(count = n, "moved OPML feeds out of config.toml into the database"),
Ok(_) => {}
Err(e) => tracing::warn!(error = ?e, "could not tidy OPML feeds out of the config"),
if let Some(admin) = ctx.db.users()?.into_iter().find(|u| u.is_admin) {
let catalogue: Vec<String> = ctx.cfg().feeds.keys().cloned().collect();
match ctx.db.adopt_catalogue(admin.id, &catalogue) {
Ok(0) => {}
Ok(n) => tracing::info!(user = %admin.name, feeds = n, "subscribed the first admin to the catalogue"),
Err(e) => tracing::error!(error = %e, "could not subscribe the first admin to the catalogue"),
}
}
match ctx.db.requeue_interrupted() {
@@ -359,7 +395,12 @@ async fn daemon(
let (tx_cmd, mut rx_cmd) = mpsc::channel::<Cmd>(64);
let web = start_web(&ctx, &config_path, web_addr, &tx_cmd, &events).await?;
let server = tokio::spawn(ipc::serve(socket.clone(), events.clone(), tx_cmd));
// status is answered by the socket itself; everything else waits its turn in the queue.
let answer: ipc::StatusFn = {
let ctx = ctx.clone();
Arc::new(move || status(&ctx))
};
let server = tokio::spawn(ipc::serve(socket.clone(), events.clone(), tx_cmd, answer));
// One command at a time: the queue is what keeps two scans from overlapping.
let mut ticker = tokio::time::interval(std::time::Duration::from_secs(60));
@@ -458,7 +499,7 @@ async fn start_web(
let mut fresh = (*cfg).clone();
fresh.web.enabled = true;
fresh.web.bind = bind.clone();
fresh.web.token = web::generate_token();
fresh.web.token = crate::auth::new_session_token();
fresh.save(config_path)?;
ctx.reload_cfg(config_path)?;
println!("web ui token generated. Open:\n http://{bind}/?token={}", fresh.web.token);
@@ -507,8 +548,9 @@ async fn add(
keywords: Vec<String>,
) -> Result<()> {
let mut cfg = (*ctx.cfg()).clone();
let url = &feed::expand_input(url);
// Includes feeds derived from an OPML, or the same show could be added twice.
if let Some(existing) = subscriptions(ctx)?.iter().find(|s| s.cfg.url == url) {
if let Some(existing) = subscriptions(ctx)?.iter().find(|s| feed::same_feed(&s.cfg.url, url)) {
anyhow::bail!("already subscribed as {:?}", existing.id);
}
let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
@@ -559,10 +601,17 @@ pub async fn add_one(
// Slugs must be unique across derived feeds too, or a new feed can collide with one
// an OPML already introduced.
let taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)?
let mut taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)?
.into_iter()
.map(|s| (s.id, s.cfg))
.collect();
// A removed feed keeps its rows, so its id is only free again for the same feed: re-adding
// it gets its history back, and a different feed does not inherit someone else's.
for (id, other) in ctx.db.feed_urls()? {
if !feed::same_feed(&other, url) {
taken.entry(id).or_insert_with(|| probe.clone());
}
}
let id = config::unique_slug(&title, &taken);
cfg.feeds.insert(id.clone(), probe);
Ok(id)
@@ -592,44 +641,94 @@ fn rm(ctx: &Ctx, config_path: &std::path::Path, feed: &str) -> Result<()> {
}
async fn import(ctx: &Ctx, config_path: &std::path::Path, file: &std::path::Path) -> Result<()> {
let mut cfg = (*ctx.cfg()).clone();
let text = std::fs::read_to_string(file)
.with_context(|| format!("reading {}", file.display()))?;
let doc = opml::OPML::from_str(&text).map_err(|e| anyhow::anyhow!("parsing OPML: {e}"))?;
// The CLI speaks for the operator, as the shared web token does.
let admin = ctx
.db
.users()?
.into_iter()
.find(|u| u.is_admin)
.ok_or_else(|| anyhow::anyhow!("no admin account to subscribe: ipx user add <name> --admin"))?;
let doc = opml::OPML::from_str(&text)
.map_err(|e| anyhow::anyhow!("{} is not OPML: {e}", file.display()))?;
let (added, had) = subscribe_opml(ctx, config_path, &doc, admin.id)?;
println!("subscribed {} to {added} feed(s); {had} already there", admin.name);
Ok(())
}
/// Subscribes one person to every feed in an OPML document, for the CLI and the web alike.
/// A feed already in the catalogue costs nothing; an unknown one is added under the OPML's
/// title rather than refetching each. Returns (newly subscribed, already subscribed).
///
/// Before accounts, importing only added unknown URLs to config.toml. Once subscriptions
/// decided what each person sees, that imported nothing at all for a feed someone else
/// already had, and a new one had no subscriber, so it was never scanned.
///
/// The caller parses the document, so each refuses a file that is not OPML in its own terms,
/// before anything is touched: a 400 from the web, a message from the CLI.
pub fn subscribe_opml(
ctx: &Ctx,
config_path: &std::path::Path,
doc: &opml::OPML,
user_id: i64,
) -> Result<(usize, usize)> {
let mut found = vec![];
collect_outlines(&doc.body.outlines, &mut found);
let mut added = 0;
let known = subscriptions(ctx)?;
let mut cfg = (*ctx.cfg()).clone();
let mut ids = vec![];
let mut grew = false;
for (title, url) in found {
if cfg.feeds.values().any(|f| f.url == url) {
continue;
}
// Name it from the OPML title rather than refetching every feed.
let id = config::unique_slug(&title, &cfg.feeds);
cfg.feeds.insert(
id.clone(),
config::Feed {
url,
folder: None,
group: None,
media_types: None,
schedule: None,
keywords: vec![],
allow_explicit: false,
auto_download: true,
max_new_per_check: None,
username: None,
password: None,
password_env: None,
},
);
println!("added {id}");
added += 1;
let existing = known
.iter()
.find(|s| s.cfg.url == url)
.map(|s| s.id.clone())
// The same URL listed twice in one file.
.or_else(|| cfg.feeds.iter().find(|(_, f)| f.url == url).map(|(id, _)| id.clone()));
let id = match existing {
Some(id) => id,
None => {
let id = config::unique_slug(&title, &cfg.feeds);
cfg.feeds.insert(
id.clone(),
config::Feed {
url,
folder: None,
group: None,
media_types: None,
schedule: None,
keywords: vec![],
allow_explicit: false,
auto_download: true,
max_new_per_check: None,
username: None,
password: None,
password_env: None,
},
);
grew = true;
id
}
};
ids.push(id);
}
cfg.save(config_path)?;
println!("{added} feed(s) imported");
Ok(())
if grew {
cfg.save(config_path)?;
ctx.reload_cfg(config_path)?;
}
let (mut added, mut had) = (0, 0);
for id in ids {
if ctx.db.subscription(user_id, &id)?.is_some() {
had += 1;
} else {
ctx.db.subscribe(user_id, &id)?;
added += 1;
}
}
Ok((added, had))
}
/// OPML nests feeds inside folder outlines, so this walks the whole tree.
@@ -747,7 +846,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool) -> Result<()> {
ctx.out.emit(Event::FeedSkip {
feed: id.clone(),
reason: format!(
"OPML: {total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing",
"{total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing",
added.len()
),
});
@@ -848,36 +947,6 @@ pub fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
Ok(out)
}
/// Moves OPML children that older versions wrote into config.toml over to the database.
/// They were never yours to edit, and 80-odd of them made the file unreadable.
fn migrate_opml_children(ctx: &Ctx) -> Result<usize> {
let cfg = (*ctx.cfg()).clone();
let children: Vec<(String, config::Feed)> = cfg
.feeds
.iter()
.filter(|(_, f)| f.group.is_some())
.map(|(id, f)| (id.clone(), f.clone()))
.collect();
if children.is_empty() {
return Ok(0);
}
let mut fresh = cfg.clone();
for (id, f) in &children {
let group = f.group.clone().unwrap_or_default();
let title = ctx
.db
.feed_summary(id)
.ok()
.and_then(|s| s.title)
.unwrap_or_else(|| id.clone());
ctx.db.upsert_managed(id, &f.url, &title, &group)?;
fresh.feeds.remove(id);
}
fresh.save(&ctx.config_path)?;
ctx.reload_cfg(&ctx.config_path)?;
Ok(children.len())
}
/// Seconds to wait before re-checking a feed.
///
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the
@@ -903,7 +972,7 @@ struct Scan {
enum Outcome {
NotModified,
Feed(Scan),
/// The URL served an OPML document, so it is a subscription list rather than a feed.
/// The URL is a list of feeds rather than a feed: an OPML, or a Patreon creator's shows.
Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
}
@@ -913,6 +982,31 @@ async fn scan_one(
feed_cfg: &config::Feed,
state: &db::HttpState,
) -> Result<Outcome> {
// A Patreon creator with more than one show is a list of feeds, like an OPML.
if feed::is_patreon_creator(&feed_cfg.url) {
match feed::patreon_shows(&ctx.client, &feed_cfg.url).await {
Ok((name, shows)) if shows.len() > 1 => {
ctx.db.touch_feed(id, &feed_cfg.url)?;
if let Some(name) = name {
ctx.db.set_title(id, &name)?;
}
// Read as one feed before it was split, it listed every show's items in one
// heap. The items go; its files and read state move to each show as the show
// lists them (`Db::adopt`), so no show comes up empty for want of a URL.
ctx.db.clear_entries(id)?;
return sync_group(ctx, id, feed_cfg, &shows).await;
}
Ok(_) => {} // One show: the creator's feed is that show.
// Already split: keep the shows it has rather than read the creator as one heap.
Err(e) if ctx.db.managed_feeds()?.iter().any(|m| m.group_id == id) => return Err(e),
Err(e) => tracing::warn!(
feed = id,
error = %format!("{e:#}"),
"could not list the Patreon shows; reading it as one feed"
),
}
}
let mut fetched = feed::fetch(
&ctx.client,
feed_cfg,
@@ -957,29 +1051,45 @@ async fn scan_one(
parsed.image.as_deref(),
)?;
let policy = policy_for(ctx, id, feed_cfg)?;
if let Some(parent) = &feed_cfg.group {
let listed: Vec<(&str, &str)> = parsed
.entries
.iter()
.flat_map(|e| e.enclosures.iter().map(move |x| (e.guid.as_str(), x.url.as_str())))
.collect();
ctx.db.adopt(parent, id, &listed)?;
}
// Verdicts are recorded in `state`, so the download queue below is just "everything still
// pending". A filter's verdict is looked at again on every scan, though: made once, at
// discovery, it outlived the setting behind it, and allowing explicit items afterwards
// changed nothing however often the feed was scanned.
let skipped = ctx.db.skipped_by_filter(id)?;
let mut scan = Scan::default();
for entry in &parsed.entries {
if ctx.db.record_entry(id, entry)? {
scan.new_entries += 1;
}
for enc in &entry.enclosures {
if !ctx.db.record_enclosure(id, &entry.guid, enc)? {
continue; // Seen before: downloaded, skipped or deliberately reaped.
}
// Filters run once, at discovery, and are recorded in `state`. The download
// queue below is then just "everything still pending".
if let Some(reason) = reject(&ctx.cfg(), feed_cfg, entry, enc) {
ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?;
let was = if ctx.db.record_enclosure(id, &entry.guid, enc)? {
None
} else if let Some(reason) = skipped.get(&enc.url) {
Some(reason.as_str())
} else {
continue; // Settled: queued, downloaded, reaped, or another feed's file.
};
let now = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc);
if now != was {
match now {
Some(reason) => ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?,
None => ctx.db.mark_enclosure(&enc.url, "pending", None)?,
}
}
}
}
// An unset per-feed cap follows the global one; 0 there means unlimited.
let budget = feed_cfg.max_new_per_check.unwrap_or_else(|| {
let g = ctx.cfg().general.max_new_per_check;
if g == 0 { usize::MAX } else { g }
});
if feed_cfg.auto_download && budget > 0 {
let budget = policy.budget;
if policy.auto_download && budget > 0 {
let cfg = ctx.cfg();
let folder = download::folder_for(&cfg, id, feed_cfg, parsed.title.as_deref());
let dest_dir = cfg.general.download_dir.join(&folder);
@@ -1057,12 +1167,6 @@ async fn scan_one(
Ok(Outcome::Feed(scan))
}
/// Brings the feed list in step with a subscribed OPML.
///
/// New entries are added under the OPML's group and folder. An entry that has gone from
/// the OPML is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
/// the UI to explain them.
async fn sync_opml(
ctx: &Arc<Ctx>,
parent_id: &str,
@@ -1073,25 +1177,44 @@ async fn sync_opml(
if let Some(title) = feed::opml_title(bytes) {
ctx.db.set_title(parent_id, &title)?;
}
sync_group(ctx, parent_id, parent, &listed).await
}
/// Brings the feed list in step with a list of feeds: a subscribed OPML, or a Patreon
/// creator's shows.
///
/// New entries are added under the list's group and folder. An entry that has gone from
/// the list is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
/// the UI to explain them.
async fn sync_group(
ctx: &Arc<Ctx>,
parent_id: &str,
parent: &config::Feed,
listed: &[(String, String)],
) -> Result<Outcome> {
let cfg = ctx.cfg();
let existing = ctx.db.managed_feeds()?;
let mut added = vec![];
for (title, url) in &listed {
for (title, url) in listed {
// Already known, whether derived or promoted into the config.
if let Some(m) = existing.iter().find(|m| &m.url == url) {
ctx.db.upsert_managed(&m.id, url, title, parent_id)?;
continue;
}
if cfg.feeds.values().any(|f| &f.url == url) {
// A Patreon show you added by hand may be spelled differently from the one listed.
if cfg.feeds.values().any(|f| feed::same_feed(&f.url, url)) {
continue;
}
// A removed feed keeps its rows, so its id is only free again for the same feed.
let known = ctx.db.feed_urls()?;
let taken: std::collections::BTreeMap<String, config::Feed> = cfg
.feeds
.keys()
.chain(existing.iter().map(|m| &m.id))
.chain(added.iter())
.chain(known.iter().filter(|(_, u)| !feed::same_feed(u, url)).map(|(id, _)| id))
.map(|id| (id.clone(), parent.clone()))
.collect();
let id = config::unique_slug(title, &taken);
@@ -1099,6 +1222,23 @@ async fn sync_opml(
added.push(id);
}
// Whoever subscribes to the OPML subscribes to what it lists: that is what taking a
// subscription means. Their own feeds are untouched.
for id in ctx
.db
.managed_feeds()?
.iter()
.filter(|m| m.group_id == parent_id)
.map(|m| m.id.clone())
.chain(std::iter::once(parent_id.to_string()))
{
for user in ctx.db.users()? {
if ctx.db.subscription(user.id, parent_id)?.is_some() {
ctx.db.subscribe(user.id, &id)?;
}
}
}
// Anything in this group the OPML no longer lists.
let mut removed = 0;
let mut kept = 0;
@@ -1125,11 +1265,12 @@ async fn sync_opml(
fn reject(
cfg: &config::Config,
feed_cfg: &config::Feed,
policy: &Policy,
entry: &feed::Entry,
enc: &feed::Enclosure,
) -> Option<&'static str> {
let url = enc.url.as_str();
if !feed_cfg.auto_download {
if !policy.auto_download {
return Some("auto_download is off");
}
// Blog feeds put the article's header image in an <enclosure>; without this a text
@@ -1141,7 +1282,7 @@ fn reject(
if !config::wanted_media(enc.mime.as_deref(), wanted) {
return Some("not audio or video");
}
if entry.explicit && !feed_cfg.allow_explicit {
if entry.explicit && !policy.allow_explicit {
return Some("explicit");
}
let categories = entry.categories.join(" ");
@@ -1151,12 +1292,79 @@ fn reject(
entry.description.as_deref().unwrap_or(""),
categories.as_str(),
];
if !download::matches_keywords(&feed_cfg.keywords, &haystacks) {
// One file serves everyone subscribed, so an item is wanted if it is wanted by
// anyone: any one person's keyword set matching is enough.
let wanted_by_someone = policy.keyword_sets.is_empty()
|| policy
.keyword_sets
.iter()
.any(|set| download::matches_keywords(set, &haystacks));
if !wanted_by_someone {
return Some("no keyword match");
}
None
}
/// What the scanner should do for a feed, merged across everyone subscribed to it. The
/// feed is fetched once and its files are downloaded once, so the merge is a union: if
/// one person wants a thing, it is fetched, and everyone else simply sees it listed.
///
/// With no subscribers at all -- a hand-written config entry nobody has claimed yet --
/// the feed's own settings stand, which is how a single-user install behaves.
pub struct Policy {
pub auto_download: bool,
pub allow_explicit: bool,
/// Empty means take everything. Otherwise one set per subscriber who filters.
pub keyword_sets: Vec<Vec<String>>,
pub budget: usize,
}
fn policy_for(ctx: &Ctx, id: &str, feed_cfg: &config::Feed) -> Result<Policy> {
let global = ctx.cfg().general.max_new_per_check;
Ok(merge_policy(&ctx.db.subscribers(id, feed_cfg.group.as_deref())?, feed_cfg, global))
}
fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Policy {
let cap = |n: Option<usize>| n.unwrap_or(if global == 0 { usize::MAX } else { global });
if subs.is_empty() {
return Policy {
auto_download: feed_cfg.auto_download,
allow_explicit: feed_cfg.allow_explicit,
keyword_sets: if feed_cfg.keywords.is_empty() {
vec![]
} else {
vec![feed_cfg.keywords.clone()]
},
budget: cap(feed_cfg.max_new_per_check),
};
}
let mut policy = Policy {
auto_download: false,
allow_explicit: false,
keyword_sets: vec![],
budget: 0,
};
for sub in subs {
if !sub.auto_download.unwrap_or(feed_cfg.auto_download) {
continue; // Not fetching for this person, so their wants add nothing.
}
policy.auto_download = true;
policy.allow_explicit |= sub.allow_explicit.unwrap_or(feed_cfg.allow_explicit);
policy.budget = policy
.budget
.max(cap(sub.max_new_per_check.map(|n| n as usize).or(feed_cfg.max_new_per_check)));
let kw = sub.keywords.clone().unwrap_or_else(|| feed_cfg.keywords.clone());
if kw.is_empty() {
// Somebody takes everything, so no filter can apply to the shared copy.
return Policy { keyword_sets: vec![], ..policy };
}
policy.keyword_sets.push(kw);
}
policy
}
async fn fetch_one(
ctx: &Arc<Ctx>,
feed_id: &str,
@@ -1352,3 +1560,79 @@ fn duration(secs: u64) -> String {
s => format!("{}d", s / 86_400),
}
}
#[cfg(test)]
mod tests {
use super::*;
fn feed() -> config::Feed {
// Whatever `ipx add` would write, which is the shape every code path sees.
let mut cfg = config::Config::default();
let f = add_one_cfg(&mut cfg, "http://x/f.xml", None, vec![]);
f
}
/// The feed entry `add` builds, without the network round trip it does for a title.
fn add_one_cfg(
_cfg: &mut config::Config,
url: &str,
folder: Option<String>,
keywords: Vec<String>,
) -> config::Feed {
config::Feed {
url: url.into(),
folder,
keywords,
allow_explicit: false,
auto_download: true,
group: None,
media_types: None,
schedule: None,
max_new_per_check: None,
username: None,
password: None,
password_env: None,
}
}
fn sub(kw: Option<&[&str]>, auto: Option<bool>, max: Option<i64>) -> db::Sub {
db::Sub {
feed_id: "f".into(),
keywords: kw.map(|k| k.iter().map(|s| s.to_string()).collect()),
auto_download: auto,
allow_explicit: None,
max_new_per_check: max,
}
}
#[test]
fn a_shared_feed_is_fetched_for_whoever_wants_the_most() {
// Nobody subscribed: the feed's own settings stand, as in a single-user install.
let p = merge_policy(&[], &feed(), 3);
assert!(p.auto_download);
assert_eq!(p.budget, 3);
assert!(p.keyword_sets.is_empty());
// Two filters: an item wanted by either of them is fetched, since one file serves
// both. The larger per-scan cap wins for the same reason.
let p = merge_policy(
&[sub(Some(&["rust"]), None, Some(2)), sub(Some(&["sqlite"]), None, Some(9))],
&feed(),
3,
);
assert_eq!(p.keyword_sets.len(), 2);
assert_eq!(p.budget, 9);
// One person taking everything removes the filter for the shared copy.
let p = merge_policy(&[sub(Some(&["rust"]), None, None), sub(Some(&[]), None, None)], &feed(), 3);
assert!(p.keyword_sets.is_empty());
// Everyone has auto-download off: nothing is fetched automatically.
let p = merge_policy(&[sub(None, Some(false), None), sub(None, Some(false), None)], &feed(), 3);
assert!(!p.auto_download);
// One of them wants it, so it is fetched.
let p = merge_policy(&[sub(None, Some(false), None), sub(None, Some(true), None)], &feed(), 3);
assert!(p.auto_download);
}
}

View File

@@ -150,33 +150,52 @@ mod tests {
}
#[test]
fn query_never_offers_flagged_files_and_prefers_read_ones() {
fn query_never_offers_a_file_anyone_starred_and_prefers_ones_everyone_read() {
// One file serves both subscribers, so it takes both of them to release it.
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO entries (feed_id, guid, first_seen, read, flagged) VALUES
('f', 'keep', 0, 1, 1),
('f', 'unread', 0, 0, 0),
('f', 'read', 0, 1, 0);
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0);
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'f'),(2,'f');
INSERT INTO entries (feed_id, guid, first_seen) VALUES
('f', 'keep', 0),
('f', 'half', 0),
('f', 'unread', 0),
('f', 'read', 0);
-- Starred by one of the two, so it stays whatever the other thinks.
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
(1, 'f', 'keep', 1, 1),
(2, 'f', 'keep', 1, 0),
(1, 'f', 'half', 1, 0),
(1, 'f', 'read', 1, 0),
(2, 'f', 'read', 1, 0);
INSERT INTO enclosures (id, feed_id, guid, url, path, bytes_done, state, downloaded_at) VALUES
(1, 'f', 'keep', 'u1', '/tmp/keep', 10, 'done', 10),
(2, 'f', 'unread', 'u2', '/tmp/unread', 10, 'done', 20),
(3, 'f', 'read', 'u3', '/tmp/read', 10, 'done', 30);",
(2, 'f', 'half', 'u2', '/tmp/half', 10, 'done', 20),
(3, 'f', 'unread', 'u3', '/tmp/unread', 10, 'done', 30),
(4, 'f', 'read', 'u4', '/tmp/read', 10, 'done', 40);",
)
.unwrap();
let got: Vec<i64> = db.reap_candidates().unwrap().iter().map(|c| c.id).collect();
assert_eq!(got, vec![3, 2], "flagged excluded; read goes before unread");
assert_eq!(
got,
vec![4, 2, 3],
"starred by anyone is never offered; read by everyone goes first, and one \
person still having it unread keeps it back with the unread ones"
);
}
#[test]
fn prune_keeps_entries_that_still_have_a_file() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO entries (feed_id, guid, first_seen, read, flagged) VALUES
('f', 'has-file', 100, 1, 0),
('f', 'no-file', 100, 1, 0),
('f', 'flagged', 100, 1, 1),
('f', 'recent', 900, 1, 0);
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1);
INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',1);
INSERT INTO entries (feed_id, guid, first_seen) VALUES
('f', 'has-file', 100),
('f', 'no-file', 100),
('f', 'flagged', 100),
('f', 'recent', 900);
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
(1, 'f', 'has-file', 'u1', '/tmp/x', 'done');",
)

File diff suppressed because it is too large Load Diff

View File

@@ -15,7 +15,7 @@ const script = html.split('<script>')[1].split('</script>')[0];
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
const missing = [];
const el = (name) => new Proxy({ style: {}, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
const el = (name) => new Proxy({ style: { setProperty(){}, getPropertyValue(){ return ''; } }, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
@@ -44,7 +44,12 @@ const ctx = {
json: () => Promise.resolve(
String(url).includes('/api/settings')
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
: []),
: String(url).includes('/api/users')
? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }]
: /\/api\/(popular|directory)/.test(String(url))
? [{ id: 'f', title: 'A Feed', image: null, subscribers: 2, subscribed: true },
{ id: 'g', title: null, image: null, subscribers: 1, subscribed: false }]
: /entries/.test(String(url)) ? { total: 0, entries: [] } : []),
}),
EventSource: function () { this.close = () => {}; },
MediaMetadata: function () {},
@@ -79,6 +84,11 @@ const drive = [
['downloadLatestModal', () => ctx.downloadLatestModal(feed)],
['removeFeed', () => ctx.removeFeed(feed)],
['prefsModal', () => ctx.prefsModal()],
['usersModal', () => ctx.usersModal()],
['opmlModal', () => ctx.opmlModal()],
['selectFeed (directory)', () => ctx.selectFeed(':directory')],
['selectFeed (popular)', () => ctx.selectFeed(':popular')],
['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
['logsModal', () => ctx.logsModal()],
// `const S` is not reachable from here: top-level const/let do not become properties
// of a vm context the way var and function declarations do.

View File

@@ -10,8 +10,8 @@ test.beforeEach(async ({ page }) => {
test('the page loads and lists the configured feeds', async ({ page }) => {
// Regression: a ReferenceError in the script left the shell rendered and the sidebar
// empty, with every handler below the error dead. Server-side checks all passed.
// Three top-level feeds in the fixture config; the OPML's child is inside a closed folder.
await expect(page.locator('.feed')).toHaveCount(4, { timeout: 15_000 });
// Four top-level feeds in the fixture config; the OPML's children are inside a closed folder.
await expect(page.locator('.feed')).toHaveCount(5, { timeout: 15_000 });
await expect(page.getByText('Test Show')).toBeVisible();
const errors = [];
page.on('pageerror', e => errors.push(e.message));
@@ -29,6 +29,18 @@ test('the theme toggle actually changes the theme', async ({ page }) => {
.not.toBe(before);
});
test('the theme button steps through dark, light and classic, and remembers', async ({ page }) => {
const theme = () => page.evaluate(() => document.documentElement.dataset.theme);
for (let i = 0; i < 3 && (await theme()) !== 'classic'; i++) await page.locator('#theme').click();
expect(await theme()).toBe('classic');
await expect(page.locator('#theme')).toHaveAttribute('title', /Classic.*Click for Dark/);
await page.reload();
await expect.poll(theme).toBe('classic');
// The 2004 Mac app set its type in Lucida Grande.
expect(await page.evaluate(() => getComputedStyle(document.body).fontFamily)).toContain('Lucida Grande');
});
test('settings opens and saves the global schedule', async ({ page }) => {
await page.locator('#prefs').click();
await expect(page.locator('#modal.on')).toBeVisible();
@@ -69,20 +81,21 @@ test('the three panes are there and the item text lands in the bottom one', asyn
await page.locator('.ep', { hasText: 'First Episode' }).click();
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
// The enclosure travels with the item, into the same pane.
await expect(page.locator('#detail .encbox')).toHaveCount(1);
// The enclosure goes to the Files pane beside the list, as the original's did.
await expect(page.locator('#files')).toBeVisible();
await expect(page.locator('#files .encbox')).toHaveCount(1);
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
// its chip rather than assuming which episode the daemon happened to fetch.
const downloaded = page.locator('.ep', { hasText: 'downloaded' }).first();
const downloaded = page.locator('.ep', { has: page.locator('.kind.here') }).first();
await downloaded.click();
await expect(page.locator('#detail audio')).toBeVisible();
await expect(page.locator('#detail .encbox .btn', { hasText: 'Save' })).toBeVisible();
await expect(page.locator('#files [data-a="play"]')).toBeVisible();
await expect(page.locator('#files .encbox [title="Save to this computer"]')).toBeVisible();
// Selecting another item replaces the pane rather than stacking.
await page.locator('.ep', { hasText: 'First Episode' }).click();
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
await expect(page.locator('#detail audio')).toHaveCount(0);
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
});
test('a downloaded file that is not audio gets no player', async ({ page }) => {
@@ -96,12 +109,12 @@ test('a downloaded file that is not audio gets no player', async ({ page }) => {
await row.click();
await expect(page.locator('#detail .dt')).toHaveText('An Article');
await expect(page.locator('#detail audio')).toHaveCount(0);
await expect(page.locator('#detail .encbox')).toContainText('image');
await expect(page.locator('#detail .encbox')).toContainText('downloaded');
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
// What it is and that it is here: one icon, green, with the words in its tooltip.
await expect(page.locator('#files .encbox .kind.here')).toHaveAttribute('title', 'image, downloaded');
// Still offered as a file, just not as an episode: viewable and keepable.
await expect(page.locator('#detail .btn', { hasText: 'Save' })).toBeVisible();
const view = page.locator('#detail a', { hasText: 'View' });
await expect(page.locator('#files [title="Save to this computer"]')).toBeVisible();
const view = page.locator('#files a[title="View in a new tab"]');
await expect(view).toHaveAttribute('target', '_blank');
await expect(view).toHaveAttribute('rel', /noopener/);
await expect(view).toHaveAttribute('href', /\/media\/\d+/);
@@ -115,9 +128,9 @@ test('an item with several enclosures lists them all', async ({ page }) => {
await expect(row).toContainText('+1 more file');
await row.click();
// The pane below lists every one: the audio and the image.
await expect(page.locator('#detail .encbox')).toHaveCount(2);
await expect(page.locator('#detail .encbox').nth(1)).toContainText('image');
// The Files pane lists every one: the audio and the image.
await expect(page.locator('#files .encbox')).toHaveCount(2);
await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
});
test('the filter tabs change what is listed', async ({ page }) => {
@@ -129,13 +142,13 @@ test('the filter tabs change what is listed', async ({ page }) => {
await page.locator('.tabs button', { hasText: 'Unread' }).first().click();
expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all);
await page.locator('.tabs button', { hasText: 'Flagged' }).first().click();
await page.locator('.tabs button', { hasText: 'Kept' }).first().click();
await expect(page.locator('#count')).toContainText('0 items');
});
test('a feed URL is editable and has a copy button', async ({ page }) => {
await page.getByText('Test Show').click();
await page.locator('.btn', { hasText: 'Settings' }).first().click();
await page.locator('#content .acts [data-a="settings"]').click();
await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
await expect(page.locator('#scopy')).toBeVisible();
@@ -174,20 +187,56 @@ test('an OPML subscription is a collapsible folder', async ({ page }) => {
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
}));
// Every row reserves the chevron slot for alignment; only a folder's is clickable.
// Only a folder has a triangle, and it is a button that says whether the folder is open.
const chev = page.locator('.feed.group .chev');
await expect(chev).toBeVisible({ timeout: 20_000 });
await expect(page.locator('.feed:not(.group) .chev')).toHaveCount(0);
await expect(chev).toHaveAttribute('aria-expanded', 'false');
// Closed by default: the child is not listed until the folder is opened.
// Closed by default: the children are not listed until the folder is opened.
const before = await page.locator('.feed').count();
await chev.click();
await expect(page.locator('.feed')).toHaveCount(before + 1);
await expect(page.locator('.feed')).toHaveCount(before + 2);
// Scoped to the sidebar: the name also appears as the page heading once selected.
await expect(page.locator('#feedlist').getByText('Grouped Show')).toBeVisible();
// The subscription's own page lists what is inside it.
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
await expect(page.locator('.childrow')).toHaveCount(1);
await expect(page.locator('.childrow')).toHaveCount(2);
});
test('inside an OPML, feeds with unread items are listed first', async ({ page }) => {
await page.evaluate(() =>
fetch('/api/fetch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
}));
const chev = page.locator('.feed.group .chev');
await expect(chev).toBeVisible({ timeout: 20_000 });
await chev.click();
const side = page.locator('#feedlist');
await expect(side.getByText('Aardvark Radio')).toBeVisible({ timeout: 20_000 });
// Other tests change read state, so set it here: Aardvark Radio read, Grouped Show not.
// Opening an item reads it; the toggle in the pane below flips it back.
await side.getByText('Aardvark Radio').click();
const aa = page.locator('.ep', { hasText: 'Aardvark Ep' });
await aa.click();
await expect(aa).toHaveClass(/read/);
await side.getByText('Grouped Show').click();
const gs = page.locator('.ep', { hasText: 'Grouped Ep' });
await gs.click();
await page.locator('#detail button[title="Mark unread"]').click();
await expect(gs).not.toHaveClass(/read/);
// Aardvark comes first alphabetically and in the OPML, so only the unread sort puts
// Grouped Show above it. The folder stays open across the reload (localStorage).
await page.reload();
const want = ['Grouped Show', 'Aardvark Radio'];
await expect(page.locator('#feedlist .feed.child b')).toHaveText(want, { timeout: 20_000 });
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
await expect(page.locator('.childrow b')).toHaveText(want);
});
test('marking an OPML subscription read covers the feeds inside it', async ({ page }) => {
@@ -204,7 +253,7 @@ test('marking an OPML subscription read covers the feeds inside it', async ({ pa
await expect(folder.locator('.badge')).not.toHaveText('0');
await folder.click();
await page.locator('#content .acts button', { hasText: 'Mark all read' }).click();
await page.locator('#content .acts [data-a="read"]').click();
await expect(folder.locator('.badge')).toHaveText('0');
});
@@ -246,9 +295,9 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
// Another test may have opened this item already, so start from a known state: the
// toggle in the text below flips it back -- which used to recurse until the stack blew.
await row().click();
await page.locator('#detail button', { hasText: 'Mark unread' }).click();
await page.locator('#detail button[title="Mark unread"]').click();
await expect(row()).not.toHaveClass(/read/);
await expect(page.locator('#detail button', { hasText: 'Mark read' })).toBeVisible();
await expect(page.locator('#detail button[title="Mark read"]')).toBeVisible();
// Opening it is reading it.
await row().click();
@@ -256,3 +305,508 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
expect(errors).toEqual([]);
});
test('the toolbar acts on the selected item', async ({ page }) => {
await page.getByText('Test Show').click();
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
await expect(row()).toBeVisible({ timeout: 20_000 });
// Nothing selected, nothing to act on.
await expect(page.locator('#tbRead')).toBeDisabled();
await row().click(); // opening it reads it
await expect(row()).toHaveClass(/read/);
await page.locator('#tbRead').click();
await expect(row()).not.toHaveClass(/read/);
await page.locator('#tbFlag').click();
await expect(row().locator('.fl')).toHaveClass(/on/);
await page.locator('#tbFlag').click(); // and back, so later tests see it unkept
await expect(row().locator('.fl')).not.toHaveClass(/on/);
// Second Episode is the one the daemon downloaded, so it plays from the toolbar.
await expect(page.locator('#tbPlay')).toBeEnabled();
await page.locator('#tbPlay').click();
await expect(page.locator('#player')).toBeVisible();
await page.locator('#pclose').click();
});
test('a second person has their own feeds and their own read state', async ({ browser }) => {
const { execFileSync } = require('child_process');
const setup = require('./global-setup');
const env = {
...process.env,
IPX_CONFIG: `${setup.root}/config/config.toml`,
IPX_DATA_DIR: `${setup.root}/data`,
};
try {
execFileSync('./target/debug/ipx', ['user', 'add', 'sam'], { input: 'sampassword', env });
} catch (e) {
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
}
// A fresh context, so none of the admin's cookies come along.
const ctx = await browser.newContext();
const page = await ctx.newPage();
await page.goto('/login');
// The sign-in page shows the icon, so it has to load before anyone has signed in.
const icon = await page.request.get('/icon.png');
expect(icon.status()).toBe(200);
expect(icon.headers()['content-type']).toBe('image/png');
await page.locator('#name').fill('sam');
await page.locator('#pw').fill('sampassword');
await page.locator('button[type=submit]').click();
await expect(page.locator('#feedlist')).toBeVisible();
// Sam subscribes to nothing yet, so sees nothing -- the admin's feeds are not theirs.
await expect(page.locator('#feedlist')).toContainText('No feeds.');
await expect(page.locator('#prefs')).toBeHidden(); // not an admin
// Hiding the button is not the guard; the server is.
expect((await page.request.get('/api/users')).status()).toBe(403);
await expect(page.locator('#logs')).toBeHidden();
expect((await page.request.get('/api/logs')).status()).toBe(403);
// Subscribing to a feed the admin already has costs no second fetch: same feed, same
// files, but Sam's own read state.
await page.locator('#addFeed').click();
await page.locator('#nurl').fill('http://127.0.0.1:8792/show.xml');
await page.locator('#nsave').click();
await expect(page.locator('.feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
await page.locator('.feed', { hasText: 'Test Show' }).click();
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
// The admin read these earlier in this file; for Sam they are all still unread.
const rows = await page.locator('.ep').count();
await page.locator('.tabs button', { hasText: 'Unread' }).click();
await expect(page.locator('.ep')).toHaveCount(rows);
// Two people now share this feed, so the page says so and Delete is honest about it.
await expect(page.locator('#content .sub').first()).toContainText('shared with 1 other person');
await ctx.close();
});
test('deleting a shared file warns that it is everyone\'s copy', async ({ page }) => {
// Admin and Sam both subscribe to Test Show by now, and the daemon downloaded a file.
await page.getByText('Test Show').click();
await page.locator('.tabs button', { hasText: 'Downloaded' }).click();
const row = page.locator('.ep').first();
await expect(row).toBeVisible({ timeout: 20_000 });
await row.click();
// An icon now; what it does, and to whom, is in its tooltip.
const del = page.locator('#files button[data-a="del"]');
await expect(del).toHaveAttribute('title', /^Delete for everyone \(shared with 1 other person/);
// Two prompts: the page's own, then the server's, because someone else has not played
// it. Accept the first, decline the second, and the file must survive.
const seen = [];
page.on('dialog', d => {
seen.push(d.message());
if (seen.length === 1) d.accept();
else d.dismiss();
});
await del.click();
await expect.poll(() => seen.length, { timeout: 10_000 }).toBe(2);
expect(seen[0]).toContain('shared with 1 other person');
expect(seen[1]).toContain('one copy of this file');
await page.reload();
await page.getByText('Test Show').click();
await page.locator('.tabs button', { hasText: 'Downloaded' }).click();
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
});
// Every row says "Admin" on its checkbox, so match the name exactly.
const userRow = (page, name) =>
page.locator('#modalCard [data-id]').filter({ has: page.locator('b', { hasText: new RegExp(`^${name}$`) }) });
async function openUsers(page) {
await page.locator('#prefs').click();
await page.locator('#gusers').click();
await expect(userRow(page, 'admin')).toBeVisible();
}
test('an admin adds someone, makes them an admin, and removes them', async ({ page }) => {
await openUsers(page);
await page.locator('#uname').fill('pat');
await page.locator('#upass').fill('patpassword');
await page.locator('#uadd').click();
const row = userRow(page, 'pat');
await expect(row).toBeVisible();
// When each account was added and last signed in; the admin signed in with the token link.
await expect(row).toContainText(/Added .* never signed in/);
await expect(userRow(page, 'admin')).toContainText(/signed in \d+m ago/);
await expect(row.locator('[data-a="admin"]')).not.toBeChecked();
await row.locator('[data-a="admin"]').check();
// Not just the box: it has to have reached the database.
await expect.poll(async () =>
(await (await page.request.get('/api/users')).json()).find(u => u.name === 'pat')?.admin
).toBe(true);
page.once('dialog', d => d.accept());
await row.locator('[data-a="rm"]').click();
await expect(row).toHaveCount(0);
});
test('the only admin cannot be demoted or removed', async ({ page }) => {
await openUsers(page);
await userRow(page, 'admin').locator('[data-a="admin"]').click();
await expect(page.locator('.toast.bad')).toContainText('only admin');
// Redrawn from the server, so the box is back.
await expect(userRow(page, 'admin').locator('[data-a="admin"]')).toBeChecked();
const me = (await (await page.request.get('/api/users')).json()).find(u => u.name === 'admin');
expect((await page.request.delete(`/api/users/${me.id}`)).status()).toBe(400);
});
test('Settings exports your OPML and imports a pasted one', async ({ page }) => {
await page.locator('#prefs').click();
const [dl] = await Promise.all([
page.waitForEvent('download'),
page.locator('#modalCard a[title="Export OPML"]').click(),
]);
expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml');
const out = require('fs').readFileSync(await dl.path(), 'utf8');
for (const f of ['show.xml', 'pics.xml', 'multi.xml', 'subs.opml']) expect(out).toContain(f);
// A feed from an OPML subscription comes back with the OPML itself, not on its own.
expect(out).not.toContain('other.xml');
// One feed new to everyone, one the admin already has.
await page.locator('#gopml').click();
await page.locator('#opmlText').fill('<opml version="2.0"><head><title>t</title></head><body>' +
'<outline text="Imported Show" xmlUrl="http://127.0.0.1:8792/imported.xml"/>' +
'<outline text="Test Show" xmlUrl="http://127.0.0.1:8792/show.xml"/></body></opml>');
await page.locator('#oimp').click();
await expect(page.locator('.toast', { hasText: 'Subscribed to' }))
.toHaveText('Subscribed to 1 feed(s), 1 you already had');
// Named from the OPML's id until the first scan reads the feed's own title.
await expect(page.locator('#feedlist .feed', { hasText: /Imported Show|imported-show/ }))
.toBeVisible({ timeout: 20_000 });
});
test('an uploaded OPML file imports, and a file that is not OPML is refused', async ({ page }) => {
await page.locator('#prefs').click();
await page.locator('#gopml').click();
const pick = page.locator('#opmlFile');
// An RSS feed is XML but not OPML: refused in the page, and the picker lets go of it.
await pick.setInputFiles({
name: 'feed.xml', mimeType: 'application/xml',
buffer: require('fs').readFileSync(require('path').join(__dirname, 'fixtures', 'show.xml')),
});
await page.locator('#oimp').click();
await expect(page.locator('.toast.bad', { hasText: 'feed.xml is not an OPML file' })).toBeVisible();
expect(await pick.evaluate(i => i.files.length)).toBe(0);
// Something that gets past the page's quick look is still refused by the server, untouched.
const sneaky = await page.request.post('/api/opml', {
data: { xml: '<rss version="2.0"><channel><title>&lt;opml&gt;</title></channel></rss>' },
});
expect(sneaky.status()).toBe(400);
expect(await sneaky.text()).toContain('not an OPML file');
// A real one. Multi Show is already the admin's, so it counts as already had.
await pick.setInputFiles({
name: 'subs.opml', mimeType: 'text/x-opml',
buffer: Buffer.from('<opml version="2.0"><head><title>t</title></head><body>' +
'<outline text="Multi Show" xmlUrl="http://127.0.0.1:8792/multi.xml"/></body></opml>'),
});
await page.locator('#oimp').click();
await expect(page.locator('.toast', { hasText: 'Subscribed to' }))
.toHaveText('Subscribed to 0 feed(s), 1 you already had');
await expect(page.locator('#modal.on')).toBeHidden();
});
test('an export from one account imports into another', async ({ page, browser }) => {
// Regression: import only added URLs the catalogue lacked and subscribed nobody, so importing
// the admin's export into a second account did nothing at all.
const { execFileSync } = require('child_process');
const setup = require('./global-setup');
const env = {
...process.env,
IPX_CONFIG: `${setup.root}/config/config.toml`,
IPX_DATA_DIR: `${setup.root}/data`,
};
try {
execFileSync('./target/debug/ipx', ['user', 'add', 'opal'], { input: 'opalpassword', env });
} catch (e) {
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
}
const ctx = await browser.newContext();
const opal = await ctx.newPage();
await opal.goto('/login');
await opal.locator('#name').fill('opal');
await opal.locator('#pw').fill('opalpassword');
await opal.locator('button[type=submit]').click();
await expect(opal.locator('#feedlist')).toContainText('No feeds.');
// Export used to hand anyone the whole catalogue. Opal has nothing yet, so gets nothing.
const empty = await opal.request.get('/api/opml');
expect(empty.status()).toBe(200);
expect(await empty.text()).not.toContain('xmlUrl');
const urlsIn = xml => [...xml.matchAll(/xmlUrl="([^"]+)"/g)].map(m => m[1]).sort();
const exported = await (await page.request.get('/api/opml')).text(); // the admin's
const urls = urlsIn(exported);
expect(urls.length).toBeGreaterThan(2);
expect(await (await opal.request.post('/api/opml', { data: { xml: exported } })).json())
.toEqual({ added: urls.length, already: 0 });
expect(await (await opal.request.post('/api/opml', { data: { xml: exported } })).json())
.toEqual({ added: 0, already: urls.length });
await opal.reload();
await expect(opal.locator('.feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// The round trip closes: opal's own export now lists what the admin's did.
expect(urlsIn(await (await opal.request.get('/api/opml')).text())).toEqual(urls);
await ctx.close();
});
test('ipx import subscribes the admin, and ipx export writes the feeds out', async () => {
// Its own config and database. The CLI works in-process, and the suite's running daemon
// reads config.toml once at start, so it would not see what the CLI added anyway.
const fs = require('fs');
const path = require('path');
const { execFileSync } = require('child_process');
const setup = require('./global-setup');
const dir = path.join(setup.root, 'cli');
fs.rmSync(dir, { recursive: true, force: true });
fs.mkdirSync(path.join(dir, 'data'), { recursive: true });
fs.writeFileSync(path.join(dir, 'config.toml'),
`[general]\ndownload_dir = "${dir}/downloads"\nsocket = "${dir}/ipx.sock"\n`);
const env = { ...process.env, IPX_CONFIG: path.join(dir, 'config.toml'), IPX_DATA_DIR: path.join(dir, 'data') };
const ipx = (args, input) => execFileSync('./target/debug/ipx', args, { env, input, encoding: 'utf8' });
ipx(['user', 'add', 'boss'], 'bosspassword'); // the first account is the admin
const opml = path.join(dir, 'in.opml');
fs.writeFileSync(opml, '<opml version="2.0"><head><title>t</title></head><body>' +
'<outline text="One" xmlUrl="http://127.0.0.1:8792/one.xml"/>' +
'<outline text="Two" xmlUrl="http://127.0.0.1:8792/two.xml"/></body></opml>');
expect(ipx(['import', opml])).toContain('subscribed boss to 2 feed(s); 0 already there');
expect(ipx(['import', opml])).toContain('subscribed boss to 0 feed(s); 2 already there');
const out = path.join(dir, 'out.opml');
ipx(['export', out]);
const xml = fs.readFileSync(out, 'utf8');
expect(xml).toContain('http://127.0.0.1:8792/one.xml');
expect(xml).toContain('http://127.0.0.1:8792/two.xml');
});
test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
const { execFileSync } = require('child_process');
const setup = require('./global-setup');
const env = {
...process.env,
IPX_CONFIG: `${setup.root}/config/config.toml`,
IPX_DATA_DIR: `${setup.root}/data`,
};
try {
execFileSync('./target/debug/ipx', ['user', 'add', 'piper'], { input: 'piperpassword', env });
} catch (e) {
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
}
const ctx = await browser.newContext();
const piper = await ctx.newPage();
await piper.goto('/login');
await piper.locator('#name').fill('piper');
await piper.locator('#pw').fill('piperpassword');
await piper.locator('button[type=submit]').click();
await expect(piper.locator('#feedlist')).toContainText('No feeds.');
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
const offered = piper.locator('#popular .childrow');
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// An OPML is listed as the feeds inside it, and a key in a URL marks someone's paid feed.
await expect(offered.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
// No URL reaches the page at all, so neither can a key, and the server holds the same line.
const listed = await (await piper.request.get('/api/popular')).text();
expect(listed).not.toContain('secret123');
expect(listed).not.toContain('.xml');
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
// Popular is the top ten of the directory, and the directory is every listed feed A to Z,
// with an OPML's feeds in place of the OPML in both.
const dir = await (await piper.request.get('/api/directory')).json();
const top = await (await piper.request.get('/api/popular')).json();
const names = dir.map(p => (p.title || p.id).toLowerCase());
expect(names).toEqual([...names].sort());
const ids = dir.map(p => p.id);
expect(ids).not.toContain('test-subscriptions');
expect(ids).toEqual(expect.arrayContaining(['grouped-show', 'aardvark-radio']));
expect(top.length).toBe(Math.min(10, dir.length));
expect(top.every(t => ids.includes(t.id))).toBe(true);
expect(ids).not.toContain('paid-show');
// Subscribe from the directory this time; the popular list shares the same rows.
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible();
await expect(offered.filter({ hasText: /Grouped Show|grouped-show/ })).toBeVisible();
await expect(offered.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
const row = async () =>
(await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
const before = await row();
expect(before.subscribed).toBe(false);
await offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
// All Subscriptions is every item from piper's feeds and only those: the admin's Picture
// Blog is not among them.
await piper.locator('#feedlist .place', { hasText: 'All Subscriptions' }).click();
const first = piper.locator('.ep', { hasText: 'First Episode' });
await expect(first).toBeVisible({ timeout: 20_000 });
await expect(first.locator('.fd')).toHaveText('Test Show');
await expect(piper.locator('.ep', { hasText: 'An Article' })).toHaveCount(0);
await expect(piper.locator('#count')).toContainText('All Subscriptions:');
await ctx.close();
});
test('adding a feed scans it straight away', async ({ page }) => {
await page.locator('#addFeed').click();
await page.locator('#nurl').fill('http://127.0.0.1:8792/fresh.xml');
await page.locator('#nsave').click();
// Nobody pressed Scan. The scheduler's tick is a minute, so this is the add scanning it.
await expect(page.locator('.ep', { hasText: 'Fresh Ep' })).toBeVisible({ timeout: 10_000 });
});
test('a deleted file looks as if it was never downloaded', async ({ page }) => {
// Other people subscribe to Picture Blog by now, so both prompts come; take them.
page.on('dialog', d => d.accept());
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
const row = page.locator('.ep', { hasText: 'An Article' });
await expect(row).toBeVisible({ timeout: 20_000 });
await row.click();
await page.locator('#files button[data-a="del"]').click();
// No "reaped", no chip at all: just the way to get it again.
await expect(row.locator('.kind.here')).toHaveCount(0);
await expect(row).not.toContainText(/reaped/i);
await row.click();
await expect(page.locator('#files')).not.toContainText(/reaped/i);
await expect(page.locator('#files button[title="Download to the server"]')).toBeVisible();
});
test('one action, one icon: the toolbar, the page and every dialog agree', async ({ page }) => {
const icon = loc => loc.locator('svg path').first().getAttribute('d');
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
// Unsubscribe is a minus in the toolbar and the feed header, never the x that closes things.
expect(await icon(page.locator('#content .acts [data-a="rm"]'))).toBe(await icon(page.locator('#tbRemove')));
// The toolbar's read and keep show the selected item's state, as its own buttons do, and follow
// a change made from the toolbar.
await page.locator('.ep').first().click();
const pair = async a => [await icon(page.locator(a === 'read' ? '#tbRead' : '#tbFlag')),
await icon(page.locator(`#detail [data-a="${a}"]`))];
for (const a of ['read', 'flag']) { const [tb, own] = await pair(a); expect(tb).toBe(own); }
const [kept] = await pair('flag');
await page.locator('#tbFlag').click();
await expect.poll(async () => { const [tb, own] = await pair('flag'); return tb === own && tb !== kept; }).toBe(true);
await page.locator('#tbFlag').click(); // leave it as it was
await expect.poll(async () => (await pair('flag'))[0]).toBe(kept);
// Every button in every dialog is an icon with its words in the tooltip.
const dialogs = [
() => page.locator('#addFeed').click(),
() => page.locator('#prefs').click(),
async () => { await page.locator('#prefs').click(); await page.locator('#gusers').click(); },
async () => { await page.locator('#prefs').click(); await page.locator('#gopml').click(); },
() => page.locator('#logs').click(),
() => page.locator('#content .acts [data-a="settings"]').click(),
() => page.locator('#content .acts [data-a="dl"]').click(),
() => page.locator('#content .acts [data-a="rm"]').click(),
];
for (const open of dialogs) {
await open();
const btns = page.locator('#modalCard .btn');
await expect(btns.first()).toBeVisible();
for (const b of await btns.all()) {
await expect(b.locator('svg')).toHaveCount(1);
await expect(b).toHaveAttribute('title', /\S/);
}
await page.keyboard.press('Escape');
await expect(page.locator('#modal.on')).toBeHidden();
}
});
test('All Subscriptions marks everything read, across every feed', async ({ page }) => {
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
await all.click();
// Earlier tests read things; make sure something is unread. Opening an item reads it, and
// its own button makes it unread again.
await page.locator('.ep').first().click();
await page.locator('#detail [data-a="read"][title="Mark unread"]').click();
await expect(all.locator('.badge')).not.toHaveText('0');
page.once('dialog', d => d.accept());
await page.locator('#content .acts [data-a="readall"]').click();
await expect(all.locator('.badge')).toHaveText('0');
await page.locator('.tabs button', { hasText: 'Unread' }).click();
await expect(page.locator('.ep')).toHaveCount(0);
});
test('the item table sorts by any column, both ways, and remembers', async ({ page }) => {
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
await all.click();
const head = k => page.locator(`#list .ephead [data-sort="${k}"]`);
const titles = () => page.locator('#eps .ep .t').allTextContents();
// Byte order on lower case, which is what SQLite gives for lower(...).
const cmp = (a, b) => (a.toLowerCase() < b.toLowerCase() ? -1 : a.toLowerCase() > b.toLowerCase() ? 1 : 0);
const sorted = (t, dir) => JSON.stringify(t) === JSON.stringify([...t].sort((a, b) => cmp(a, b) * dir));
await expect(page.locator('#eps .ep').nth(2)).toBeVisible({ timeout: 20_000 });
expect(new Set(await titles()).size).toBeGreaterThan(2); // or both orders would prove nothing
await expect(head('title')).toHaveText('Title');
await head('title').click();
await expect.poll(async () => sorted(await titles(), 1)).toBe(true);
await head('title').click();
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
// Kept across a reload.
await page.reload();
await all.click();
await expect(head('title').locator('.arr.desc')).toBeVisible();
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
// Size has its own column; the file column is just what the file is.
await expect(page.locator('#eps .ep .size', { hasText: /\d/ }).first()).toBeVisible();
await expect(page.locator('#eps .ep .file', { hasText: /\d/ })).toHaveCount(0);
});
test('play in the Files pane plays once, in the player bar', async ({ page }) => {
// Regression: the pane had an <audio> of its own, and playing it started the player bar too,
// so the same file played twice at once.
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
await page.locator('.ep', { has: page.locator('.kind.here') }).first().click();
await page.locator('#files [data-a="play"]').click();
await expect(page.locator('#player')).toBeVisible();
await expect(page.locator('audio')).toHaveCount(1); // the player bar's, and nothing else
await page.locator('#pclose').click();
});
test('someone the proxy signs in never sees the password page, and signs out through the proxy', async ({ page, browser }) => {
// Signed in with the token, not by the proxy: Sign out stays ipx's own.
expect((await (await page.request.get('/api/me')).json()).sign_out).toBeNull();
const ctx = await browser.newContext({ extraHTTPHeaders: { 'X-Test-User': 'proxied@example.com' } });
const proxied = await ctx.newPage();
// Regression: after Sign out, the password form showed to someone the proxy still vouched for.
await proxied.goto('/login');
await expect(proxied).toHaveURL(/:8791\/$/);
await expect(proxied.locator('#who')).toContainText('proxied@example.com');
expect(await (await proxied.request.get('/api/me')).json())
.toMatchObject({ name: 'proxied@example.com', sign_out: '/signed-out-by-the-proxy' });
await proxied.locator('#signout').click();
await expect(proxied).toHaveURL(/\/signed-out-by-the-proxy$/);
await ctx.close();
});

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Aardvark Radio</title><link>http://127.0.0.1:8792/</link>
<description>Inside the OPML, and first in it and alphabetically.</description>
<item><title>Aardvark Ep</title><guid>aa-1</guid><description>x</description></item>
</channel></rss>

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Fresh Show</title><link>http://127.0.0.1:8792/</link>
<description>Added in the browser suite, and scanned by adding it.</description>
<item><title>Fresh Ep</title><guid>fresh-1</guid><description>x</description></item>
</channel></rss>

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Imported Show</title><link>http://127.0.0.1:8792/</link>
<description>Only ever arrives through an OPML import.</description>
<item><title>Imported Ep</title><guid>imp-1</guid><description>x</description></item>
</channel></rss>

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Paid Show</title><link>http://127.0.0.1:8792/</link>
<description>Subscribed with a key in its URL, so it must never be offered to anyone else.</description>
<item><title>Paid Ep</title><guid>paid-1</guid><description>x</description></item>
</channel></rss>

View File

@@ -1,4 +1,5 @@
<opml version="2.0"><head><title>Test Subscriptions</title></head>
<body><outline text="Folder">
<outline type="rss" text="Aardvark Radio" xmlUrl="http://127.0.0.1:8792/aardvark.xml"/>
<outline type="rss" text="Grouped Show" xmlUrl="http://127.0.0.1:8792/other.xml"/>
</outline></body></opml>
</outline></body></opml>

View File

@@ -10,7 +10,15 @@ const TOKEN = 'testtokentesttokentesttoken12345'; // fixed, so tests need not
// Called from playwright.config.js at load time, NOT as globalSetup: Playwright starts
// webServer *before* globalSetup, so a config written there does not exist yet when the
// daemon launches -- it would fall back to the real config and fight the live daemon.
// Playwright imports this config again in every worker process, so prepare() runs more
// than once per suite. Wiping on the second call deleted the data directory out from under
// the running daemon: it kept serving from the unlinked inode, while anything else opening
// that path -- the CLI, a query -- got a brand new empty database and disagreed with it.
function prepare() {
// Only the process that launches the run may wipe. A worker gets TEST_WORKER_INDEX.
if (process.env.TEST_WORKER_INDEX !== undefined || process.env.PW_WORKER_INDEX !== undefined) {
return;
}
fs.rmSync(root, { recursive: true, force: true });
for (const d of ['config', 'data', 'downloads']) {
fs.mkdirSync(path.join(root, d), { recursive: true });
@@ -29,6 +37,10 @@ enabled = false
enabled = true
bind = "127.0.0.1:8791"
token = "${TOKEN}"
# The proxy path, for tests that send the header themselves: the daemon sees them at 127.0.0.1.
trusted_header = "X-Test-User"
trusted_proxies = ["127.0.0.1"]
sign_out_url = "/signed-out-by-the-proxy"
[feeds.test-show]
url = "http://127.0.0.1:8792/show.xml"
@@ -47,6 +59,11 @@ auto_download = true
[feeds.test-subscriptions]
url = "http://127.0.0.1:8792/subs.opml"
auto_download = false
# A key in its URL, like a Patreon feed: someone's paid subscription, never offered to others.
[feeds.paid-show]
url = "http://127.0.0.1:8792/paid.xml?auth=secret123"
auto_download = false
`);
}

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long