37 Commits

Author SHA1 Message Date
2af57065c6 Release 0.5.1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:15:53 +00:00
564b011c7a Give the folder triangle a wider gutter
The feed list's left padding grows from 8 to 16 px and the triangle's
button from 16 to 24 px wide, so it is no longer cramped against the
folder's art. Everything in the list shifts together, so feeds still line
up with the places above.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:13:32 +00:00
9eb7aadced Release 0.5.0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:06:23 +00:00
c6bceaef37 Docs: a slow migration and a CLI run at the same time
Every ipx command migrates when it opens the database, so the healthcheck
collided with the daemon while it dropped the old entries columns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:59:05 +00:00
dc63d6acaf Cut what the audit found: dead columns, one-time upgrades, three deps
Works through TODO.md from the 2026-09-12 over-engineering audit. Drops the
entries.read/flagged/position columns (migrate() removes them from older
databases), migrate_opml_children, the legacy interval_mins key, the
contrib/ systemd units, test-only Db wrappers, a duplicate token generator,
redundant logbuf visitors, unused page state and CSS, and the infer, dirs
and tokio-stream dependencies. The icon is served once as /icon.png instead
of inlined four times, taking about 94 KB off the two pages.

The adoption's subscription half was not dead: it gives a fresh install's
first admin the config's feeds. It stays as adopt_catalogue, now tested.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:55:44 +00:00
8937f35f00 TODO.md: cleared, the design pass is done
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:31:08 +00:00
0990f2a90d Design pass on the web UI: amber for new, EQ bars, keyboard feed list
Works through TODO.md from the 2026-09-11 review. Unread badges, dots and
download bars take the icon's amber; the playing item is marked by EQ bars
that move only while it plays. The feed list is usable from the keyboard,
focus rings show everywhere, and folders get a mosaic of their shows' art
with the triangle hung in the margin. Sentence-case labels, fewer bold
weights, tinted initials tiles, shorter header lines, "Kept" everywhere,
and the list gets the room the empty panes had. Reduced motion is honoured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-11 22:24:41 +00:00
0cc002cdfa todo.md: folders in the sidebar; drop the system-theme item
The disclosure triangle, looked at closely: the feed list cannot be
reached from the keyboard (span and div, no tabindex), every feed is
pushed 28 px right for a slot only folders use, the target is 18 px,
shows barely nest under their folder, a folder looks like a feed, and
a selected feed's placeholder tile vanishes in Dark and Light.

Following the system light or dark setting is off the list, by
choice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 19:39:49 +00:00
2c9e899762 todo.md: the design review's list, to do later
A review of the web UI against screenshots of every view in all three
themes. The palette and Classic carry the iPodderX identity; Dark and
Light do not. First three: amber for new (unread badges, dots,
download progress), EQ bars as the playing marker, and toolbar focus
rings that overflow:hidden currently clips. The rest is weights,
sentence-case labels, pane sizes, header lines, wording, system theme,
reduced motion, art, sign-in and the Settings export/import icons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 18:53:42 +00:00
2e416f96cf Patreon creators split into their shows; filters follow settings
A Patreon token pasted into Add feed, or a creator link without
&show=, becomes a folder of that creator's shows, found through
Patreon's web API and kept in step like a subscribed OPML (sync_group,
split out of sync_opml). A creator already read as one feed is split
too: each show takes over the files and read state it held
(Db::adopt). A creator with one show stays a plain feed.

Filter verdicts are judged again every scan, so turning on Allow
explicit brings skipped items back. Add feed has an explicit box.
Feeds in a group follow your settings on the group, as its dialog
said. A new feed no longer takes the id of a removed one at a
different URL and shows its old items. See CHANGELOG.md [Unreleased]
and docs/history.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 18:24:47 +00:00
9269aa99f7 README: a short overview that points into docs/
It described the layout from before 0.4.0 (items across the top, a
player below), and carried long sections on OPML, the log view and
this server's own deploy steps, all of which docs/ and CLAUDE.md cover.
It now says what ipx does, how to run it with Docker or from source,
the first sign-in, the TLS caveat, where the docs are, and the tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:36:34 +00:00
ae8123250b Release 0.4.0
The original iPodderX layout (toolbar, places, item table, Files pane),
the Classic theme, Directory and Popular, All Subscriptions with mark
everything read, sortable columns and a Size column, one meaning per
icon across the UI, and fixes for the double play, Escape in dialogs,
the dark-theme password box and paid feeds listed in Popular. See
CHANGELOG.md [0.4.0] and docs/history.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:35:08 +00:00
2c34a144ba History: the icon pass, sorting and the double play; fix a stale changelog line
docs/history.md gets the long-form entry CLAUDE.md asks for: what the
UI pass found, the three bugs it turned up (Escape inside a dialog's
text box, the white password box, the stray dot), why sorting is done
on the server, and how the Files pane came to play a file twice.
CHANGELOG's Added line for the item table named the old columns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:32:41 +00:00
669e8b5124 Files pane plays through the player bar, once
The pane drew its own <audio controls> for a downloaded file, and its
onplay also started the player bar, so one click played the same file
twice at once. The pane now has a play button (with the file's type
icon, like the other rows) that hands that exact file to the player
bar, the only player. play() takes the file, so another of an item's
files starts from its top instead of resuming the first. Dead CSS for
the pane's <audio> removed.

Test: play in the Files pane leaves one <audio> on the page, the bar's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:30:03 +00:00
5362436766 Sortable item table, Size in its own column, Subscribed as an icon
- Every column heading sorts (kept, title, feed, file type, size,
  published); a second click reverses it. The server sorts through a
  fixed whitelist (order_sql), so it covers the whole list, not the
  fifty loaded; the choice is remembered in the browser.
- Size is its own column and shows KB for small files instead of
  "0 MB". The Item heading is Title.
- Popular/Directory/Add feed: Subscribed is a green circle-check.
- Tests: every sort column runs and orders both ways (db); the table
  sorts by title both ways and remembers across a reload (browser).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:17:16 +00:00
0efc49519c One icon per meaning across the UI; mark everything read in All Subscriptions
- All Subscriptions' header checks every feed and marks everything read
  (POST /api/read-all, the same feeds the view lists); it asks first.
- Minus unsubscribes everywhere (the feed header's x read as "close"),
  x only closes or cancels, plus adds/subscribes/imports, and a dialog's
  confirm carries its action's icon. Remaining word buttons, the player
  and the folder arrow are Font Awesome 7.3.1 icons.
- Toolbar grouped by what it acts on (add, unsubscribe, scan | play,
  read, keep); read and keep show the selected item's state.
- The OPML subscription page uses the same header as a feed.
- Fixed: Escape ignored inside a dialog's text box (Add feed could not
  be closed with it), white password box in the dark theme, stray dot
  in an undated item's details.
- Tests: one action one icon across toolbar, page and all 8 dialogs;
  All Subscriptions mark everything read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:05:47 +00:00
57dcba2d1a One file icon, green when downloaded; mark unread is an envelope
- The separate check (and warning) beside a file's type icon is gone:
  the type icon itself is green once the file is downloaded and red when
  the download failed, with the details in its tooltip. One icon per row
  keeps the column lined up. On Classic's blue selection they are a
  lighter green and red rather than white.
- Mark unread under an item's title was a solid circle, which read as a
  record button. It is Font Awesome's closed envelope now.
- Drops the unused circle-check, circle-exclamation and circle icons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:20:32 +00:00
e95cccc66f Icons are Font Awesome Free, embedded as SVG
The ICON set is now Font Awesome Free 7.3.1: the 29 icons the page uses,
taken from svgs/solid and svgs/regular at that tag and embedded as SVG
paths, 12.8 KB in all. No webfont to download and nothing fetched from a
CDN. The CC BY 4.0 attribution is above the set, and in the README.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:14:51 +00:00
7295be8b25 Drawn icons throughout; file state and type as icons, no PENDING
- Every button's icon is drawn from one small SVG set (ICON), in bold
  strokes of the button's own colour, instead of font characters: ⟳ ⤓ ↗
  and the like came out thin and tiny and differed from font to font.
  Static buttons name theirs with data-icon. Keep is a flag everywhere.
- A file's state is an icon: a check when downloaded, a warning with the
  error in its tooltip when it failed, nothing while it waits. Its type
  (audio, video, image, pdf, torrent, other) is an icon with the word in
  its tooltip. The DOWNLOADED and PENDING chips are gone, which also
  fixes them being hard to read on Classic's blue selection.
- Tests find state and type by their tooltips.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:11:15 +00:00
f1b0d97b81 Classic: white text across a selected row, white lists
A selected row's file size stayed grey on the Aqua blue; the whole row
is white now. The directory's rows took the source list's pale
blue-grey; lists were white in the original.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:03:04 +00:00
ae47e31a97 Classic theme after the 2004 Mac app; header and dialog buttons as icons
- A Classic theme beside Dark and Light: brushed-metal toolbar and
  status bar, a pale blue-grey source list, Aqua blue for whatever is
  selected, red unread badges, a striped table with blue titles, a blue
  bar behind the item's title, and Lucida Grande. The theme button steps
  through all three and remembers the choice.
- The feed and OPML headers' buttons (scan, download latest, mark all
  read, settings, unsubscribe) and the Settings, feed settings and
  Download latest dialogs' buttons (save, download, cancel) are icons,
  with the words in title and aria-label.
- Tests: the theme button reaches Classic and it survives a reload; the
  header buttons are found by action.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:00:30 +00:00
8a309eb652 Files pane and item buttons are icons, with the words in tooltips
Save, delete, view and download in the Files pane, and mark read, keep
and open the original under an item's title, are icons now. The word is
in each one's title and aria-label, so it is still there on hover and
for screen readers. Tests find them by title or action, not text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:57:03 +00:00
1a2b0d87c6 CLAUDE.md: what to do when Docker Hub rate-limits the image build
The build asks Docker Hub about its two base images every time unless
they are stored locally, and this host has no Docker Hub login, so a busy
day ends in 429 Too Many Requests. Pulling them from mirror.gcr.io and
tagging them locally lets the build go ahead without asking.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:49:58 +00:00
d7fac2d0e7 Never list paid-feed services; scan on add; no "reaped"; slimmer header
- Security: feeds from Patreon, Supercast, Supporting Cast, Glow and
  Memberful are never listed in Popular or the Directory. A Supercast
  feed keeps its key in the URL's path, which the query check missed, so
  it was being listed.
- Adding a feed queues a scan of it, and an OPML import that added feeds
  scans what is due, so items show without pressing Scan.
- A file deleted to save space, or by hand, looks as if it was never
  downloaded: no "reaped" chip, just the Download button. The retention
  summary says "deleted".
- The feed header keeps its title and stats to one line each and wraps
  its buttons; a single feed's table drops the Feed column.
- Tests: adding a feed shows its item without Scan; a deleted file shows
  no "reaped"; paid-feed hosts and acast public ids in the unit test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:47:03 +00:00
df9b7645d6 The original iPodderX layout: toolbar, places, item table, Files pane
- A toolbar across the window with the original's groups: add and
  unsubscribe, play, mark read and keep for the selected item, scan, a
  search box for what is showing, and Settings and Log (admins only).
- Directory, Popular and All Subscriptions sit at the top of the feed
  list and open in the main pane; the Popular and Directory buttons and
  their dialogs are gone.
- All Subscriptions lists every item from every feed you subscribe to:
  GET /api/entries, the per-feed query with its scope widened. The
  enclosure lookup after it matches files to rows by feed and guid, since
  a page can now span feeds.
- Items are a table (unread, kept, item, feed, file, published) with a
  Files pane beside it, the text below, and a status bar with totals. On
  a phone the files follow the text and the table is title and date.
- Tests: enclosures are checked in #files; the toolbar's read, keep and
  play act on the selected item; All Subscriptions holds only your feeds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:39:05 +00:00
f3825cfc57 Popular is a top 10; a Directory lists every listed feed A to Z
- GET /api/popular returns the ten most subscribed feeds. The new GET
  /api/directory returns every feed that may be listed, sorted by name,
  from the same list: everyone counted, you included, never a URL, never
  a private feed or a feed inside an OPML. POST /api/popular/{id} still
  subscribes to anything on it.
- A Directory button sits beside Popular; both open the same list
  screen. The sidebar toolbar wraps rather than squeezing four buttons.
- Tests: the directory is A to Z, Popular is its top ten, Paid Show is
  in neither, and subscribing works from the directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:13:16 +00:00
c0f4b0bcb2 Popular button in the sidebar; the popular list counts everyone
- A Popular button beside + Feed opens the popular list directly; the
  Add feed dialog keeps it too.
- The list counts every subscriber, you included. Your own feeds stay on
  it, marked Subscribed, and clicking one opens it. Private feeds and
  feeds inside an OPML are still never listed, for anyone.
- GET /api/popular rows carry `subscribed`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:05:24 +00:00
8b8b48302b Popular on this server; changelog follows Keep a Changelog; 0.3.0
- Add feed lists what other accounts subscribe to, most subscribers
  first, and subscribes you by id (GET /api/popular, POST
  /api/popular/{id}). Rows never carry a URL. Feeds from an OPML and
  anything that looks private (a login, credentials in the URL, a key
  such as auth= or token=) are never listed, and the subscribe route
  checks the id against the same list.
- CHANGELOG.md follows Keep a Changelog 1.1.0: 0.1.0 (2026-09-09, the
  CLI), 0.2.0 (2026-09-10, the web UI), 0.3.0 (2026-09-11, accounts and
  sharing). The long-form entries moved unchanged to docs/history.md.
- Cargo.toml is 0.3.0. CLAUDE.md says how to add an entry and cut a
  release.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 13:56:57 +00:00
5d3fdde4da Upload an OPML file to import; tests for every way in and out
- The import screen has a file picker beside the paste box. The page
  reads the file, checks it looks like OPML before sending, and clears
  the picker when it is refused and after it is imported. The file is
  sent as text and never written to disk on the server.
- The server parses the OPML before touching anything and answers 400
  "that is not an OPML file" (was a 500). subscribe_opml takes a parsed
  document, so ipx import also refuses a non-OPML file by name.
- Tests: Settings' Export OPML download and paste import; uploading an
  RSS file (refused) and a real OPML; the server's 400; the admin's
  export round-tripped into a second account; ipx import/export in a
  scratch config.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 13:42:38 +00:00
8784d0a3fd OPML import subscribes you; export lists only your feeds
Import predated accounts: it only added URLs missing from config.toml
and subscribed nobody. Importing another account's export did nothing
("Imported 0 feed(s)"), and a genuinely new feed had no subscriber, so
it was never scanned. Web and CLI import now share subscribe_opml,
which subscribes the caller (the CLI: the first admin) to every feed in
the file and reports new vs already-subscribed.

Export wrote the whole catalogue to anyone signed in, including other
people's private feed URLs. It now lists only your own subscriptions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 12:53:29 +00:00
5e95557cbb User admin in the web UI, admin-only log, unread-first OPML feeds
- Settings > Manage users: add an account (password, or none for proxy
  sign-in), toggle admin, remove. Backed by GET/POST /api/users and
  PATCH/DELETE /api/users/{id}, 403 for non-admins. The only admin
  cannot be demoted or removed.
- GET /api/logs is admin-only and the Log button is hidden for others;
  the log names every account, feed and failed sign-in.
- Feeds inside an OPML list those with unread items first, in the
  sidebar folder and on the subscription's page.
- Deploying is now buildx --push to 192.168.1.130:5000 and recreating
  the ipodderx service of the Arcane project content; CLAUDE.md and the
  README's Docker section say so.
- Tests: Playwright for user admin, the last-admin guard, 403s for a
  non-admin and the unread ordering (new Aardvark Radio fixture); a unit
  test for last_admin; the smoke test drives usersModal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 12:43:28 +00:00
6114add4a6 Documentation: docs/, a changelog, and CLAUDE.md
PROGRESS.md becomes CHANGELOG.md with the finished step lists moved to an
appendix. The README is an overview pointing at docs/: configuration,
cli, users, sso (refreshed for accounts and admin-only settings), and
architecture. CLAUDE.md collects what working on this code actually
requires -- pkill -x not -f, the page being compiled in, the dead columns
on entries, the Playwright worker that deleted its own database.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 03:00:25 +00:00
c47c224372 Pruning respects a star from anyone
prune_entries still guarded on entries.flagged, which nothing writes
since read state moved to entry_state -- so starring a text item with no
file would not have saved it from the age sweep. It follows the reaper's
rule now, and takes orphaned read state with whatever it deletes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 02:41:12 +00:00
686851b448 Warn before deleting a file other people share
A feed with other subscribers labels the button Delete for everyone and
names them in the confirmation. The server decides: if anyone else has
starred the item or not played it, DELETE returns 409 with the reason and
only ?force=true proceeds. A feed's header says when it is shared, which
answers why a file nobody here asked for exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 02:37:07 +00:00
f0d03c79c8 first commit 2026-09-11 02:31:09 +00:00
7df4ee7dde Retention follows per-user read and starred
reap_candidates still read entries.read/flagged, which nothing writes
since read state moved to entry_state -- so starring no longer protected
a file and the read-first ordering was dead. One file serves every
subscriber, so anyone starring it keeps it, and it counts as read only
once everyone subscribed has read it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 02:25:24 +00:00
d46ec73261 Per-user read state and subscriptions
Read, starred and position move to entry_state; subscriptions carry each
person's keywords, auto-download, explicit and per-scan limit. The feed
list and unread counts are per person, and the existing library is
adopted by the admin on first start.

The feed URL, folder and schedule stay shared and admin-only: one file
serves everyone, so they describe the file rather than a preference.
Scanning merges subscribers' wants -- anyone wanting an item is enough --
via merge_policy, which is pure and tested.

Also: the test fixture wiped its data directory from every Playwright
worker, deleting the database out from under the running daemon.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
2026-09-11 02:17:16 +00:00
34 changed files with 4653 additions and 1051 deletions

274
CHANGELOG.md Normal file
View File

@@ -0,0 +1,274 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
The long form, with what was wrong before and how it was found, is in
[docs/history.md](docs/history.md).
## [Unreleased]
## [0.5.1] - 2026-09-12
### Fixed
- The triangle that opens an OPML or Patreon folder was cramped against the folder's art. It has
more room now, and a wider target to click.
## [0.5.0] - 2026-09-12
### Added
- A Patreon token pasted into Add feed, or a creator's RSS link without `&show=`, becomes a folder
of that creator's shows, kept in step on every scan like a subscribed OPML. A creator with only
one show stays a plain feed. One already added as a single long feed is split into its shows on
its next scan, keeping its files and what you had read.
- Add feed has an "Allow items marked explicit" box, so a new feed's first scan no longer skips
every explicit item.
### Changed
- Unread counts, unread dots and download progress are amber, the colour of the icon's EQ bars.
Blue is kept for the primary action and links, so a count no longer looks like a button.
- What is playing is marked by small EQ bars, in its row and in the player. They move only while it
plays.
- A folder in the sidebar shows its first four shows' art as a mosaic, and its shows sit under its
title. Only folders have a triangle, so every feed lines up with Directory and Popular above.
- Feeds without art get initials in a colour of their own, instead of all the same grey.
- The Flagged tab is called Kept, as the Keep button and Settings already said.
- A feed's header is one short line; when it checks next is in its tooltip.
- The item list takes more of the window, and the Files pane shows only when the item has files.
- Column headings and tags are in sentence case, and fewer things are bold.
- Unsubscribe is a round button beside the feed's other actions.
- Export and Import in Settings say what they do.
- The sign-in page shows the original icon large.
- Nothing animates when your system asks for reduced motion.
- The pages are about 90 KB smaller: the icon is served once instead of written into each.
- A web token generated for a new install is 64 characters instead of 32.
- The README is a short overview of what ipx does and how to run it, and points into `docs/` for
the rest. It still described the layout from before 0.4.0.
### Removed
- The systemd units in `contrib/`. Run ipx with Docker, or point a unit of your own at
`ipx daemon`.
- Upgrading from before 0.3.0 directly: what was read, kept or part-played before accounts is no
longer carried over to the admin, and OPML feeds that old versions wrote into `config.toml` are
no longer moved out of it. Upgrade through 0.4.0 first.
- `interval_mins` in `config.toml` is ignored; use `schedule`.
### Fixed
- The feed list works from the keyboard: Tab reaches every feed, Enter opens it, and Right and Left
open and close a folder. Every button shows where the focus is, including in the toolbar, which
used to clip the ring.
- "1 items" reads "1 item".
- A selected feed without art no longer loses its initials tile in Dark and Light.
- The player shows the feed's initials when there is no art, not the episode's.
- Turning on Allow explicit, or changing keywords or auto-download, brings back what those settings
had skipped on the feed's next scan. Before, an item was judged once, when first seen, and a
skipped one stayed skipped whatever you changed.
- Feeds inside an OPML or a Patreon creator follow your settings on the folder unless you set their
own, as the folder's settings dialog said they did. Before, the folder's settings reached nothing
inside it.
- A new feed no longer takes the name of one you removed earlier and shows that feed's old items.
Re-adding the same feed still gets its old name, and its history, back.
## [0.4.0] - 2026-09-11
### Added
- A Classic theme after the 2004 Mac app, beside Dark and Light: brushed-metal toolbar, Aqua
blue selection, red unread badges, a striped table and Lucida Grande. The theme button steps
through all three and remembers the choice.
- A toolbar across the top, after the original iPodderX: add and unsubscribe, play, mark read
and keep for the selected item, scan, a search box for what is showing, and Settings and Log.
- Directory, Popular and All Subscriptions at the top of the feed list, opening in the main pane.
Directory lists every feed anyone here subscribes to, A to Z (`GET /api/directory`). All
Subscriptions lists every item from every feed you subscribe to (`GET /api/entries`).
- Items show as a table (unread, kept, title, feed, file, size, published) with a Files pane beside
it, and a status bar with the totals.
- Mark everything read from All Subscriptions, across every feed you subscribe to
(`POST /api/read-all`). It asks first. All Subscriptions can also check every feed from its header.
- Click a column heading in the item table to sort by it (kept, title, feed, file type, size,
published); click again to reverse. The server sorts, so it covers the whole list, not just the
fifty shown, and the choice is remembered.
### Changed
- Popular shows the top 10, not 20, and counts everyone, you included. Your own feeds stay on it,
marked Subscribed, and clicking one opens it.
- Adding a feed scans it straight away, and an OPML import that added feeds scans them, so their
items show without pressing Scan.
- A file deleted to save space, or by hand, looks as if it was never downloaded: no "reaped"
label, just the Download button. The retention summary says "deleted", not "reaped".
- Buttons are icons, with the words in their tooltips: the Files pane (save, delete, view,
download), an item's own buttons (mark read, keep, open the original), the feed header (scan,
download latest, mark all read, settings, unsubscribe), and the Settings, feed settings and
Download latest dialogs (save, download, cancel). The icons are Font Awesome Free, embedded as
SVG: only the ones used, no font to download, and nothing fetched from anyone else. They
replace font characters such as ⟳ ⤓ ↗, which came out thin and tiny and differed from font to
font. Keep is a flag everywhere, as it was in the original, and mark unread is an envelope.
- One meaning per icon. Minus unsubscribes, x closes or cancels, plus adds or subscribes, and a
dialog's confirm button carries the icon of what it does. The feed header's unsubscribe was an x
and read as closing the page. The remaining word buttons are icons too:
- Log, Add feed, Users, Unsubscribe and OPML.
- Popular's Subscribe, Copy and Sign out.
- The Subscribed label in Popular, the Directory and Add feed, which is now a green check.
- The player's back, play, forward and close, which were font characters, and the folder arrow.
- The toolbar's read and keep buttons show the selected item's state, with the same icons as the
item's own buttons. Play, read and keep sit together, and Scan sits with add and unsubscribe.
- An OPML subscription's page has the same header as a feed's, with its buttons in the same places.
- The item table's size has its own column, apart from the file's type, and shows KB for small
files instead of "0 MB". The Item heading is now Title.
- A file's type is an icon (audio, video, image, PDF, torrent, other), green once it is
downloaded and red when the download failed, with the details in its tooltip. One icon per
row keeps the column lined up. The DOWNLOADED and PENDING labels are gone.
### Fixed
- Playing a file from the Files pane played it twice at once, in the pane and in the player bar.
The pane has a play button now, and the player bar is the only player.
- The password box in Manage users was white in the dark theme.
- An item with no date showed a stray dot in its details.
- Escape did not close a dialog while the cursor was in one of its boxes, so Add feed, which opens
in its URL box, could not be closed with Escape.
### Security
- Feeds from paid-feed services (Patreon, Supercast, Supporting Cast, Glow, Memberful) are never
listed in Popular or the Directory. A Supercast feed, which keeps its key in the URL's path
rather than the query, was being listed.
## [0.3.0] - 2026-09-11
### Added
- Add feed lists what other people on this server subscribe to, most subscribers first, and
subscribes you by id (`GET /api/popular`, `POST /api/popular/{id}`). Feeds from an OPML, and
feeds with a login or a key in their URL, are never listed.
- Upload an OPML file to import, beside the paste box. The page checks it looks like OPML before
sending it and clears the picker afterwards.
- Settings → Manage users: add and remove accounts, and choose who is an admin
(`GET`/`POST /api/users`, `PATCH`/`DELETE /api/users/{id}`).
- Per-user subscriptions, and per-user read, starred and playback state. The existing library is
adopted by the admin on first start.
- Subscribing to a feed someone else already has costs no second fetch and no second copy. Scanning
merges every subscriber's wants.
- Delete on a shared feed reads **Delete for everyone**, and the server answers `409` while anyone
else has starred the item or not played it (`?force=true` overrides).
- A shared feed's header says how many other people read it.
- `docs/` for configuration, the CLI, users, SSO and architecture, and `CLAUDE.md` for anyone
working on the code.
- Browser tests for OPML import and export by every route, user admin, unread ordering, and
`ipx import`/`ipx export`.
### Changed
- Feeds inside an OPML subscription list the ones with unread items first.
- OPML import subscribes you to every feed in the file. `ipx import` subscribes the first admin.
- OPML export lists only your own subscriptions.
- Production runs as a Docker image pushed to `192.168.1.130:5000` and recreated with
`docker compose`.
- This changelog follows Keep a Changelog. The long-form entries moved to `docs/history.md`.
### Fixed
- Importing another account's OPML export subscribed nobody and reported "Imported 0 feed(s)".
Feeds it added had no subscriber, so they were never scanned.
- Importing something that is not OPML answered `500`. It is now `400` "that is not an OPML file",
refused before anything changes.
- Starring stopped protecting a file from the quota and age sweeps once read state became per-user.
### Security
- The log is admin-only (`GET /api/logs` answers `403`, and the Log button is hidden). It names
every account, every feed and every failed sign-in.
- OPML export no longer hands anyone signed in the whole catalogue, including other people's
private feed URLs.
## [0.2.0] - 2026-09-10
### Added
- Web UI served by the daemon: plain HTML and JS compiled into the binary, with feeds, items,
filters, search, sanitised show notes and live progress over SSE.
- Player bar with resume, speed, keyboard shortcuts and lock-screen controls.
- Three-pane layout: feeds beside, items above, and the selected item's text and files below.
- Phone layout.
- Accounts and sign-in: Argon2id passwords, session cookies, `ipx user add|list|passwd|rm`, and a
trusted proxy header for Cloudflare Zero Trust or Authentik (`docs/sso.md`).
- Subscribing to an OPML: it is re-read every scan and its feeds show as a folder. A feed dropped
from it is removed unless it has downloads.
- Scheduling: a global interval with per-feed overrides (`every 30m`, `4h`, `1d`, `2w`).
- `[general] media_types`, default audio and video, and `max_new_per_check`, default 3.
- Every enclosure of an item, a View link for files that are not audio or video, and per-item
artwork.
- In-app log view with Daemon I/O, Scans and HTTP tabs.
- Mark all read on an OPML subscription.
- Editable feed URL with a copy button.
- Docker image whose healthcheck goes through the control socket.
- The iPodderX name, icon, and a colour scheme taken from the icon.
- `tests/page-smoke.js` and a Playwright browser suite.
### Changed
- Global settings and scan schedules are admin-only. The per-feed schedule picker is gone.
- Feeds from an OPML live in the database, not `config.toml`.
- Opening an item marks it read. Playing it marks it read only at the end or past 90%.
- "Episodes" became "items", since half the library is text.
- A burst of scan events causes one refresh, not one per feed.
- All is the default filter. OPML import and export, Settings and Log moved out of the header.
- Torrents run detached, two at a time.
### Fixed
- Download fetched the next queued episodes instead of the one clicked.
- Pressing play made an item vanish from the Unread list.
- Clearing a folder, schedule or cap from the UI did nothing.
- The daemon ignored SIGTERM until the current download finished.
- A missing function stopped the page script and left the whole UI dead.
- One download painted progress on every pending row.
- A torrent could freeze scanning for up to an hour.
- Downloading from an OPML feed failed with "belongs to unsubscribed feed".
- Image enclosures were downloaded, counted as episodes and given a play button.
- A feed whose entries had been deleted stayed empty, because the server kept answering `304`.
- An item with several enclosures kept only the last.
- OPML folders rendered open by default.
- Mark read in the text pane recursed until the stack overflowed.
- The Unread, Downloaded and Flagged filters answered `500` without a search term.
- An OPML subscription always showed 0 unread.
- Folder names kept doubled spaces where separators were stripped.
- Sidebar rows had four different left edges.
### Security
- The web UI needs a token or a sign-in. The token is compared in constant time, and an empty
token refuses to serve.
- Show notes are sanitised with `ammonia`.
- A proxy's user header is honoured only from an address in `trusted_proxies`.
- A feed URL must be http(s), so `file:///etc/passwd` is refused.
- Download folders are sanitised per path segment, so `../../etc/Show` cannot climb out.
## [0.1.0] - 2026-09-09
### Added
- `ipx`, a Rust rewrite of the iPodderX engine: TOML config, SQLite state, and `ipx list`, `add`,
`rm` and `fetch`.
- RSS and Atom parsing with conditional GET, `<ttl>` and basic auth.
- Streaming downloads with explicit, keyword and per-scan filters, deduplicated by enclosure URL.
- Quota and age retention that never touches a starred file, and `ipx reap [--dry-run]`.
- `ipx daemon` with a JSON-lines Unix socket. CLI commands proxy to a running daemon.
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.1...main
[0.5.1]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.0...v0.5.1
[0.5.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.4.0...v0.5.0
[0.4.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...v0.4.0
[0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0
[0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0
[0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0

148
CLAUDE.md Normal file
View File

@@ -0,0 +1,148 @@
# Working on ipodderx-rs
Notes for whoever picks this up next. Read [docs/architecture.md](docs/architecture.md) for how the
thing is built; this file is about working on it without repeating mistakes that have already been
made here.
## Where things are
Production is the `iPodderX` container on Tower (192.168.1.130), the `ipodderx` service of the
Arcane project `content`: `/mnt/fast/arcane/projects/content/compose.yaml`. That file is what runs;
`docker-compose.yml` in this repo is a copy, and editing it changes nothing in production.
| | Host | In the container |
|---|---|---|
| Image | `192.168.1.130:5000/ipodderx:latest` | |
| Config | `/mnt/fast/appdata/ipodderx/config.toml` | `/config/config.toml` |
| Database | `/mnt/user/ipodderx/state.db` | `/data/state.db` |
| Downloads | `/mnt/user/ipodderx/downloads` | `/downloads` |
| Web UI | `192.168.1.130:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel | `0.0.0.0:8099` |
Deploying a change is: build and push the image, then pull it and recreate the container.
```sh
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml pull ipodderx
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml up -d ipodderx
docker logs --tail 20 iPodderX
```
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
**A build that fails with `429 Too Many Requests` on a base image** is Docker Hub rate-limiting
this host. There is no Docker Hub login here, and the build asks about `debian:bookworm-slim` and
`rust:1-slim-bookworm` every time unless they are already stored locally. Pull them from Google's
mirror and tag them; the build then uses the local copies without asking Docker Hub:
```sh
docker pull mirror.gcr.io/library/debian:bookworm-slim
docker tag mirror.gcr.io/library/debian:bookworm-slim debian:bookworm-slim
docker pull mirror.gcr.io/library/rust:1-slim-bookworm
docker tag mirror.gcr.io/library/rust:1-slim-bookworm rust:1-slim-bookworm
```
Run those again now and then, or the local copies go stale.
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
the worker is alive, not just the web port. The container restarts on its own after a reboot.
Before the container, ipx ran by hand in code-server, with its files in `/config/.config/ipx/` and
`/config/.local/share/ipx/`. Those are still there and the container does not read them. If you run
a daemon by hand for testing, stop it with **`pkill -x ipx`, never `pkill -f ipx`**. `-f` matches
the shell running the command and kills the session (exit 144). This has happened more than once.
## Before you touch the page
`web/index.html` is `include_str!`d into the binary, so **every page change needs a rebuild** before
it is visible. It is one file: markup, CSS and script.
After any edit to it:
```sh
node tests/page-smoke.js
```
That loads the script against a stub DOM and checks every selector it wires at load actually
exists. It exists because a patch once anchored on a deleted function, `String.replace` silently
matched nothing, and the whole UI died with a `ReferenceError` while every server-side test passed.
Patching that file by guessing an anchor string has failed repeatedly. Read the exact block first
(`sed -n 'START,ENDp'`), match it verbatim, and assert the replacement happened rather than hoping.
## Tests
```sh
cargo test # ~51 tests: parsing, filters, retention, schedules, SQL, per-user state
node tests/page-smoke.js
npx playwright test # 16 browser tests against a real daemon on fixture feeds
```
Things about the browser suite that have cost time:
* It starts **its own daemon and database** under `/tmp/ipx-ui-test`, wiped once per run. Playwright
re-imports the config in every worker, so `prepare()` guards on `TEST_WORKER_INDEX` — without
that guard a worker deleted the database out from under the running daemon, which then kept
serving from the unlinked inode while everything else saw an empty file.
* Tests **share that daemon and run in order**. A test that opens an item marks it read and changes
what later tests see. Write assertions that do not depend on what ran before, or normalise the
state first.
* Fixture feeds must not share an enclosure URL, because `enclosures.url` is globally unique and
whichever feed is scanned first claims it.
* `webServer` starts **before** `globalSetup`, which is why the fixture config is written at
config-load time instead.
Non-trivial logic leaves one runnable check behind. Pure functions (`merge_policy`, `pick`,
`matches_keywords`, `parse_interval`) are the easiest place to put it.
## Things that are true and easy to get wrong
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
* **Read state lives in `entry_state`, per user, and nowhere else.** `entries` had `read`, `flagged`
and `position` columns from before accounts; two bugs came from queries still reading them
(retention, and the entry pruner), and `migrate()` now drops them.
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
derived and never written to config.
* **One fetch serves everyone**, so scan policy is a union of subscribers' wants (`merge_policy`).
Anyone wanting an item is enough to fetch it.
* **The UI hiding a control is not enforcement.** Admin-only actions check `user.is_admin` in the
handler and return `403`.
* **A `tokio::select!` only races its branches at the point of selection.** A long download has to
watch the shutdown channel itself; the daemon ignored SIGTERM for exactly this reason.
* Only one daemon per socket. Removing the socket file defeats the guard and you get two daemons
fighting over the database, with the stale one still holding the port.
* `/api/settings` answering `200` does **not** mean the worker is alive — it is a different task.
Probe the control socket (`ipx status`) to check that.
* **Every `ipx` command runs `migrate()` when it opens the database**, the healthcheck's
`ipx status` included. A migration that rewrites a big table (`DROP COLUMN`) takes seconds on
production, and a command run meanwhile fails with `migrating schema`. It changes nothing; wait
for `daemon started` in the log. Copy `state.db` aside before deploying one.
## House style
Comments explain **why**, not what. If a line looks odd, the comment says what went wrong without
it. No emoji, no exclamation marks, no "obviously". Prose in the UI and docs is plain English and
addressed to the person using it.
Every change gets one line under `## [Unreleased]` in [CHANGELOG.md](CHANGELOG.md), in its
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) group: Added, Changed, Deprecated,
Removed, Fixed or Security. Say it the way someone using ipx would notice it. When there is more to
say, such as what was wrong before or what it cost to find out, write it up at the top of
[docs/history.md](docs/history.md), dated. That record has been more useful than the git log more
than once.
Cutting a release: rename `[Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD` and open a new empty
`[Unreleased]` above it, bump `version` in `Cargo.toml`, tag the commit `vX.Y.Z`, and update the
compare links at the bottom of the changelog.
Deliberate simplifications get a `ponytail:` comment naming the ceiling and the upgrade path, e.g.
`// ponytail: global connection mutex, move to a pool if feed count makes it contend`.
## Known gaps
* Cloudflare's `Cf-Access-Jwt-Assertion` is not verified — ipx trusts the hop plus `trusted_proxies`
(documented in [docs/sso.md](docs/sso.md)).
* A feed's `<description>` subtitle is dropped whenever `content:encoded` exists, which loses
Substack-style subtitles.

34
Cargo.lock generated
View File

@@ -436,17 +436,6 @@ dependencies = [
"shlex", "shlex",
] ]
[[package]]
name = "cfb"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a347dcabdae9c31b0825fd6a8bed285ec9c2acb89c47827126d52fa4f59cece3"
dependencies = [
"fnv",
"uuid",
"web-time",
]
[[package]] [[package]]
name = "cfg-if" name = "cfg-if"
version = "1.0.4" version = "1.0.4"
@@ -867,15 +856,6 @@ dependencies = [
"dirs-sys", "dirs-sys",
] ]
[[package]]
name = "dirs"
version = "7.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d57d423b3c82e89b9a24ca3091fee61f456a26edbd28d26c65906f4bc1dcd8f"
dependencies = [
"dirs-sys",
]
[[package]] [[package]]
name = "dirs-sys" name = "dirs-sys"
version = "0.5.0" version = "0.5.0"
@@ -1608,15 +1588,6 @@ dependencies = [
"serde_core", "serde_core",
] ]
[[package]]
name = "infer"
version = "0.22.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f4200d433cbd5178df7797c9c2e75b348b728e39631cf14520d1e2fc424201f4"
dependencies = [
"cfb",
]
[[package]] [[package]]
name = "intervaltree" name = "intervaltree"
version = "0.2.7" version = "0.2.7"
@@ -1634,7 +1605,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
[[package]] [[package]]
name = "ipx" name = "ipx"
version = "0.1.0" version = "0.5.1"
dependencies = [ dependencies = [
"ammonia", "ammonia",
"anyhow", "anyhow",
@@ -1643,9 +1614,7 @@ dependencies = [
"axum", "axum",
"chrono", "chrono",
"clap", "clap",
"dirs",
"futures-util", "futures-util",
"infer",
"librqbit", "librqbit",
"opml", "opml",
"percent-encoding", "percent-encoding",
@@ -1656,7 +1625,6 @@ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
"tokio", "tokio",
"tokio-stream",
"toml", "toml",
"tower", "tower",
"tower-http 0.7.1", "tower-http 0.7.1",

View File

@@ -1,6 +1,6 @@
[package] [package]
name = "ipx" name = "ipx"
version = "0.1.0" version = "0.5.1"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
@@ -11,9 +11,7 @@ atom_syndication = "0.12.10"
axum = "0.8.9" axum = "0.8.9"
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] } chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
clap = { version = "4.6.6", features = ["derive"] } clap = { version = "4.6.6", features = ["derive"] }
dirs = "7.0.0"
futures-util = { version = "0.3.34", default-features = false, features = ["std"] } futures-util = { version = "0.3.34", default-features = false, features = ["std"] }
infer = "0.22.0"
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] } librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
opml = "1.1.6" opml = "1.1.6"
percent-encoding = "2.3.2" percent-encoding = "2.3.2"
@@ -24,7 +22,6 @@ rusqlite = { version = "0.40.2", features = ["bundled"] }
serde = { version = "1.0.229", features = ["derive"] } serde = { version = "1.0.229", features = ["derive"] }
serde_json = "1.0.151" serde_json = "1.0.151"
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] } tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
tokio-stream = { version = "0.1.19", features = ["sync"] }
toml = "1.1.5" toml = "1.1.5"
tower = { version = "0.5.3", features = ["util"] } tower = { version = "0.5.3", features = ["util"] }
tower-http = { version = "0.7.1", features = ["fs"] } tower-http = { version = "0.7.1", features = ["fs"] }

259
README.md
View File

@@ -1,238 +1,83 @@
# ipodderx-rs # ipodderx-rs
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent), A self-hosted podcatcher for a household. It checks your feeds, downloads the episodes, and serves
files them into per-feed folders, and reaps old episodes to stay under a disk quota. a web UI modelled on the 2004 Mac app **iPodderX**, for any number of people sharing one copy of
Runs as a one-shot CLI or as a daemon with a Unix-socket JSON event stream for a UI to attach to. the files. One Rust binary, `ipx`, is both the daemon and the command line.
## Lineage It is a rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python engine
behind iPodderX (2004-2008, Ray Slakinski & August Trometer).
This is a modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the ## What it does
Python 2 engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), which was
open-sourced under the MIT License in 2010.
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download - **The web UI.** It has a toolbar, and a feed list that opens with Directory, Popular and All
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the Subscriptions. Items sit in a sortable table with a Files pane, and there is a player bar. It
oldest-first disk quota reaper. comes in Dark, Light and Classic themes, and works on a phone.
- **Several people, one copy.** Each person has their own subscriptions and their own read, kept
and playback state. There is one file on disk per episode, however many people want it. People
sign in with a password or through a proxy (Cloudflare Zero Trust or Authentik), and admins
manage accounts and settings.
- **Scanning.** Feeds are checked on a schedule, globally or per feed, and a feed's own TTL is
honoured. Keyword, explicit-content and media-type filters decide what is downloaded, with a cap
on new downloads per scan.
- **Downloads.** Files come over HTTP or BitTorrent and are filed into a folder per feed.
Retention deletes the oldest files to stay under a disk quota or an age limit, and never touches
an item someone has kept.
- **OPML.** You can import and export your own subscriptions. You can also subscribe to an OPML
URL, which keeps a whole list in step as a folder.
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows ## Run it
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
3DES-encrypted preferences, and the `printMSG` stdout protocol (replaced by a JSON-lines socket).
## Quickstart With Docker:
```sh ```sh
cargo install --path . docker build -t ipodderx .
docker compose up -d
ipx add https://atp.fm/rss # names the feed from its own title
ipx list
ipx fetch # scan now
ipx daemon # or run continuously, honouring each feed's <ttl>
``` ```
Config lives at `~/.config/ipx/config.toml` (mode 0600, since it may hold feed passwords); `docker-compose.yml` is set up for the author's own server. Point its `image` and its three volumes
state at `~/.local/share/ipx/state.db`. Override with `IPX_CONFIG` and `IPX_DATA_DIR`. (`/config`, `/data` and `/downloads`) at yours first. The UI is on port 8099. BitTorrent uses 6881
Set `IPX_LOG=ipx=debug` for verbose logging on stderr. over TCP and UDP. Files are written as `PUID`/`PGID`, 99:100 by default.
## Commands From source:
| command | what it does |
|---|---|
| `ipx add <url> [--folder X] [--keywords a,b]` | subscribe; the id comes from the feed title |
| `ipx rm <feed>` | unsubscribe; downloads and history are kept |
| `ipx list` / `ipx status` | subscriptions and their state |
| `ipx fetch [FEED] [--force]` | scan; `--force` ignores the TTL |
| `ipx reap [--dry-run]` | run retention now |
| `ipx import/export <file.opml>` | move subscriptions in or out |
| `ipx daemon` | scheduler plus the control socket |
Any command with a wire form probes the socket first: if a daemon is running it does the work,
and the CLI just renders the events it streams back. `--local` forces in-process execution.
## Configuration
```toml
[general]
download_dir = "~/Podcasts"
socket = "/run/user/1000/ipx.sock" # default: $XDG_RUNTIME_DIR/ipx.sock
interval_mins = 60 # default poll; a feed's own <ttl> wins when longer
organize = "feed" # "feed" | "date"
max_total_gb = 50 # 0 = unlimited
max_age_days = 30 # 0 = keep forever
max_new_per_check = 3 # per feed, per scan. 0 = unlimited (pulls whole back catalogues)
media_types = ["audio", "video"] # what downloads automatically. Anything else is still
# listed and can be fetched by hand -- blog feeds put article
# images in enclosures. Empty takes everything.
[torrent]
enabled = true
seed_ratio = 1.0 # stop seeding at this ratio ...
seed_time_mins = 60 # ... or after this long, whichever comes first
port_range = "6881-6889"
stall_mins = 30 # give up on a torrent making no progress
[feeds.atp]
url = "https://atp.fm/rss"
folder = "Accidental Tech Podcast" # default: the feed title
keywords = ["deep dive"] # OR across keywords, AND within one
allow_explicit = false
auto_download = true
max_new_per_check = 3 # the rest wait for the next scan
username = "ray" # optional HTTP basic auth
password_env = "IPX_ATP_PASS" # or a literal `password`
```
Retention keeps files that are `flagged` in the database, and deletes read episodes before unread
ones, oldest first.
## Socket protocol
Newline-delimited JSON over a Unix socket, both directions.
```sh ```sh
$ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock cargo build --release
{"ev":"feed_start","feed":"atp"} ./target/release/ipx daemon
{"ev":"progress","feed":"atp","url":"...","file":"ep1.mp3","done":8192,"total":3000000}
{"ev":"download_done","feed":"atp","url":"...","path":"...","bytes":3000000}
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
{"ev":"scan_done","feeds":1}
``` ```
Commands: `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `status`. The first start creates **admin / ipodderx**. Sign in at `/login`, then change it:
Events: `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
`scan_done`, `reap_done` and `status` are terminal -- a client that asked for work stops there.
Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy ```sh
daemon also sees that daemon's other work. echo -n 'a good password' | ipx user passwd admin
```
## OPML The UI is plain HTTP, so put TLS in front of it if it is reachable from outside your network.
Two different things, both supported: ## Documentation
**Importing and exporting** a file copies subscriptions in or out once — `ipx import subs.opml`, | | |
`ipx export subs.opml`, or the OPML button in the UI.
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
other feed; every scan re-reads it and keeps your feed list in step. An OPML is recognised by its
content, so a URL without a `.opml` extension still works.
The feeds inside it are **not written to `config.toml`** — the OPML is the source of truth, so they
are re-derived each scan and held in the database. Your config keeps only what you chose. They show
as a collapsible folder in the sidebar, download into one nested folder, and newly listed ones are
scanned in the same run rather than waiting for the next interval. They inherit the subscription's
settings; change anything on one and it gets its own config entry from then on.
When a feed drops out of the OPML upstream:
| it has downloads | what happens |
|---|---| |---|---|
| no | unsubscribed and removed from the config | | [docs/configuration.md](docs/configuration.md) | Every config key, path and environment variable |
| yes | kept, flagged in the UI as no longer listed | | [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
A downloaded file is never left behind with nothing explaining where it came from. | [docs/sso.md](docs/sso.md) | Signing in through Cloudflare Zero Trust or Authentik |
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, control socket, HTTP API |
## Web UI | [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
| [docs/history.md](docs/history.md) | How it was built, with what was wrong and why |
```toml | [CLAUDE.md](CLAUDE.md) | Notes for working on the code, including how production is deployed |
[web]
enabled = true
bind = "0.0.0.0:8080" # 127.0.0.1:8080 by default
token = "" # generated and written back on first run
```
`ipx daemon` then serves it in the same process (`ipx daemon --web ADDR` overrides the bind for one
run). On first start it mints a token, saves it to config.toml, and prints the URL to open:
```
web ui token generated. Open:
http://0.0.0.0:8080/?token=1f4c…
```
`?token=` sets a year-long cookie, so you only paste it once per browser. Everything is behind that
token, including `/media/...` — a cookie rather than a header precisely because an `<audio>` element
cannot send headers.
Browse feeds, read show notes, play episodes in the browser (Range requests are served, so seeking
works), download or delete individual files, mark episodes read or flag them to keep, and edit a
feed's folder/keywords/explicit/auto-download/limit settings. Config edits are written to
config.toml and hot-reloaded — no daemon restart.
Show notes are feed-supplied HTML from an untrusted source; they are sanitized with `ammonia`
server-side before they reach the page.
**It is plain HTTP.** On a LAN bind, the token and everything else crosses the network in the
clear — and a feed URL can itself contain a credential (Patreon's, for one, carries an auth token).
Put it behind a reverse proxy with TLS if that matters to you.
## Layout
Feeds down the side; the selected feed's episodes across the top; the selected episode's text and
its enclosures in the pane below, which is where you play, download or delete them. The divider
between the two panes drags and the position is remembered. An OPML subscription's page instead
lists the feeds inside it.
An item may carry several enclosures. All of them appear in the pane below; the row summarises the
one you would act on and notes how many others there are. Anything that is not audio or video gets a
View link — opening the publisher's copy, or the local one once downloaded — rather than a player.
## Log view
The **Log** button in the sidebar shows the running daemon's output live, in four tabs:
**Daemon I/O** is the control protocol itself — every command in and every event out, as JSON;
**Scans** is feed and download activity; **HTTP** is web requests; **All** is everything: feed scans, downloads,
torrent activity and every HTTP request, with level and text filters and a copy button. It reads a
2000-line ring buffer held inside the process (`/api/logs`), not a file — so it works the same under
Docker, where logs go to stdout and there is no file to tail. The buffer keeps `debug` even when the terminal does not, so protocol traffic and routine
skips are there without making stderr unreadable — `IPX_UI_LOG` changes what it captures and
`IPX_LOG` what reaches stderr.
## Tests ## Tests
```sh ```sh
cargo test # the server: parsing, filters, retention, schedules, SQL cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
node tests/page-smoke.js # the page script loads without throwing node tests/page-smoke.js # the page script loads without throwing
npx playwright test # a real browser against a real daemon npx playwright test # a real browser against a real daemon on fixture feeds
``` ```
The Rust tests cover the server and the smoke test catches a script that fails to load, but neither `npm install` gets the test runner, and `npx playwright install --with-deps chromium` gets the
can see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which browser.
is what has actually slipped through. The Playwright suite drives a headless browser against a
daemon started on fixture feeds, and each test maps to a bug that reached a user: the page rendering
empty, a dead theme toggle, settings not persisting, episode metadata, filter tabs, the feed URL
field and its copy button, the log tabs, and OPML folders.
`npm install` gets the test runner; the browser itself comes from
`npx playwright install --with-deps chromium` (in `install.sh`).
## Docker
```sh
docker compose up -d # builds the image and starts it
docker compose logs -f ipx # the token is printed on first start
```
`docker-compose.yml` mounts `./config`, `./data` and a downloads directory, publishes 8099 for the
UI and 6881 (TCP **and** UDP — DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
land owned the way Unraid shares expect. On first start the entrypoint writes a config bound to
`0.0.0.0`, since a container's loopback is not reachable from outside it, and prints the URL with
its generated token.
The healthcheck runs `ipx status`, which goes through the control socket to the command worker — so
it catches a daemon that is alive but wedged, not merely one that has died.
## Running it as a service
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
rather run periodic scans with no daemon (in which case there is no socket for a UI to attach to).
## License ## License
MIT. See [LICENSE](LICENSE). MIT, see [LICENSE](LICENSE). The icons are [Font Awesome Free](https://fontawesome.com) 7.3.1 by
@fontawesome, under [CC BY 4.0](https://fontawesome.com/license/free), embedded as SVG.
## Signing in
A new database starts with **admin / ipodderx** at `/login`; change it with
`echo -n <password> | ipx user passwd admin`. Accounts are managed with `ipx user add|list|passwd|rm`.
To let Cloudflare Zero Trust or Authentik do the authenticating instead, see
[docs/sso.md](docs/sso.md).

30
TODO.md Normal file
View File

@@ -0,0 +1,30 @@
# To do
## Cut what is no longer needed
From a whole-repo audit for over-engineering on 2026-09-12. Biggest cut first.
- [x] **Pre-accounts adoption and the dead `entries` columns.** The copy of the old read state into
`entry_state` and the `entries.read`, `flagged` and `position` columns are gone. Its other half,
subscribing the first admin to the catalogue, was not dead and stays as `adopt_catalogue`.
(`src/db.rs`, `src/main.rs`)
- [x] **`contrib/` systemd units.** From before the container; nothing points at them.
- [x] **`migrate_opml_children`.** A one-time move of OPML children out of `config.toml` that has
run. Delete it and its call. (`src/main.rs`)
- [x] **The legacy `interval_mins` key.** Production uses `schedule`. Delete the field, the fallback
in `General::interval` and its test. (`src/config.rs`)
- [x] **`Db::entries` and `Db::count_entries`.** One-line wrappers only the tests call; the tests
call `entries_in` and `count_in` instead. (`src/db.rs`)
- [x] **`web::generate_token`.** Repeats `auth::new_session_token`. Use that. (`src/web.rs`)
- [x] **Page leftovers.** `globalEvery`, `S.busy`, `S.limit`, `unitOptions`' `firstLabel`, `--r`,
`.ep.open`, the phone `.ep .art`, the duplicate phone `.fhead.slim{flex-wrap}`, the second
`#sidebar{z-index}`, and the `on()` helper. (`web/index.html`)
- [x] **`logbuf` visitors.** `record_i64`, `record_u64` and `record_bool` repeat what `Visit`'s
defaults already do through `record_debug`. (`src/logbuf.rs`)
- [x] **The `infer` dependency.** Its torrent check is the `d8:announce` test on the next line.
- [x] **The `dirs` dependency.** `XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME` from `std::env`.
- [x] **The `tokio-stream` dependency.** `futures_util::stream::unfold` over the broadcast receiver.
- [x] **The icon inlined four times.** About 94 KB of base64 across both pages; serve it once as
`/icon.png` from `include_bytes!`, open without signing in like `/login`.
After these: `cargo test`, `node tests/page-smoke.js`, `npx playwright test`.

View File

@@ -1,7 +0,0 @@
[Unit]
Description=ipx feed scan (one shot)
[Service]
Type=oneshot
ExecStart=%h/.cargo/bin/ipx fetch
Environment=IPX_LOG=ipx=info

View File

@@ -1,18 +0,0 @@
# User unit: install to ~/.config/systemd/user/ipx.service, then
# systemctl --user enable --now ipx
# The socket lands in $XDG_RUNTIME_DIR/ipx.sock by default, so a UI running as the
# same user can attach without extra configuration.
[Unit]
Description=ipx podcatcher
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=%h/.cargo/bin/ipx daemon
Restart=on-failure
RestartSec=30
Environment=IPX_LOG=ipx=info
[Install]
WantedBy=default.target

View File

@@ -1,16 +0,0 @@
# Alternative to the daemon: a periodic one-shot scan, closer to how the original
# iPodderX agent was driven. Use this OR ipx.service, not both -- with no daemon
# running there is no socket, so a UI cannot attach.
#
# Install ipx-scan.service and ipx.timer to ~/.config/systemd/user/, then
# systemctl --user enable --now ipx.timer
[Unit]
Description=Periodic ipx feed scan
[Timer]
OnBootSec=5min
OnUnitActiveSec=1h
Persistent=true
[Install]
WantedBy=timers.target

142
docs/architecture.md Normal file
View File

@@ -0,0 +1,142 @@
# How it works
One binary, `ipx`. `ipx daemon` runs three things in one process: a scheduler, a Unix-socket
control server, and the web UI. Everything else is a CLI that either does the work itself or hands
it to a running daemon.
## Modules
| File | Responsibility | What it replaced in the Python |
|---|---|---|
| `src/main.rs` | CLI, dispatch, scan loop, download policy | `iPXAgent.py` |
| `src/config.rs` | TOML load/save, `General`/`Feed`/`Web`, intervals, slugs | `iPXSettings.py`, `feeds.plist` |
| `src/db.rs` | SQLite schema, migrations, every query | `.ipxd` plists, `history.dat`, `qmcache.dat` |
| `src/feed.rs` | Conditional GET, RSS/Atom/OPML parsing | `FeedData.__getFeed/__getEntries` |
| `src/download.rs` | Streaming download, naming, type sniffing, placement | `iPXDownloader.getFile` |
| `src/torrent.rs` | librqbit session, seeding limits, stall abort | vendored BitTorrent 4.2.1 |
| `src/retention.rs` | Quota and age sweeps | `iPXQuotaManager.py` |
| `src/ipc.rs` | Event and command types, the socket server | `printMSG` on stdout |
| `src/auth.rs` | Argon2id hashing, session tokens, header names | — |
| `src/web.rs` | axum: HTTP API, auth, SSE, media streaming | — |
| `src/logbuf.rs` | Ring buffer behind the UI's Log view | — |
| `web/index.html` | The whole front end, `include_str!`d into the binary | — |
The page is compiled in, so **editing `web/index.html` needs a rebuild**.
## A scan
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
re-derived into the database each scan, never written to config.toml. A Patreon creator link
(a token, no `show=`) with more than one show is treated the same way, before any fetch: its
shows come from Patreon's web API and each becomes a derived feed.
4. Record entries. A changed title or description flips the item back to unread.
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
a reason. What a filter skipped is judged again every scan, so a change of settings brings it
back. A feed in a group takes your settings on the group for anything you have not set on it.
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
or an error page — and is deleted.
## Data model
```
feeds id, url, title, image, etag, last_modified, last_checked, ttl_mins,
last_error, orphaned, group_id, managed
entries feed_id, guid, title, link, published, description, first_seen,
image, duration, episode, season PK (feed_id, guid)
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
bytes_done, downloaded_at, last_error
users id, name, pass_hash, is_admin, created
sessions token, user_id, created, seen
subscriptions user_id, feed_id, keywords, auto_download, allow_explicit,
max_new_per_check, created PK (user_id, feed_id)
entry_state user_id, feed_id, guid, read, flagged, position
PK (user_id, feed_id, guid)
```
Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position` columns from
before accounts; two bugs came from queries still reading them, and `migrate()` drops them from an
older database.
Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in
`migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the
same path as `Db::open`, so a migration-only column cannot pass tests while missing in production.
## Control socket
Newline-delimited JSON, both directions, over `$XDG_RUNTIME_DIR/ipx.sock`.
```sh
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
{"ev":"feed_start","feed":"atp"}
{"ev":"progress","feed":"atp","enclosure":42,"file":"ep1.mp3","done":8192,"total":3000000}
{"ev":"download_done","feed":"atp","enclosure":42,"path":"…","bytes":3000000}
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
{"ev":"scan_done","feeds":1}
```
**Commands**`fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `download`
(`enclosure`), `status`.
**Events**`feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
`scan_done`, `reap_done` and `status` are terminal: a client that asked for work stops reading
there.
Progress carries the enclosure id, without which a UI cannot tell one download from another and
ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast,
so a client attached to a busy daemon also sees that daemon's other work.
Inside the process the same events go over a `tokio::broadcast`; commands arrive on an `mpsc` and
are handled by a single worker, so nothing races over the same download. Shutdown is a `watch`
channel raced *inside* each job — `tokio::select!` only races branches at the point of selection,
so a long download had to be able to notice the signal itself.
## HTTP API
Everything below `/api` needs a signed-in user; the browser gets a redirect to `/login`, anything
else a `401`.
| Route | |
|---|---|
| `GET /` | the app |
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable, sortable (`sort` = kept, title, feed, type, size or published; `dir` = asc or desc) |
| `GET /api/entries` | the same, across every feed you subscribe to (All Subscriptions) |
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
| `POST /api/read-all` | everything read in every feed you subscribe to (All Subscriptions) |
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, kept, position |
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
| `POST /api/fetch` | |
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
| `GET /api/events` | SSE, the same broadcast the socket carries |
| `GET /api/logs` | admin-only; the ring buffer, with a sequence cursor |
| `GET /media/{id}` | the file, with Range support so seeking works |
Show notes are feed-supplied HTML from an untrusted source, sanitized with `ammonia` server-side
before they reach the page.
## Testing
```sh
cargo test # parsing, filters, retention, schedules, SQL, per-user isolation
node tests/page-smoke.js # the page script loads and every selector it wires at load exists
npx playwright test # a real browser against a real daemon on fixture feeds
```
The Rust tests cannot see a wrong selector, a handler that runs and does nothing, or a page that
renders empty — which is what has actually reached users. Each Playwright case maps to a bug that
did.
The suite starts its own daemon and database under `/tmp/ipx-ui-test`, wiped once per run. Tests
share that daemon and run in order, so a test that marks something read changes what later tests
see — make assertions that do not depend on earlier ones.

91
docs/cli.md Normal file
View File

@@ -0,0 +1,91 @@
# Command line
```
ipx [--config PATH] [--local] <command>
```
Every command that has a wire form probes the control socket first: if a daemon is running, the
daemon does the work and the CLI just renders the events it streams back. That is deliberate — two
processes must never download the same thing. `--local` forces the work to happen in-process.
| Command | What it does |
|---|---|
| `ipx list` | Subscriptions and their state |
| `ipx status` | Counts: feeds, pending, downloaded |
| `ipx fetch [FEED] [--force]` | Scan everything, or one feed. `--force` ignores the TTL |
| `ipx add <url> [--folder X] [--keywords a,b]` | Subscribe; the id comes from the feed title |
| `ipx rm <feed>` | Unsubscribe; downloads and history are kept |
| `ipx import <file.opml>` / `ipx export <file.opml>` | Move subscriptions in or out. Import subscribes the first admin, as the shared web token does; in the web UI it subscribes whoever is signed in |
| `ipx reap [--dry-run]` | Run retention now |
| `ipx user <add\|list\|passwd\|rm>` | Accounts for the web UI |
| `ipx daemon [--web ADDR]` | Scheduler, control socket and web UI |
## Accounts
Passwords are read from **stdin**, so they miss the shell history and any `ps` listing.
```sh
echo -n 'a good password' | ipx user add ray # local account
ipx user add ray@example.com --no-password # signs in through the proxy only
echo -n 'a good password' | ipx user passwd admin # change a password
ipx user list # who exists, and how each signs in
ipx user rm sam # account, subscriptions and read state
```
The first account created is an admin; later ones are ordinary users. A database with no accounts
at all gets **admin / ipodderx** on the next daemon start, announced in the log — change it.
To avoid even the command line, read it interactively:
```sh
read -s PW && echo -n "$PW" | ipx user passwd admin
```
## Scanning
```sh
ipx fetch # everything due
ipx fetch atp --force # one feed, ignoring its TTL and schedule
```
A scan: conditional GET (`If-None-Match` / `If-Modified-Since`), parse, record new entries, apply
the filters, then download up to the per-scan cap, newest first. A feed nothing has changed in
answers `304` and costs one request.
## Retention
```sh
ipx reap --dry-run # what would go, oldest first
ipx reap # actually delete
```
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
with no file are pruned from the database. **An item anyone kept keeps its file**, and one only counts
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
stops the next scan fetching it again.
## The daemon
```sh
ipx daemon # scheduler + socket + web UI
ipx daemon --web 0.0.0.0:8099 # override the configured bind for one run
```
One daemon per socket; a second refuses to start rather than fight over the database. It shuts down
cleanly on SIGTERM, including mid-download.
To kill it, match the binary exactly:
```sh
pkill -x ipx
```
`pkill -f ipx` matches the shell running the command too, and kills your own session.
## Talking to it directly
```sh
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
```
See [architecture.md](architecture.md#control-socket) for the protocol.

116
docs/configuration.md Normal file
View File

@@ -0,0 +1,116 @@
# Configuration
One TOML file, read at startup and re-read whenever the web UI writes to it — most changes take
effect without a restart. Default location `$XDG_CONFIG_HOME/ipx/config.toml`
(`~/.config/ipx/config.toml`), overridden with `--config` or `$IPX_CONFIG`.
| What | Where | Override |
|---|---|---|
| Config | `~/.config/ipx/config.toml` | `--config`, `$IPX_CONFIG` |
| Database | `~/.local/share/ipx/state.db` | `$IPX_DATA_DIR` |
| Control socket | `$XDG_RUNTIME_DIR/ipx.sock` | `[general] socket` |
| Downloads | `[general] download_dir` | — |
`~` is expanded in paths. The database is SQLite in WAL mode; back it up by copying `state.db`
while the daemon is stopped, or with `sqlite3 state.db .backup`.
## `[general]`
```toml
[general]
download_dir = "~/Podcasts"
socket = "/run/user/1000/ipx.sock"
schedule = "every 1h" # "every 30m", "every 4h", "2d", "90" (minutes)
organize = "feed" # "feed" | "date"
max_total_gb = 50 # 0 = unlimited
max_age_days = 30 # 0 = keep forever
max_new_per_check = 3 # per feed, per scan. 0 = unlimited
media_types = ["audio", "video"]
```
* **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
pad. Kept items are never deleted, and a file only counts as read once every subscriber has
read it. `0` disables it entirely.
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
Kept ones stay. `0` disables it.
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
fetched 216 files and 22 GB in one scan.
* **`media_types`** — top-level MIME types taken automatically. Anything else is still listed and
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
without this the disk fills with artwork. Empty takes everything.
## `[torrent]`
```toml
[torrent]
enabled = true
seed_ratio = 1.0 # stop seeding at this ratio ...
seed_time_mins = 60 # ... or after this long, whichever comes first
port_range = "6881-6889"
stall_mins = 30 # give up on a torrent making no progress
```
A `.torrent` body is handed to the torrent path whatever MIME type it was advertised as. Torrents
run on their own tasks (two at a time) so a slow swarm never blocks a scan.
## `[web]`
```toml
[web]
enabled = true
bind = "0.0.0.0:8099" # 127.0.0.1:8080 by default
token = "" # generated and saved on first run
trusted_header = "" # e.g. "Cf-Access-Authenticated-User-Email"
trusted_proxies = ["127.0.0.1", "::1"]
auto_create_users = true
session_days = 30
```
* **`token`** — the shared secret, which signs in as the **admin**. `?token=…` sets a cookie, so
you paste it once per browser. It is what the Docker healthcheck and any scripts use.
* **`trusted_header`** — a header naming the signed-in user, set by whatever fronts ipx. Empty
disables that path. See [sso.md](sso.md).
* **`trusted_proxies`** — addresses allowed to assert that header, and the entire security boundary
for it. Name the proxy, never a subnet.
* **`auto_create_users`** — create an account the first time the proxy vouches for a new name.
* **`session_days`** — sign a session out after this long without a request.
It is plain HTTP. On a LAN bind everything crosses the network in the clear — and a feed URL can
itself carry a credential. Put TLS in front of it if that matters.
## `[feeds.<id>]`
The table key is the feed id: stable, human-readable, and used in paths and the API. `ipx add`
derives it from the feed title.
```toml
[feeds.atp]
url = "https://atp.fm/rss"
folder = "Accidental Tech Podcast" # default: the feed title
schedule = "every 6h" # overrides [general] for this feed
media_types = ["audio"] # overrides [general] for this feed
username = "ray" # HTTP basic auth
password_env = "IPX_ATP_PASS" # preferred over a literal `password`
```
With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
`max_new_per_check` live on each person's subscription in the database**, not here — the values in
config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
and are the same for everyone. See [users.md](users.md).
Feeds derived from a subscribed OPML are **not** written here: the OPML is the source of truth and
they are re-derived on every scan. Editing one in the UI promotes it to a real config entry.
## Environment
| Variable | Effect |
|---|---|
| `IPX_CONFIG` | Config file path |
| `IPX_DATA_DIR` | Directory holding `state.db` |
| `IPX_LOG` | What reaches stderr (`ipx=debug`, `ipx::scan=debug`, …) |
| `IPX_UI_LOG` | What the in-process log buffer captures for the UI's Log view |
| `http_proxy` / `https_proxy` | Honoured for feed and enclosure fetches |

View File

@@ -1,58 +1,414 @@
# Progress # How ipx was built
Running record of what has actually landed. Newest entry first. The long-form record, newest first: what landed each day, what was wrong before, and what it cost
The full design and step list live in the plan file at to find out. [CHANGELOG.md](../CHANGELOG.md) has the short version, by release; this is where the
`/config/.claude/plans/i-want-to-create-playful-quiche.md`. reasoning lives. New write-ups go at the top.
## Build order See [README.md](../README.md) for what the thing is.
- [x] **1. Repo skeleton** — git init (`main`), `cargo init --name ipx`, deps pinned, LICENSE, ## 2026-09-12 — Cutting what had outlived its reason
README, this file.
- [x] **2. `config.rs` + `db.rs`** — TOML config structs + SQLite schema.
- [x] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries.
- [x] **4. `download.rs`** — downloads, filters, dedupe.
- [x] **5. `retention.rs`** — oldest-first quota + age reaper.
- [x] **6. `ipc.rs` + daemon** — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon.
- [x] **7. `torrent.rs`** — librqbit, seed to ratio/time, stall abort. (swarm download unverified —
see the step 7 entry)
- [x] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd units, README.
### Phase 2 — web front end A whole-repo audit for over-engineering listed twelve things to cut, and all of them went.
Decided with Ray: axum serving plain HTML/JS (no WASM toolchain), running **inside the daemon** - **Upgrades from before accounts.** `migrate_opml_children` moved OPML feeds that old versions
process so it reads SQLite and the event bus directly, LAN-bindable with a shared token. wrote into `config.toml` out to the database, and ran at every daemon start to do nothing after
the first. Production ran it in 0.3.0; anything older has to pass through 0.4.0.
- **Half of the adoption, and not the other half.** The audit called `adopt_existing_library` a
one-time migration and it was cut whole. It did two jobs: copy the old read state into
`entry_state`, which was dead, and subscribe the first admin to the whole catalogue while nobody
subscribed to anything, which is how a fresh install's first account gets `config.toml`'s feeds.
The browser suite caught it at once, signing in to an empty sidebar; `cargo test` had no idea.
The second job is back as `adopt_catalogue`, with a unit test of its own.
- **The dead `entries` columns.** `read`, `flagged` and `position` moved to `entry_state` with
accounts. The adoption's copy was their last reader, but `record_entry` still wrote them, and
still reset `read` when a title changed, which nothing looked at. Two bugs came from queries
reading them. `migrate()` now drops them from an existing database (SQLite has had `DROP COLUMN`
since 3.35), and a test builds an old table to prove it. On production each drop rewrote the
66 MB `entries` table, about four seconds apiece, so the first start took thirteen. An
`ipx status` run in that window failed with `migrating schema`: every `ipx` command migrates when
it opens the database, and it collided with the daemon doing the same. A failed `ALTER TABLE`
changes nothing, and the database had been copied to `backup/` first anyway.
- **`interval_mins`**, which `schedule` replaced. An old config that still has the key loads; the
key is ignored, and the config test carries it to keep that true.
- **Three dependencies.** `infer` was only asked whether a file is a torrent, and the check after
it already looked for `d8:announce`, which is what `infer` looks for. `dirs` was three lookups of
`XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME`. `tokio-stream` wrapped the broadcast receiver for
the event stream; `futures_util::stream::unfold` does the same, lagging clients included.
- **Two token generators.** The web token came from a copy of the session-token code, with a
clock fallback on top. It uses `auth::new_session_token` now, and is 64 characters.
- **The icon inlined four times**, 23 KB of base64 each, into both pages. It is `/icon.png` now,
outside the sign-in wall with `/login`, since the sign-in page shows it.
- Also: the `contrib/` systemd units from before the container, `Db::entries` and
`Db::count_entries` that only the tests called, three `logbuf` visitors that repeated the trait's
defaults, and unused state, a helper and dead CSS in the page.
- [x] **9. Config hot-reload + web skeleton.** `Ctx.cfg` becomes `RwLock<Arc<Config>>` so the UI can ## 2026-09-11 — A design pass on the web UI
edit feeds without a daemon restart. `[web]` config section (enabled/bind/token, token
auto-generated and saved on first run). axum server started by `ipx daemon`, token checked by
middleware, `?token=` sets a cookie so `<audio>` requests authenticate too.
*Done when:* `ipx daemon` serves a page on the configured bind, and a wrong token gets 401.
- [x] **10. Browsing.** `/api/feeds`, `/api/feeds/:id/entries`, entry detail. Descriptions are
untrusted feed HTML — sanitized with `ammonia` before they reach the page.
*Done when:* the Glass Cannon feed's 131 entries browse and read correctly.
- [x] **11. Media actions.** Range-request audio streaming (`tower-http` ServeFile) so seeking
works, download-on-demand for a pending enclosure, delete a file, mark read/flagged.
*Done when:* an episode plays and seeks in a browser, and delete reaps the row.
- [x] **12. Feed configuration.** Add/remove feeds and edit folder, keywords, allow_explicit,
auto_download, max_new_per_check from the UI, written back to config.toml and hot-reloaded.
*Done when:* flipping allow_explicit in the UI takes effect on the next scan with no restart.
- [x] **13. Live progress + polish.** SSE from the existing broadcast bus so downloads show live.
README section, screenshot-free usage notes.
*Done when:* starting a fetch from the UI shows progress advancing without a reload.
Note: `read`/`flagged` finally get a writer here. Retention orders by them (see the step 5 entry), A review against screenshots of every view in all three themes found that Dark and Light read as a
and until now nothing set them. generic dark dashboard: one pale blue did every job, most labels were bold, and nothing led. The
list it produced, in `TODO.md`, was worked through in one go. What was worth knowing:
## Smoke tests - **Amber means new.** Badges, unread dots and download bars take the icon's EQ amber; blue is left
for the primary action and links. Light's amber was `#b06f10`, which gives white text 4.1:1,
short of AA for 11 px bold. It is `#9a5f0a` now, 5.2:1.
- **EQ bars mark what is playing.** Three `<i>` bars stand at 60, 100 and 40 % and animate only
while `body.playing` is set. The first version left the animation on but paused, expecting each
bar to hold a different frame. The frames it held were within a pixel of each other, and on
screen the bars read as three dots. Under reduced motion one rule drops every animation and
transition, which leaves the bars standing.
- **The focus ring was clipped.** `.tgroup` and `#topbar` both set `overflow:hidden`, so a ring
drawn outside a toolbar button was cut off. Rings inside clipping parents are inset instead.
- **The feed list could not be used from the keyboard at all.** Rows were `<div>`s and the triangle
a `<span>`, so Tab went from the feed filter to Sign out. Rows now take focus, the triangle is a
`<button aria-expanded>`, and `renderFeeds` puts focus back on the same row after redrawing,
since every live update replaces every row. The list's own key handler stops Space and the
arrows from reaching the player's shortcuts on the document.
- **The triangle hangs in the margin.** Every row used to reserve an 18 px slot for it, pushing a
hundred feeds 28 px right of the places above for the sake of two folders. It is now absolutely
placed in the row's left padding, the full height of the row, so a near miss no longer opens
the folder's page.
- **A selected tile vanished** because the initials tile and the selected row were both `--raise`.
Tiles now mix their tint into `--bg`, which no row uses.
- **The Files pane hides itself** with `#split:has(>#files[hidden])`, which collapses its column.
The phone layout already hides the pane, and a zero-width extra track there is harmless.
- **Flagged became Kept** in the tab, and in the server's refusal to delete a file someone else
kept. The filter value and the column stay `flagged`; renaming those buys nothing.
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`. ## 2026-09-11 — A Patreon creator is a list of shows
2. `ipx fetch` again → no re-download, feed skipped for TTL.
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events; Ray asked whether ipx could sync with Patreon. Not in full. The documented API (v2, the
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel. `identity.memberships` scope) lists the creators you back and whether each has a feed (`has_rss`),
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded. but no resource carries the `auth` token that makes a feed URL work. That token only comes from the
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time. creator's page. It is also one per membership, not one per account: techpod's differs from Glass
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to Cannon's, so no single token finds everything you back.
`reaped`.
What does work is one creator at a time, which is what Ray wanted for Glass Cannon and its 33 shows:
- `patreon.com/rss?auth=<token>`, with no creator named, returns that token's creator. Its self link,
about 660 bytes in, gives the campaign by number (`/rss/369921`). Patreon ignores `Range` here, so
ipx reads the stream until the number appears and hangs up, instead of taking all 2.8 MB.
- A show's `show=` number is a Patreon collection. Asked anonymously, the collection listing
(`/api/collection?filter[campaign_id]=`) and a post's `collections` both hide patron-only ones: you
get "FAQ". `/api/campaigns/<id>?include=shows` lists every show, anonymously, in one response.
- Every spelling works: `rss/glasscannon?auth=…&show=N`, `rss/369921?…` and `rss?auth=…&show=N` all
serve the same 131 items. Enclosure URLs are the same in the creator feed and the show feed, and
stable between fetches.
That last point shaped the design. `enclosures.url` is unique, so whichever feed is scanned first owns
the file. The first cut only asked a creator for its shows while it had no entries of its own, so that a
creator already read as a plain feed, holding every show's episodes, would never be split into shows
that came up empty. Within the hour that was the wrong call: Glass Cannon had gone into production on
the build before this one, been read as one feed of 2,385 items, and the rule kept it that way. Finding
anything in that heap was the problem Ray wanted solved.
So a creator with more than one show is always a group, run through the same sync as an OPML
(`sync_group`, split out of `sync_opml`). When it becomes one, its items are cleared and each show
takes over the enclosures the creator holds as the show lists them (`Db::adopt`), downloaded files and
everyone's read state included. One show leaves it a plain feed, which is what techpod already was. If
the shows cannot be listed, a creator already split fails the scan rather than being read as one heap;
one that never was is read as one feed until they can be. An answer without a `shows` list is an error,
not "no shows".
**Filter verdicts follow the settings.** Ray also reported that turning on Allow explicit and
rescanning brought nothing back. An item was judged once, when first seen, and `skipped` was final. The
2026-09-10 entry below saw it coming ("worth a `ipx retry <feed>` command if this bites"). It bit:
2,166 Glass Cannon items and all 88 of Shadowdark's sat at `skipped: explicit` with the setting on.
Every scan now runs the filters again over what they skipped (not over `torrents disabled`, which is not
a filter's call) and requeues what they now let through. Only that direction: a queued item is never
pulled back, because Download latest and a manual download both work by queueing.
**Two gaps beside it.** Add feed had no explicit box, so every new feed's first scan skipped all its
explicit items; it has one now, stored on your subscription like the feed dialog's. And a feed inside a
group ignored your settings on the group, though the group's dialog said they were inherited: settings
live on each person's subscription, and nothing read the group's. `Db::subscribers` now fills what you
have not set on the feed from your subscription to the group, and the feed list shows the same.
**A name that was already used.** Replaying the split on a copy of the production database left one
show with a Supercast episode in it. "Glass Cannon Live! Ascension | Pathfinder 2E" slugs to
`glass-cannon-live-ascension-pathfinder-2`, the id of a Supercast feed of the same show that had been
removed. Removing a feed keeps its rows on purpose, so that re-adding it does not fetch the back
catalogue again, but choosing a new id only checked config.toml and derived feeds. The Patreon show took
the old id and everything still filed under it. An id is now also taken when the database has a feed by
that id at a different URL; the same URL may still have it back, which is the re-add case.
Shows already added by hand are matched by token and show number, not by exact URL (`same_feed`), so a
bare token does not add Get in the Trunk and Shadowdark a second time under another spelling.
The show listing is Patreon's own undocumented web API. If it changes, only finding new shows stops.
## 2026-09-11 — One meaning per icon, sortable columns, and one player
Ray asked for a pass over the whole UI: consistent icons, and buttons placed next to what they act
on. The pass started from screenshots of every view and dialog, taken against the browser suite's
own daemon and fixtures. Several things did not agree:
- Unsubscribe was a minus in the toolbar but an x in the feed header. There it read as "close this
page", and x also meant Cancel in every dialog.
- Half the dialogs still had word buttons: Log, Add feed, Users, Unsubscribe, OPML, Popular's
Subscribe, the Copy buttons and Sign out.
- The player bar and the folder arrow were font characters, and so was the row's "playing" marker.
Those came out thin and varied from font to font.
- The OPML subscription page laid out its header differently from a feed's, and All Subscriptions
had no buttons at all.
The rule now is one meaning per icon. Minus unsubscribes, x closes or cancels, plus adds, subscribes
or imports, and a dialog's confirm button carries the icon of what it does. The words go in the
tooltip. The toolbar is grouped by what it acts on: add, unsubscribe and scan act on feeds; play,
read and keep act on the selected item. Its read and keep buttons now show that item's state, as
the item's own buttons already did. A browser test opens all eight dialogs and fails on any button
without an icon and a tooltip. It also checks that the toolbar and the page use the same icon for
the same action.
The pass turned up three bugs:
- **Escape ignored inside a dialog's text box.** The keydown handler returned early for any key
typed in a text box, and Escape sat below that check. So Add feed, which opens with the cursor in
its URL box, could not be closed with Escape. It showed up because the screenshot run stalled:
the dialog stayed open, and every later click landed on its overlay until the run timed out.
- **A white password box in the dark theme.** The input rule listed text, search and number
fields, but not password ones.
- **A stray "• •" under an item with no date.** Each detail carried its own trailing dot, whether
or not the next one was there.
All Subscriptions gained Mark everything read, as `POST /api/read-all`. It marks the feeds from the
person's subscriptions, which is the same set the view's query is scoped by. It asks first, since
one click covers every feed.
**Sorting** happens on the server. A list arrives fifty at a time, so sorting in the page would
only have reordered what was loaded. The column name picks one of a fixed set of SQL expressions
(`order_sql`), so nothing the browser sends reaches the query, and an unknown name means newest
first. Ties also fall back to newest first, so "Load more" does not skip or repeat items at a page
boundary. A database test runs every column both ways. Size came out of the File column into its
own, and small files show KB, since whole megabytes made every article image "0 MB".
**The double play.** Ray found that pressing play in the Files pane played the file twice at once.
The pane drew its own `<audio controls>`, and its `onplay` handler also started the player bar on
the same file. That meant two audio elements from one click. The pane now has a play button that
hands that exact file to the player bar, the only player. `play()` now takes the file. Its check
for resuming compares the file as well as the item, so another of an item's files starts from its
beginning instead of carrying on with the first. A test plays from the pane and checks the page
has one `<audio>` element.
---
## 2026-09-11 — The original's layout
Ray pointed at a screenshot of the Mac app (techpp.com, 2012) and asked for its panes, its
toolbar, and a Directory that lives in the feed list rather than behind a button. What it had, and
what ipx now does:
- A toolbar across the window: subscribe and unsubscribe, play, flag, refresh, and a search box
scoped to the feed on show. ipx's has the same groups, acting on the selected feed and item, with
Settings and Log at the right end, for admins only.
- A source list opening with Directory, Playlist Builder and All Subscriptions above the feeds.
ipx has Directory, Popular and All Subscriptions there, opening in the main pane. Playlist
Builder is left out, since nothing here builds playlists.
- The entries as a table, with a Files pane beside it and the entry below. The columns are unread,
kept, the item, its feed, its file and when. Sorting by column is not done yet.
- A status bar with the totals for what is on show.
All Subscriptions needed one new endpoint, `GET /api/entries`. It is the per-feed query with
`feed_id = ?` swapped for the person's subscriptions. The enclosure lookup that follows it used to
filter by feed as well; a page can now span feeds, so each file is matched to its row by feed and
guid instead.
A phone has no room for a pane beside the table, so there the files follow the item's text in the
full-screen reader, and the table drops to title and date.
---
## 2026-09-11 — Popular on this server
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
the list was fed. After every successful download, `iPXClass.py:1249` called `doPing`, which sent
the enclosure URL and the feed URL, unencoded and over plain HTTP, to
`directory.iPodderX.com/feedData/survey/files`. It was on by default, and only the Mac app's
preferences could switch it off (`anonFeedback`). The directory itself lived on that server and in
the Cocoa app, and both are gone.
ipx keeps the idea and drops the ping. Add feed lists what other accounts on this server subscribe
to, ranked by subscriber count. The old ping would have sent a Patreon feed's `auth=` key along
with everything else, so this is built around not doing that:
- A row carries an id, a title, artwork and a count. No URL reaches the page, so no key can.
- Subscribing goes by id through `POST /api/popular/{id}`, which checks the id against the same
list. Guessing a private feed's id gets a 400.
- Anything that looks private is never listed: a login configured for the feed, credentials in
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
`unlisted` flag is the upgrade if that happens.
It happened the same day. The first screenshot of the new Directory listed "Glass Cannon Live!
Ascension (for Ray Slakinski)", a Supercast feed at `feeds.supercast.com/feeds/<key>`. Treating
any long path segment as a key would have hidden public feeds too: acast's show ids look the
same. So paid-feed services are named instead (Patreon, Supercast, Supporting Cast, Glow,
Memberful), and any feed from one of them is private whatever its URL looks like. The per-feed
flag is still the upgrade for a service not on that list.
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
they would bury everything anyone chose on purpose.
---
## 2026-09-11 — Import an OPML file by uploading it
The import screen now has a file picker as well as the paste box. The page reads the file and
checks it looks like OPML before sending anything. If it doesn't, the page says so and clears the
picker, and it also clears the picker after an import. The file travels as text and is never
written to disk on the server, so there is nothing to clean up there.
The server now parses the OPML before touching anything, and returns `400` "that is not an OPML
file" instead of a `500`. `subscribe_opml` takes a parsed document, so `ipx import` also refuses a
non-OPML file by name before changing anything.
---
## 2026-09-11 — OPML import and export are per person
Importing an OPML now subscribes you to every feed in it. Feeds already in the catalogue cost
nothing, and unknown ones are added under the OPML's title. Before this, import only added URLs
missing from `config.toml` and subscribed nobody. So importing an export from another account did
nothing at all, and a genuinely new feed had no subscriber and was never scanned. The page said
"Imported 0 feed(s)". `ipx import` had the same gap; it now subscribes the first admin. Both go
through `subscribe_opml`.
Export now lists only your own subscriptions. It used to write out the whole catalogue to anyone
signed in, including other people's feeds and any private URLs in them.
Tests now cover every way in and out:
- Settings' Export OPML download and paste-to-import screen.
- A round trip that imports the admin's actual export into a second account and checks both
exports match. That round trip is exactly what failed.
- `ipx import` and `ipx export`, run in their own scratch config and database.
---
## 2026-09-11 — The log is admin-only
`GET /api/logs` now returns `403` to anyone who is not an admin, and the page hides the Log button
from them. Before this, every signed-in person could read the whole log. That includes every
account's name, every feed anyone subscribes to, and every failed sign-in. `/api/events` stays open
to everyone, because it carries the scan progress each person's page shows.
---
## 2026-09-11 — Managing users from the web
Settings has a **Manage users…** screen for an admin. From it you can add someone, with a password
or with none for someone the proxy signs in, tick or untick Admin, and remove an account. It is
backed by `GET/POST /api/users` and `PATCH/DELETE /api/users/{id}`, which return `403` for anyone
who is not an admin. The only admin cannot be demoted or removed, because nobody would then be
able to manage accounts except from the CLI on the box. Before this, accounts could only be managed
with `ipx user`.
---
## 2026-09-11 — Unread feeds first inside an OPML
An OPML subscription's feeds, both in the sidebar folder and on its own page, now list the ones with
unread items first. They were listed alphabetically, so with dozens of feeds the few with anything
new were scattered through the list. Within each half the order is still alphabetical. The browser
suite's fixture OPML gained a second feed, Aardvark Radio, which sorts first by name and by position,
so the new test only passes if unread wins.
---
## 2026-09-11 — Deploying is a Docker image
Production moved from a hand-started daemon in code-server to the `iPodderX` container in the Arcane
project `content`. `CLAUDE.md` now deploys by pushing to the registry at `192.168.1.130:5000` and
recreating that one service with `docker compose`. The old instructions copied a binary over a
process nobody supervised, so it did not come back after a reboot. Two known gaps are gone: the
image does support accounts, and the entrypoint drops to `99:100`, so downloads are no longer
owned by root. The README's Docker section said `docker compose up -d` builds the image and named the
service `ipx`; the compose file pulls from the registry and the service is `ipodderx`.
---
## 2026-09-11 — Documentation
`PROGRESS.md` became this changelog; the finished step lists moved to an appendix. The README is now
an overview that points at `docs/`: [configuration](configuration.md) (every key, path and
environment variable), [cli](cli.md), [users](users.md) (what several people share),
[sso](sso.md) (refreshed for accounts and the admin-only settings), and
[architecture](architecture.md) (modules, the scan, the schema, the socket protocol, the HTTP
API). `CLAUDE.md` collects what someone working on the code needs to know and has already been
learned the hard way here -- `pkill -x` not `-f`, the page being compiled in, the dead columns on
`entries`, the Playwright worker that deleted its own database.
---
## 2026-09-11 — Pruning respects a star from anyone
Same defect as the reaper, one function along: `prune_entries` guarded on `entries.flagged`, which
nothing has written since read state moved to `entry_state`. Starring a text item with no file would
not have saved it from the age sweep. It follows the reaper's rule now, and takes orphaned read
state with whatever it deletes.
---
## 2026-09-11 — Step D: one file, and everyone who wants it
The last of it, which is all about telling the truth before acting:
* **Delete says whose file it is.** A feed with other subscribers labels the button *Delete for
everyone*, and the confirmation names how many people share it. The server then has the last
word: if anyone else has starred the item or not played it yet, `DELETE /api/enclosures/{id}`
returns **409** with the reason in plain words, and only `?force=true` goes through. So the
rule is enforced where it matters rather than in the page that happens to be asking.
* **A feed's header says it is shared** -- "shared with 1 other person" -- which is the answer to
"why is there a file here I never asked for": someone else's subscription fetched it, and one
copy serves you both.
* Retention already respects the same rule from the entry before this one: starred by anyone keeps
it, read by everyone releases it.
`others_wanting` is tested with three subscribers disagreeing, and a browser test walks the whole
delete flow: the label, both prompts, declining the second, and the file still being there.
---
## 2026-09-11 — Retention caught up with per-user state
Moving read and starred into `entry_state` left `reap_candidates` reading `entries.read` and
`entries.flagged`, which nothing writes any more: **starring stopped protecting a file**, and the
"delete the ones already read first" ordering was dead. Harmless while the quota and age limits are
0 -- the reaper deletes nothing at all then -- but it would have bitten the moment one was set.
One file serves every subscriber, so both tests are now about all of them: **anyone** starring it
keeps it, and it only counts as read once **everyone** subscribed has read it. A file whose feed
nobody subscribes to has nobody left to keep it, so it sorts with the read ones. Tested with two
subscribers disagreeing.
---
## 2026-09-11 — Steps B and C: what is yours, what is everyone's
Read, starred and playback position moved out of `entries` into `entry_state (user_id, feed_id,
guid, ...)`; subscriptions became rows in `subscriptions (user_id, feed_id, ...)` carrying **your**
keywords, auto-download, explicit and per-scan limit. The feed list, unread counts, filters and
mark-all-read are all per person now. On first start the existing library is adopted by the admin:
2438 read/starred items and all 86 feeds, so nothing was lost.
The split follows from the file being shared:
* **Yours**: read state, starred, position, keywords, auto-download, explicit, per-scan limit,
and which feeds you see at all.
* **Everyone's**: the feed URL, its download folder, and when it is scanned -- there is one copy of
a file however many people subscribe, so those describe the file, not a preference. Admin-only,
refused with a 403 for anyone else rather than merely hidden.
Scanning merges the subscribers' wants, because one fetch and one file serve them all: an item is
downloaded if **anyone** wants it (any one person's keyword set matching is enough, and one person
taking everything removes the filter), auto-download is on if anyone has it on, and the per-scan cap
is the largest anyone asked for. `merge_policy` is a pure function with a test covering each of
those. Subscribing to a feed someone already has costs no second fetch and no second copy on disk;
unsubscribing takes it off your list alone, and only when the last subscriber leaves does the feed
stop being scanned.
**A test-harness bug worth naming**: Playwright imports the config in every worker, so the fixture's
`prepare()` ran again mid-run and deleted the data directory out from under the daemon. The daemon
kept serving from the unlinked inode while the CLI and any query opened a fresh empty database at
the same path -- which looked exactly like sign-in being broken. Only the launching process wipes
now (a worker has `TEST_WORKER_INDEX`).
--- ---
@@ -107,34 +463,6 @@ who is signed in above the sidebar footer, with a sign-out, and a 401 sends the
Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B. Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B.
--- ---
## Multi-user — the plan
Decided with Ray: **stay on SQLite** (Postgres was considered and dropped -- it is a deployment
choice, not a capability one, and nothing here contends for writes). Sign-in is either a local
username and password or the Cloudflare Zero Trust that already fronts `ipodderx.sdf1.net`, which
puts the authenticated identity in `Cf-Access-Authenticated-User-Email`. Feeds, items and files are **shared**; read state and subscriptions are **per user**.
The point of sharing: two people subscribed to the same show cost one fetch, one parse, and one file
on disk. `enclosures.url` is already globally UNIQUE, so the file half is nearly free.
- [x] **A. Users, sessions, sign-in.** `users` + `sessions` tables, Argon2 hashing, session cookie,
`ipx user add|list|passwd|rm`. A proxy header (`trusted_header` in `[web]`) signs in and
optionally creates a user -- honoured only from a `trusted_proxies` address, so a LAN client
cannot simply assert it. The existing shared token keeps working and resolves to the admin, so
the healthcheck and any scripts survive. Login page for direct access.
- [ ] **B. Per-user read state.** `entry_state(user_id, feed_id, guid, read, flagged, position)`;
the current columns on `entries` migrate into the first user's rows. Unread counts, filters and
playback position all become per user.
- [ ] **C. Per-user subscriptions.** `subscriptions(user_id, feed_id)`. config.toml stays the feed
catalogue; the UI lists only what you subscribe to. Adding a feed someone else already has costs
nothing. A feed nobody subscribes to stops being scanned but keeps its files.
- [ ] **D. One file, many users.** Auto-download when *any* subscriber wants it; retention never
deletes a file another user has starred or not yet played; deleting a download says so when
someone else still has it.
---
## 2026-09-10 — Items, not episodes ## 2026-09-10 — Items, not episodes
Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search
@@ -1185,3 +1513,37 @@ Gotcha worth keeping: three feature names in the plan were wrong against current
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls` `rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
is the fix. Whole tree is rustls-only now, no C TLS dependency. is the fix. Whole tree is rustls-only now, no C TLS dependency.
Next: step 2 — `config.rs` + `db.rs`. (done) Next: step 2 — `config.rs` + `db.rs`. (done)
---
## How this was built
Three planned phases, all complete. The dated entries above are the record of each.
**The engine (steps 1-8)** — repo skeleton with pinned dependencies; `config.rs` + `db.rs` (TOML
config, SQLite schema); `feed.rs` (conditional GET, RSS-then-Atom parsing); `download.rs`
(streaming downloads, filters, URL dedupe); `retention.rs` (oldest-first quota and age reaper);
`ipc.rs` plus the daemon (Unix-socket JSON lines, TTL scheduler, CLI proxying to a running daemon);
`torrent.rs` (librqbit, seeding to a ratio or a time, stall abort); OPML import/export and polish.
**The web front end (steps 9-13)** — axum serving plain HTML and JS, no WASM toolchain, running
*inside* the daemon so it reads SQLite and the event bus directly: config hot-reload and the `[web]`
section; browsing feeds and entries with `ammonia`-sanitized show notes; media actions (Range
streaming, download on demand, delete, read/starred); feed configuration written back to
config.toml; live progress over SSE from the existing broadcast bus.
**Multi-user (steps A-D)** — accounts, sessions and sign-in (Argon2id, session cookie, a trusted
proxy header for Cloudflare Zero Trust); per-user read state; per-user subscriptions; and the rules
that follow from one file serving everyone. SQLite stayed: Postgres was considered and dropped, as
nothing here contends for writes.
### The live checks each step had to pass
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
2. `ipx fetch` again → no re-download, feed skipped for TTL.
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
`reaped`.

View File

@@ -35,7 +35,15 @@ Restart the daemon after editing. Accounts made this way have **no password**: t
arrive through the proxy. `ipx user list` marks them `proxy only`. arrive through the proxy. `ipx user list` marks them `proxy only`.
The first account created is an admin. Every later one is an ordinary user, and an ordinary user The first account created is an admin. Every later one is an ordinary user, and an ordinary user
cannot change global settings or how often feeds are scanned. Promote someone with: cannot change global settings, a feed's URL or folder, or how often feeds are scanned: the API
refuses those with a `403`, not just the UI. Everything else about a feed (which items they want,
whether to fetch them, how many at a time) is theirs alone; see [users.md](users.md).
Somebody arriving through the proxy for the first time starts with **no feeds**, because
subscriptions are per person. Adding a feed someone else already reads costs no second fetch and no
second copy on disk.
Promote someone with:
```sh ```sh
ipx user list ipx user list
@@ -43,7 +51,9 @@ echo -n 'a good password' | ipx user passwd <name> # optional: also lets them
``` ```
Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN
when the tunnel is down. A brand new database starts with **admin / ipodderx** — change it. when the tunnel is down. So does the shared `[web] token`, which signs in as the admin: that is
what the Docker healthcheck uses, and the way back in if you lock yourself out. A brand new database
starts with **admin / ipodderx** — change it.
--- ---
@@ -206,7 +216,13 @@ ipx user rm sam # remove the account
Set `auto_create_users = false` once everyone who should have an account has one. After that the Set `auto_create_users = false` once everyone who should have an account has one. After that the
proxy vouching for an unknown name is logged and refused, rather than quietly making an account. proxy vouching for an unknown name is logged and refused, rather than quietly making an account.
Pre-create people instead with `ipx user add <name> --no-password`, using exactly the name the
header will carry (Cloudflare sends the email address, lower-cased).
Scanning intervals, the disk quota, retention and the download folder are **admin-only** — the Scanning intervals, the disk quota, retention, the download folder and a feed's URL are
Settings button is hidden for everyone else, and the API refuses the change even if the request is **admin-only**: the Settings button is hidden for everyone else, and the API refuses the change even
made by hand. Ordinary users still control their own folders, keywords and downloads per feed. if the request is made by hand. Everyone controls their own keywords, auto-download, explicit
setting and per-scan cap, along with their own read state and which feeds they see.
See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.

97
docs/users.md Normal file
View File

@@ -0,0 +1,97 @@
# Accounts, and what several people share
ipx serves any number of people from one copy of the data. The rule that decides everything else:
**there is one file on disk per enclosure URL.** Two people subscribed to the same show cost one
fetch, one parse and one file.
## What is yours, what is everyone's
| Yours alone | The same for everyone |
|---|---|
| Read, kept, playback position | The feed's URL |
| Which feeds you see at all | Its download folder |
| Keywords, auto-download, explicit, per-scan cap | When it is scanned |
| | The file on disk |
The right-hand column describes the feed and the file rather than a preference — two people wanting
different folders would mean two copies. Those three are **admin-only**, and the API returns `403`
for anyone else rather than merely hiding the controls.
## How the scanner merges everyone's wants
One fetch serves every subscriber, so the policy is a union:
* an item is downloaded if **anyone** wants it — one person's keyword set matching is enough, and
one person with no keywords removes the filter for that feed entirely
* auto-download is on if **anyone** has it on
* the per-scan cap is the **largest** anyone asked for
So "auto-download off" means *I don't cause downloads*, not *I never see them*. If someone else's
subscription pulls an item, you see it listed as downloaded and can play it, because the enclosure
is shared.
## Deleting
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
for everyone** and names them in the confirmation, and the server has the last word: if anyone else
has kept the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
reason, and only `?force=true` goes through.
Retention follows the same rule: an item anyone kept keeps its file, and it counts as read only once
every subscriber has read it.
## Signing in
Three ways, tried in order of how specific the claim is:
1. **A proxy header** naming the user — Cloudflare Zero Trust or Authentik. Honoured only from an
address in `trusted_proxies`. See [sso.md](sso.md).
2. **A session cookie** from signing in at `/login`. Argon2id hashes, sessions in the database,
idle timeout `session_days`.
3. **The shared `[web] token`**, which signs in as the admin — this is what the Docker healthcheck
and any scripts use.
A database with no accounts creates **admin / ipodderx** on the next daemon start and says so in
the log. Change it:
```sh
echo -n 'a good password' | ipx user passwd admin
```
## Adding someone
```sh
echo -n 'their password' | ipx user add sam
```
They sign in at `/login` and start with **no feeds**: subscriptions are per person. Adding a feed
someone else already has costs nothing — no second fetch, no second copy — it just appears on their
list with their own read state. Unsubscribing removes it from their list alone; only when the last
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
re-subscribing does not pull the back catalogue again.
**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
included. Directory lists every one of them A to Z. Your own feeds are marked Subscribed.
It shows a title, artwork and a count, never a URL or who reads it. Feeds from an
OPML subscription are left out, since they come with the OPML. So is anything that looks private: a
login configured for the feed, credentials in its URL, or a key such as `auth=` or `token=` in the
query, or a feed from a paid-feed service such as Patreon or Supercast, which put the key in the
path. Those are someone's paid subscriptions, and listing them would let anyone here read what they
pay for.
An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none
for someone the proxy signs in), tick or untick Admin, or remove an account. Removing one takes its
subscriptions and read state with it; downloaded files stay. The only admin cannot be demoted or
removed there, so there is always someone who can manage the rest.
## Admin
The first account is an admin. An admin can change global settings (scanning interval, quota,
retention, media types, download folder), a feed's URL, folder and schedule, and who has an account
and who else is an admin, and read the log, which names everyone's feeds and sign-ins. Everyone else
gets the Settings and Log buttons hidden and a `403` if they ask anyway.
```sh
ipx user list # the admin column says who
```

View File

@@ -26,9 +26,6 @@ pub struct General {
/// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d". /// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d".
/// A feed's own `schedule` overrides this. /// A feed's own `schedule` overrides this.
pub schedule: String, pub schedule: String,
/// Superseded by `schedule`. Still read so existing configs keep working.
#[serde(skip_serializing_if = "Option::is_none")]
pub interval_mins: Option<u64>,
pub organize: Organize, pub organize: Organize,
/// 0 = unlimited. /// 0 = unlimited.
pub max_total_gb: f64, pub max_total_gb: f64,
@@ -153,7 +150,6 @@ impl Default for General {
download_dir: home().join("Podcasts"), download_dir: home().join("Podcasts"),
socket: default_socket(), socket: default_socket(),
schedule: "every 60m".into(), schedule: "every 60m".into(),
interval_mins: None,
organize: Organize::Feed, organize: Organize::Feed,
max_total_gb: 0.0, max_total_gb: 0.0,
max_age_days: 0, max_age_days: 0,
@@ -176,8 +172,8 @@ impl Default for Torrent {
} }
impl General { impl General {
/// Minutes between checks. Falls back to the legacy `interval_mins`, then to an hour. /// Minutes between checks, or an hour when `schedule` is empty or unreadable. A malformed
/// A malformed value warns rather than stopping the daemon. /// value warns rather than stopping the daemon.
pub fn interval(&self) -> u64 { pub fn interval(&self) -> u64 {
if let Some(n) = parse_interval(&self.schedule) { if let Some(n) = parse_interval(&self.schedule) {
return n; return n;
@@ -185,7 +181,7 @@ impl General {
if !self.schedule.trim().is_empty() { if !self.schedule.trim().is_empty() {
tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default"); tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default");
} }
self.interval_mins.filter(|n| *n > 0).unwrap_or(60) 60
} }
} }
@@ -278,9 +274,7 @@ pub fn config_path() -> PathBuf {
if let Ok(p) = std::env::var("IPX_CONFIG") { if let Ok(p) = std::env::var("IPX_CONFIG") {
return PathBuf::from(p); return PathBuf::from(p);
} }
dirs::config_dir() xdg("XDG_CONFIG_HOME", ".config").join("ipx/config.toml")
.unwrap_or_else(|| home().join(".config"))
.join("ipx/config.toml")
} }
/// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`. /// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`.
@@ -288,9 +282,7 @@ pub fn data_dir() -> PathBuf {
if let Ok(p) = std::env::var("IPX_DATA_DIR") { if let Ok(p) = std::env::var("IPX_DATA_DIR") {
return PathBuf::from(p); return PathBuf::from(p);
} }
dirs::data_dir() xdg("XDG_DATA_HOME", ".local/share").join("ipx")
.unwrap_or_else(|| home().join(".local/share"))
.join("ipx")
} }
fn default_socket() -> PathBuf { fn default_socket() -> PathBuf {
@@ -346,8 +338,16 @@ pub fn unique_slug(text: &str, taken: &BTreeMap<String, Feed>) -> String {
(2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap() (2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap()
} }
/// `$var`, or `~/fallback` when it is unset or empty, as the XDG base directory spec says.
fn xdg(var: &str, fallback: &str) -> PathBuf {
std::env::var_os(var)
.filter(|v| !v.is_empty())
.map(PathBuf::from)
.unwrap_or_else(|| home().join(fallback))
}
fn home() -> PathBuf { fn home() -> PathBuf {
dirs::home_dir().unwrap_or_else(|| PathBuf::from(".")) std::env::var_os("HOME").map(PathBuf::from).unwrap_or_else(|| PathBuf::from("."))
} }
fn expand_tilde(p: &Path) -> PathBuf { fn expand_tilde(p: &Path) -> PathBuf {
@@ -367,6 +367,8 @@ mod tests {
r#" r#"
[general] [general]
download_dir = "/tmp/pods" download_dir = "/tmp/pods"
# A key older versions read. An old config that still has it has to load.
interval_mins = 45
[feeds.example] [feeds.example]
url = "https://example.com/feed.xml" url = "https://example.com/feed.xml"
@@ -403,22 +405,17 @@ mod tests {
} }
#[test] #[test]
fn interval_falls_back_through_legacy_then_default() { fn interval_falls_back_to_an_hour() {
let mut g = General::default(); let mut g = General::default();
assert_eq!(g.interval(), 60, "the default schedule"); assert_eq!(g.interval(), 60, "the default schedule");
g.schedule = "every 15m".into(); g.schedule = "every 15m".into();
assert_eq!(g.interval(), 15); assert_eq!(g.interval(), 15);
// A config written before `schedule` existed still works. // Empty or garbage must not stop the daemon.
g.schedule = String::new(); g.schedule = String::new();
g.interval_mins = Some(45); assert_eq!(g.interval(), 60);
assert_eq!(g.interval(), 45);
// Garbage must not stop the daemon.
g.schedule = "whenever".into(); g.schedule = "whenever".into();
assert_eq!(g.interval(), 45);
g.interval_mins = None;
assert_eq!(g.interval(), 60); assert_eq!(g.interval(), 60);
} }

768
src/db.rs
View File

@@ -40,14 +40,10 @@ CREATE TABLE IF NOT EXISTS entries (
published INTEGER, published INTEGER,
description TEXT, description TEXT,
first_seen INTEGER NOT NULL, first_seen INTEGER NOT NULL,
read INTEGER NOT NULL DEFAULT 0,
flagged INTEGER NOT NULL DEFAULT 0,
image TEXT, image TEXT,
duration INTEGER, duration INTEGER,
episode INTEGER, episode INTEGER,
season INTEGER, season INTEGER,
-- Seconds into the audio, so playback resumes where it was left.
position INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (feed_id, guid) PRIMARY KEY (feed_id, guid)
); );
@@ -80,6 +76,31 @@ CREATE TABLE IF NOT EXISTS users (
created INTEGER NOT NULL created INTEGER NOT NULL
); );
-- What one person wants from a feed. The feed, its items and its files are shared; this
-- is the part that is not. NULL in a column means: follow the feed's own setting.
CREATE TABLE IF NOT EXISTS subscriptions (
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
feed_id TEXT NOT NULL,
keywords TEXT,
auto_download INTEGER,
allow_explicit INTEGER,
max_new_per_check INTEGER,
created INTEGER NOT NULL,
PRIMARY KEY (user_id, feed_id)
);
-- Read, kept and how far in. One row per person per item, created on first touch;
-- an item nobody has touched has no row at all, which is what unread means.
CREATE TABLE IF NOT EXISTS entry_state (
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
feed_id TEXT NOT NULL,
guid TEXT NOT NULL,
read INTEGER NOT NULL DEFAULT 0,
flagged INTEGER NOT NULL DEFAULT 0,
position INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (user_id, feed_id, guid)
);
CREATE TABLE IF NOT EXISTS sessions ( CREATE TABLE IF NOT EXISTS sessions (
token TEXT PRIMARY KEY, token TEXT PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
@@ -88,6 +109,16 @@ CREATE TABLE IF NOT EXISTS sessions (
); );
"; ";
/// One person's wants for one feed. `None` in a field means the feed's own setting stands.
#[derive(Debug, Clone, Default)]
pub struct Sub {
pub feed_id: String,
pub keywords: Option<Vec<String>>,
pub auto_download: Option<bool>,
pub allow_explicit: Option<bool>,
pub max_new_per_check: Option<i64>,
}
/// Someone who can sign in. `pass_hash` is None for an account that only ever arrives /// Someone who can sign in. `pass_hash` is None for an account that only ever arrives
/// through the proxy. /// through the proxy.
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
@@ -108,8 +139,8 @@ pub struct Managed {
pub orphaned: bool, pub orphaned: bool,
} }
/// Adds columns that later versions introduced. CREATE TABLE IF NOT EXISTS does nothing to /// Adds the columns later versions introduced and drops the ones they retired. CREATE TABLE IF
/// a table that already exists, so an installed database needs them added explicitly. /// NOT EXISTS does nothing to a table that already exists, so an installed database needs both.
fn migrate(conn: &Connection) -> Result<()> { fn migrate(conn: &Connection) -> Result<()> {
let wanted: &[(&str, &str, &str)] = &[ let wanted: &[(&str, &str, &str)] = &[
("feeds", "image", "TEXT"), ("feeds", "image", "TEXT"),
@@ -120,18 +151,29 @@ fn migrate(conn: &Connection) -> Result<()> {
("entries", "duration", "INTEGER"), ("entries", "duration", "INTEGER"),
("entries", "episode", "INTEGER"), ("entries", "episode", "INTEGER"),
("entries", "season", "INTEGER"), ("entries", "season", "INTEGER"),
("entries", "position", "INTEGER NOT NULL DEFAULT 0"),
]; ];
for (table, column, ty) in wanted { // Read state from before accounts, long since moved to entry_state. Two bugs came from
// queries still reading these after they stopped meaning anything, so they go.
let retired: &[(&str, &str)] = &[("entries", "read"), ("entries", "flagged"), ("entries", "position")];
let has = |table: &str, column: &str| -> Result<bool> {
let mut stmt = conn.prepare(&format!("PRAGMA table_info({table})"))?; let mut stmt = conn.prepare(&format!("PRAGMA table_info({table})"))?;
let existing: Vec<String> = stmt let names = stmt
.query_map([], |r| r.get::<_, String>(1))? .query_map([], |r| r.get::<_, String>(1))?
.collect::<rusqlite::Result<Vec<_>>>()?; .collect::<rusqlite::Result<Vec<_>>>()?;
if !existing.iter().any(|c| c == column) { Ok(names.iter().any(|c| c == column))
};
for (table, column, ty) in wanted {
if !has(table, column)? {
tracing::info!(table, column, "adding column"); tracing::info!(table, column, "adding column");
conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?; conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?;
} }
} }
for (table, column) in retired {
if has(table, column)? {
tracing::info!(table, column, "dropping column");
conn.execute_batch(&format!("ALTER TABLE {table} DROP COLUMN {column}"))?;
}
}
Ok(()) Ok(())
} }
@@ -310,22 +352,19 @@ impl Db {
/// Returns true when this entry had not been seen before. /// Returns true when this entry had not been seen before.
/// ///
/// A changed description or title flips `read` back to 0, which is what the original's
/// textDiff dance was ultimately for -- minus the diff markup, which the UI can do.
pub fn record_entry(&self, feed_id: &str, e: &crate::feed::Entry) -> Result<bool> { pub fn record_entry(&self, feed_id: &str, e: &crate::feed::Entry) -> Result<bool> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let inserted = conn.execute( let inserted = conn.execute(
"INSERT OR IGNORE INTO entries "INSERT OR IGNORE INTO entries
(feed_id, guid, title, link, published, description, first_seen, read, flagged, (feed_id, guid, title, link, published, description, first_seen,
image, duration, episode, season) image, duration, episode, season)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, 0, 0, ?8, ?9, ?10, ?11)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11)",
rusqlite::params![ rusqlite::params![
feed_id, e.guid, e.title, e.link, e.published, e.description, now(), feed_id, e.guid, e.title, e.link, e.published, e.description, now(),
e.image, e.duration, e.episode, e.season e.image, e.duration, e.episode, e.season
], ],
)?; )?;
if inserted == 0 { if inserted == 0 {
// The SET expressions see the pre-update row, so this compares old vs new.
conn.execute( conn.execute(
"UPDATE entries SET "UPDATE entries SET
title = coalesce(?3, title), title = coalesce(?3, title),
@@ -333,8 +372,7 @@ impl Db {
image = coalesce(?5, image), image = coalesce(?5, image),
duration = coalesce(?6, duration), duration = coalesce(?6, duration),
episode = coalesce(?7, episode), episode = coalesce(?7, episode),
season = coalesce(?8, season), season = coalesce(?8, season)
read = CASE WHEN description IS NOT ?4 OR title IS NOT ?3 THEN 0 ELSE read END
WHERE feed_id = ?1 AND guid = ?2", WHERE feed_id = ?1 AND guid = ?2",
rusqlite::params![ rusqlite::params![
feed_id, e.guid, e.title, e.description, feed_id, e.guid, e.title, e.description,
@@ -431,23 +469,31 @@ pub struct Candidate {
} }
impl Db { impl Db {
/// Files on disk, flagged ones excluded, read before unread and oldest first within /// Files on disk that may be deleted to get back under quota: starred by nobody,
/// each group. /// with the ones everybody has finished going first, oldest first within each group.
/// ///
/// The Python intended `read = 1 AND flagged = 0` but never achieved it (a missing /// One file serves every subscriber, so both tests are about all of them: **anyone**
/// plistlib import and an `EntreiesData` typo meant the filter always threw). Requiring /// starring it keeps it, and it only counts as read when **everyone** subscribed has
/// `read = 1` outright would be just as dead here, since nothing marks episodes read /// read it. A file whose feed nobody subscribes to has no one left to keep it, so it
/// until a UI exists -- so `flagged` is the keep-forever marker, and `read` only decides /// sorts with the read ones.
/// what goes first. ///
/// (The Python intended `read = 1 AND flagged = 0` but never achieved it -- a missing
/// plistlib import and an `EntreiesData` typo meant the filter always threw.)
pub fn reap_candidates(&self) -> Result<Vec<Candidate>> { pub fn reap_candidates(&self) -> Result<Vec<Candidate>> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare( let mut stmt = conn.prepare(
"SELECT e.id, e.url, e.path, e.bytes_done, "SELECT e.id, e.url, e.path, e.bytes_done, coalesce(e.downloaded_at, 0),
coalesce(e.downloaded_at, 0), coalesce(n.read, 0), coalesce(n.flagged, 0) CASE WHEN coalesce(readers.n, 0) >= coalesce(subs.n, 0) THEN 1 ELSE 0 END
FROM enclosures e FROM enclosures e
LEFT JOIN entries n ON n.feed_id = e.feed_id AND n.guid = e.guid LEFT JOIN (SELECT feed_id, count(*) n FROM subscriptions GROUP BY feed_id) subs
WHERE e.path IS NOT NULL AND coalesce(n.flagged, 0) = 0 ON subs.feed_id = e.feed_id
ORDER BY coalesce(n.read, 0) DESC, coalesce(e.downloaded_at, 0) ASC, e.id ASC", LEFT JOIN (SELECT feed_id, guid, count(*) n FROM entry_state
WHERE read = 1 GROUP BY feed_id, guid) readers
ON readers.feed_id = e.feed_id AND readers.guid = e.guid
WHERE e.path IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM entry_state s
WHERE s.feed_id = e.feed_id AND s.guid = e.guid AND s.flagged = 1)
ORDER BY 6 DESC, coalesce(e.downloaded_at, 0) ASC, e.id ASC",
)?; )?;
let rows = stmt let rows = stmt
.query_map([], |r| { .query_map([], |r| {
@@ -493,24 +539,40 @@ impl Db {
pub fn prune_entries(&self, older_than: i64) -> Result<usize> { pub fn prune_entries(&self, older_than: i64) -> Result<usize> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let n = conn.execute( let n = conn.execute(
"DELETE FROM entries WHERE flagged = 0 "DELETE FROM entries
AND coalesce(published, first_seen) < ?1 WHERE coalesce(published, first_seen) < ?1
AND NOT EXISTS ( AND NOT EXISTS (
SELECT 1 FROM enclosures e SELECT 1 FROM enclosures e
WHERE e.feed_id = entries.feed_id AND e.guid = entries.guid WHERE e.feed_id = entries.feed_id AND e.guid = entries.guid
AND e.path IS NOT NULL)", AND e.path IS NOT NULL)
-- Starred by anyone keeps it, the same rule the reaper follows.
AND NOT EXISTS (
SELECT 1 FROM entry_state s
WHERE s.feed_id = entries.feed_id AND s.guid = entries.guid
AND s.flagged = 1)",
[older_than], [older_than],
)?; )?;
// Whatever went takes everyone's read state with it, rather than leaving rows
// pointing at an item that no longer exists.
conn.execute(
"DELETE FROM entry_state WHERE NOT EXISTS (
SELECT 1 FROM entries e
WHERE e.feed_id = entry_state.feed_id AND e.guid = entry_state.guid)",
[],
)?;
Ok(n) Ok(n)
} }
} }
impl Db { impl Db {
pub fn unread_count(&self, feed_id: &str) -> Result<i64> { pub fn unread_count(&self, user_id: i64, feed_id: &str) -> Result<i64> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
Ok(conn.query_row( Ok(conn.query_row(
"SELECT count(*) FROM entries WHERE feed_id = ?1 AND read = 0", "SELECT count(*) FROM entries e
[feed_id], LEFT JOIN entry_state s
ON s.user_id = ?2 AND s.feed_id = e.feed_id AND s.guid = e.guid
WHERE e.feed_id = ?1 AND coalesce(s.read, 0) = 0",
rusqlite::params![feed_id, user_id],
|r| r.get(0), |r| r.get(0),
)?) )?)
} }
@@ -550,6 +612,36 @@ pub struct EntryRow {
const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2 const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2
OR lower(coalesce(e.description, '')) LIKE ?2)"; OR lower(coalesce(e.description, '')) LIKE ?2)";
/// Which feeds a query covers: one, or every feed the person subscribes to. Both forms
/// mention `?1`, since binding a parameter the statement does not use is an error.
fn scope_sql(feed_id: Option<&str>, user_param: u8) -> String {
match feed_id {
Some(_) => "e.feed_id = ?1".into(),
None => format!(
"?1 IS NULL AND e.feed_id IN (SELECT feed_id FROM subscriptions WHERE user_id = ?{user_param})"
),
}
}
/// The item table's ORDER BY. The column name picks one of these fixed expressions, so nothing
/// the caller sends reaches the query, and anything unrecognised is newest first. Ties fall back
/// to newest first too, so a page boundary is stable across "Load more".
///
/// ponytail: file type and size look at the item's first and largest file. The row shows the file
/// it summarises, which is almost always that one; sort by that one if they ever disagree.
pub fn order_sql(col: &str, dir: &str) -> String {
let expr = match col {
"kept" => "coalesce(s.flagged, 0)",
"title" => "lower(coalesce(e.title, ''))",
"feed" => "(SELECT lower(coalesce(f.title, f.id)) FROM feeds f WHERE f.id = e.feed_id)",
"type" => "(SELECT min(x.mime) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
"size" => "(SELECT max(x.length) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
_ => "coalesce(e.published, e.first_seen)",
};
let dir = if dir == "asc" { "ASC" } else { "DESC" };
format!("{expr} {dir}, coalesce(e.published, e.first_seen) DESC, e.rowid DESC")
}
/// Which slice of a feed the UI is asking for. /// Which slice of a feed the UI is asking for.
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Filter { pub enum Filter {
@@ -574,8 +666,8 @@ impl Filter {
fn sql(self) -> &'static str { fn sql(self) -> &'static str {
match self { match self {
Self::All => "1=1", Self::All => "1=1",
Self::Unread => "e.read = 0", Self::Unread => "coalesce(s.read, 0) = 0",
Self::Flagged => "e.flagged = 1", Self::Flagged => "coalesce(s.flagged, 0) = 1",
Self::Downloaded => { Self::Downloaded => {
"EXISTS (SELECT 1 FROM enclosures x "EXISTS (SELECT 1 FROM enclosures x
WHERE x.feed_id = e.feed_id AND x.guid = e.guid AND x.path IS NOT NULL)" WHERE x.feed_id = e.feed_id AND x.guid = e.guid AND x.path IS NOT NULL)"
@@ -598,27 +690,34 @@ pub struct EncRow {
} }
impl Db { impl Db {
/// One page of a feed's entries, newest first, each with its enclosures attached. /// One page of entries, each with its enclosures attached: one feed's, or every feed the
/// `search` matches title and description, case-insensitively. /// person subscribes to when `feed_id` is None (All Subscriptions). `search` matches title
pub fn entries( /// and description, case-insensitively.
pub fn entries_in(
&self, &self,
feed_id: &str, user_id: i64,
feed_id: Option<&str>,
filter: Filter, filter: Filter,
search: Option<&str>, search: Option<&str>,
offset: i64, offset: i64,
limit: i64, limit: i64,
order: &str,
) -> Result<Vec<EntryRow>> { ) -> Result<Vec<EntryRow>> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let like = search let like = search
.map(|q| format!("%{}%", q.trim().to_lowercase())) .map(|q| format!("%{}%", q.trim().to_lowercase()))
.unwrap_or_default(); .unwrap_or_default();
let sql = format!( let sql = format!(
"SELECT e.guid, e.feed_id, e.title, e.link, e.published, e.description, e.read, "SELECT e.guid, e.feed_id, e.title, e.link, e.published, e.description,
e.flagged, e.image, e.duration, e.episode, e.season, e.position coalesce(s.read, 0), coalesce(s.flagged, 0), e.image, e.duration,
e.episode, e.season, coalesce(s.position, 0)
FROM entries e FROM entries e
WHERE e.feed_id = ?1 AND {} AND {SEARCH} LEFT JOIN entry_state s
ORDER BY coalesce(e.published, e.first_seen) DESC, e.rowid DESC ON s.user_id = ?5 AND s.feed_id = e.feed_id AND s.guid = e.guid
WHERE {} AND {} AND {SEARCH}
ORDER BY {order}
LIMIT ?4 OFFSET ?3", LIMIT ?4 OFFSET ?3",
scope_sql(feed_id, 5),
filter.sql() filter.sql()
); );
let mut stmt = conn.prepare(&sql)?; let mut stmt = conn.prepare(&sql)?;
@@ -641,21 +740,21 @@ impl Db {
}) })
}; };
let mut rows: Vec<EntryRow> = stmt let mut rows: Vec<EntryRow> = stmt
.query_map(rusqlite::params![feed_id, like, offset, limit], map)? .query_map(rusqlite::params![feed_id, like, offset, limit, user_id], map)?
.collect::<rusqlite::Result<Vec<_>>>()?; .collect::<rusqlite::Result<Vec<_>>>()?;
if rows.is_empty() { if rows.is_empty() {
return Ok(rows); return Ok(rows);
} }
// Only the guids on this page, so a feed with thousands of entries stays cheap. // Only the guids on this page, so a feed with thousands of entries stays cheap. A page
// can span feeds, so each file is matched to its row by feed as well as guid, below.
let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(","); let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(",");
let sql = format!( let sql = format!(
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error "SELECT id, feed_id, guid, url, mime, length, path, state, last_error
FROM enclosures WHERE feed_id = ? AND guid IN ({placeholders}) ORDER BY id" FROM enclosures WHERE guid IN ({placeholders}) ORDER BY id"
); );
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len() + 1); let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len());
params.push(&feed_id);
for row in &rows { for row in &rows {
params.push(&row.guid); params.push(&row.guid);
} }
@@ -677,37 +776,249 @@ impl Db {
.collect::<rusqlite::Result<Vec<_>>>()?; .collect::<rusqlite::Result<Vec<_>>>()?;
for enc in encs { for enc in encs {
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid) { if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid && r.feed_id == enc.feed_id) {
row.enclosures.push(enc); row.enclosures.push(enc);
} }
} }
Ok(rows) Ok(rows)
} }
/// How many entries match, so the UI knows whether there is another page. /// How many entries `entries_in` would page through, so the UI knows whether there is more.
pub fn count_entries(&self, feed_id: &str, filter: Filter, search: Option<&str>) -> Result<i64> { pub fn count_in(
&self,
user_id: i64,
feed_id: Option<&str>,
filter: Filter,
search: Option<&str>,
) -> Result<i64> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let like = search let like = search
.map(|q| format!("%{}%", q.trim().to_lowercase())) .map(|q| format!("%{}%", q.trim().to_lowercase()))
.unwrap_or_default(); .unwrap_or_default();
let sql = format!( let sql = format!(
"SELECT count(*) FROM entries e WHERE e.feed_id = ?1 AND {} AND {SEARCH}", "SELECT count(*) FROM entries e
LEFT JOIN entry_state s
ON s.user_id = ?3 AND s.feed_id = e.feed_id AND s.guid = e.guid
WHERE {} AND {} AND {SEARCH}",
scope_sql(feed_id, 3),
filter.sql() filter.sql()
); );
Ok(conn.query_row(&sql, rusqlite::params![feed_id, like], |r| r.get(0))?) Ok(conn.query_row(&sql, rusqlite::params![feed_id, like, user_id], |r| r.get(0))?)
} }
/// Where playback got to, so it resumes there next time. /// Where playback got to, so it resumes there next time -- for this listener only.
pub fn set_position(&self, feed_id: &str, guid: &str, secs: i64) -> Result<()> { pub fn set_position(&self, user_id: i64, feed_id: &str, guid: &str, secs: i64) -> Result<()> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
conn.execute( conn.execute(
"UPDATE entries SET position = ?3 WHERE feed_id = ?1 AND guid = ?2", "INSERT INTO entry_state (user_id, feed_id, guid, position) VALUES (?1, ?2, ?3, ?4)
rusqlite::params![feed_id, guid, secs.max(0)], ON CONFLICT(user_id, feed_id, guid) DO UPDATE SET position = excluded.position",
rusqlite::params![user_id, feed_id, guid, secs.max(0)],
)?; )?;
Ok(()) Ok(())
} }
/// Marks every entry in a feed read, for the "mark all read" button. /// The first admin starts subscribed to the whole catalogue: whoever wrote config.toml meant
/// to read those feeds, and without this a fresh install signs in to an empty sidebar. Runs
/// only while nobody subscribes to anything, so an unsubscribe is never undone.
pub fn adopt_catalogue(&self, user_id: i64, catalogue: &[String]) -> Result<usize> {
let conn = self.conn.lock().unwrap();
let already: i64 =
conn.query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0))?;
if already > 0 {
return Ok(0);
}
for id in catalogue {
conn.execute(
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
params![user_id, id, now()],
)?;
}
// Feeds that exist only in the database (OPML children) count too.
conn.execute(
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created)
SELECT ?1, id, ?2 FROM feeds",
params![user_id, now()],
)?;
Ok(catalogue.len())
}
// ---- subscriptions ----
/// What this person wants from a feed. Absent means they do not subscribe at all.
pub fn subscription(&self, user_id: i64, feed_id: &str) -> Result<Option<Sub>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT keywords, auto_download, allow_explicit, max_new_per_check
FROM subscriptions WHERE user_id = ?1 AND feed_id = ?2",
)?;
let mut rows = stmt.query(params![user_id, feed_id])?;
Ok(match rows.next()? {
Some(r) => Some(Sub {
feed_id: feed_id.to_string(),
keywords: r
.get::<_, Option<String>>(0)?
.and_then(|j| serde_json::from_str(&j).ok()),
auto_download: r.get::<_, Option<i64>>(1)?.map(|v| v != 0),
allow_explicit: r.get::<_, Option<i64>>(2)?.map(|v| v != 0),
max_new_per_check: r.get(3)?,
}),
None => None,
})
}
/// Every feed this person subscribes to, with their settings.
pub fn subscriptions_for(&self, user_id: i64) -> Result<Vec<Sub>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT feed_id, keywords, auto_download, allow_explicit, max_new_per_check
FROM subscriptions WHERE user_id = ?1",
)?;
let out = stmt
.query_map([user_id], |r| {
Ok(Sub {
feed_id: r.get(0)?,
keywords: r
.get::<_, Option<String>>(1)?
.and_then(|j| serde_json::from_str(&j).ok()),
auto_download: r.get::<_, Option<i64>>(2)?.map(|v| v != 0),
allow_explicit: r.get::<_, Option<i64>>(3)?.map(|v| v != 0),
max_new_per_check: r.get(4)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(out)
}
/// Everyone's settings for one feed. The scanner merges these into what it fetches
/// and downloads, since one file serves the lot.
/// In a group, whatever someone has not set on the feed itself comes from their
/// subscription to the group, as the group's settings dialog has always said it does.
pub fn subscribers(&self, feed_id: &str, group: Option<&str>) -> Result<Vec<Sub>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT coalesce(c.keywords, p.keywords), coalesce(c.auto_download, p.auto_download),
coalesce(c.allow_explicit, p.allow_explicit),
coalesce(c.max_new_per_check, p.max_new_per_check)
FROM subscriptions c
LEFT JOIN subscriptions p ON p.user_id = c.user_id AND p.feed_id = ?2
WHERE c.feed_id = ?1",
)?;
let out = stmt
.query_map(params![feed_id, group], |r| {
Ok(Sub {
feed_id: feed_id.to_string(),
keywords: r
.get::<_, Option<String>>(0)?
.and_then(|j| serde_json::from_str(&j).ok()),
auto_download: r.get::<_, Option<i64>>(1)?.map(|v| v != 0),
allow_explicit: r.get::<_, Option<i64>>(2)?.map(|v| v != 0),
max_new_per_check: r.get(3)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(out)
}
/// Subscribers per feed, for the whole catalogue in one query -- the feed list would
/// otherwise ask once per feed.
pub fn subscriber_counts(&self) -> Result<std::collections::HashMap<String, i64>> {
let conn = self.conn.lock().unwrap();
let mut stmt =
conn.prepare("SELECT feed_id, count(*) FROM subscriptions GROUP BY feed_id")?;
let out = stmt
.query_map([], |r| Ok((r.get::<_, String>(0)?, r.get::<_, i64>(1)?)))?
.collect::<rusqlite::Result<std::collections::HashMap<_, _>>>()?;
Ok(out)
}
/// Who else would miss this file: subscribers other than `user_id` who have starred
/// the item or have not read it yet. Deleting is deleting their copy too.
pub fn others_wanting(&self, enclosure_id: i64, user_id: i64) -> Result<(i64, i64)> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT
sum(CASE WHEN coalesce(st.flagged, 0) = 1 THEN 1 ELSE 0 END),
sum(CASE WHEN coalesce(st.read, 0) = 0 THEN 1 ELSE 0 END)
FROM enclosures e
JOIN subscriptions s ON s.feed_id = e.feed_id AND s.user_id != ?2
LEFT JOIN entry_state st
ON st.user_id = s.user_id AND st.feed_id = e.feed_id AND st.guid = e.guid
WHERE e.id = ?1",
)?;
let (starred, unread) = stmt.query_row(params![enclosure_id, user_id], |r| {
Ok((r.get::<_, Option<i64>>(0)?.unwrap_or(0), r.get::<_, Option<i64>>(1)?.unwrap_or(0)))
})?;
Ok((starred, unread))
}
pub fn subscribe(&self, user_id: i64, feed_id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
params![user_id, feed_id, now()],
)?;
Ok(())
}
pub fn unsubscribe(&self, user_id: i64, feed_id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"DELETE FROM subscriptions WHERE user_id = ?1 AND feed_id = ?2",
params![user_id, feed_id],
)?;
Ok(())
}
/// How many people want this feed. Nobody means it stops being scanned.
pub fn subscriber_count(&self, feed_id: &str) -> Result<i64> {
let conn = self.conn.lock().unwrap();
Ok(conn.query_row(
"SELECT count(*) FROM subscriptions WHERE feed_id = ?1",
[feed_id],
|r| r.get(0),
)?)
}
/// Overwrites one person's settings for a feed. A None field means: follow the feed.
pub fn set_subscription(&self, user_id: i64, sub: &Sub) -> Result<()> {
let conn = self.conn.lock().unwrap();
let kw = sub
.keywords
.as_ref()
.map(|k| serde_json::to_string(k))
.transpose()?;
conn.execute(
"INSERT INTO subscriptions
(user_id, feed_id, keywords, auto_download, allow_explicit, max_new_per_check, created)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
ON CONFLICT(user_id, feed_id) DO UPDATE SET
keywords = excluded.keywords,
auto_download = excluded.auto_download,
allow_explicit = excluded.allow_explicit,
max_new_per_check = excluded.max_new_per_check",
params![
user_id,
sub.feed_id,
kw,
sub.auto_download.map(|v| v as i64),
sub.allow_explicit.map(|v| v as i64),
sub.max_new_per_check,
now()
],
)?;
Ok(())
}
/// Feeds with at least one subscriber. What the scanner walks.
pub fn subscribed_feed_ids(&self) -> Result<Vec<String>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare("SELECT DISTINCT feed_id FROM subscriptions")?;
let out = stmt
.query_map([], |r| r.get::<_, String>(0))?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(out)
}
// ---- users and sessions ---- // ---- users and sessions ----
pub fn create_user(&self, name: &str, pass_hash: Option<&str>, admin: bool) -> Result<i64> { pub fn create_user(&self, name: &str, pass_hash: Option<&str>, admin: bool) -> Result<i64> {
@@ -827,11 +1138,19 @@ impl Db {
/// Marks every entry of the given feeds read. Takes a list because an OPML subscription /// Marks every entry of the given feeds read. Takes a list because an OPML subscription
/// holds no entries itself -- marking it read means the feeds inside it. /// holds no entries itself -- marking it read means the feeds inside it.
pub fn mark_all_read(&self, feed_ids: &[String]) -> Result<usize> { pub fn mark_all_read(&self, user_id: i64, feed_ids: &[String]) -> Result<usize> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let mut n = 0; let mut n = 0;
for id in feed_ids { for id in feed_ids {
n += conn.execute("UPDATE entries SET read = 1 WHERE feed_id = ?1 AND read = 0", [id])?; n += conn.execute(
"INSERT INTO entry_state (user_id, feed_id, guid, read)
SELECT ?1, e.feed_id, e.guid, 1 FROM entries e
LEFT JOIN entry_state s
ON s.user_id = ?1 AND s.feed_id = e.feed_id AND s.guid = e.guid
WHERE e.feed_id = ?2 AND coalesce(s.read, 0) = 0
ON CONFLICT(user_id, feed_id, guid) DO UPDATE SET read = 1",
rusqlite::params![user_id, id],
)?;
} }
Ok(n) Ok(n)
} }
@@ -876,14 +1195,28 @@ impl Db {
.optional()?) .optional()?)
} }
/// `read` and `flagged` finally get a writer: retention orders by them. /// Read and starred, per person. The row is created on first touch.
pub fn set_entry_flag(&self, feed_id: &str, guid: &str, field: EntryFlag, on: bool) -> Result<()> { pub fn set_entry_flag(
&self,
user_id: i64,
feed_id: &str,
guid: &str,
field: EntryFlag,
on: bool,
) -> Result<()> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let sql = match field { let col = match field {
EntryFlag::Read => "UPDATE entries SET read = ?3 WHERE feed_id = ?1 AND guid = ?2", EntryFlag::Read => "read",
EntryFlag::Flagged => "UPDATE entries SET flagged = ?3 WHERE feed_id = ?1 AND guid = ?2", EntryFlag::Flagged => "flagged",
}; };
conn.execute(sql, rusqlite::params![feed_id, guid, on as i64])?; conn.execute(
&format!(
"INSERT INTO entry_state (user_id, feed_id, guid, {col})
VALUES (?1, ?2, ?3, ?4)
ON CONFLICT(user_id, feed_id, guid) DO UPDATE SET {col} = excluded.{col}"
),
rusqlite::params![user_id, feed_id, guid, on as i64],
)?;
Ok(()) Ok(())
} }
@@ -969,6 +1302,71 @@ impl Db {
Ok(()) Ok(())
} }
/// Empties a feed of its items, leaving its files alone.
pub fn clear_entries(&self, feed_id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("DELETE FROM entries WHERE feed_id = ?1", [feed_id])?;
Ok(())
}
/// Every feed the database holds rows for, as (id, url), removed ones included.
pub fn feed_urls(&self) -> Result<Vec<(String, String)>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare("SELECT id, coalesce(url, '') FROM feeds")?;
let out = stmt
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))?
.collect::<rusqlite::Result<_>>()?;
Ok(out)
}
/// Hands a feed in a group the enclosures its parent holds, as (guid, url), with everyone's
/// read state for them. A Patreon creator read as one feed before it was split into shows
/// owns every show's files, and `enclosures.url` is unique, so without this each show would
/// list its items with nothing to play.
pub fn adopt(&self, parent: &str, child: &str, listed: &[(&str, &str)]) -> Result<()> {
let mut conn = self.conn.lock().unwrap();
let holds: bool = conn.query_row(
"SELECT EXISTS (SELECT 1 FROM enclosures WHERE feed_id = ?1)",
[parent],
|r| r.get(0),
)?;
if !holds {
return Ok(()); // An OPML, or a creator already shared out.
}
let tx = conn.transaction()?;
for &(guid, url) in listed {
let moved = tx.execute(
"UPDATE enclosures SET feed_id = ?3, guid = ?4 WHERE url = ?1 AND feed_id = ?2",
params![url, parent, child, guid],
)?;
if moved == 1 {
// Patreon gives a post the same guid in every feed it appears in.
tx.execute(
"UPDATE OR IGNORE entry_state SET feed_id = ?2 WHERE feed_id = ?1 AND guid = ?3",
params![parent, child, guid],
)?;
}
}
tx.commit()?;
Ok(())
}
/// A feed's enclosures skipped by one of its filters, by URL, with the reason: the verdicts a
/// change of settings can overturn. A torrent held back while torrents are off is not a
/// filter's call.
pub fn skipped_by_filter(&self, feed_id: &str) -> Result<std::collections::HashMap<String, String>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT url, last_error FROM enclosures
WHERE feed_id = ?1 AND state = 'skipped' AND last_error IS NOT NULL
AND last_error != 'torrents disabled'",
)?;
let out = stmt
.query_map([feed_id], |r| Ok((r.get(0)?, r.get(1)?)))?
.collect::<rusqlite::Result<_>>()?;
Ok(out)
}
pub fn set_orphaned(&self, feed_id: &str, on: bool) -> Result<()> { pub fn set_orphaned(&self, feed_id: &str, on: bool) -> Result<()> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
conn.execute( conn.execute(
@@ -1021,6 +1419,100 @@ pub fn now() -> i64 {
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn every_sort_column_runs_and_orders_both_ways() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(1,'g',0);
INSERT INTO feeds (id, url, title) VALUES ('f','u','Zebra'),('g','v','Aardvark');
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
('f','a','banana',100),('g','b','Apple',200),('f','c','cherry',300);
INSERT INTO enclosures (id, feed_id, guid, url, mime, length, state) VALUES
(1,'f','a','u1','audio/mpeg',300,'pending'),(2,'g','b','u2','image/png',10,'pending'),
(3,'f','c','u3','video/mp4',2000,'pending');",
)
.unwrap();
let order = |col: &str, dir: &str| -> Vec<String> {
db.entries_in(1, None, Filter::All, None, 0, 50, &order_sql(col, dir))
.unwrap()
.into_iter()
.map(|e| e.guid)
.collect()
};
assert_eq!(order("title", "asc"), ["b", "a", "c"], "Apple, banana, cherry: case folded");
assert_eq!(order("title", "desc"), ["c", "a", "b"]);
assert_eq!(order("feed", "asc"), ["b", "c", "a"], "Aardvark, then Zebra's newest first");
assert_eq!(order("type", "asc"), ["a", "b", "c"], "audio, image, video");
assert_eq!(order("size", "desc"), ["c", "a", "b"]);
assert_eq!(order("published", "desc"), ["c", "b", "a"]);
db.set_entry_flag(1, "f", "a", EntryFlag::Flagged, true).unwrap();
assert_eq!(order("kept", "desc")[0], "a");
// An unknown column or direction is newest first; the name itself never reaches the SQL.
assert_eq!(order("title; DROP TABLE entries", "sideways"), ["c", "b", "a"]);
assert!(!order_sql("x'; --", "asc").contains("x'"));
}
#[test]
fn deleting_a_shared_file_asks_about_everyone_else() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0),(3,'kit',0,0);
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0),(3,'f',0);
INSERT INTO entries (feed_id, guid, first_seen) VALUES ('f','a',0);
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
(1,'f','a','u1','/tmp/a','done');",
)
.unwrap();
// Nobody has touched it: both others still have it unplayed.
assert_eq!(db.others_wanting(1, 1).unwrap(), (0, 2));
// Sam reads it, Kit stars it.
db.set_entry_flag(2, "f", "a", EntryFlag::Read, true).unwrap();
db.set_entry_flag(3, "f", "a", EntryFlag::Flagged, true).unwrap();
assert_eq!(db.others_wanting(1, 1).unwrap(), (1, 1), "one starred it, one has not played it");
// Asking as Kit, only Ray and Sam count -- and Kit's own star is not a reason to
// warn Kit.
db.set_entry_flag(1, "f", "a", EntryFlag::Read, true).unwrap();
assert_eq!(db.others_wanting(1, 3).unwrap(), (0, 0));
}
#[test]
fn read_state_belongs_to_one_person() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
('f','a','One',100),('f','b','Two',200);",
)
.unwrap();
assert_eq!(db.unread_count(1, "f").unwrap(), 2);
assert_eq!(db.unread_count(2, "f").unwrap(), 2);
db.set_entry_flag(1, "f", "a", EntryFlag::Read, true).unwrap();
assert_eq!(db.unread_count(1, "f").unwrap(), 1, "ray read one of them");
assert_eq!(db.unread_count(2, "f").unwrap(), 2, "sam has read nothing");
// Starring and position are just as private.
db.set_entry_flag(1, "f", "b", EntryFlag::Flagged, true).unwrap();
db.set_position(2, "f", "b", 42).unwrap();
let order = order_sql("published", "desc");
let page = |user| db.entries_in(user, Some("f"), Filter::All, None, 0, 50, &order).unwrap();
let (ray, sam) = (page(1), page(2));
let ray_b = ray.iter().find(|e| e.guid == "b").unwrap();
let sam_b = sam.iter().find(|e| e.guid == "b").unwrap();
assert!(ray_b.flagged && ray_b.position == 0);
assert!(!sam_b.flagged && sam_b.position == 42);
// Marking a whole feed read is likewise one person's business.
assert_eq!(db.mark_all_read(2, &["f".to_string()]).unwrap(), 2);
assert_eq!(db.unread_count(2, "f").unwrap(), 0);
assert_eq!(db.unread_count(1, "f").unwrap(), 1);
}
#[test] #[test]
fn schema_is_idempotent_and_summary_handles_unknown_feeds() { fn schema_is_idempotent_and_summary_handles_unknown_feeds() {
let db = Db::memory().unwrap(); let db = Db::memory().unwrap();
@@ -1033,6 +1525,49 @@ mod tests {
assert_eq!(sum.downloaded, 0); assert_eq!(sum.downloaded, 0);
} }
#[test]
fn an_old_database_loses_the_retired_read_columns() {
let conn = Connection::open_in_memory().unwrap();
conn.execute_batch(
"CREATE TABLE entries (feed_id TEXT NOT NULL, guid TEXT NOT NULL,
first_seen INTEGER NOT NULL, read INTEGER NOT NULL DEFAULT 0,
flagged INTEGER NOT NULL DEFAULT 0, position INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (feed_id, guid));",
)
.unwrap();
// The same order as open(): the schema leaves the old table alone, migrate() fixes it.
conn.execute_batch(SCHEMA).unwrap();
migrate(&conn).unwrap();
let cols: Vec<String> = conn
.prepare("PRAGMA table_info(entries)")
.unwrap()
.query_map([], |r| r.get(1))
.unwrap()
.collect::<rusqlite::Result<_>>()
.unwrap();
assert!(!cols.iter().any(|c| ["read", "flagged", "position"].contains(&c.as_str())), "{cols:?}");
assert!(cols.iter().any(|c| c == "image"), "and it still gains the newer ones");
}
#[test]
fn the_first_admin_starts_with_the_catalogue_and_only_once() {
// Cutting this along with the dead read columns left the browser suite's admin with an
// empty sidebar: it is how a fresh install's first account gets config.toml's feeds.
let db = Db::memory().unwrap();
db.exec_for_test("INSERT INTO users (id, name, is_admin, created) VALUES (1,'admin',1,0);")
.unwrap();
let subs = || -> i64 {
db.conn.lock().unwrap().query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0)).unwrap()
};
let catalogue = ["a".to_string(), "b".to_string()];
assert_eq!(db.adopt_catalogue(1, &catalogue).unwrap(), 2);
assert_eq!(subs(), 2);
// Once anyone subscribes to anything it never runs again, so an unsubscribe sticks.
db.unsubscribe(1, "a").unwrap();
assert_eq!(db.adopt_catalogue(1, &catalogue).unwrap(), 0);
assert_eq!(subs(), 1);
}
#[test] #[test]
fn every_filter_works_with_and_without_a_search_term() { fn every_filter_works_with_and_without_a_search_term() {
// Regression: the search clause used to be omitted when no term was given, while // Regression: the search clause used to be omitted when no term was given, while
@@ -1040,32 +1575,38 @@ mod tests {
// so plain filtering failed with "Wrong number of parameters passed to query". // so plain filtering failed with "Wrong number of parameters passed to query".
let db = Db::memory().unwrap(); let db = Db::memory().unwrap();
db.exec_for_test( db.exec_for_test(
"INSERT INTO entries (feed_id, guid, title, description, first_seen, read, flagged) VALUES "INSERT INTO entries (feed_id, guid, title, description, first_seen) VALUES
('f','a','Alpha dive','notes one',100,0,0), ('f','a','Alpha dive','notes one',100),
('f','b','Beta', 'notes two',200,1,0), ('f','b','Beta', 'notes two',200),
('f','c','Gamma dive','notes three',300,1,1); ('f','c','Gamma dive','notes three',300);
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
(1,'f','b','u1','/tmp/b','done');", (1,'f','b','u1','/tmp/b','done');
-- Read and starred belong to a person now, so say which one.
INSERT INTO users (id, name, is_admin, created) VALUES (7,'reader',1,0);
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
(7,'f','b',1,0),
(7,'f','c',1,1);",
) )
.unwrap(); .unwrap();
for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] { for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] {
// Both paths must run without erroring, and agree with each other. // Both paths must run without erroring, and agree with each other.
let rows = db.entries("f", f, None, 0, 50).unwrap(); let order = order_sql("published", "desc");
let n = db.count_entries("f", f, None).unwrap(); let rows = db.entries_in(7, Some("f"), f, None, 0, 50, &order).unwrap();
let n = db.count_in(7, Some("f"), f, None).unwrap();
assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page"); assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page");
let rows = db.entries("f", f, Some("dive"), 0, 50).unwrap(); let rows = db.entries_in(7, Some("f"), f, Some("dive"), 0, 50, &order).unwrap();
let n = db.count_entries("f", f, Some("dive")).unwrap(); let n = db.count_in(7, Some("f"), f, Some("dive")).unwrap();
assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees"); assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees");
} }
assert_eq!(db.count_entries("f", Filter::All, None).unwrap(), 3); assert_eq!(db.count_in(7, Some("f"), Filter::All, None).unwrap(), 3);
assert_eq!(db.count_entries("f", Filter::Unread, None).unwrap(), 1); assert_eq!(db.count_in(7, Some("f"), Filter::Unread, None).unwrap(), 1);
assert_eq!(db.count_entries("f", Filter::Downloaded, None).unwrap(), 1); assert_eq!(db.count_in(7, Some("f"), Filter::Downloaded, None).unwrap(), 1);
assert_eq!(db.count_entries("f", Filter::Flagged, None).unwrap(), 1); assert_eq!(db.count_in(7, Some("f"), Filter::Flagged, None).unwrap(), 1);
assert_eq!(db.count_entries("f", Filter::All, Some("dive")).unwrap(), 2); assert_eq!(db.count_in(7, Some("f"), Filter::All, Some("dive")).unwrap(), 2);
assert_eq!(db.count_entries("f", Filter::All, Some("NOTES two")).unwrap(), 1, assert_eq!(db.count_in(7, Some("f"), Filter::All, Some("NOTES two")).unwrap(), 1,
"search is case-insensitive and covers the description"); "search is case-insensitive and covers the description");
} }
@@ -1107,6 +1648,61 @@ mod tests {
assert_eq!(state(4), "done"); assert_eq!(state(4), "done");
} }
#[test]
fn a_show_takes_over_what_its_creator_held() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
INSERT INTO enclosures (id, feed_id, guid, url, state, path, last_error) VALUES
(1,'creator','a','u1','done','/x/a.mp3',NULL),
(2,'creator','b','u2','skipped',NULL,'explicit'),
(3,'creator','c','u3','skipped',NULL,'explicit'),
(4,'other','d','u4','skipped',NULL,'torrents disabled');
INSERT INTO entry_state (user_id, feed_id, guid, read) VALUES (1,'creator','a',1);",
)
.unwrap();
db.adopt("creator", "show", &[("a", "u1"), ("b", "u2"), ("d", "u4")]).unwrap();
{
let conn = db.conn.lock().unwrap();
let owner = |id: i64| -> String {
conn.query_row("SELECT feed_id FROM enclosures WHERE id = ?1", [id], |r| r.get(0)).unwrap()
};
assert_eq!(owner(1), "show", "a downloaded file moves with its item");
assert_eq!(owner(2), "show");
assert_eq!(owner(3), "creator", "this show does not list it");
assert_eq!(owner(4), "other", "only the parent's are taken");
let read: String = conn
.query_row("SELECT feed_id FROM entry_state WHERE user_id = 1 AND guid = 'a'", [], |r| r.get(0))
.unwrap();
assert_eq!(read, "show", "what you had read stays read");
}
// Only a filter's verdict can be overturned by a change of settings.
let skipped = db.skipped_by_filter("show").unwrap();
assert_eq!(skipped.get("u2").map(String::as_str), Some("explicit"));
assert_eq!(skipped.len(), 1);
assert!(db.skipped_by_filter("other").unwrap().is_empty(), "torrents disabled is not a filter");
}
#[test]
fn a_feed_in_a_group_follows_your_settings_on_the_group() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
INSERT INTO subscriptions (user_id, feed_id, allow_explicit, created) VALUES
(1,'group',1,0),(1,'show',NULL,0),(2,'group',1,0),(2,'show',0,0);",
)
.unwrap();
let explicit = |group| -> Vec<Option<bool>> {
let mut v: Vec<_> =
db.subscribers("show", group).unwrap().into_iter().map(|s| s.allow_explicit).collect();
v.sort();
v
};
assert_eq!(explicit(Some("group")), [Some(false), Some(true)], "ray inherits; sam's own choice on the show wins");
assert_eq!(explicit(None), [None, Some(false)], "outside a group nothing is inherited");
}
#[test] #[test]
fn enclosure_url_is_the_dedupe_key() { fn enclosure_url_is_the_dedupe_key() {
let db = Db::memory().unwrap(); let db = Db::memory().unwrap();

View File

@@ -223,7 +223,7 @@ enum Sniffed {
/// 2008 and so always answered 'data'. /// 2008 and so always answered 'data'.
async fn sniff(path: &Path) -> Result<Sniffed> { async fn sniff(path: &Path) -> Result<Sniffed> {
let head = read_head(path, 512).await?; let head = read_head(path, 512).await?;
if infer::is(&head, "torrent") || head.starts_with(b"d8:announce") || head.starts_with(b"d7:") { if head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
return Ok(Sniffed::Torrent); return Ok(Sniffed::Torrent);
} }
let text = String::from_utf8_lossy(&head); let text = String::from_utf8_lossy(&head);

View File

@@ -117,6 +117,103 @@ pub fn opml_title(bytes: &[u8]) -> Option<String> {
.filter(|t| !t.is_empty()) .filter(|t| !t.is_empty())
} }
/// The token and show of a Patreon feed link, or None for any other URL.
///
/// Patreon gives each patron one token per creator. With no show it stands for the creator,
/// whose feed carries every show at once.
fn patreon_parts(url: &str) -> Option<(String, Option<String>)> {
let u = url::Url::parse(url).ok()?;
if !matches!(u.host_str()?, "patreon.com" | "www.patreon.com") || !u.path().starts_with("/rss") {
return None;
}
let param = |name: &str| u.query_pairs().find(|(k, _)| k == name).map(|(_, v)| v.into_owned());
Some((param("auth")?, param("show")))
}
/// A Patreon link naming a creator but no show.
pub fn is_patreon_creator(url: &str) -> bool {
matches!(patreon_parts(url), Some((_, None)))
}
/// What was typed into Add feed, as a URL. A bare Patreon token is taken as its creator's
/// feed, since the token alone says whose it is.
pub fn expand_input(input: &str) -> String {
let s = input.trim();
let token = s.len() >= 20 && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_');
if token { format!("https://www.patreon.com/rss?auth={s}") } else { s.to_owned() }
}
/// Whether two URLs are the same feed. One Patreon show has several spellings -- by the
/// creator's name, by number, or with no creator at all -- and the token and show are what
/// identify it.
pub fn same_feed(a: &str, b: &str) -> bool {
a == b || patreon_parts(a).is_some_and(|p| Some(p) == patreon_parts(b))
}
/// A Patreon creator's name and shows, each show as (title, feed URL).
///
/// ponytail: Patreon's own web API, undocumented, asked without signing in. If it changes,
/// finding shows stops and the show feeds already found keep working. The documented API
/// needs an OAuth client per install and does not list shows.
pub async fn patreon_shows(
client: &reqwest::Client,
url: &str,
) -> Result<(Option<String>, Vec<(String, String)>)> {
// The creator feed names its campaign by number in its self link, a few hundred bytes in.
// The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
let mut resp = client.get(url).send().await.context("connecting")?;
if !resp.status().is_success() {
return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
}
let mut head = Vec::new();
while patreon_campaign(&head).is_none() && head.len() < 64 * 1024 {
let Some(chunk) = resp.chunk().await.context("reading the feed")? else { break };
head.extend_from_slice(&chunk);
}
let campaign = patreon_campaign(&head)
.ok_or_else(|| anyhow!("the Patreon feed does not say whose it is"))?;
let api = format!(
"https://www.patreon.com/api/campaigns/{campaign}\
?include=shows&fields%5Bcampaign%5D=name&fields%5Bcollection%5D=title"
);
let resp = client.get(api).send().await.context("asking Patreon for the shows")?;
if !resp.status().is_success() {
return Err(anyhow!("Patreon would not list the shows: HTTP {}", resp.status()));
}
let (name, shows) = parse_patreon_shows(&resp.bytes().await.context("reading the shows")?)?;
Ok((name, shows.into_iter().map(|(id, title)| (title, format!("{url}&show={id}"))).collect()))
}
/// The campaign number in the start of a Patreon feed.
fn patreon_campaign(head: &[u8]) -> Option<String> {
let text = String::from_utf8_lossy(head);
text.match_indices("patreon.com/rss/").find_map(|(i, m)| {
let id: String = text[i + m.len()..].chars().take_while(char::is_ascii_digit).collect();
(!id.is_empty()).then_some(id)
})
}
/// A campaign's name and its shows as (id, title), from Patreon's JSON:API answer.
fn parse_patreon_shows(json: &[u8]) -> Result<(Option<String>, Vec<(String, String)>)> {
let v: serde_json::Value = serde_json::from_slice(json).context("Patreon's answer is not JSON")?;
// Missing is not the same as none. Read as no shows, the creator feed would be scanned as
// a plain feed, claim every show's files, and leave the shows empty once the list returned.
let ids = v["data"]["relationships"]["shows"]["data"]
.as_array()
.ok_or_else(|| anyhow!("Patreon's answer does not list the shows"))?;
let title = |id: &str| -> Option<String> {
let show = v["included"].as_array()?.iter().find(|x| x["type"] == "collection" && x["id"] == id)?;
show["attributes"]["title"].as_str().map(|t| t.trim().to_owned())
};
let shows = ids
.iter()
.filter_map(|s| s["id"].as_str())
.map(|id| (id.to_owned(), title(id).unwrap_or_else(|| format!("Show {id}"))))
.collect();
Ok((v["data"]["attributes"]["name"].as_str().map(str::to_owned), shows))
}
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`. /// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> { pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
match rss::Channel::read_from(bytes) { match rss::Channel::read_from(bytes) {
@@ -550,6 +647,45 @@ mod tests {
assert!(!is_opml(include_bytes!("../tests/data/atom.xml"))); assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
} }
#[test]
fn a_patreon_creator_is_a_list_of_its_shows() {
let tok = "AbCdEfGhIjKlMnOpQrStUvWxYz012_-9";
assert_eq!(expand_input(&format!(" {tok} ")), format!("https://www.patreon.com/rss?auth={tok}"));
assert_eq!(expand_input("https://example.com/rss"), "https://example.com/rss");
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss/glasscannon?auth={tok}")));
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss?auth={tok}")));
assert!(!is_patreon_creator(&format!("https://www.patreon.com/rss/x?auth={tok}&show=1")), "one show is a feed");
assert!(!is_patreon_creator(&format!("https://example.com/rss?auth={tok}")));
// The show you already have by name is the one a bare token would add by number.
assert!(same_feed(
&format!("https://www.patreon.com/rss/glasscannon?auth={tok}&show=2073588"),
&format!("https://www.patreon.com/rss?auth={tok}&show=2073588"),
));
assert!(!same_feed(
&format!("https://www.patreon.com/rss?auth={tok}&show=1"),
&format!("https://www.patreon.com/rss?auth={tok}&show=2"),
));
// The self link carries the campaign by number, whichever spelling was asked for.
let head = br#"<rss><channel><link>https://www.patreon.com/glasscannon</link>
<atom:link href="https://www.patreon.com/rss/369921?auth=t" rel="self"/>"#;
assert_eq!(patreon_campaign(head).as_deref(), Some("369921"));
assert_eq!(patreon_campaign(b"<rss><channel><title>T"), None);
let json = br#"{"data":{"id":"369921","type":"campaign","attributes":{"name":"The Glass Cannon Network"},
"relationships":{"shows":{"data":[{"id":"2073588","type":"collection"},{"id":"2073636","type":"collection"}]}}},
"included":[{"id":"2073588","type":"collection","attributes":{"title":"Get in the Trunk "}},
{"id":"2073636","type":"collection","attributes":{"title":"Shadowdark"}}]}"#;
let (name, shows) = parse_patreon_shows(json).unwrap();
assert_eq!(name.as_deref(), Some("The Glass Cannon Network"));
assert_eq!(shows, [("2073588".into(), "Get in the Trunk".into()), ("2073636".into(), "Shadowdark".into())]);
// An answer that stops naming the shows is an error, never "this creator has none".
assert!(parse_patreon_shows(br#"{"data":{"attributes":{"name":"X"}}}"#).is_err());
}
#[test] #[test]
fn an_item_may_carry_several_enclosures() { fn an_item_may_carry_several_enclosures() {
// The rss crate keeps only one per item -- the last -- so these come from the XML. // The rss crate keeps only one per item -- the last -- so these come from the XML.

View File

@@ -62,10 +62,10 @@ impl Event {
Event::DownloadDone { path, .. } => format!(" saved {path}"), Event::DownloadDone { path, .. } => format!(" saved {path}"),
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"), Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
Event::Reaped { path, bytes } => { Event::Reaped { path, bytes } => {
format!("reap {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0) format!("deleted {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
} }
Event::ReapDone { files, bytes } => format!( Event::ReapDone { files, bytes } => format!(
"reaped {files} file(s), {:.1} MB", "deleted {files} old file(s), {:.1} MB",
*bytes as f64 / 1_048_576.0 *bytes as f64 / 1_048_576.0
), ),
Event::Status { feeds, pending, downloaded } => { Event::Status { feeds, pending, downloaded } => {

View File

@@ -111,18 +111,11 @@ impl Visit for Collect {
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) { fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
self.add(field, format!("{value:?}")); self.add(field, format!("{value:?}"));
} }
// Numbers and bools reach record_debug through the trait's defaults, which prints them the
// same way. A string would print quoted there, hence its own method.
fn record_str(&mut self, field: &Field, value: &str) { fn record_str(&mut self, field: &Field, value: &str) {
self.add(field, value.to_owned()); self.add(field, value.to_owned());
} }
fn record_i64(&mut self, field: &Field, value: i64) {
self.add(field, value.to_string());
}
fn record_u64(&mut self, field: &Field, value: u64) {
self.add(field, value.to_string());
}
fn record_bool(&mut self, field: &Field, value: bool) {
self.add(field, value.to_string());
}
} }
#[cfg(test)] #[cfg(test)]

View File

@@ -334,8 +334,7 @@ async fn daemon(
anyhow::bail!("a daemon is already listening on {}", socket.display()); anyhow::bail!("a daemon is already listening on {}", socket.display());
} }
// A database with nobody in it cannot be signed into, and an install that predates // A database with nobody in it cannot be signed into.
// accounts still has to serve its owner. Both get the same starting point.
if ctx.db.users()?.is_empty() { if ctx.db.users()?.is_empty() {
ctx.db.create_user("admin", Some(&crate::auth::hash_password(DEFAULT_PASSWORD)?), true)?; ctx.db.create_user("admin", Some(&crate::auth::hash_password(DEFAULT_PASSWORD)?), true)?;
tracing::warn!( tracing::warn!(
@@ -344,10 +343,13 @@ async fn daemon(
); );
} }
match migrate_opml_children(&ctx) { if let Some(admin) = ctx.db.users()?.into_iter().find(|u| u.is_admin) {
Ok(n) if n > 0 => tracing::info!(count = n, "moved OPML feeds out of config.toml into the database"), let catalogue: Vec<String> = ctx.cfg().feeds.keys().cloned().collect();
Ok(_) => {} match ctx.db.adopt_catalogue(admin.id, &catalogue) {
Err(e) => tracing::warn!(error = ?e, "could not tidy OPML feeds out of the config"), Ok(0) => {}
Ok(n) => tracing::info!(user = %admin.name, feeds = n, "subscribed the first admin to the catalogue"),
Err(e) => tracing::error!(error = %e, "could not subscribe the first admin to the catalogue"),
}
} }
match ctx.db.requeue_interrupted() { match ctx.db.requeue_interrupted() {
@@ -458,7 +460,7 @@ async fn start_web(
let mut fresh = (*cfg).clone(); let mut fresh = (*cfg).clone();
fresh.web.enabled = true; fresh.web.enabled = true;
fresh.web.bind = bind.clone(); fresh.web.bind = bind.clone();
fresh.web.token = web::generate_token(); fresh.web.token = crate::auth::new_session_token();
fresh.save(config_path)?; fresh.save(config_path)?;
ctx.reload_cfg(config_path)?; ctx.reload_cfg(config_path)?;
println!("web ui token generated. Open:\n http://{bind}/?token={}", fresh.web.token); println!("web ui token generated. Open:\n http://{bind}/?token={}", fresh.web.token);
@@ -507,8 +509,9 @@ async fn add(
keywords: Vec<String>, keywords: Vec<String>,
) -> Result<()> { ) -> Result<()> {
let mut cfg = (*ctx.cfg()).clone(); let mut cfg = (*ctx.cfg()).clone();
let url = &feed::expand_input(url);
// Includes feeds derived from an OPML, or the same show could be added twice. // Includes feeds derived from an OPML, or the same show could be added twice.
if let Some(existing) = subscriptions(ctx)?.iter().find(|s| s.cfg.url == url) { if let Some(existing) = subscriptions(ctx)?.iter().find(|s| feed::same_feed(&s.cfg.url, url)) {
anyhow::bail!("already subscribed as {:?}", existing.id); anyhow::bail!("already subscribed as {:?}", existing.id);
} }
let id = add_one(ctx, &mut cfg, url, folder, keywords).await?; let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
@@ -559,10 +562,17 @@ pub async fn add_one(
// Slugs must be unique across derived feeds too, or a new feed can collide with one // Slugs must be unique across derived feeds too, or a new feed can collide with one
// an OPML already introduced. // an OPML already introduced.
let taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)? let mut taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)?
.into_iter() .into_iter()
.map(|s| (s.id, s.cfg)) .map(|s| (s.id, s.cfg))
.collect(); .collect();
// A removed feed keeps its rows, so its id is only free again for the same feed: re-adding
// it gets its history back, and a different feed does not inherit someone else's.
for (id, other) in ctx.db.feed_urls()? {
if !feed::same_feed(&other, url) {
taken.entry(id).or_insert_with(|| probe.clone());
}
}
let id = config::unique_slug(&title, &taken); let id = config::unique_slug(&title, &taken);
cfg.feeds.insert(id.clone(), probe); cfg.feeds.insert(id.clone(), probe);
Ok(id) Ok(id)
@@ -592,44 +602,94 @@ fn rm(ctx: &Ctx, config_path: &std::path::Path, feed: &str) -> Result<()> {
} }
async fn import(ctx: &Ctx, config_path: &std::path::Path, file: &std::path::Path) -> Result<()> { async fn import(ctx: &Ctx, config_path: &std::path::Path, file: &std::path::Path) -> Result<()> {
let mut cfg = (*ctx.cfg()).clone();
let text = std::fs::read_to_string(file) let text = std::fs::read_to_string(file)
.with_context(|| format!("reading {}", file.display()))?; .with_context(|| format!("reading {}", file.display()))?;
let doc = opml::OPML::from_str(&text).map_err(|e| anyhow::anyhow!("parsing OPML: {e}"))?; // The CLI speaks for the operator, as the shared web token does.
let admin = ctx
.db
.users()?
.into_iter()
.find(|u| u.is_admin)
.ok_or_else(|| anyhow::anyhow!("no admin account to subscribe: ipx user add <name> --admin"))?;
let doc = opml::OPML::from_str(&text)
.map_err(|e| anyhow::anyhow!("{} is not OPML: {e}", file.display()))?;
let (added, had) = subscribe_opml(ctx, config_path, &doc, admin.id)?;
println!("subscribed {} to {added} feed(s); {had} already there", admin.name);
Ok(())
}
/// Subscribes one person to every feed in an OPML document, for the CLI and the web alike.
/// A feed already in the catalogue costs nothing; an unknown one is added under the OPML's
/// title rather than refetching each. Returns (newly subscribed, already subscribed).
///
/// Before accounts, importing only added unknown URLs to config.toml. Once subscriptions
/// decided what each person sees, that imported nothing at all for a feed someone else
/// already had, and a new one had no subscriber, so it was never scanned.
///
/// The caller parses the document, so each refuses a file that is not OPML in its own terms,
/// before anything is touched: a 400 from the web, a message from the CLI.
pub fn subscribe_opml(
ctx: &Ctx,
config_path: &std::path::Path,
doc: &opml::OPML,
user_id: i64,
) -> Result<(usize, usize)> {
let mut found = vec![]; let mut found = vec![];
collect_outlines(&doc.body.outlines, &mut found); collect_outlines(&doc.body.outlines, &mut found);
let mut added = 0; let known = subscriptions(ctx)?;
let mut cfg = (*ctx.cfg()).clone();
let mut ids = vec![];
let mut grew = false;
for (title, url) in found { for (title, url) in found {
if cfg.feeds.values().any(|f| f.url == url) { let existing = known
continue; .iter()
} .find(|s| s.cfg.url == url)
// Name it from the OPML title rather than refetching every feed. .map(|s| s.id.clone())
let id = config::unique_slug(&title, &cfg.feeds); // The same URL listed twice in one file.
cfg.feeds.insert( .or_else(|| cfg.feeds.iter().find(|(_, f)| f.url == url).map(|(id, _)| id.clone()));
id.clone(), let id = match existing {
config::Feed { Some(id) => id,
url, None => {
folder: None, let id = config::unique_slug(&title, &cfg.feeds);
group: None, cfg.feeds.insert(
media_types: None, id.clone(),
schedule: None, config::Feed {
keywords: vec![], url,
allow_explicit: false, folder: None,
auto_download: true, group: None,
max_new_per_check: None, media_types: None,
username: None, schedule: None,
password: None, keywords: vec![],
password_env: None, allow_explicit: false,
}, auto_download: true,
); max_new_per_check: None,
println!("added {id}"); username: None,
added += 1; password: None,
password_env: None,
},
);
grew = true;
id
}
};
ids.push(id);
} }
cfg.save(config_path)?; if grew {
println!("{added} feed(s) imported"); cfg.save(config_path)?;
Ok(()) ctx.reload_cfg(config_path)?;
}
let (mut added, mut had) = (0, 0);
for id in ids {
if ctx.db.subscription(user_id, &id)?.is_some() {
had += 1;
} else {
ctx.db.subscribe(user_id, &id)?;
added += 1;
}
}
Ok((added, had))
} }
/// OPML nests feeds inside folder outlines, so this walks the whole tree. /// OPML nests feeds inside folder outlines, so this walks the whole tree.
@@ -747,7 +807,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool) -> Result<()> {
ctx.out.emit(Event::FeedSkip { ctx.out.emit(Event::FeedSkip {
feed: id.clone(), feed: id.clone(),
reason: format!( reason: format!(
"OPML: {total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing", "{total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing",
added.len() added.len()
), ),
}); });
@@ -848,36 +908,6 @@ pub fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
Ok(out) Ok(out)
} }
/// Moves OPML children that older versions wrote into config.toml over to the database.
/// They were never yours to edit, and 80-odd of them made the file unreadable.
fn migrate_opml_children(ctx: &Ctx) -> Result<usize> {
let cfg = (*ctx.cfg()).clone();
let children: Vec<(String, config::Feed)> = cfg
.feeds
.iter()
.filter(|(_, f)| f.group.is_some())
.map(|(id, f)| (id.clone(), f.clone()))
.collect();
if children.is_empty() {
return Ok(0);
}
let mut fresh = cfg.clone();
for (id, f) in &children {
let group = f.group.clone().unwrap_or_default();
let title = ctx
.db
.feed_summary(id)
.ok()
.and_then(|s| s.title)
.unwrap_or_else(|| id.clone());
ctx.db.upsert_managed(id, &f.url, &title, &group)?;
fresh.feeds.remove(id);
}
fresh.save(&ctx.config_path)?;
ctx.reload_cfg(&ctx.config_path)?;
Ok(children.len())
}
/// Seconds to wait before re-checking a feed. /// Seconds to wait before re-checking a feed.
/// ///
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the /// A per-feed schedule is an explicit instruction and wins outright. Without one, the
@@ -903,7 +933,7 @@ struct Scan {
enum Outcome { enum Outcome {
NotModified, NotModified,
Feed(Scan), Feed(Scan),
/// The URL served an OPML document, so it is a subscription list rather than a feed. /// The URL is a list of feeds rather than a feed: an OPML, or a Patreon creator's shows.
Opml { added: Vec<String>, removed: usize, kept: usize, total: usize }, Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
} }
@@ -913,6 +943,31 @@ async fn scan_one(
feed_cfg: &config::Feed, feed_cfg: &config::Feed,
state: &db::HttpState, state: &db::HttpState,
) -> Result<Outcome> { ) -> Result<Outcome> {
// A Patreon creator with more than one show is a list of feeds, like an OPML.
if feed::is_patreon_creator(&feed_cfg.url) {
match feed::patreon_shows(&ctx.client, &feed_cfg.url).await {
Ok((name, shows)) if shows.len() > 1 => {
ctx.db.touch_feed(id, &feed_cfg.url)?;
if let Some(name) = name {
ctx.db.set_title(id, &name)?;
}
// Read as one feed before it was split, it listed every show's items in one
// heap. The items go; its files and read state move to each show as the show
// lists them (`Db::adopt`), so no show comes up empty for want of a URL.
ctx.db.clear_entries(id)?;
return sync_group(ctx, id, feed_cfg, &shows).await;
}
Ok(_) => {} // One show: the creator's feed is that show.
// Already split: keep the shows it has rather than read the creator as one heap.
Err(e) if ctx.db.managed_feeds()?.iter().any(|m| m.group_id == id) => return Err(e),
Err(e) => tracing::warn!(
feed = id,
error = %format!("{e:#}"),
"could not list the Patreon shows; reading it as one feed"
),
}
}
let mut fetched = feed::fetch( let mut fetched = feed::fetch(
&ctx.client, &ctx.client,
feed_cfg, feed_cfg,
@@ -957,29 +1012,45 @@ async fn scan_one(
parsed.image.as_deref(), parsed.image.as_deref(),
)?; )?;
let policy = policy_for(ctx, id, feed_cfg)?;
if let Some(parent) = &feed_cfg.group {
let listed: Vec<(&str, &str)> = parsed
.entries
.iter()
.flat_map(|e| e.enclosures.iter().map(move |x| (e.guid.as_str(), x.url.as_str())))
.collect();
ctx.db.adopt(parent, id, &listed)?;
}
// Verdicts are recorded in `state`, so the download queue below is just "everything still
// pending". A filter's verdict is looked at again on every scan, though: made once, at
// discovery, it outlived the setting behind it, and allowing explicit items afterwards
// changed nothing however often the feed was scanned.
let skipped = ctx.db.skipped_by_filter(id)?;
let mut scan = Scan::default(); let mut scan = Scan::default();
for entry in &parsed.entries { for entry in &parsed.entries {
if ctx.db.record_entry(id, entry)? { if ctx.db.record_entry(id, entry)? {
scan.new_entries += 1; scan.new_entries += 1;
} }
for enc in &entry.enclosures { for enc in &entry.enclosures {
if !ctx.db.record_enclosure(id, &entry.guid, enc)? { let was = if ctx.db.record_enclosure(id, &entry.guid, enc)? {
continue; // Seen before: downloaded, skipped or deliberately reaped. None
} } else if let Some(reason) = skipped.get(&enc.url) {
// Filters run once, at discovery, and are recorded in `state`. The download Some(reason.as_str())
// queue below is then just "everything still pending". } else {
if let Some(reason) = reject(&ctx.cfg(), feed_cfg, entry, enc) { continue; // Settled: queued, downloaded, reaped, or another feed's file.
ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?; };
let now = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc);
if now != was {
match now {
Some(reason) => ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?,
None => ctx.db.mark_enclosure(&enc.url, "pending", None)?,
}
} }
} }
} }
// An unset per-feed cap follows the global one; 0 there means unlimited. let budget = policy.budget;
let budget = feed_cfg.max_new_per_check.unwrap_or_else(|| { if policy.auto_download && budget > 0 {
let g = ctx.cfg().general.max_new_per_check;
if g == 0 { usize::MAX } else { g }
});
if feed_cfg.auto_download && budget > 0 {
let cfg = ctx.cfg(); let cfg = ctx.cfg();
let folder = download::folder_for(&cfg, id, feed_cfg, parsed.title.as_deref()); let folder = download::folder_for(&cfg, id, feed_cfg, parsed.title.as_deref());
let dest_dir = cfg.general.download_dir.join(&folder); let dest_dir = cfg.general.download_dir.join(&folder);
@@ -1057,12 +1128,6 @@ async fn scan_one(
Ok(Outcome::Feed(scan)) Ok(Outcome::Feed(scan))
} }
/// Brings the feed list in step with a subscribed OPML.
///
/// New entries are added under the OPML's group and folder. An entry that has gone from
/// the OPML is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
/// the UI to explain them.
async fn sync_opml( async fn sync_opml(
ctx: &Arc<Ctx>, ctx: &Arc<Ctx>,
parent_id: &str, parent_id: &str,
@@ -1073,25 +1138,44 @@ async fn sync_opml(
if let Some(title) = feed::opml_title(bytes) { if let Some(title) = feed::opml_title(bytes) {
ctx.db.set_title(parent_id, &title)?; ctx.db.set_title(parent_id, &title)?;
} }
sync_group(ctx, parent_id, parent, &listed).await
}
/// Brings the feed list in step with a list of feeds: a subscribed OPML, or a Patreon
/// creator's shows.
///
/// New entries are added under the list's group and folder. An entry that has gone from
/// the list is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
/// the UI to explain them.
async fn sync_group(
ctx: &Arc<Ctx>,
parent_id: &str,
parent: &config::Feed,
listed: &[(String, String)],
) -> Result<Outcome> {
let cfg = ctx.cfg(); let cfg = ctx.cfg();
let existing = ctx.db.managed_feeds()?; let existing = ctx.db.managed_feeds()?;
let mut added = vec![]; let mut added = vec![];
for (title, url) in &listed { for (title, url) in listed {
// Already known, whether derived or promoted into the config. // Already known, whether derived or promoted into the config.
if let Some(m) = existing.iter().find(|m| &m.url == url) { if let Some(m) = existing.iter().find(|m| &m.url == url) {
ctx.db.upsert_managed(&m.id, url, title, parent_id)?; ctx.db.upsert_managed(&m.id, url, title, parent_id)?;
continue; continue;
} }
if cfg.feeds.values().any(|f| &f.url == url) { // A Patreon show you added by hand may be spelled differently from the one listed.
if cfg.feeds.values().any(|f| feed::same_feed(&f.url, url)) {
continue; continue;
} }
// A removed feed keeps its rows, so its id is only free again for the same feed.
let known = ctx.db.feed_urls()?;
let taken: std::collections::BTreeMap<String, config::Feed> = cfg let taken: std::collections::BTreeMap<String, config::Feed> = cfg
.feeds .feeds
.keys() .keys()
.chain(existing.iter().map(|m| &m.id)) .chain(existing.iter().map(|m| &m.id))
.chain(added.iter()) .chain(added.iter())
.chain(known.iter().filter(|(_, u)| !feed::same_feed(u, url)).map(|(id, _)| id))
.map(|id| (id.clone(), parent.clone())) .map(|id| (id.clone(), parent.clone()))
.collect(); .collect();
let id = config::unique_slug(title, &taken); let id = config::unique_slug(title, &taken);
@@ -1099,6 +1183,23 @@ async fn sync_opml(
added.push(id); added.push(id);
} }
// Whoever subscribes to the OPML subscribes to what it lists: that is what taking a
// subscription means. Their own feeds are untouched.
for id in ctx
.db
.managed_feeds()?
.iter()
.filter(|m| m.group_id == parent_id)
.map(|m| m.id.clone())
.chain(std::iter::once(parent_id.to_string()))
{
for user in ctx.db.users()? {
if ctx.db.subscription(user.id, parent_id)?.is_some() {
ctx.db.subscribe(user.id, &id)?;
}
}
}
// Anything in this group the OPML no longer lists. // Anything in this group the OPML no longer lists.
let mut removed = 0; let mut removed = 0;
let mut kept = 0; let mut kept = 0;
@@ -1125,11 +1226,12 @@ async fn sync_opml(
fn reject( fn reject(
cfg: &config::Config, cfg: &config::Config,
feed_cfg: &config::Feed, feed_cfg: &config::Feed,
policy: &Policy,
entry: &feed::Entry, entry: &feed::Entry,
enc: &feed::Enclosure, enc: &feed::Enclosure,
) -> Option<&'static str> { ) -> Option<&'static str> {
let url = enc.url.as_str(); let url = enc.url.as_str();
if !feed_cfg.auto_download { if !policy.auto_download {
return Some("auto_download is off"); return Some("auto_download is off");
} }
// Blog feeds put the article's header image in an <enclosure>; without this a text // Blog feeds put the article's header image in an <enclosure>; without this a text
@@ -1141,7 +1243,7 @@ fn reject(
if !config::wanted_media(enc.mime.as_deref(), wanted) { if !config::wanted_media(enc.mime.as_deref(), wanted) {
return Some("not audio or video"); return Some("not audio or video");
} }
if entry.explicit && !feed_cfg.allow_explicit { if entry.explicit && !policy.allow_explicit {
return Some("explicit"); return Some("explicit");
} }
let categories = entry.categories.join(" "); let categories = entry.categories.join(" ");
@@ -1151,12 +1253,79 @@ fn reject(
entry.description.as_deref().unwrap_or(""), entry.description.as_deref().unwrap_or(""),
categories.as_str(), categories.as_str(),
]; ];
if !download::matches_keywords(&feed_cfg.keywords, &haystacks) { // One file serves everyone subscribed, so an item is wanted if it is wanted by
// anyone: any one person's keyword set matching is enough.
let wanted_by_someone = policy.keyword_sets.is_empty()
|| policy
.keyword_sets
.iter()
.any(|set| download::matches_keywords(set, &haystacks));
if !wanted_by_someone {
return Some("no keyword match"); return Some("no keyword match");
} }
None None
} }
/// What the scanner should do for a feed, merged across everyone subscribed to it. The
/// feed is fetched once and its files are downloaded once, so the merge is a union: if
/// one person wants a thing, it is fetched, and everyone else simply sees it listed.
///
/// With no subscribers at all -- a hand-written config entry nobody has claimed yet --
/// the feed's own settings stand, which is how a single-user install behaves.
pub struct Policy {
pub auto_download: bool,
pub allow_explicit: bool,
/// Empty means take everything. Otherwise one set per subscriber who filters.
pub keyword_sets: Vec<Vec<String>>,
pub budget: usize,
}
fn policy_for(ctx: &Ctx, id: &str, feed_cfg: &config::Feed) -> Result<Policy> {
let global = ctx.cfg().general.max_new_per_check;
Ok(merge_policy(&ctx.db.subscribers(id, feed_cfg.group.as_deref())?, feed_cfg, global))
}
fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Policy {
let cap = |n: Option<usize>| n.unwrap_or(if global == 0 { usize::MAX } else { global });
if subs.is_empty() {
return Policy {
auto_download: feed_cfg.auto_download,
allow_explicit: feed_cfg.allow_explicit,
keyword_sets: if feed_cfg.keywords.is_empty() {
vec![]
} else {
vec![feed_cfg.keywords.clone()]
},
budget: cap(feed_cfg.max_new_per_check),
};
}
let mut policy = Policy {
auto_download: false,
allow_explicit: false,
keyword_sets: vec![],
budget: 0,
};
for sub in subs {
if !sub.auto_download.unwrap_or(feed_cfg.auto_download) {
continue; // Not fetching for this person, so their wants add nothing.
}
policy.auto_download = true;
policy.allow_explicit |= sub.allow_explicit.unwrap_or(feed_cfg.allow_explicit);
policy.budget = policy
.budget
.max(cap(sub.max_new_per_check.map(|n| n as usize).or(feed_cfg.max_new_per_check)));
let kw = sub.keywords.clone().unwrap_or_else(|| feed_cfg.keywords.clone());
if kw.is_empty() {
// Somebody takes everything, so no filter can apply to the shared copy.
return Policy { keyword_sets: vec![], ..policy };
}
policy.keyword_sets.push(kw);
}
policy
}
async fn fetch_one( async fn fetch_one(
ctx: &Arc<Ctx>, ctx: &Arc<Ctx>,
feed_id: &str, feed_id: &str,
@@ -1352,3 +1521,79 @@ fn duration(secs: u64) -> String {
s => format!("{}d", s / 86_400), s => format!("{}d", s / 86_400),
} }
} }
#[cfg(test)]
mod tests {
use super::*;
fn feed() -> config::Feed {
// Whatever `ipx add` would write, which is the shape every code path sees.
let mut cfg = config::Config::default();
let f = add_one_cfg(&mut cfg, "http://x/f.xml", None, vec![]);
f
}
/// The feed entry `add` builds, without the network round trip it does for a title.
fn add_one_cfg(
_cfg: &mut config::Config,
url: &str,
folder: Option<String>,
keywords: Vec<String>,
) -> config::Feed {
config::Feed {
url: url.into(),
folder,
keywords,
allow_explicit: false,
auto_download: true,
group: None,
media_types: None,
schedule: None,
max_new_per_check: None,
username: None,
password: None,
password_env: None,
}
}
fn sub(kw: Option<&[&str]>, auto: Option<bool>, max: Option<i64>) -> db::Sub {
db::Sub {
feed_id: "f".into(),
keywords: kw.map(|k| k.iter().map(|s| s.to_string()).collect()),
auto_download: auto,
allow_explicit: None,
max_new_per_check: max,
}
}
#[test]
fn a_shared_feed_is_fetched_for_whoever_wants_the_most() {
// Nobody subscribed: the feed's own settings stand, as in a single-user install.
let p = merge_policy(&[], &feed(), 3);
assert!(p.auto_download);
assert_eq!(p.budget, 3);
assert!(p.keyword_sets.is_empty());
// Two filters: an item wanted by either of them is fetched, since one file serves
// both. The larger per-scan cap wins for the same reason.
let p = merge_policy(
&[sub(Some(&["rust"]), None, Some(2)), sub(Some(&["sqlite"]), None, Some(9))],
&feed(),
3,
);
assert_eq!(p.keyword_sets.len(), 2);
assert_eq!(p.budget, 9);
// One person taking everything removes the filter for the shared copy.
let p = merge_policy(&[sub(Some(&["rust"]), None, None), sub(Some(&[]), None, None)], &feed(), 3);
assert!(p.keyword_sets.is_empty());
// Everyone has auto-download off: nothing is fetched automatically.
let p = merge_policy(&[sub(None, Some(false), None), sub(None, Some(false), None)], &feed(), 3);
assert!(!p.auto_download);
// One of them wants it, so it is fetched.
let p = merge_policy(&[sub(None, Some(false), None), sub(None, Some(true), None)], &feed(), 3);
assert!(p.auto_download);
}
}

View File

@@ -150,33 +150,52 @@ mod tests {
} }
#[test] #[test]
fn query_never_offers_flagged_files_and_prefers_read_ones() { fn query_never_offers_a_file_anyone_starred_and_prefers_ones_everyone_read() {
// One file serves both subscribers, so it takes both of them to release it.
let db = Db::memory().unwrap(); let db = Db::memory().unwrap();
db.exec_for_test( db.exec_for_test(
"INSERT INTO entries (feed_id, guid, first_seen, read, flagged) VALUES "INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
('f', 'keep', 0, 1, 1), INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0);
('f', 'unread', 0, 0, 0), INSERT INTO entries (feed_id, guid, first_seen) VALUES
('f', 'read', 0, 1, 0); ('f', 'keep', 0),
('f', 'half', 0),
('f', 'unread', 0),
('f', 'read', 0);
-- Starred by one of the two, so it stays whatever the other thinks.
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
(1, 'f', 'keep', 1, 1),
(2, 'f', 'keep', 1, 0),
(1, 'f', 'half', 1, 0),
(1, 'f', 'read', 1, 0),
(2, 'f', 'read', 1, 0);
INSERT INTO enclosures (id, feed_id, guid, url, path, bytes_done, state, downloaded_at) VALUES INSERT INTO enclosures (id, feed_id, guid, url, path, bytes_done, state, downloaded_at) VALUES
(1, 'f', 'keep', 'u1', '/tmp/keep', 10, 'done', 10), (1, 'f', 'keep', 'u1', '/tmp/keep', 10, 'done', 10),
(2, 'f', 'unread', 'u2', '/tmp/unread', 10, 'done', 20), (2, 'f', 'half', 'u2', '/tmp/half', 10, 'done', 20),
(3, 'f', 'read', 'u3', '/tmp/read', 10, 'done', 30);", (3, 'f', 'unread', 'u3', '/tmp/unread', 10, 'done', 30),
(4, 'f', 'read', 'u4', '/tmp/read', 10, 'done', 40);",
) )
.unwrap(); .unwrap();
let got: Vec<i64> = db.reap_candidates().unwrap().iter().map(|c| c.id).collect(); let got: Vec<i64> = db.reap_candidates().unwrap().iter().map(|c| c.id).collect();
assert_eq!(got, vec![3, 2], "flagged excluded; read goes before unread"); assert_eq!(
got,
vec![4, 2, 3],
"starred by anyone is never offered; read by everyone goes first, and one \
person still having it unread keeps it back with the unread ones"
);
} }
#[test] #[test]
fn prune_keeps_entries_that_still_have_a_file() { fn prune_keeps_entries_that_still_have_a_file() {
let db = Db::memory().unwrap(); let db = Db::memory().unwrap();
db.exec_for_test( db.exec_for_test(
"INSERT INTO entries (feed_id, guid, first_seen, read, flagged) VALUES "INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
('f', 'has-file', 100, 1, 0), INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',1);
('f', 'no-file', 100, 1, 0), INSERT INTO entries (feed_id, guid, first_seen) VALUES
('f', 'flagged', 100, 1, 1), ('f', 'has-file', 100),
('f', 'recent', 900, 1, 0); ('f', 'no-file', 100),
('f', 'flagged', 100),
('f', 'recent', 900);
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
(1, 'f', 'has-file', 'u1', '/tmp/x', 'done');", (1, 'f', 'has-file', 'u1', '/tmp/x', 'done');",
) )

View File

@@ -12,9 +12,7 @@ use axum::{
}, },
routing::{delete, get, patch, post}, routing::{delete, get, patch, post},
}; };
use futures_util::StreamExt;
use serde::Deserialize; use serde::Deserialize;
use tokio_stream::wrappers::BroadcastStream;
use tower::ServiceExt; use tower::ServiceExt;
use tower_http::services::ServeFile; use tower_http::services::ServeFile;
use serde::Serialize; use serde::Serialize;
@@ -35,28 +33,6 @@ pub struct WebState {
pub events: broadcast::Sender<Event>, pub events: broadcast::Sender<Event>,
} }
/// A 32-hex-character shared secret, generated when config.toml has none.
///
/// ponytail: /dev/urandom rather than a CSPRNG crate -- 16 bytes, once, on a Unix-only
/// binary. Falls back to the clock only if urandom is somehow unreadable, which would be a
/// weak token, so that case is logged loudly.
pub fn generate_token() -> String {
use std::io::Read;
let mut bytes = [0u8; 16];
match std::fs::File::open("/dev/urandom").and_then(|mut f| f.read_exact(&mut bytes)) {
Ok(()) => {}
Err(e) => {
tracing::error!(error = %e, "could not read /dev/urandom; token is NOT secure");
let n = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0);
bytes[..8].copy_from_slice(&n.to_le_bytes());
}
}
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
pub fn router(state: WebState) -> Router { pub fn router(state: WebState) -> Router {
Router::new() Router::new()
.route("/", get(index)) .route("/", get(index))
@@ -65,15 +41,22 @@ pub fn router(state: WebState) -> Router {
.route("/api/feeds", get(feeds).post(add_feed)) .route("/api/feeds", get(feeds).post(add_feed))
.route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed)) .route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
.route("/api/feeds/{id}/entries", get(entries)) .route("/api/feeds/{id}/entries", get(entries))
.route("/api/entries", get(all_entries))
.route("/api/entries/{feed_id}/{guid}/flags", post(set_flags)) .route("/api/entries/{feed_id}/{guid}/flags", post(set_flags))
.route("/api/entries/{feed_id}/{guid}/position", post(set_position)) .route("/api/entries/{feed_id}/{guid}/position", post(set_position))
.route("/api/feeds/{id}/read-all", post(read_all)) .route("/api/feeds/{id}/read-all", post(read_all))
.route("/api/read-all", post(read_all_mine))
.route("/api/feeds/{id}/download-latest", post(download_latest)) .route("/api/feeds/{id}/download-latest", post(download_latest))
.route("/api/enclosures/{id}/download", post(download_now)) .route("/api/enclosures/{id}/download", post(download_now))
.route("/api/enclosures/{id}", delete(delete_file)) .route("/api/enclosures/{id}", delete(delete_file))
.route("/api/fetch", post(fetch_now)) .route("/api/fetch", post(fetch_now))
.route("/api/opml", get(export_opml).post(import_opml)) .route("/api/opml", get(export_opml).post(import_opml))
.route("/api/settings", get(get_settings).patch(patch_settings)) .route("/api/settings", get(get_settings).patch(patch_settings))
.route("/api/popular", get(get_popular))
.route("/api/directory", get(get_directory))
.route("/api/popular/{id}", post(subscribe_popular))
.route("/api/users", get(list_users).post(add_user))
.route("/api/users/{id}", patch(patch_user).delete(remove_user))
.route("/api/logs", get(logs)) .route("/api/logs", get(logs))
.route("/api/events", get(events)) .route("/api/events", get(events))
.route("/media/{id}", get(media)) .route("/media/{id}", get(media))
@@ -81,6 +64,7 @@ pub fn router(state: WebState) -> Router {
// Signing in cannot require being signed in, so these sit outside the auth layer. // Signing in cannot require being signed in, so these sit outside the auth layer.
.route("/login", get(login_page)) .route("/login", get(login_page))
.route("/api/login", post(login)) .route("/api/login", post(login))
.route("/icon.png", get(icon))
.layer(middleware::from_fn(access_log)) .layer(middleware::from_fn(access_log))
.with_state(state) .with_state(state)
} }
@@ -303,10 +287,138 @@ async fn me(user: crate::db::User) -> Json<serde_json::Value> {
Json(serde_json::json!({ "name": user.name, "admin": user.is_admin })) Json(serde_json::json!({ "name": user.name, "admin": user.is_admin }))
} }
// ---- accounts: admin only ----
fn require_admin(user: &crate::db::User) -> Result<(), ApiError> {
if user.is_admin {
Ok(())
} else {
Err(ApiError::forbidden("only an admin manages accounts"))
}
}
/// Demoting or removing this account would leave nobody able to manage anyone, and the only
/// way back would be `ipx user` on the box.
fn last_admin(users: &[crate::db::User], id: i64) -> bool {
let admins: Vec<i64> = users.iter().filter(|u| u.is_admin).map(|u| u.id).collect();
admins == [id]
}
async fn list_users(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<serde_json::Value>, ApiError> {
require_admin(&user)?;
let users: Vec<_> = state
.ctx
.db
.users()?
.iter()
.map(|u| {
serde_json::json!({
"id": u.id, "name": u.name, "admin": u.is_admin, "password": u.pass_hash.is_some(),
})
})
.collect();
Ok(Json(serde_json::json!(users)))
}
#[derive(Deserialize)]
struct NewUser {
name: String,
#[serde(default)]
password: String,
#[serde(default)]
admin: bool,
}
async fn add_user(
State(state): State<WebState>,
user: crate::db::User,
Json(body): Json<NewUser>,
) -> Result<StatusCode, ApiError> {
require_admin(&user)?;
// The same rules as a name a proxy vouches for, so either way of signing in finds it.
let name = crate::auth::name_from_header(&body.name).ok_or_else(|| {
ApiError::bad_request("a name is required, without commas, semicolons or line breaks")
})?;
if state.ctx.db.user_by_name(&name)?.is_some() {
return Err(ApiError::bad_request(format!("{name} already exists")));
}
// No password is someone the proxy signs in, as with `ipx user add --no-password`.
let hash = if body.password.is_empty() {
None
} else {
Some(crate::auth::hash_password(&body.password).map_err(|e| ApiError::bad_request(format!("{e:#}")))?)
};
state.ctx.db.create_user(&name, hash.as_deref(), body.admin)?;
tracing::info!(by = %user.name, user = %name, admin = body.admin, "account added");
Ok(StatusCode::CREATED)
}
#[derive(Deserialize)]
struct UserPatch {
admin: bool,
}
async fn patch_user(
State(state): State<WebState>,
user: crate::db::User,
Path(id): Path<i64>,
Json(body): Json<UserPatch>,
) -> Result<StatusCode, ApiError> {
require_admin(&user)?;
let users = state.ctx.db.users()?;
let target = users
.iter()
.find(|u| u.id == id)
.ok_or_else(|| ApiError::bad_request(format!("no account with id {id}")))?;
if !body.admin && last_admin(&users, id) {
return Err(ApiError::bad_request(format!(
"{} is the only admin; make someone else an admin first",
target.name
)));
}
state.ctx.db.set_admin(id, body.admin)?;
tracing::info!(by = %user.name, user = %target.name, admin = body.admin, "admin changed");
Ok(StatusCode::NO_CONTENT)
}
async fn remove_user(
State(state): State<WebState>,
user: crate::db::User,
Path(id): Path<i64>,
) -> Result<StatusCode, ApiError> {
require_admin(&user)?;
let users = state.ctx.db.users()?;
let target = users
.iter()
.find(|u| u.id == id)
.ok_or_else(|| ApiError::bad_request(format!("no account with id {id}")))?;
if last_admin(&users, id) {
return Err(ApiError::bad_request(format!(
"{} is the only admin; make someone else an admin first",
target.name
)));
}
state.ctx.db.delete_user(id)?;
tracing::info!(by = %user.name, user = %target.name, "account removed");
Ok(StatusCode::NO_CONTENT)
}
async fn login_page() -> Html<&'static str> { async fn login_page() -> Html<&'static str> {
Html(include_str!("../web/login.html")) Html(include_str!("../web/login.html"))
} }
/// The 2004 icon, served once for both pages rather than inlined as base64 into each. The
/// sign-in page shows it, so it sits outside the auth layer with /login.
async fn icon() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "image/png"), (header::CACHE_CONTROL, "max-age=86400")],
include_bytes!("../web/ipodderx-icon.png").as_slice(),
)
}
fn constant_time_eq(a: &str, b: &str) -> bool { fn constant_time_eq(a: &str, b: &str) -> bool {
let (a, b) = (a.as_bytes(), b.as_bytes()); let (a, b) = (a.as_bytes(), b.as_bytes());
if a.len() != b.len() { if a.len() != b.len() {
@@ -346,15 +458,33 @@ struct FeedRow {
entries: i64, entries: i64,
downloaded: i64, downloaded: i64,
unread: i64, unread: i64,
/// Including you. More than one means every file here is shared.
subscribers: i64,
} }
async fn feeds(State(state): State<WebState>) -> Result<Json<Vec<FeedRow>>, ApiError> { async fn feeds(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<Vec<FeedRow>>, ApiError> {
let cfg = state.ctx.cfg(); let cfg = state.ctx.cfg();
// Config entries plus the feeds derived from OPML subscriptions. // Config entries plus the feeds derived from OPML subscriptions -- the catalogue.
// What comes back is only the part of it this person subscribes to.
let subs = crate::subscriptions(&state.ctx)?; let subs = crate::subscriptions(&state.ctx)?;
let mut out = Vec::with_capacity(subs.len()); let mine: std::collections::HashMap<String, crate::db::Sub> = state
.ctx
.db
.subscriptions_for(user.id)?
.into_iter()
.map(|s| (s.feed_id.clone(), s))
.collect();
let counts = state.ctx.db.subscriber_counts()?;
let mut out = Vec::with_capacity(mine.len());
for sub in &subs { for sub in &subs {
let (id, feed) = (&sub.id, &sub.cfg); let (id, feed) = (&sub.id, &sub.cfg);
// In a group, what you have not set on the feed comes from your settings on the group,
// the same fallback the scanner uses (`Db::subscribers`).
let up = feed.group.as_deref().and_then(|g| mine.get(g));
let Some(mine) = mine.get(id) else { continue };
let s = state.ctx.db.feed_summary(id)?; let s = state.ctx.db.feed_summary(id)?;
let st = state.ctx.db.http_state(id)?; let st = state.ctx.db.http_state(id)?;
out.push(FeedRow { out.push(FeedRow {
@@ -363,10 +493,24 @@ async fn feeds(State(state): State<WebState>) -> Result<Json<Vec<FeedRow>>, ApiE
title: s.title, title: s.title,
image: s.image, image: s.image,
folder: feed.folder.clone(), folder: feed.folder.clone(),
keywords: feed.keywords.clone(), keywords: mine
allow_explicit: feed.allow_explicit, .keywords
auto_download: feed.auto_download, .clone()
max_new_per_check: feed.max_new_per_check, .or_else(|| up.and_then(|u| u.keywords.clone()))
.unwrap_or_else(|| feed.keywords.clone()),
allow_explicit: mine
.allow_explicit
.or(up.and_then(|u| u.allow_explicit))
.unwrap_or(feed.allow_explicit),
auto_download: mine
.auto_download
.or(up.and_then(|u| u.auto_download))
.unwrap_or(feed.auto_download),
max_new_per_check: mine
.max_new_per_check
.or(up.and_then(|u| u.max_new_per_check))
.map(|n| n as usize)
.or(feed.max_new_per_check),
group: feed.group.clone(), group: feed.group.clone(),
orphaned: s.orphaned, orphaned: s.orphaned,
// Derived from an OPML and not written to config until you change something. // Derived from an OPML and not written to config until you change something.
@@ -384,12 +528,118 @@ async fn feeds(State(state): State<WebState>) -> Result<Json<Vec<FeedRow>>, ApiE
last_error: s.last_error, last_error: s.last_error,
entries: s.entries, entries: s.entries,
downloaded: s.downloaded, downloaded: s.downloaded,
unread: state.ctx.db.unread_count(id)?, unread: state.ctx.db.unread_count(user.id, id)?,
subscribers: counts.get(id).copied().unwrap_or(0),
}); });
} }
Ok(Json(out)) Ok(Json(out))
} }
// ---- popular on this server ----
/// A feed that carries a credential is someone's paid or private subscription. Listing it
/// would let anyone signed in subscribe to it and read what they pay for.
///
/// ponytail: a heuristic. A key hidden in the path of a host not in `PAID_HOSTS` gets
/// through; a per-feed `unlisted` flag is the upgrade if that happens again.
fn looks_private(feed: &crate::config::Feed) -> bool {
if feed.username.is_some() || feed.password.is_some() || feed.password_env.is_some() {
return true;
}
let Ok(u) = url::Url::parse(&feed.url) else { return true };
// Paid-feed services put the subscriber's key in the path as often as in the query, and a
// long path segment alone proves nothing (acast's public show ids look the same). So a
// feed from one of them is private whatever its URL looks like. Supercast is why this
// exists: `feeds.supercast.com/feeds/<key>` reached the directory before it did.
let host = u.host_str().unwrap_or("");
PAID_HOSTS.iter().any(|h| host == *h || host.ends_with(&format!(".{h}")))
|| !u.username().is_empty()
|| u.password().is_some()
|| u.query_pairs().any(|(k, _)| {
let k = k.to_ascii_lowercase();
["auth", "token", "key", "secret", "pass", "sig", "session", "user", "uid"]
.iter()
.any(|w| k.contains(w))
})
}
/// Services whose feeds are always one subscriber's own.
const PAID_HOSTS: &[&str] =
&["patreon.com", "supercast.com", "supportingcast.fm", "glow.fm", "memberful.com"];
/// Only an id, a title, artwork and a count: never a URL, which is where a key would be.
#[derive(Serialize)]
struct PopularRow {
id: String,
title: Option<String>,
image: Option<String>,
subscribers: i64,
/// Yours already. Everyone counts, you included, so your own feeds are listed too.
subscribed: bool,
}
/// Every feed that may be listed, with everyone counted, you included, most subscribers
/// first. Popular is the top of it, the directory is all of it, and it is all that
/// `subscribe_popular` will subscribe you to.
fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
let db = &state.ctx.db;
let mine: std::collections::HashSet<String> =
db.subscriptions_for(user_id)?.into_iter().map(|s| s.feed_id).collect();
let counts = db.subscriber_counts()?;
let mut out = vec![];
for s in crate::subscriptions(&state.ctx)? {
let n = counts.get(&s.id).copied().unwrap_or(0);
// A feed from an OPML rides on the OPML: everyone subscribed to it counts every feed
// inside, which would bury everything anyone chose on purpose.
let from_opml = s.managed || s.cfg.group.is_some();
if n == 0 || from_opml || looks_private(&s.cfg) {
continue;
}
let sum = db.feed_summary(&s.id)?;
let subscribed = mine.contains(&s.id);
out.push(PopularRow { id: s.id, title: sum.title, image: sum.image, subscribers: n, subscribed });
}
out.sort_by(|a, b| b.subscribers.cmp(&a.subscribers).then_with(|| sort_name(a).cmp(&sort_name(b))));
Ok(out)
}
async fn get_popular(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<Vec<PopularRow>>, ApiError> {
let mut rows = popular(&state, user.id)?;
rows.truncate(10);
Ok(Json(rows))
}
/// Every feed that may be listed, A to Z.
async fn get_directory(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<Vec<PopularRow>>, ApiError> {
let mut rows = popular(&state, user.id)?;
rows.sort_by_key(sort_name);
Ok(Json(rows))
}
fn sort_name(p: &PopularRow) -> String {
p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase()
}
/// Subscribes by id, because the list never shows a URL. Checked against the same list, so
/// a guessed id cannot reach a private feed.
async fn subscribe_popular(
State(state): State<WebState>,
user: crate::db::User,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, ApiError> {
if !popular(&state, user.id)?.iter().any(|p| p.id == id) {
return Err(ApiError::bad_request(format!("{id:?} is not on the popular list")));
}
state.ctx.db.subscribe(user.id, &id)?;
Ok(Json(serde_json::json!({ "id": id })))
}
/// Validates a replacement feed URL: present, parseable, and not already subscribed under /// Validates a replacement feed URL: present, parseable, and not already subscribed under
/// a different id. Returns the trimmed URL. /// a different id. Returns the trimmed URL.
fn check_url( fn check_url(
@@ -454,6 +704,47 @@ impl IntoResponse for ApiError {
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn a_feed_with_a_credential_is_never_popular() {
let f = |url: &str| crate::config::Feed {
url: url.into(),
folder: None,
group: None,
media_types: None,
schedule: None,
keywords: vec![],
allow_explicit: false,
auto_download: true,
max_new_per_check: None,
username: None,
password: None,
password_env: None,
};
assert!(!looks_private(&f("https://feeds.twit.tv/twit.xml")));
assert!(!looks_private(&f("https://example.com/rss?format=mp3")));
// Patreon's shape: the key is a query parameter.
assert!(looks_private(&f("https://www.patreon.com/rss/x?auth=abc123&show=2073588")));
assert!(looks_private(&f("https://example.com/rss?api_key=abc")));
// Paid-feed services put the key in the path; the host gives them away.
assert!(looks_private(&f("https://feeds.supercast.com/feeds/abcdefghijklmnopqrstuvwx")));
assert!(looks_private(&f("https://someshow.supportingcast.fm/content/abc123.rss")));
// A long path segment alone is not a key: acast's public show ids look the same.
assert!(!looks_private(&f("https://feeds.acast.com/public/shows/0123456789abcdef01234567")));
assert!(looks_private(&f("https://ray:hunter2@example.com/rss")));
assert!(looks_private(&f("not a url")), "unparseable is not safe to list");
let mut basic = f("https://example.com/rss");
basic.username = Some("ray".into());
assert!(looks_private(&basic), "a feed with a login configured");
}
#[test]
fn only_the_last_admin_is_protected() {
let u = |id, is_admin| crate::db::User { id, name: format!("u{id}"), pass_hash: None, is_admin };
assert!(last_admin(&[u(1, true), u(2, false)], 1));
assert!(!last_admin(&[u(1, true), u(2, true)], 1), "another admin remains");
assert!(!last_admin(&[u(1, true), u(2, false)], 2), "not an admin at all");
}
#[test] #[test]
fn token_comparison_rejects_mismatches_and_length_differences() { fn token_comparison_rejects_mismatches_and_length_differences() {
assert!(constant_time_eq("abc123", "abc123")); assert!(constant_time_eq("abc123", "abc123"));
@@ -512,15 +803,6 @@ mod tests {
"another feed already has that URL" "another feed already has that URL"
); );
} }
#[test]
fn generated_tokens_are_32_hex_chars_and_not_repeated() {
let a = generate_token();
let b = generate_token();
assert_eq!(a.len(), 32);
assert!(a.chars().all(|c| c.is_ascii_hexdigit()));
assert_ne!(a, b);
}
} }
#[derive(Deserialize)] #[derive(Deserialize)]
@@ -533,6 +815,12 @@ struct Page {
filter: Option<String>, filter: Option<String>,
#[serde(default)] #[serde(default)]
q: Option<String>, q: Option<String>,
/// A column name and asc or desc. Anything unrecognised is newest first: the name picks a
/// fixed expression in the query and never reaches it itself.
#[serde(default)]
sort: Option<String>,
#[serde(default)]
dir: Option<String>,
} }
fn fifty() -> i64 { fn fifty() -> i64 {
@@ -548,21 +836,44 @@ struct EntryPage {
async fn entries( async fn entries(
State(state): State<WebState>, State(state): State<WebState>,
Path(id): Path<String>, Path(id): Path<String>,
user: crate::db::User,
Query(page): Query<Page>, Query(page): Query<Page>,
) -> Result<Json<EntryPage>, ApiError> {
entry_page(&state, user.id, Some(&id), &page)
}
/// Every subscribed feed's items together, newest first: All Subscriptions.
async fn all_entries(
State(state): State<WebState>,
user: crate::db::User,
Query(page): Query<Page>,
) -> Result<Json<EntryPage>, ApiError> {
entry_page(&state, user.id, None, &page)
}
/// One feed's page of items, or every subscribed feed's when `feed` is None.
fn entry_page(
state: &WebState,
user_id: i64,
feed: Option<&str>,
page: &Page,
) -> Result<Json<EntryPage>, ApiError> { ) -> Result<Json<EntryPage>, ApiError> {
let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all")); let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all"));
let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty()); let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty());
let mut rows = state let db = &state.ctx.db;
.ctx let order = crate::db::order_sql(
.db page.sort.as_deref().unwrap_or("published"),
.entries(&id, filter, search, page.offset, page.limit.clamp(1, 200))?; page.dir.as_deref().unwrap_or("desc"),
);
let mut rows =
db.entries_in(user_id, feed, filter, search, page.offset, page.limit.clamp(1, 200), &order)?;
// Feed HTML is untrusted: it reaches the page only after ammonia has been through it. // Feed HTML is untrusted: it reaches the page only after ammonia has been through it.
for row in &mut rows { for row in &mut rows {
if let Some(d) = &row.description { if let Some(d) = &row.description {
row.description = Some(ammonia::clean(d)); row.description = Some(ammonia::clean(d));
} }
} }
let total = state.ctx.db.count_entries(&id, filter, search)?; let total = db.count_in(user_id, feed, filter, search)?;
Ok(Json(EntryPage { total, entries: rows })) Ok(Json(EntryPage { total, entries: rows }))
} }
@@ -573,26 +884,63 @@ struct NewFeed {
folder: Option<String>, folder: Option<String>,
#[serde(default)] #[serde(default)]
keywords: Vec<String>, keywords: Vec<String>,
#[serde(default)]
allow_explicit: bool,
}
/// The Add feed dialog's explicit box. Like everything on a feed's own dialog it is yours, so it
/// goes on your subscription, and before the first scan, which would otherwise skip every
/// explicit item.
fn explicit_on_add(state: &WebState, user_id: i64, feed_id: &str, allow: bool) -> Result<(), ApiError> {
if allow {
let sub = crate::db::Sub { feed_id: feed_id.to_owned(), allow_explicit: Some(true), ..Default::default() };
state.ctx.db.set_subscription(user_id, &sub)?;
}
Ok(())
} }
async fn add_feed( async fn add_feed(
State(state): State<WebState>, State(state): State<WebState>,
user: crate::db::User,
Json(body): Json<NewFeed>, Json(body): Json<NewFeed>,
) -> Result<Json<serde_json::Value>, ApiError> { ) -> Result<Json<serde_json::Value>, ApiError> {
let mut cfg = (*state.ctx.cfg()).clone(); let mut cfg = (*state.ctx.cfg()).clone();
// Derived feeds count as subscribed: adding one an OPML already lists would duplicate it. let url = crate::feed::expand_input(&body.url);
// Someone else may already have it. Then adding costs nothing: no second fetch, no
// second copy on disk, just another name against the same feed.
if let Some(existing) = crate::subscriptions(&state.ctx)? if let Some(existing) = crate::subscriptions(&state.ctx)?
.into_iter() .into_iter()
.find(|s| s.cfg.url == body.url) .find(|s| crate::feed::same_feed(&s.cfg.url, &url))
{ {
return Ok(Json(serde_json::json!({ "id": existing.id, "existing": true }))); let already = state.ctx.db.subscription(user.id, &existing.id)?.is_some();
state.ctx.db.subscribe(user.id, &existing.id)?;
if !already {
explicit_on_add(&state, user.id, &existing.id, body.allow_explicit)?;
}
scan_soon(&state, Some(existing.id.clone())).await;
return Ok(Json(
serde_json::json!({ "id": existing.id, "existing": already }),
));
} }
let id = crate::add_one(&state.ctx, &mut cfg, &body.url, body.folder, body.keywords).await?; let id = crate::add_one(&state.ctx, &mut cfg, &url, body.folder, body.keywords).await?;
cfg.save(&state.config_path)?; cfg.save(&state.config_path)?;
state.ctx.reload_cfg(&state.config_path)?; state.ctx.reload_cfg(&state.config_path)?;
state.ctx.db.subscribe(user.id, &id)?;
explicit_on_add(&state, user.id, &id, body.allow_explicit)?;
scan_soon(&state, Some(id.clone())).await;
Ok(Json(serde_json::json!({ "id": id, "existing": false }))) Ok(Json(serde_json::json!({ "id": id, "existing": false })))
} }
/// Queues a scan, so a feed just added shows its items without anyone pressing Scan now.
/// `None` scans whatever is due, which a feed never checked always is. The add has
/// succeeded either way, so a daemon not taking commands is only logged.
async fn scan_soon(state: &WebState, feed: Option<String>) {
let force = feed.is_some();
if state.cmds.send(Command::Fetch { feed, force }).await.is_err() {
tracing::warn!("could not queue a scan: the daemon is not accepting commands");
}
}
/// Absent means "leave alone"; JSON `null` means "clear this". /// Absent means "leave alone"; JSON `null` means "clear this".
/// ///
/// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a /// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a
@@ -626,10 +974,46 @@ async fn patch_feed(
user: crate::db::User, user: crate::db::User,
Json(body): Json<FeedPatch>, Json(body): Json<FeedPatch>,
) -> Result<StatusCode, ApiError> { ) -> Result<StatusCode, ApiError> {
// How often a feed is polled is the operator's call: it costs bandwidth, it is what // What one person wants -- which items, whether to fetch them, how many at a time --
// publishers notice, and one impatient setting affects everyone reading the feed. // is theirs. It goes on their subscription and nobody else sees the change.
if body.schedule.is_some() && !user.is_admin { if state.ctx.db.subscription(user.id, &id)?.is_some() {
return Err(ApiError::forbidden("only an admin sets when feeds are scanned")); let mut mine = state
.ctx
.db
.subscription(user.id, &id)?
.unwrap_or_else(|| crate::db::Sub { feed_id: id.clone(), ..Default::default() });
let mut touched = false;
if let Some(v) = body.keywords.clone() {
mine.keywords = Some(v.into_iter().filter(|k| !k.trim().is_empty()).collect());
touched = true;
}
if let Some(v) = body.allow_explicit {
mine.allow_explicit = Some(v);
touched = true;
}
if let Some(v) = body.auto_download {
mine.auto_download = Some(v);
touched = true;
}
if let Some(v) = body.max_new_per_check {
mine.max_new_per_check = v.map(|n| n as i64);
touched = true;
}
if touched {
state.ctx.db.set_subscription(user.id, &mine)?;
}
}
// The rest describes the feed itself -- where its files land, its address, when it is
// polled -- and there is one of those however many people read it.
let feed_level = body.url.is_some() || body.folder.is_some() || body.schedule.is_some();
if !feed_level {
return Ok(StatusCode::NO_CONTENT);
}
if !user.is_admin {
return Err(ApiError::forbidden(
"the feed's address, folder and schedule are the same for everyone, so only an admin changes them",
));
} }
let mut cfg = (*state.ctx.cfg()).clone(); let mut cfg = (*state.ctx.cfg()).clone();
@@ -675,18 +1059,6 @@ async fn patch_feed(
if let Some(v) = body.folder { if let Some(v) = body.folder {
feed.folder = v.filter(|s| !s.trim().is_empty()); feed.folder = v.filter(|s| !s.trim().is_empty());
} }
if let Some(v) = body.keywords {
feed.keywords = v.into_iter().filter(|k| !k.trim().is_empty()).collect();
}
if let Some(v) = body.allow_explicit {
feed.allow_explicit = v;
}
if let Some(v) = body.auto_download {
feed.auto_download = v;
}
if let Some(v) = body.max_new_per_check {
feed.max_new_per_check = v;
}
cfg.save(&state.config_path)?; cfg.save(&state.config_path)?;
state.ctx.reload_cfg(&state.config_path)?; state.ctx.reload_cfg(&state.config_path)?;
if url_changed { if url_changed {
@@ -700,7 +1072,23 @@ async fn patch_feed(
async fn remove_feed( async fn remove_feed(
State(state): State<WebState>, State(state): State<WebState>,
Path(id): Path<String>, Path(id): Path<String>,
user: crate::db::User,
) -> Result<StatusCode, ApiError> { ) -> Result<StatusCode, ApiError> {
// Unsubscribing is personal: it takes the feed off your list and leaves everyone
// else's alone.
state.ctx.db.unsubscribe(user.id, &id)?;
for child in crate::subscriptions(&state.ctx)?
.iter()
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
{
state.ctx.db.unsubscribe(user.id, &child.id)?;
}
if state.ctx.db.subscriber_count(&id)? > 0 {
return Ok(StatusCode::NO_CONTENT);
}
// Nobody is left: the feed stops being scanned. Its files and history stay, so if
// someone subscribes again they do not pull the back catalogue a second time.
let mut cfg = (*state.ctx.cfg()).clone(); let mut cfg = (*state.ctx.cfg()).clone();
if cfg.feeds.remove(&id).is_none() { if cfg.feeds.remove(&id).is_none() {
// A derived feed: forget it here, though the OPML will list it again on the next // A derived feed: forget it here, though the OPML will list it again on the next
@@ -708,7 +1096,6 @@ async fn remove_feed(
state.ctx.db.drop_managed(&id)?; state.ctx.db.drop_managed(&id)?;
return Ok(StatusCode::NO_CONTENT); return Ok(StatusCode::NO_CONTENT);
} }
// Downloads and history stay, so re-adding does not re-pull the back catalogue.
cfg.save(&state.config_path)?; cfg.save(&state.config_path)?;
state.ctx.reload_cfg(&state.config_path)?; state.ctx.reload_cfg(&state.config_path)?;
Ok(StatusCode::NO_CONTENT) Ok(StatusCode::NO_CONTENT)
@@ -723,14 +1110,15 @@ struct Flags {
async fn set_flags( async fn set_flags(
State(state): State<WebState>, State(state): State<WebState>,
Path((feed_id, guid)): Path<(String, String)>, Path((feed_id, guid)): Path<(String, String)>,
user: crate::db::User,
Json(body): Json<Flags>, Json(body): Json<Flags>,
) -> Result<StatusCode, ApiError> { ) -> Result<StatusCode, ApiError> {
use crate::db::EntryFlag; use crate::db::EntryFlag;
if let Some(v) = body.read { if let Some(v) = body.read {
state.ctx.db.set_entry_flag(&feed_id, &guid, EntryFlag::Read, v)?; state.ctx.db.set_entry_flag(user.id, &feed_id, &guid, EntryFlag::Read, v)?;
} }
if let Some(v) = body.flagged { if let Some(v) = body.flagged {
state.ctx.db.set_entry_flag(&feed_id, &guid, EntryFlag::Flagged, v)?; state.ctx.db.set_entry_flag(user.id, &feed_id, &guid, EntryFlag::Flagged, v)?;
} }
Ok(StatusCode::NO_CONTENT) Ok(StatusCode::NO_CONTENT)
} }
@@ -759,15 +1147,46 @@ async fn download_now(
Ok(StatusCode::ACCEPTED) Ok(StatusCode::ACCEPTED)
} }
#[derive(Deserialize)]
struct Force {
#[serde(default)]
force: bool,
}
async fn delete_file( async fn delete_file(
State(state): State<WebState>, State(state): State<WebState>,
Path(id): Path<i64>, Path(id): Path<i64>,
user: crate::db::User,
Query(q): Query<Force>,
) -> Result<StatusCode, ApiError> { ) -> Result<StatusCode, ApiError> {
let enc = state let enc = state
.ctx .ctx
.db .db
.enclosure(id)? .enclosure(id)?
.ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?; .ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
// There is one copy of the file: deleting it deletes everyone's. Say so before doing
// it, once, and let them decide.
if !q.force {
let (starred, unread) = state.ctx.db.others_wanting(id, user.id)?;
let people = |n: i64| if n == 1 { "person".to_string() } else { format!("{n} people") };
let complaint = match (starred, unread) {
(0, 0) => None,
(0, u) => Some(format!("{} subscribed to this feed {} not played it yet", people(u), if u == 1 { "has" } else { "have" })),
(st, 0) => Some(format!("another {} kept it", people(st))),
(st, u) => Some(format!(
"another {} kept it, and {} not played it yet",
people(st),
if u == 1 { "one person has".to_string() } else { format!("{u} have") }
)),
};
if let Some(why) = complaint {
return Err(ApiError {
error: anyhow::Error::msg(format!("There is one copy of this file and {why}.")),
status: StatusCode::CONFLICT,
});
}
}
if let Some(path) = &enc.path if let Some(path) = &enc.path
&& let Err(e) = std::fs::remove_file(path) && let Err(e) = std::fs::remove_file(path)
&& e.kind() != std::io::ErrorKind::NotFound && e.kind() != std::io::ErrorKind::NotFound
@@ -805,9 +1224,20 @@ async fn fetch_now(
/// The same broadcast the socket clients read, as server-sent events. /// The same broadcast the socket clients read, as server-sent events.
async fn events(State(state): State<WebState>) -> Sse<impl futures_util::Stream<Item = Result<SseEvent, std::convert::Infallible>>> { async fn events(State(state): State<WebState>) -> Sse<impl futures_util::Stream<Item = Result<SseEvent, std::convert::Infallible>>> {
let stream = BroadcastStream::new(state.events.subscribe()).filter_map(|ev| async move { // A client that falls behind skips what it missed rather than being cut off.
let ev = ev.ok()?; let stream = futures_util::stream::unfold(state.events.subscribe(), |mut rx| async move {
Some(Ok(SseEvent::default().data(serde_json::to_string(&ev).ok()?))) loop {
match rx.recv().await {
Ok(ev) => {
if let Ok(data) = serde_json::to_string(&ev) {
let ev = Ok::<_, std::convert::Infallible>(SseEvent::default().data(data));
return Some((ev, rx));
}
}
Err(broadcast::error::RecvError::Lagged(_)) => {}
Err(broadcast::error::RecvError::Closed) => return None,
}
}
}); });
Sse::new(stream).keep_alive(axum::response::sse::KeepAlive::default()) Sse::new(stream).keep_alive(axum::response::sse::KeepAlive::default())
} }
@@ -838,15 +1268,17 @@ struct Position {
async fn set_position( async fn set_position(
State(state): State<WebState>, State(state): State<WebState>,
Path((feed_id, guid)): Path<(String, String)>, Path((feed_id, guid)): Path<(String, String)>,
user: crate::db::User,
Json(body): Json<Position>, Json(body): Json<Position>,
) -> Result<StatusCode, ApiError> { ) -> Result<StatusCode, ApiError> {
state.ctx.db.set_position(&feed_id, &guid, body.secs)?; state.ctx.db.set_position(user.id, &feed_id, &guid, body.secs)?;
Ok(StatusCode::NO_CONTENT) Ok(StatusCode::NO_CONTENT)
} }
async fn read_all( async fn read_all(
State(state): State<WebState>, State(state): State<WebState>,
Path(id): Path<String>, Path(id): Path<String>,
user: crate::db::User,
) -> Result<Json<serde_json::Value>, ApiError> { ) -> Result<Json<serde_json::Value>, ApiError> {
// A subscription's own row has no entries, so marking it read means everything under it. // A subscription's own row has no entries, so marking it read means everything under it.
let mut ids = vec![id.clone()]; let mut ids = vec![id.clone()];
@@ -856,7 +1288,19 @@ async fn read_all(
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str())) .filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
.map(|s| s.id), .map(|s| s.id),
); );
let n = state.ctx.db.mark_all_read(&ids)?; let n = state.ctx.db.mark_all_read(user.id, &ids)?;
Ok(Json(serde_json::json!({ "marked": n })))
}
/// Everything read in every feed you subscribe to: exactly what All Subscriptions lists, since
/// that view is scoped by the same subscriptions.
async fn read_all_mine(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<serde_json::Value>, ApiError> {
let ids: Vec<String> =
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
let n = state.ctx.db.mark_all_read(user.id, &ids)?;
Ok(Json(serde_json::json!({ "marked": n }))) Ok(Json(serde_json::json!({ "marked": n })))
} }
@@ -890,8 +1334,14 @@ async fn download_latest(
} }
/// Subscriptions as OPML, so they can move to another podcast app. /// Subscriptions as OPML, so they can move to another podcast app.
async fn export_opml(State(state): State<WebState>) -> Result<Response, ApiError> { async fn export_opml(
let cfg = state.ctx.cfg(); State(state): State<WebState>,
user: crate::db::User,
) -> Result<Response, ApiError> {
// Yours, not the whole catalogue: other people's feeds, and any private URLs in them, are
// not yours to download. This used to export config.toml to whoever asked.
let mine: std::collections::HashSet<String> =
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
let mut doc = opml::OPML { let mut doc = opml::OPML {
head: Some(opml::Head { head: Some(opml::Head {
title: Some("ipx subscriptions".into()), title: Some("ipx subscriptions".into()),
@@ -899,15 +1349,19 @@ async fn export_opml(State(state): State<WebState>) -> Result<Response, ApiError
}), }),
..Default::default() ..Default::default()
}; };
for (id, feed) in &cfg.feeds { for s in crate::subscriptions(&state.ctx)? {
// A feed from an OPML subscription comes back with the OPML itself.
if s.managed || !mine.contains(&s.id) {
continue;
}
let title = state let title = state
.ctx .ctx
.db .db
.feed_summary(id) .feed_summary(&s.id)
.ok() .ok()
.and_then(|s| s.title) .and_then(|sum| sum.title)
.unwrap_or_else(|| id.clone()); .unwrap_or_else(|| s.id.clone());
doc.add_feed(&title, &feed.url); doc.add_feed(&title, &s.cfg.url);
} }
let xml = doc.to_string().map_err(|e| anyhow::anyhow!("writing OPML: {e}"))?; let xml = doc.to_string().map_err(|e| anyhow::anyhow!("writing OPML: {e}"))?;
Ok(( Ok((
@@ -930,42 +1384,18 @@ struct OpmlBody {
async fn import_opml( async fn import_opml(
State(state): State<WebState>, State(state): State<WebState>,
user: crate::db::User,
Json(body): Json<OpmlBody>, Json(body): Json<OpmlBody>,
) -> Result<Json<serde_json::Value>, ApiError> { ) -> Result<Json<serde_json::Value>, ApiError> {
// Refused before anything is touched. Nothing reaches the disk either way: an uploaded
// file arrives as text, is read here, and is gone when the request ends.
let doc = opml::OPML::from_str(&body.xml) let doc = opml::OPML::from_str(&body.xml)
.map_err(|e| anyhow::anyhow!("that does not parse as OPML: {e}"))?; .map_err(|e| ApiError::bad_request(format!("that is not an OPML file: {e}")))?;
let mut found = vec![]; let (added, already) = crate::subscribe_opml(&state.ctx, &state.config_path, &doc, user.id)?;
crate::collect_outlines(&doc.body.outlines, &mut found); if added > 0 {
scan_soon(&state, None).await;
let mut cfg = (*state.ctx.cfg()).clone();
let mut added = 0;
for (title, url) in found {
if cfg.feeds.values().any(|f| f.url == url) {
continue;
}
let id = crate::config::unique_slug(&title, &cfg.feeds);
cfg.feeds.insert(
id,
crate::config::Feed {
url,
folder: None,
group: None,
media_types: None,
schedule: None,
keywords: vec![],
allow_explicit: false,
auto_download: true,
max_new_per_check: None,
username: None,
password: None,
password_env: None,
},
);
added += 1;
} }
cfg.save(&state.config_path)?; Ok(Json(serde_json::json!({ "added": added, "already": already })))
state.ctx.reload_cfg(&state.config_path)?;
Ok(Json(serde_json::json!({ "added": added })))
} }
#[derive(Serialize)] #[derive(Serialize)]
@@ -1018,8 +1448,6 @@ async fn patch_settings(
))); )));
} }
cfg.general.schedule = sched; cfg.general.schedule = sched;
// The legacy key would otherwise keep shadowing intent in the file.
cfg.general.interval_mins = None;
} }
if let Some(v) = body.max_new_per_check { if let Some(v) = body.max_new_per_check {
cfg.general.max_new_per_check = v; cfg.general.max_new_per_check = v;
@@ -1061,9 +1489,13 @@ struct LogPage {
latest: u64, latest: u64,
} }
async fn logs(Query(q): Query<LogQuery>) -> Json<LogPage> { async fn logs(user: crate::db::User, Query(q): Query<LogQuery>) -> Result<Json<LogPage>, ApiError> {
// The log names every account, every feed and every failed sign-in, not just yours.
if !user.is_admin {
return Err(ApiError::forbidden("only an admin reads the log"));
}
let (lines, latest) = crate::logbuf::since(q.after, q.limit.clamp(1, 2000)); let (lines, latest) = crate::logbuf::since(q.after, q.limit.clamp(1, 2000));
Json(LogPage { lines, latest }) Ok(Json(LogPage { lines, latest }))
} }
/// One line per HTTP request, so the web side shows up in the same log as the daemon. /// One line per HTTP request, so the web side shows up in the same log as the daemon.

View File

@@ -15,7 +15,7 @@ const script = html.split('<script>')[1].split('</script>')[0];
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1])); const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
const missing = []; const missing = [];
const el = (name) => new Proxy({ style: {}, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } }, const el = (name) => new Proxy({ style: { setProperty(){}, getPropertyValue(){ return ''; } }, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null, value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){}, appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){}, setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
@@ -44,7 +44,12 @@ const ctx = {
json: () => Promise.resolve( json: () => Promise.resolve(
String(url).includes('/api/settings') String(url).includes('/api/settings')
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 } ? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
: []), : String(url).includes('/api/users')
? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }]
: /\/api\/(popular|directory)/.test(String(url))
? [{ id: 'f', title: 'A Feed', image: null, subscribers: 2, subscribed: true },
{ id: 'g', title: null, image: null, subscribers: 1, subscribed: false }]
: /entries/.test(String(url)) ? { total: 0, entries: [] } : []),
}), }),
EventSource: function () { this.close = () => {}; }, EventSource: function () { this.close = () => {}; },
MediaMetadata: function () {}, MediaMetadata: function () {},
@@ -79,6 +84,11 @@ const drive = [
['downloadLatestModal', () => ctx.downloadLatestModal(feed)], ['downloadLatestModal', () => ctx.downloadLatestModal(feed)],
['removeFeed', () => ctx.removeFeed(feed)], ['removeFeed', () => ctx.removeFeed(feed)],
['prefsModal', () => ctx.prefsModal()], ['prefsModal', () => ctx.prefsModal()],
['usersModal', () => ctx.usersModal()],
['opmlModal', () => ctx.opmlModal()],
['selectFeed (directory)', () => ctx.selectFeed(':directory')],
['selectFeed (popular)', () => ctx.selectFeed(':popular')],
['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
['logsModal', () => ctx.logsModal()], ['logsModal', () => ctx.logsModal()],
// `const S` is not reachable from here: top-level const/let do not become properties // `const S` is not reachable from here: top-level const/let do not become properties
// of a vm context the way var and function declarations do. // of a vm context the way var and function declarations do.

View File

@@ -10,8 +10,8 @@ test.beforeEach(async ({ page }) => {
test('the page loads and lists the configured feeds', async ({ page }) => { test('the page loads and lists the configured feeds', async ({ page }) => {
// Regression: a ReferenceError in the script left the shell rendered and the sidebar // Regression: a ReferenceError in the script left the shell rendered and the sidebar
// empty, with every handler below the error dead. Server-side checks all passed. // empty, with every handler below the error dead. Server-side checks all passed.
// Three top-level feeds in the fixture config; the OPML's child is inside a closed folder. // Four top-level feeds in the fixture config; the OPML's children are inside a closed folder.
await expect(page.locator('.feed')).toHaveCount(4, { timeout: 15_000 }); await expect(page.locator('.feed')).toHaveCount(5, { timeout: 15_000 });
await expect(page.getByText('Test Show')).toBeVisible(); await expect(page.getByText('Test Show')).toBeVisible();
const errors = []; const errors = [];
page.on('pageerror', e => errors.push(e.message)); page.on('pageerror', e => errors.push(e.message));
@@ -29,6 +29,18 @@ test('the theme toggle actually changes the theme', async ({ page }) => {
.not.toBe(before); .not.toBe(before);
}); });
test('the theme button steps through dark, light and classic, and remembers', async ({ page }) => {
const theme = () => page.evaluate(() => document.documentElement.dataset.theme);
for (let i = 0; i < 3 && (await theme()) !== 'classic'; i++) await page.locator('#theme').click();
expect(await theme()).toBe('classic');
await expect(page.locator('#theme')).toHaveAttribute('title', /Classic.*Click for Dark/);
await page.reload();
await expect.poll(theme).toBe('classic');
// The 2004 Mac app set its type in Lucida Grande.
expect(await page.evaluate(() => getComputedStyle(document.body).fontFamily)).toContain('Lucida Grande');
});
test('settings opens and saves the global schedule', async ({ page }) => { test('settings opens and saves the global schedule', async ({ page }) => {
await page.locator('#prefs').click(); await page.locator('#prefs').click();
await expect(page.locator('#modal.on')).toBeVisible(); await expect(page.locator('#modal.on')).toBeVisible();
@@ -69,20 +81,21 @@ test('the three panes are there and the item text lands in the bottom one', asyn
await page.locator('.ep', { hasText: 'First Episode' }).click(); await page.locator('.ep', { hasText: 'First Episode' }).click();
await expect(page.locator('#detail .dt')).toHaveText('First Episode'); await expect(page.locator('#detail .dt')).toHaveText('First Episode');
// The enclosure travels with the item, into the same pane. // The enclosure goes to the Files pane beside the list, as the original's did.
await expect(page.locator('#detail .encbox')).toHaveCount(1); await expect(page.locator('#files')).toBeVisible();
await expect(page.locator('#files .encbox')).toHaveCount(1);
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by // Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
// its chip rather than assuming which episode the daemon happened to fetch. // its chip rather than assuming which episode the daemon happened to fetch.
const downloaded = page.locator('.ep', { hasText: 'downloaded' }).first(); const downloaded = page.locator('.ep', { has: page.locator('.kind.here') }).first();
await downloaded.click(); await downloaded.click();
await expect(page.locator('#detail audio')).toBeVisible(); await expect(page.locator('#files [data-a="play"]')).toBeVisible();
await expect(page.locator('#detail .encbox .btn', { hasText: 'Save' })).toBeVisible(); await expect(page.locator('#files .encbox [title="Save to this computer"]')).toBeVisible();
// Selecting another item replaces the pane rather than stacking. // Selecting another item replaces the pane rather than stacking.
await page.locator('.ep', { hasText: 'First Episode' }).click(); await page.locator('.ep', { hasText: 'First Episode' }).click();
await expect(page.locator('#detail .dt')).toHaveText('First Episode'); await expect(page.locator('#detail .dt')).toHaveText('First Episode');
await expect(page.locator('#detail audio')).toHaveCount(0); await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
}); });
test('a downloaded file that is not audio gets no player', async ({ page }) => { test('a downloaded file that is not audio gets no player', async ({ page }) => {
@@ -96,12 +109,12 @@ test('a downloaded file that is not audio gets no player', async ({ page }) => {
await row.click(); await row.click();
await expect(page.locator('#detail .dt')).toHaveText('An Article'); await expect(page.locator('#detail .dt')).toHaveText('An Article');
await expect(page.locator('#detail audio')).toHaveCount(0); await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
await expect(page.locator('#detail .encbox')).toContainText('image'); // What it is and that it is here: one icon, green, with the words in its tooltip.
await expect(page.locator('#detail .encbox')).toContainText('downloaded'); await expect(page.locator('#files .encbox .kind.here')).toHaveAttribute('title', 'image, downloaded');
// Still offered as a file, just not as an episode: viewable and keepable. // Still offered as a file, just not as an episode: viewable and keepable.
await expect(page.locator('#detail .btn', { hasText: 'Save' })).toBeVisible(); await expect(page.locator('#files [title="Save to this computer"]')).toBeVisible();
const view = page.locator('#detail a', { hasText: 'View' }); const view = page.locator('#files a[title="View in a new tab"]');
await expect(view).toHaveAttribute('target', '_blank'); await expect(view).toHaveAttribute('target', '_blank');
await expect(view).toHaveAttribute('rel', /noopener/); await expect(view).toHaveAttribute('rel', /noopener/);
await expect(view).toHaveAttribute('href', /\/media\/\d+/); await expect(view).toHaveAttribute('href', /\/media\/\d+/);
@@ -115,9 +128,9 @@ test('an item with several enclosures lists them all', async ({ page }) => {
await expect(row).toContainText('+1 more file'); await expect(row).toContainText('+1 more file');
await row.click(); await row.click();
// The pane below lists every one: the audio and the image. // The Files pane lists every one: the audio and the image.
await expect(page.locator('#detail .encbox')).toHaveCount(2); await expect(page.locator('#files .encbox')).toHaveCount(2);
await expect(page.locator('#detail .encbox').nth(1)).toContainText('image'); await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
}); });
test('the filter tabs change what is listed', async ({ page }) => { test('the filter tabs change what is listed', async ({ page }) => {
@@ -129,13 +142,13 @@ test('the filter tabs change what is listed', async ({ page }) => {
await page.locator('.tabs button', { hasText: 'Unread' }).first().click(); await page.locator('.tabs button', { hasText: 'Unread' }).first().click();
expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all); expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all);
await page.locator('.tabs button', { hasText: 'Flagged' }).first().click(); await page.locator('.tabs button', { hasText: 'Kept' }).first().click();
await expect(page.locator('#count')).toContainText('0 items'); await expect(page.locator('#count')).toContainText('0 items');
}); });
test('a feed URL is editable and has a copy button', async ({ page }) => { test('a feed URL is editable and has a copy button', async ({ page }) => {
await page.getByText('Test Show').click(); await page.getByText('Test Show').click();
await page.locator('.btn', { hasText: 'Settings' }).first().click(); await page.locator('#content .acts [data-a="settings"]').click();
await expect(page.locator('#surl')).toHaveValue(/show\.xml/); await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
await expect(page.locator('#scopy')).toBeVisible(); await expect(page.locator('#scopy')).toBeVisible();
@@ -174,20 +187,56 @@ test('an OPML subscription is a collapsible folder', async ({ page }) => {
body: JSON.stringify({ feed: 'test-subscriptions', force: true }), body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
})); }));
// Every row reserves the chevron slot for alignment; only a folder's is clickable. // Only a folder has a triangle, and it is a button that says whether the folder is open.
const chev = page.locator('.feed.group .chev'); const chev = page.locator('.feed.group .chev');
await expect(chev).toBeVisible({ timeout: 20_000 }); await expect(chev).toBeVisible({ timeout: 20_000 });
await expect(page.locator('.feed:not(.group) .chev')).toHaveCount(0);
await expect(chev).toHaveAttribute('aria-expanded', 'false');
// Closed by default: the child is not listed until the folder is opened. // Closed by default: the children are not listed until the folder is opened.
const before = await page.locator('.feed').count(); const before = await page.locator('.feed').count();
await chev.click(); await chev.click();
await expect(page.locator('.feed')).toHaveCount(before + 1); await expect(page.locator('.feed')).toHaveCount(before + 2);
// Scoped to the sidebar: the name also appears as the page heading once selected. // Scoped to the sidebar: the name also appears as the page heading once selected.
await expect(page.locator('#feedlist').getByText('Grouped Show')).toBeVisible(); await expect(page.locator('#feedlist').getByText('Grouped Show')).toBeVisible();
// The subscription's own page lists what is inside it. // The subscription's own page lists what is inside it.
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click(); await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
await expect(page.locator('.childrow')).toHaveCount(1); await expect(page.locator('.childrow')).toHaveCount(2);
});
test('inside an OPML, feeds with unread items are listed first', async ({ page }) => {
await page.evaluate(() =>
fetch('/api/fetch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
}));
const chev = page.locator('.feed.group .chev');
await expect(chev).toBeVisible({ timeout: 20_000 });
await chev.click();
const side = page.locator('#feedlist');
await expect(side.getByText('Aardvark Radio')).toBeVisible({ timeout: 20_000 });
// Other tests change read state, so set it here: Aardvark Radio read, Grouped Show not.
// Opening an item reads it; the toggle in the pane below flips it back.
await side.getByText('Aardvark Radio').click();
const aa = page.locator('.ep', { hasText: 'Aardvark Ep' });
await aa.click();
await expect(aa).toHaveClass(/read/);
await side.getByText('Grouped Show').click();
const gs = page.locator('.ep', { hasText: 'Grouped Ep' });
await gs.click();
await page.locator('#detail button[title="Mark unread"]').click();
await expect(gs).not.toHaveClass(/read/);
// Aardvark comes first alphabetically and in the OPML, so only the unread sort puts
// Grouped Show above it. The folder stays open across the reload (localStorage).
await page.reload();
const want = ['Grouped Show', 'Aardvark Radio'];
await expect(page.locator('#feedlist .feed.child b')).toHaveText(want, { timeout: 20_000 });
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
await expect(page.locator('.childrow b')).toHaveText(want);
}); });
test('marking an OPML subscription read covers the feeds inside it', async ({ page }) => { test('marking an OPML subscription read covers the feeds inside it', async ({ page }) => {
@@ -204,7 +253,7 @@ test('marking an OPML subscription read covers the feeds inside it', async ({ pa
await expect(folder.locator('.badge')).not.toHaveText('0'); await expect(folder.locator('.badge')).not.toHaveText('0');
await folder.click(); await folder.click();
await page.locator('#content .acts button', { hasText: 'Mark all read' }).click(); await page.locator('#content .acts [data-a="read"]').click();
await expect(folder.locator('.badge')).toHaveText('0'); await expect(folder.locator('.badge')).toHaveText('0');
}); });
@@ -246,9 +295,9 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
// Another test may have opened this item already, so start from a known state: the // Another test may have opened this item already, so start from a known state: the
// toggle in the text below flips it back -- which used to recurse until the stack blew. // toggle in the text below flips it back -- which used to recurse until the stack blew.
await row().click(); await row().click();
await page.locator('#detail button', { hasText: 'Mark unread' }).click(); await page.locator('#detail button[title="Mark unread"]').click();
await expect(row()).not.toHaveClass(/read/); await expect(row()).not.toHaveClass(/read/);
await expect(page.locator('#detail button', { hasText: 'Mark read' })).toBeVisible(); await expect(page.locator('#detail button[title="Mark read"]')).toBeVisible();
// Opening it is reading it. // Opening it is reading it.
await row().click(); await row().click();
@@ -256,3 +305,482 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
expect(errors).toEqual([]); expect(errors).toEqual([]);
}); });
test('the toolbar acts on the selected item', async ({ page }) => {
await page.getByText('Test Show').click();
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
await expect(row()).toBeVisible({ timeout: 20_000 });
// Nothing selected, nothing to act on.
await expect(page.locator('#tbRead')).toBeDisabled();
await row().click(); // opening it reads it
await expect(row()).toHaveClass(/read/);
await page.locator('#tbRead').click();
await expect(row()).not.toHaveClass(/read/);
await page.locator('#tbFlag').click();
await expect(row().locator('.fl')).toHaveClass(/on/);
await page.locator('#tbFlag').click(); // and back, so later tests see it unkept
await expect(row().locator('.fl')).not.toHaveClass(/on/);
// Second Episode is the one the daemon downloaded, so it plays from the toolbar.
await expect(page.locator('#tbPlay')).toBeEnabled();
await page.locator('#tbPlay').click();
await expect(page.locator('#player')).toBeVisible();
await page.locator('#pclose').click();
});
test('a second person has their own feeds and their own read state', async ({ browser }) => {
const { execFileSync } = require('child_process');
const setup = require('./global-setup');
const env = {
...process.env,
IPX_CONFIG: `${setup.root}/config/config.toml`,
IPX_DATA_DIR: `${setup.root}/data`,
};
try {
execFileSync('./target/debug/ipx', ['user', 'add', 'sam'], { input: 'sampassword', env });
} catch (e) {
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
}
// A fresh context, so none of the admin's cookies come along.
const ctx = await browser.newContext();
const page = await ctx.newPage();
await page.goto('/login');
// The sign-in page shows the icon, so it has to load before anyone has signed in.
const icon = await page.request.get('/icon.png');
expect(icon.status()).toBe(200);
expect(icon.headers()['content-type']).toBe('image/png');
await page.locator('#name').fill('sam');
await page.locator('#pw').fill('sampassword');
await page.locator('button[type=submit]').click();
await expect(page.locator('#feedlist')).toBeVisible();
// Sam subscribes to nothing yet, so sees nothing -- the admin's feeds are not theirs.
await expect(page.locator('#feedlist')).toContainText('No feeds.');
await expect(page.locator('#prefs')).toBeHidden(); // not an admin
// Hiding the button is not the guard; the server is.
expect((await page.request.get('/api/users')).status()).toBe(403);
await expect(page.locator('#logs')).toBeHidden();
expect((await page.request.get('/api/logs')).status()).toBe(403);
// Subscribing to a feed the admin already has costs no second fetch: same feed, same
// files, but Sam's own read state.
await page.locator('#addFeed').click();
await page.locator('#nurl').fill('http://127.0.0.1:8792/show.xml');
await page.locator('#nsave').click();
await expect(page.locator('.feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
await page.locator('.feed', { hasText: 'Test Show' }).click();
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
// The admin read these earlier in this file; for Sam they are all still unread.
const rows = await page.locator('.ep').count();
await page.locator('.tabs button', { hasText: 'Unread' }).click();
await expect(page.locator('.ep')).toHaveCount(rows);
// Two people now share this feed, so the page says so and Delete is honest about it.
await expect(page.locator('#content .sub').first()).toContainText('shared with 1 other person');
await ctx.close();
});
test('deleting a shared file warns that it is everyone\'s copy', async ({ page }) => {
// Admin and Sam both subscribe to Test Show by now, and the daemon downloaded a file.
await page.getByText('Test Show').click();
await page.locator('.tabs button', { hasText: 'Downloaded' }).click();
const row = page.locator('.ep').first();
await expect(row).toBeVisible({ timeout: 20_000 });
await row.click();
// An icon now; what it does, and to whom, is in its tooltip.
const del = page.locator('#files button[data-a="del"]');
await expect(del).toHaveAttribute('title', /^Delete for everyone \(shared with 1 other person/);
// Two prompts: the page's own, then the server's, because someone else has not played
// it. Accept the first, decline the second, and the file must survive.
const seen = [];
page.on('dialog', d => {
seen.push(d.message());
if (seen.length === 1) d.accept();
else d.dismiss();
});
await del.click();
await expect.poll(() => seen.length, { timeout: 10_000 }).toBe(2);
expect(seen[0]).toContain('shared with 1 other person');
expect(seen[1]).toContain('one copy of this file');
await page.reload();
await page.getByText('Test Show').click();
await page.locator('.tabs button', { hasText: 'Downloaded' }).click();
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
});
// Every row says "Admin" on its checkbox, so match the name exactly.
const userRow = (page, name) =>
page.locator('#modalCard [data-id]').filter({ has: page.locator('b', { hasText: new RegExp(`^${name}$`) }) });
async function openUsers(page) {
await page.locator('#prefs').click();
await page.locator('#gusers').click();
await expect(userRow(page, 'admin')).toBeVisible();
}
test('an admin adds someone, makes them an admin, and removes them', async ({ page }) => {
await openUsers(page);
await page.locator('#uname').fill('pat');
await page.locator('#upass').fill('patpassword');
await page.locator('#uadd').click();
const row = userRow(page, 'pat');
await expect(row).toBeVisible();
await expect(row.locator('[data-a="admin"]')).not.toBeChecked();
await row.locator('[data-a="admin"]').check();
// Not just the box: it has to have reached the database.
await expect.poll(async () =>
(await (await page.request.get('/api/users')).json()).find(u => u.name === 'pat')?.admin
).toBe(true);
page.once('dialog', d => d.accept());
await row.locator('[data-a="rm"]').click();
await expect(row).toHaveCount(0);
});
test('the only admin cannot be demoted or removed', async ({ page }) => {
await openUsers(page);
await userRow(page, 'admin').locator('[data-a="admin"]').click();
await expect(page.locator('.toast.bad')).toContainText('only admin');
// Redrawn from the server, so the box is back.
await expect(userRow(page, 'admin').locator('[data-a="admin"]')).toBeChecked();
const me = (await (await page.request.get('/api/users')).json()).find(u => u.name === 'admin');
expect((await page.request.delete(`/api/users/${me.id}`)).status()).toBe(400);
});
test('Settings exports your OPML and imports a pasted one', async ({ page }) => {
await page.locator('#prefs').click();
const [dl] = await Promise.all([
page.waitForEvent('download'),
page.locator('#modalCard a[title="Export OPML"]').click(),
]);
expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml');
const out = require('fs').readFileSync(await dl.path(), 'utf8');
for (const f of ['show.xml', 'pics.xml', 'multi.xml', 'subs.opml']) expect(out).toContain(f);
// A feed from an OPML subscription comes back with the OPML itself, not on its own.
expect(out).not.toContain('other.xml');
// One feed new to everyone, one the admin already has.
await page.locator('#gopml').click();
await page.locator('#opmlText').fill('<opml version="2.0"><head><title>t</title></head><body>' +
'<outline text="Imported Show" xmlUrl="http://127.0.0.1:8792/imported.xml"/>' +
'<outline text="Test Show" xmlUrl="http://127.0.0.1:8792/show.xml"/></body></opml>');
await page.locator('#oimp').click();
await expect(page.locator('.toast', { hasText: 'Subscribed to' }))
.toHaveText('Subscribed to 1 feed(s), 1 you already had');
// Named from the OPML's id until the first scan reads the feed's own title.
await expect(page.locator('#feedlist .feed', { hasText: /Imported Show|imported-show/ }))
.toBeVisible({ timeout: 20_000 });
});
test('an uploaded OPML file imports, and a file that is not OPML is refused', async ({ page }) => {
await page.locator('#prefs').click();
await page.locator('#gopml').click();
const pick = page.locator('#opmlFile');
// An RSS feed is XML but not OPML: refused in the page, and the picker lets go of it.
await pick.setInputFiles({
name: 'feed.xml', mimeType: 'application/xml',
buffer: require('fs').readFileSync(require('path').join(__dirname, 'fixtures', 'show.xml')),
});
await page.locator('#oimp').click();
await expect(page.locator('.toast.bad', { hasText: 'feed.xml is not an OPML file' })).toBeVisible();
expect(await pick.evaluate(i => i.files.length)).toBe(0);
// Something that gets past the page's quick look is still refused by the server, untouched.
const sneaky = await page.request.post('/api/opml', {
data: { xml: '<rss version="2.0"><channel><title>&lt;opml&gt;</title></channel></rss>' },
});
expect(sneaky.status()).toBe(400);
expect(await sneaky.text()).toContain('not an OPML file');
// A real one. Multi Show is already the admin's, so it counts as already had.
await pick.setInputFiles({
name: 'subs.opml', mimeType: 'text/x-opml',
buffer: Buffer.from('<opml version="2.0"><head><title>t</title></head><body>' +
'<outline text="Multi Show" xmlUrl="http://127.0.0.1:8792/multi.xml"/></body></opml>'),
});
await page.locator('#oimp').click();
await expect(page.locator('.toast', { hasText: 'Subscribed to' }))
.toHaveText('Subscribed to 0 feed(s), 1 you already had');
await expect(page.locator('#modal.on')).toBeHidden();
});
test('an export from one account imports into another', async ({ page, browser }) => {
// Regression: import only added URLs the catalogue lacked and subscribed nobody, so importing
// the admin's export into a second account did nothing at all.
const { execFileSync } = require('child_process');
const setup = require('./global-setup');
const env = {
...process.env,
IPX_CONFIG: `${setup.root}/config/config.toml`,
IPX_DATA_DIR: `${setup.root}/data`,
};
try {
execFileSync('./target/debug/ipx', ['user', 'add', 'opal'], { input: 'opalpassword', env });
} catch (e) {
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
}
const ctx = await browser.newContext();
const opal = await ctx.newPage();
await opal.goto('/login');
await opal.locator('#name').fill('opal');
await opal.locator('#pw').fill('opalpassword');
await opal.locator('button[type=submit]').click();
await expect(opal.locator('#feedlist')).toContainText('No feeds.');
// Export used to hand anyone the whole catalogue. Opal has nothing yet, so gets nothing.
const empty = await opal.request.get('/api/opml');
expect(empty.status()).toBe(200);
expect(await empty.text()).not.toContain('xmlUrl');
const urlsIn = xml => [...xml.matchAll(/xmlUrl="([^"]+)"/g)].map(m => m[1]).sort();
const exported = await (await page.request.get('/api/opml')).text(); // the admin's
const urls = urlsIn(exported);
expect(urls.length).toBeGreaterThan(2);
expect(await (await opal.request.post('/api/opml', { data: { xml: exported } })).json())
.toEqual({ added: urls.length, already: 0 });
expect(await (await opal.request.post('/api/opml', { data: { xml: exported } })).json())
.toEqual({ added: 0, already: urls.length });
await opal.reload();
await expect(opal.locator('.feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// The round trip closes: opal's own export now lists what the admin's did.
expect(urlsIn(await (await opal.request.get('/api/opml')).text())).toEqual(urls);
await ctx.close();
});
test('ipx import subscribes the admin, and ipx export writes the feeds out', async () => {
// Its own config and database. The CLI works in-process, and the suite's running daemon
// reads config.toml once at start, so it would not see what the CLI added anyway.
const fs = require('fs');
const path = require('path');
const { execFileSync } = require('child_process');
const setup = require('./global-setup');
const dir = path.join(setup.root, 'cli');
fs.rmSync(dir, { recursive: true, force: true });
fs.mkdirSync(path.join(dir, 'data'), { recursive: true });
fs.writeFileSync(path.join(dir, 'config.toml'),
`[general]\ndownload_dir = "${dir}/downloads"\nsocket = "${dir}/ipx.sock"\n`);
const env = { ...process.env, IPX_CONFIG: path.join(dir, 'config.toml'), IPX_DATA_DIR: path.join(dir, 'data') };
const ipx = (args, input) => execFileSync('./target/debug/ipx', args, { env, input, encoding: 'utf8' });
ipx(['user', 'add', 'boss'], 'bosspassword'); // the first account is the admin
const opml = path.join(dir, 'in.opml');
fs.writeFileSync(opml, '<opml version="2.0"><head><title>t</title></head><body>' +
'<outline text="One" xmlUrl="http://127.0.0.1:8792/one.xml"/>' +
'<outline text="Two" xmlUrl="http://127.0.0.1:8792/two.xml"/></body></opml>');
expect(ipx(['import', opml])).toContain('subscribed boss to 2 feed(s); 0 already there');
expect(ipx(['import', opml])).toContain('subscribed boss to 0 feed(s); 2 already there');
const out = path.join(dir, 'out.opml');
ipx(['export', out]);
const xml = fs.readFileSync(out, 'utf8');
expect(xml).toContain('http://127.0.0.1:8792/one.xml');
expect(xml).toContain('http://127.0.0.1:8792/two.xml');
});
test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
const { execFileSync } = require('child_process');
const setup = require('./global-setup');
const env = {
...process.env,
IPX_CONFIG: `${setup.root}/config/config.toml`,
IPX_DATA_DIR: `${setup.root}/data`,
};
try {
execFileSync('./target/debug/ipx', ['user', 'add', 'piper'], { input: 'piperpassword', env });
} catch (e) {
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
}
const ctx = await browser.newContext();
const piper = await ctx.newPage();
await piper.goto('/login');
await piper.locator('#name').fill('piper');
await piper.locator('#pw').fill('piperpassword');
await piper.locator('button[type=submit]').click();
await expect(piper.locator('#feedlist')).toContainText('No feeds.');
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
const offered = piper.locator('#popular .childrow');
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// An OPML's own feeds ride on the OPML, and a key in a URL marks someone's paid feed.
await expect(offered.filter({ hasText: /Grouped Show|grouped-show/ })).toHaveCount(0);
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
// No URL reaches the page at all, so neither can a key, and the server holds the same line.
const listed = await (await piper.request.get('/api/popular')).text();
expect(listed).not.toContain('secret123');
expect(listed).not.toContain('.xml');
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
// Popular is the top ten of the directory, and the directory is every listed feed, A to Z.
const dir = await (await piper.request.get('/api/directory')).json();
const top = await (await piper.request.get('/api/popular')).json();
const names = dir.map(p => (p.title || p.id).toLowerCase());
expect(names).toEqual([...names].sort());
expect(top.length).toBe(Math.min(10, dir.length));
expect(top.every(t => dir.some(d => d.id === t.id))).toBe(true);
expect(dir.map(p => p.id)).not.toContain('paid-show');
// Subscribe from the directory this time; the popular list shares the same rows.
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible();
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
const row = async () =>
(await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
const before = await row();
expect(before.subscribed).toBe(false);
await offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
// All Subscriptions is every item from piper's feeds and only those: the admin's Picture
// Blog is not among them.
await piper.locator('#feedlist .place', { hasText: 'All Subscriptions' }).click();
const first = piper.locator('.ep', { hasText: 'First Episode' });
await expect(first).toBeVisible({ timeout: 20_000 });
await expect(first.locator('.fd')).toHaveText('Test Show');
await expect(piper.locator('.ep', { hasText: 'An Article' })).toHaveCount(0);
await expect(piper.locator('#count')).toContainText('All Subscriptions:');
await ctx.close();
});
test('adding a feed scans it straight away', async ({ page }) => {
await page.locator('#addFeed').click();
await page.locator('#nurl').fill('http://127.0.0.1:8792/fresh.xml');
await page.locator('#nsave').click();
// Nobody pressed Scan. The scheduler's tick is a minute, so this is the add scanning it.
await expect(page.locator('.ep', { hasText: 'Fresh Ep' })).toBeVisible({ timeout: 10_000 });
});
test('a deleted file looks as if it was never downloaded', async ({ page }) => {
// Other people subscribe to Picture Blog by now, so both prompts come; take them.
page.on('dialog', d => d.accept());
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
const row = page.locator('.ep', { hasText: 'An Article' });
await expect(row).toBeVisible({ timeout: 20_000 });
await row.click();
await page.locator('#files button[data-a="del"]').click();
// No "reaped", no chip at all: just the way to get it again.
await expect(row.locator('.kind.here')).toHaveCount(0);
await expect(row).not.toContainText(/reaped/i);
await row.click();
await expect(page.locator('#files')).not.toContainText(/reaped/i);
await expect(page.locator('#files button[title="Download to the server"]')).toBeVisible();
});
test('one action, one icon: the toolbar, the page and every dialog agree', async ({ page }) => {
const icon = loc => loc.locator('svg path').first().getAttribute('d');
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
// Unsubscribe is a minus in the toolbar and the feed header, never the x that closes things.
expect(await icon(page.locator('#content .acts [data-a="rm"]'))).toBe(await icon(page.locator('#tbRemove')));
// The toolbar's read and keep show the selected item's state, as its own buttons do, and follow
// a change made from the toolbar.
await page.locator('.ep').first().click();
const pair = async a => [await icon(page.locator(a === 'read' ? '#tbRead' : '#tbFlag')),
await icon(page.locator(`#detail [data-a="${a}"]`))];
for (const a of ['read', 'flag']) { const [tb, own] = await pair(a); expect(tb).toBe(own); }
const [kept] = await pair('flag');
await page.locator('#tbFlag').click();
await expect.poll(async () => { const [tb, own] = await pair('flag'); return tb === own && tb !== kept; }).toBe(true);
await page.locator('#tbFlag').click(); // leave it as it was
await expect.poll(async () => (await pair('flag'))[0]).toBe(kept);
// Every button in every dialog is an icon with its words in the tooltip.
const dialogs = [
() => page.locator('#addFeed').click(),
() => page.locator('#prefs').click(),
async () => { await page.locator('#prefs').click(); await page.locator('#gusers').click(); },
async () => { await page.locator('#prefs').click(); await page.locator('#gopml').click(); },
() => page.locator('#logs').click(),
() => page.locator('#content .acts [data-a="settings"]').click(),
() => page.locator('#content .acts [data-a="dl"]').click(),
() => page.locator('#content .acts [data-a="rm"]').click(),
];
for (const open of dialogs) {
await open();
const btns = page.locator('#modalCard .btn');
await expect(btns.first()).toBeVisible();
for (const b of await btns.all()) {
await expect(b.locator('svg')).toHaveCount(1);
await expect(b).toHaveAttribute('title', /\S/);
}
await page.keyboard.press('Escape');
await expect(page.locator('#modal.on')).toBeHidden();
}
});
test('All Subscriptions marks everything read, across every feed', async ({ page }) => {
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
await all.click();
// Earlier tests read things; make sure something is unread. Opening an item reads it, and
// its own button makes it unread again.
await page.locator('.ep').first().click();
await page.locator('#detail [data-a="read"][title="Mark unread"]').click();
await expect(all.locator('.badge')).not.toHaveText('0');
page.once('dialog', d => d.accept());
await page.locator('#content .acts [data-a="readall"]').click();
await expect(all.locator('.badge')).toHaveText('0');
await page.locator('.tabs button', { hasText: 'Unread' }).click();
await expect(page.locator('.ep')).toHaveCount(0);
});
test('the item table sorts by any column, both ways, and remembers', async ({ page }) => {
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
await all.click();
const head = k => page.locator(`#list .ephead [data-sort="${k}"]`);
const titles = () => page.locator('#eps .ep .t').allTextContents();
// Byte order on lower case, which is what SQLite gives for lower(...).
const cmp = (a, b) => (a.toLowerCase() < b.toLowerCase() ? -1 : a.toLowerCase() > b.toLowerCase() ? 1 : 0);
const sorted = (t, dir) => JSON.stringify(t) === JSON.stringify([...t].sort((a, b) => cmp(a, b) * dir));
await expect(page.locator('#eps .ep').nth(2)).toBeVisible({ timeout: 20_000 });
expect(new Set(await titles()).size).toBeGreaterThan(2); // or both orders would prove nothing
await expect(head('title')).toHaveText('Title');
await head('title').click();
await expect.poll(async () => sorted(await titles(), 1)).toBe(true);
await head('title').click();
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
// Kept across a reload.
await page.reload();
await all.click();
await expect(head('title').locator('.arr.desc')).toBeVisible();
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
// Size has its own column; the file column is just what the file is.
await expect(page.locator('#eps .ep .size', { hasText: /\d/ }).first()).toBeVisible();
await expect(page.locator('#eps .ep .file', { hasText: /\d/ })).toHaveCount(0);
});
test('play in the Files pane plays once, in the player bar', async ({ page }) => {
// Regression: the pane had an <audio> of its own, and playing it started the player bar too,
// so the same file played twice at once.
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
await page.locator('.ep', { has: page.locator('.kind.here') }).first().click();
await page.locator('#files [data-a="play"]').click();
await expect(page.locator('#player')).toBeVisible();
await expect(page.locator('audio')).toHaveCount(1); // the player bar's, and nothing else
await page.locator('#pclose').click();
});

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Aardvark Radio</title><link>http://127.0.0.1:8792/</link>
<description>Inside the OPML, and first in it and alphabetically.</description>
<item><title>Aardvark Ep</title><guid>aa-1</guid><description>x</description></item>
</channel></rss>

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Fresh Show</title><link>http://127.0.0.1:8792/</link>
<description>Added in the browser suite, and scanned by adding it.</description>
<item><title>Fresh Ep</title><guid>fresh-1</guid><description>x</description></item>
</channel></rss>

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Imported Show</title><link>http://127.0.0.1:8792/</link>
<description>Only ever arrives through an OPML import.</description>
<item><title>Imported Ep</title><guid>imp-1</guid><description>x</description></item>
</channel></rss>

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Paid Show</title><link>http://127.0.0.1:8792/</link>
<description>Subscribed with a key in its URL, so it must never be offered to anyone else.</description>
<item><title>Paid Ep</title><guid>paid-1</guid><description>x</description></item>
</channel></rss>

View File

@@ -1,4 +1,5 @@
<opml version="2.0"><head><title>Test Subscriptions</title></head> <opml version="2.0"><head><title>Test Subscriptions</title></head>
<body><outline text="Folder"> <body><outline text="Folder">
<outline type="rss" text="Aardvark Radio" xmlUrl="http://127.0.0.1:8792/aardvark.xml"/>
<outline type="rss" text="Grouped Show" xmlUrl="http://127.0.0.1:8792/other.xml"/> <outline type="rss" text="Grouped Show" xmlUrl="http://127.0.0.1:8792/other.xml"/>
</outline></body></opml> </outline></body></opml>

View File

@@ -10,7 +10,15 @@ const TOKEN = 'testtokentesttokentesttoken12345'; // fixed, so tests need not
// Called from playwright.config.js at load time, NOT as globalSetup: Playwright starts // Called from playwright.config.js at load time, NOT as globalSetup: Playwright starts
// webServer *before* globalSetup, so a config written there does not exist yet when the // webServer *before* globalSetup, so a config written there does not exist yet when the
// daemon launches -- it would fall back to the real config and fight the live daemon. // daemon launches -- it would fall back to the real config and fight the live daemon.
// Playwright imports this config again in every worker process, so prepare() runs more
// than once per suite. Wiping on the second call deleted the data directory out from under
// the running daemon: it kept serving from the unlinked inode, while anything else opening
// that path -- the CLI, a query -- got a brand new empty database and disagreed with it.
function prepare() { function prepare() {
// Only the process that launches the run may wipe. A worker gets TEST_WORKER_INDEX.
if (process.env.TEST_WORKER_INDEX !== undefined || process.env.PW_WORKER_INDEX !== undefined) {
return;
}
fs.rmSync(root, { recursive: true, force: true }); fs.rmSync(root, { recursive: true, force: true });
for (const d of ['config', 'data', 'downloads']) { for (const d of ['config', 'data', 'downloads']) {
fs.mkdirSync(path.join(root, d), { recursive: true }); fs.mkdirSync(path.join(root, d), { recursive: true });
@@ -47,6 +55,11 @@ auto_download = true
[feeds.test-subscriptions] [feeds.test-subscriptions]
url = "http://127.0.0.1:8792/subs.opml" url = "http://127.0.0.1:8792/subs.opml"
auto_download = false auto_download = false
# A key in its URL, like a Patreon feed: someone's paid subscription, never offered to others.
[feeds.paid-show]
url = "http://127.0.0.1:8792/paid.xml?auth=secret123"
auto_download = false
`); `);
} }

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long