27 Commits

Author SHA1 Message Date
2af57065c6 Release 0.5.1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:15:53 +00:00
564b011c7a Give the folder triangle a wider gutter
The feed list's left padding grows from 8 to 16 px and the triangle's
button from 16 to 24 px wide, so it is no longer cramped against the
folder's art. Everything in the list shifts together, so feeds still line
up with the places above.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:13:32 +00:00
9eb7aadced Release 0.5.0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 02:06:23 +00:00
c6bceaef37 Docs: a slow migration and a CLI run at the same time
Every ipx command migrates when it opens the database, so the healthcheck
collided with the daemon while it dropped the old entries columns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:59:05 +00:00
dc63d6acaf Cut what the audit found: dead columns, one-time upgrades, three deps
Works through TODO.md from the 2026-09-12 over-engineering audit. Drops the
entries.read/flagged/position columns (migrate() removes them from older
databases), migrate_opml_children, the legacy interval_mins key, the
contrib/ systemd units, test-only Db wrappers, a duplicate token generator,
redundant logbuf visitors, unused page state and CSS, and the infer, dirs
and tokio-stream dependencies. The icon is served once as /icon.png instead
of inlined four times, taking about 94 KB off the two pages.

The adoption's subscription half was not dead: it gives a fresh install's
first admin the config's feeds. It stays as adopt_catalogue, now tested.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:55:44 +00:00
8937f35f00 TODO.md: cleared, the design pass is done
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-12 01:31:08 +00:00
0990f2a90d Design pass on the web UI: amber for new, EQ bars, keyboard feed list
Works through TODO.md from the 2026-09-11 review. Unread badges, dots and
download bars take the icon's amber; the playing item is marked by EQ bars
that move only while it plays. The feed list is usable from the keyboard,
focus rings show everywhere, and folders get a mosaic of their shows' art
with the triangle hung in the margin. Sentence-case labels, fewer bold
weights, tinted initials tiles, shorter header lines, "Kept" everywhere,
and the list gets the room the empty panes had. Reduced motion is honoured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
2026-09-11 22:24:41 +00:00
0cc002cdfa todo.md: folders in the sidebar; drop the system-theme item
The disclosure triangle, looked at closely: the feed list cannot be
reached from the keyboard (span and div, no tabindex), every feed is
pushed 28 px right for a slot only folders use, the target is 18 px,
shows barely nest under their folder, a folder looks like a feed, and
a selected feed's placeholder tile vanishes in Dark and Light.

Following the system light or dark setting is off the list, by
choice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 19:39:49 +00:00
2c9e899762 todo.md: the design review's list, to do later
A review of the web UI against screenshots of every view in all three
themes. The palette and Classic carry the iPodderX identity; Dark and
Light do not. First three: amber for new (unread badges, dots,
download progress), EQ bars as the playing marker, and toolbar focus
rings that overflow:hidden currently clips. The rest is weights,
sentence-case labels, pane sizes, header lines, wording, system theme,
reduced motion, art, sign-in and the Settings export/import icons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 18:53:42 +00:00
2e416f96cf Patreon creators split into their shows; filters follow settings
A Patreon token pasted into Add feed, or a creator link without
&show=, becomes a folder of that creator's shows, found through
Patreon's web API and kept in step like a subscribed OPML (sync_group,
split out of sync_opml). A creator already read as one feed is split
too: each show takes over the files and read state it held
(Db::adopt). A creator with one show stays a plain feed.

Filter verdicts are judged again every scan, so turning on Allow
explicit brings skipped items back. Add feed has an explicit box.
Feeds in a group follow your settings on the group, as its dialog
said. A new feed no longer takes the id of a removed one at a
different URL and shows its old items. See CHANGELOG.md [Unreleased]
and docs/history.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
2026-09-11 18:24:47 +00:00
9269aa99f7 README: a short overview that points into docs/
It described the layout from before 0.4.0 (items across the top, a
player below), and carried long sections on OPML, the log view and
this server's own deploy steps, all of which docs/ and CLAUDE.md cover.
It now says what ipx does, how to run it with Docker or from source,
the first sign-in, the TLS caveat, where the docs are, and the tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:36:34 +00:00
ae8123250b Release 0.4.0
The original iPodderX layout (toolbar, places, item table, Files pane),
the Classic theme, Directory and Popular, All Subscriptions with mark
everything read, sortable columns and a Size column, one meaning per
icon across the UI, and fixes for the double play, Escape in dialogs,
the dark-theme password box and paid feeds listed in Popular. See
CHANGELOG.md [0.4.0] and docs/history.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:35:08 +00:00
2c34a144ba History: the icon pass, sorting and the double play; fix a stale changelog line
docs/history.md gets the long-form entry CLAUDE.md asks for: what the
UI pass found, the three bugs it turned up (Escape inside a dialog's
text box, the white password box, the stray dot), why sorting is done
on the server, and how the Files pane came to play a file twice.
CHANGELOG's Added line for the item table named the old columns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:32:41 +00:00
669e8b5124 Files pane plays through the player bar, once
The pane drew its own <audio controls> for a downloaded file, and its
onplay also started the player bar, so one click played the same file
twice at once. The pane now has a play button (with the file's type
icon, like the other rows) that hands that exact file to the player
bar, the only player. play() takes the file, so another of an item's
files starts from its top instead of resuming the first. Dead CSS for
the pane's <audio> removed.

Test: play in the Files pane leaves one <audio> on the page, the bar's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:30:03 +00:00
5362436766 Sortable item table, Size in its own column, Subscribed as an icon
- Every column heading sorts (kept, title, feed, file type, size,
  published); a second click reverses it. The server sorts through a
  fixed whitelist (order_sql), so it covers the whole list, not the
  fifty loaded; the choice is remembered in the browser.
- Size is its own column and shows KB for small files instead of
  "0 MB". The Item heading is Title.
- Popular/Directory/Add feed: Subscribed is a green circle-check.
- Tests: every sort column runs and orders both ways (db); the table
  sorts by title both ways and remembers across a reload (browser).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:17:16 +00:00
0efc49519c One icon per meaning across the UI; mark everything read in All Subscriptions
- All Subscriptions' header checks every feed and marks everything read
  (POST /api/read-all, the same feeds the view lists); it asks first.
- Minus unsubscribes everywhere (the feed header's x read as "close"),
  x only closes or cancels, plus adds/subscribes/imports, and a dialog's
  confirm carries its action's icon. Remaining word buttons, the player
  and the folder arrow are Font Awesome 7.3.1 icons.
- Toolbar grouped by what it acts on (add, unsubscribe, scan | play,
  read, keep); read and keep show the selected item's state.
- The OPML subscription page uses the same header as a feed.
- Fixed: Escape ignored inside a dialog's text box (Add feed could not
  be closed with it), white password box in the dark theme, stray dot
  in an undated item's details.
- Tests: one action one icon across toolbar, page and all 8 dialogs;
  All Subscriptions mark everything read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
2026-09-11 17:05:47 +00:00
57dcba2d1a One file icon, green when downloaded; mark unread is an envelope
- The separate check (and warning) beside a file's type icon is gone:
  the type icon itself is green once the file is downloaded and red when
  the download failed, with the details in its tooltip. One icon per row
  keeps the column lined up. On Classic's blue selection they are a
  lighter green and red rather than white.
- Mark unread under an item's title was a solid circle, which read as a
  record button. It is Font Awesome's closed envelope now.
- Drops the unused circle-check, circle-exclamation and circle icons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:20:32 +00:00
e95cccc66f Icons are Font Awesome Free, embedded as SVG
The ICON set is now Font Awesome Free 7.3.1: the 29 icons the page uses,
taken from svgs/solid and svgs/regular at that tag and embedded as SVG
paths, 12.8 KB in all. No webfont to download and nothing fetched from a
CDN. The CC BY 4.0 attribution is above the set, and in the README.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:14:51 +00:00
7295be8b25 Drawn icons throughout; file state and type as icons, no PENDING
- Every button's icon is drawn from one small SVG set (ICON), in bold
  strokes of the button's own colour, instead of font characters: ⟳ ⤓ ↗
  and the like came out thin and tiny and differed from font to font.
  Static buttons name theirs with data-icon. Keep is a flag everywhere.
- A file's state is an icon: a check when downloaded, a warning with the
  error in its tooltip when it failed, nothing while it waits. Its type
  (audio, video, image, pdf, torrent, other) is an icon with the word in
  its tooltip. The DOWNLOADED and PENDING chips are gone, which also
  fixes them being hard to read on Classic's blue selection.
- Tests find state and type by their tooltips.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:11:15 +00:00
f1b0d97b81 Classic: white text across a selected row, white lists
A selected row's file size stayed grey on the Aqua blue; the whole row
is white now. The directory's rows took the source list's pale
blue-grey; lists were white in the original.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:03:04 +00:00
ae47e31a97 Classic theme after the 2004 Mac app; header and dialog buttons as icons
- A Classic theme beside Dark and Light: brushed-metal toolbar and
  status bar, a pale blue-grey source list, Aqua blue for whatever is
  selected, red unread badges, a striped table with blue titles, a blue
  bar behind the item's title, and Lucida Grande. The theme button steps
  through all three and remembers the choice.
- The feed and OPML headers' buttons (scan, download latest, mark all
  read, settings, unsubscribe) and the Settings, feed settings and
  Download latest dialogs' buttons (save, download, cancel) are icons,
  with the words in title and aria-label.
- Tests: the theme button reaches Classic and it survives a reload; the
  header buttons are found by action.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 15:00:30 +00:00
8a309eb652 Files pane and item buttons are icons, with the words in tooltips
Save, delete, view and download in the Files pane, and mark read, keep
and open the original under an item's title, are icons now. The word is
in each one's title and aria-label, so it is still there on hover and
for screen readers. Tests find them by title or action, not text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:57:03 +00:00
1a2b0d87c6 CLAUDE.md: what to do when Docker Hub rate-limits the image build
The build asks Docker Hub about its two base images every time unless
they are stored locally, and this host has no Docker Hub login, so a busy
day ends in 429 Too Many Requests. Pulling them from mirror.gcr.io and
tagging them locally lets the build go ahead without asking.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:49:58 +00:00
d7fac2d0e7 Never list paid-feed services; scan on add; no "reaped"; slimmer header
- Security: feeds from Patreon, Supercast, Supporting Cast, Glow and
  Memberful are never listed in Popular or the Directory. A Supercast
  feed keeps its key in the URL's path, which the query check missed, so
  it was being listed.
- Adding a feed queues a scan of it, and an OPML import that added feeds
  scans what is due, so items show without pressing Scan.
- A file deleted to save space, or by hand, looks as if it was never
  downloaded: no "reaped" chip, just the Download button. The retention
  summary says "deleted".
- The feed header keeps its title and stats to one line each and wraps
  its buttons; a single feed's table drops the Feed column.
- Tests: adding a feed shows its item without Scan; a deleted file shows
  no "reaped"; paid-feed hosts and acast public ids in the unit test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:47:03 +00:00
df9b7645d6 The original iPodderX layout: toolbar, places, item table, Files pane
- A toolbar across the window with the original's groups: add and
  unsubscribe, play, mark read and keep for the selected item, scan, a
  search box for what is showing, and Settings and Log (admins only).
- Directory, Popular and All Subscriptions sit at the top of the feed
  list and open in the main pane; the Popular and Directory buttons and
  their dialogs are gone.
- All Subscriptions lists every item from every feed you subscribe to:
  GET /api/entries, the per-feed query with its scope widened. The
  enclosure lookup after it matches files to rows by feed and guid, since
  a page can now span feeds.
- Items are a table (unread, kept, item, feed, file, published) with a
  Files pane beside it, the text below, and a status bar with totals. On
  a phone the files follow the text and the table is title and date.
- Tests: enclosures are checked in #files; the toolbar's read, keep and
  play act on the selected item; All Subscriptions holds only your feeds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:39:05 +00:00
f3825cfc57 Popular is a top 10; a Directory lists every listed feed A to Z
- GET /api/popular returns the ten most subscribed feeds. The new GET
  /api/directory returns every feed that may be listed, sorted by name,
  from the same list: everyone counted, you included, never a URL, never
  a private feed or a feed inside an OPML. POST /api/popular/{id} still
  subscribes to anything on it.
- A Directory button sits beside Popular; both open the same list
  screen. The sidebar toolbar wraps rather than squeezing four buttons.
- Tests: the directory is A to Z, Popular is its top ten, Paid Show is
  in neither, and subscribing works from the directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:13:16 +00:00
c0f4b0bcb2 Popular button in the sidebar; the popular list counts everyone
- A Popular button beside + Feed opens the popular list directly; the
  Add feed dialog keeps it too.
- The list counts every subscriber, you included. Your own feeds stay on
  it, marked Subscribed, and clicking one opens it. Private feeds and
  feeds inside an OPML are still never listed, for anyone.
- GET /api/popular rows carry `subscribed`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
2026-09-11 14:05:24 +00:00
27 changed files with 2188 additions and 749 deletions

View File

@@ -10,6 +10,138 @@ The long form, with what was wrong before and how it was found, is in
## [Unreleased] ## [Unreleased]
## [0.5.1] - 2026-09-12
### Fixed
- The triangle that opens an OPML or Patreon folder was cramped against the folder's art. It has
more room now, and a wider target to click.
## [0.5.0] - 2026-09-12
### Added
- A Patreon token pasted into Add feed, or a creator's RSS link without `&show=`, becomes a folder
of that creator's shows, kept in step on every scan like a subscribed OPML. A creator with only
one show stays a plain feed. One already added as a single long feed is split into its shows on
its next scan, keeping its files and what you had read.
- Add feed has an "Allow items marked explicit" box, so a new feed's first scan no longer skips
every explicit item.
### Changed
- Unread counts, unread dots and download progress are amber, the colour of the icon's EQ bars.
Blue is kept for the primary action and links, so a count no longer looks like a button.
- What is playing is marked by small EQ bars, in its row and in the player. They move only while it
plays.
- A folder in the sidebar shows its first four shows' art as a mosaic, and its shows sit under its
title. Only folders have a triangle, so every feed lines up with Directory and Popular above.
- Feeds without art get initials in a colour of their own, instead of all the same grey.
- The Flagged tab is called Kept, as the Keep button and Settings already said.
- A feed's header is one short line; when it checks next is in its tooltip.
- The item list takes more of the window, and the Files pane shows only when the item has files.
- Column headings and tags are in sentence case, and fewer things are bold.
- Unsubscribe is a round button beside the feed's other actions.
- Export and Import in Settings say what they do.
- The sign-in page shows the original icon large.
- Nothing animates when your system asks for reduced motion.
- The pages are about 90 KB smaller: the icon is served once instead of written into each.
- A web token generated for a new install is 64 characters instead of 32.
- The README is a short overview of what ipx does and how to run it, and points into `docs/` for
the rest. It still described the layout from before 0.4.0.
### Removed
- The systemd units in `contrib/`. Run ipx with Docker, or point a unit of your own at
`ipx daemon`.
- Upgrading from before 0.3.0 directly: what was read, kept or part-played before accounts is no
longer carried over to the admin, and OPML feeds that old versions wrote into `config.toml` are
no longer moved out of it. Upgrade through 0.4.0 first.
- `interval_mins` in `config.toml` is ignored; use `schedule`.
### Fixed
- The feed list works from the keyboard: Tab reaches every feed, Enter opens it, and Right and Left
open and close a folder. Every button shows where the focus is, including in the toolbar, which
used to clip the ring.
- "1 items" reads "1 item".
- A selected feed without art no longer loses its initials tile in Dark and Light.
- The player shows the feed's initials when there is no art, not the episode's.
- Turning on Allow explicit, or changing keywords or auto-download, brings back what those settings
had skipped on the feed's next scan. Before, an item was judged once, when first seen, and a
skipped one stayed skipped whatever you changed.
- Feeds inside an OPML or a Patreon creator follow your settings on the folder unless you set their
own, as the folder's settings dialog said they did. Before, the folder's settings reached nothing
inside it.
- A new feed no longer takes the name of one you removed earlier and shows that feed's old items.
Re-adding the same feed still gets its old name, and its history, back.
## [0.4.0] - 2026-09-11
### Added
- A Classic theme after the 2004 Mac app, beside Dark and Light: brushed-metal toolbar, Aqua
blue selection, red unread badges, a striped table and Lucida Grande. The theme button steps
through all three and remembers the choice.
- A toolbar across the top, after the original iPodderX: add and unsubscribe, play, mark read
and keep for the selected item, scan, a search box for what is showing, and Settings and Log.
- Directory, Popular and All Subscriptions at the top of the feed list, opening in the main pane.
Directory lists every feed anyone here subscribes to, A to Z (`GET /api/directory`). All
Subscriptions lists every item from every feed you subscribe to (`GET /api/entries`).
- Items show as a table (unread, kept, title, feed, file, size, published) with a Files pane beside
it, and a status bar with the totals.
- Mark everything read from All Subscriptions, across every feed you subscribe to
(`POST /api/read-all`). It asks first. All Subscriptions can also check every feed from its header.
- Click a column heading in the item table to sort by it (kept, title, feed, file type, size,
published); click again to reverse. The server sorts, so it covers the whole list, not just the
fifty shown, and the choice is remembered.
### Changed
- Popular shows the top 10, not 20, and counts everyone, you included. Your own feeds stay on it,
marked Subscribed, and clicking one opens it.
- Adding a feed scans it straight away, and an OPML import that added feeds scans them, so their
items show without pressing Scan.
- A file deleted to save space, or by hand, looks as if it was never downloaded: no "reaped"
label, just the Download button. The retention summary says "deleted", not "reaped".
- Buttons are icons, with the words in their tooltips: the Files pane (save, delete, view,
download), an item's own buttons (mark read, keep, open the original), the feed header (scan,
download latest, mark all read, settings, unsubscribe), and the Settings, feed settings and
Download latest dialogs (save, download, cancel). The icons are Font Awesome Free, embedded as
SVG: only the ones used, no font to download, and nothing fetched from anyone else. They
replace font characters such as ⟳ ⤓ ↗, which came out thin and tiny and differed from font to
font. Keep is a flag everywhere, as it was in the original, and mark unread is an envelope.
- One meaning per icon. Minus unsubscribes, x closes or cancels, plus adds or subscribes, and a
dialog's confirm button carries the icon of what it does. The feed header's unsubscribe was an x
and read as closing the page. The remaining word buttons are icons too:
- Log, Add feed, Users, Unsubscribe and OPML.
- Popular's Subscribe, Copy and Sign out.
- The Subscribed label in Popular, the Directory and Add feed, which is now a green check.
- The player's back, play, forward and close, which were font characters, and the folder arrow.
- The toolbar's read and keep buttons show the selected item's state, with the same icons as the
item's own buttons. Play, read and keep sit together, and Scan sits with add and unsubscribe.
- An OPML subscription's page has the same header as a feed's, with its buttons in the same places.
- The item table's size has its own column, apart from the file's type, and shows KB for small
files instead of "0 MB". The Item heading is now Title.
- A file's type is an icon (audio, video, image, PDF, torrent, other), green once it is
downloaded and red when the download failed, with the details in its tooltip. One icon per
row keeps the column lined up. The DOWNLOADED and PENDING labels are gone.
### Fixed
- Playing a file from the Files pane played it twice at once, in the pane and in the player bar.
The pane has a play button now, and the player bar is the only player.
- The password box in Manage users was white in the dark theme.
- An item with no date showed a stray dot in its details.
- Escape did not close a dialog while the cursor was in one of its boxes, so Add feed, which opens
in its URL box, could not be closed with Escape.
### Security
- Feeds from paid-feed services (Patreon, Supercast, Supporting Cast, Glow, Memberful) are never
listed in Popular or the Directory. A Supercast feed, which keeps its key in the URL's path
rather than the query, was being listed.
## [0.3.0] - 2026-09-11 ## [0.3.0] - 2026-09-11
### Added ### Added
@@ -133,7 +265,10 @@ The long form, with what was wrong before and how it was found, is in
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout. - Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`. - `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...main [unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.1...main
[0.5.1]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.0...v0.5.1
[0.5.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.4.0...v0.5.0
[0.4.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...v0.4.0
[0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0 [0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0
[0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0 [0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0
[0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0 [0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0

View File

@@ -30,6 +30,20 @@ docker logs --tail 20 iPodderX
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich **Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has. included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
**A build that fails with `429 Too Many Requests` on a base image** is Docker Hub rate-limiting
this host. There is no Docker Hub login here, and the build asks about `debian:bookworm-slim` and
`rust:1-slim-bookworm` every time unless they are already stored locally. Pull them from Google's
mirror and tag them; the build then uses the local copies without asking Docker Hub:
```sh
docker pull mirror.gcr.io/library/debian:bookworm-slim
docker tag mirror.gcr.io/library/debian:bookworm-slim debian:bookworm-slim
docker pull mirror.gcr.io/library/rust:1-slim-bookworm
docker tag mirror.gcr.io/library/rust:1-slim-bookworm rust:1-slim-bookworm
```
Run those again now and then, or the local copies go stale.
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
the worker is alive, not just the web port. The container restarts on its own after a reboot. the worker is alive, not just the web port. The container restarts on its own after a reboot.
@@ -85,9 +99,9 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every * **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
subscriber. Two feeds publishing the same URL means only the first one scanned shows it. subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
* **`entries.read`, `entries.flagged` and `entries.position` are dead columns.** Read state lives in * **Read state lives in `entry_state`, per user, and nowhere else.** `entries` had `read`, `flagged`
`entry_state` per user. Two bugs have already come from queries still reading the old ones and `position` columns from before accounts; two bugs came from queries still reading them
(retention, and the entry pruner) — grep before adding a third. (retention, and the entry pruner), and `migrate()` now drops them.
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once; * **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are `subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
derived and never written to config. derived and never written to config.
@@ -101,6 +115,10 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
fighting over the database, with the stale one still holding the port. fighting over the database, with the stale one still holding the port.
* `/api/settings` answering `200` does **not** mean the worker is alive — it is a different task. * `/api/settings` answering `200` does **not** mean the worker is alive — it is a different task.
Probe the control socket (`ipx status`) to check that. Probe the control socket (`ipx status`) to check that.
* **Every `ipx` command runs `migrate()` when it opens the database**, the healthcheck's
`ipx status` included. A migration that rewrites a big table (`DROP COLUMN`) takes seconds on
production, and a command run meanwhile fails with `migrating schema`. It changes nothing; wait
for `daemon started` in the log. Copy `state.db` aside before deploying one.
## House style ## House style

34
Cargo.lock generated
View File

@@ -436,17 +436,6 @@ dependencies = [
"shlex", "shlex",
] ]
[[package]]
name = "cfb"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a347dcabdae9c31b0825fd6a8bed285ec9c2acb89c47827126d52fa4f59cece3"
dependencies = [
"fnv",
"uuid",
"web-time",
]
[[package]] [[package]]
name = "cfg-if" name = "cfg-if"
version = "1.0.4" version = "1.0.4"
@@ -867,15 +856,6 @@ dependencies = [
"dirs-sys", "dirs-sys",
] ]
[[package]]
name = "dirs"
version = "7.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d57d423b3c82e89b9a24ca3091fee61f456a26edbd28d26c65906f4bc1dcd8f"
dependencies = [
"dirs-sys",
]
[[package]] [[package]]
name = "dirs-sys" name = "dirs-sys"
version = "0.5.0" version = "0.5.0"
@@ -1608,15 +1588,6 @@ dependencies = [
"serde_core", "serde_core",
] ]
[[package]]
name = "infer"
version = "0.22.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f4200d433cbd5178df7797c9c2e75b348b728e39631cf14520d1e2fc424201f4"
dependencies = [
"cfb",
]
[[package]] [[package]]
name = "intervaltree" name = "intervaltree"
version = "0.2.7" version = "0.2.7"
@@ -1634,7 +1605,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
[[package]] [[package]]
name = "ipx" name = "ipx"
version = "0.3.0" version = "0.5.1"
dependencies = [ dependencies = [
"ammonia", "ammonia",
"anyhow", "anyhow",
@@ -1643,9 +1614,7 @@ dependencies = [
"axum", "axum",
"chrono", "chrono",
"clap", "clap",
"dirs",
"futures-util", "futures-util",
"infer",
"librqbit", "librqbit",
"opml", "opml",
"percent-encoding", "percent-encoding",
@@ -1656,7 +1625,6 @@ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
"tokio", "tokio",
"tokio-stream",
"toml", "toml",
"tower", "tower",
"tower-http 0.7.1", "tower-http 0.7.1",

View File

@@ -1,6 +1,6 @@
[package] [package]
name = "ipx" name = "ipx"
version = "0.3.0" version = "0.5.1"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
@@ -11,9 +11,7 @@ atom_syndication = "0.12.10"
axum = "0.8.9" axum = "0.8.9"
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] } chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
clap = { version = "4.6.6", features = ["derive"] } clap = { version = "4.6.6", features = ["derive"] }
dirs = "7.0.0"
futures-util = { version = "0.3.34", default-features = false, features = ["std"] } futures-util = { version = "0.3.34", default-features = false, features = ["std"] }
infer = "0.22.0"
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] } librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
opml = "1.1.6" opml = "1.1.6"
percent-encoding = "2.3.2" percent-encoding = "2.3.2"
@@ -24,7 +22,6 @@ rusqlite = { version = "0.40.2", features = ["bundled"] }
serde = { version = "1.0.229", features = ["derive"] } serde = { version = "1.0.229", features = ["derive"] }
serde_json = "1.0.151" serde_json = "1.0.151"
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] } tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
tokio-stream = { version = "0.1.19", features = ["sync"] }
toml = "1.1.5" toml = "1.1.5"
tower = { version = "0.5.3", features = ["util"] } tower = { version = "0.5.3", features = ["util"] }
tower-http = { version = "0.7.1", features = ["fs"] } tower-http = { version = "0.7.1", features = ["fs"] }

141
README.md
View File

@@ -1,120 +1,83 @@
# ipodderx-rs # ipodderx-rs
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent), files them A self-hosted podcatcher for a household. It checks your feeds, downloads the episodes, and serves
into per-feed folders, and reaps old files to stay under a disk quota. Runs as a one-shot CLI or as a web UI modelled on the 2004 Mac app **iPodderX**, for any number of people sharing one copy of
a daemon with a web UI, serving any number of people from one copy of the data. the files. One Rust binary, `ipx`, is both the daemon and the command line.
## Lineage It is a rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python engine
behind iPodderX (2004-2008, Ray Slakinski & August Trometer).
A modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python 2 ## What it does
engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), open-sourced under the MIT
License in 2010.
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download - **The web UI.** It has a toolbar, and a feed list that opens with Directory, Popular and All
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the Subscriptions. Items sit in a sortable table with a Files pane, and there is a player bar. It
oldest-first disk quota reaper. comes in Dark, Light and Classic themes, and works on a phone.
- **Several people, one copy.** Each person has their own subscriptions and their own read, kept
and playback state. There is one file on disk per episode, however many people want it. People
sign in with a password or through a proxy (Cloudflare Zero Trust or Authentik), and admins
manage accounts and settings.
- **Scanning.** Feeds are checked on a schedule, globally or per feed, and a feed's own TTL is
honoured. Keyword, explicit-content and media-type filters decide what is downloaded, with a cap
on new downloads per scan.
- **Downloads.** Files come over HTTP or BitTorrent and are filed into a folder per feed.
Retention deletes the oldest files to stay under a disk quota or an age limit, and never touches
an item someone has kept.
- **OPML.** You can import and export your own subscriptions. You can also subscribe to an OPML
URL, which keeps a whole list in step as a folder.
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows ## Run it
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
3DES-encrypted preferences, and the `printMSG` stdout protocol -- replaced by a JSON-lines socket.
## Quick start With Docker:
```sh
docker build -t ipodderx .
docker compose up -d
```
`docker-compose.yml` is set up for the author's own server. Point its `image` and its three volumes
(`/config`, `/data` and `/downloads`) at yours first. The UI is on port 8099. BitTorrent uses 6881
over TCP and UDP. Files are written as `PUID`/`PGID`, 99:100 by default.
From source:
```sh ```sh
cargo build --release cargo build --release
install -m755 target/release/ipx ~/.cargo/bin/ ./target/release/ipx daemon
ipx add https://atp.fm/rss # subscribe
ipx fetch # scan and download
ipx daemon # scheduler, control socket and web UI
``` ```
On first start with `[web] enabled = true` the daemon mints a token, writes it to config.toml and The first start creates **admin / ipodderx**. Sign in at `/login`, then change it:
prints the URL to open. A database with no accounts starts with **admin / ipodderx** at `/login` --
change it with `echo -n '<password>' | ipx user passwd admin`. ```sh
echo -n 'a good password' | ipx user passwd admin
```
The UI is plain HTTP, so put TLS in front of it if it is reachable from outside your network.
## Documentation ## Documentation
| | | | | |
|---|---| |---|---|
| [docs/configuration.md](docs/configuration.md) | Every config key, paths, environment variables | | [docs/configuration.md](docs/configuration.md) | Every config key, path and environment variable |
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` | | [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
| [docs/users.md](docs/users.md) | Accounts, and what several people share | | [docs/users.md](docs/users.md) | Accounts, and what several people share |
| [docs/sso.md](docs/sso.md) | Cloudflare Zero Trust or Authentik in front | | [docs/sso.md](docs/sso.md) | Signing in through Cloudflare Zero Trust or Authentik |
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, socket, HTTP API | | [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, control socket, HTTP API |
| [CHANGELOG.md](CHANGELOG.md) | What changed, by release | | [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
| [docs/history.md](docs/history.md) | How it was built: the long form, with what was wrong and why | | [docs/history.md](docs/history.md) | How it was built, with what was wrong and why |
| [CLAUDE.md](CLAUDE.md) | Notes for anyone (or anything) working on the code | | [CLAUDE.md](CLAUDE.md) | Notes for working on the code, including how production is deployed |
## The web UI
`ipx daemon` serves it in the same process, so it reads SQLite and the event bus directly.
Feeds down the side; the selected feed's items across the top; the selected item's text and its
enclosures below, which is where you play, download or delete them. The divider drags and its
position is remembered. Playback serves Range requests, so seeking works. An OPML subscription is a
collapsible folder whose page lists the feeds inside it.
An item may carry several enclosures; all of them appear below, and anything that is not audio or
video gets a View link rather than a player -- the publisher's copy until it is downloaded, the
local one after. Opening an item marks it read. Show notes are untrusted feed HTML, sanitized with
`ammonia` server-side before they reach the page.
The **Log** button shows the running daemon live in four tabs: *Daemon I/O* is the control protocol
itself, every command in and event out; *Scans* is feed and download activity; *HTTP* is web
requests; *All* is everything, with level and text filters and a copy button. It reads a ring buffer
held in the process, not a file, so it works the same under Docker.
It is plain HTTP. On a LAN bind the token and everything else cross the network in the clear, and a
feed URL can itself carry a credential. Put TLS in front of it if that matters.
## OPML
**Importing and exporting** a file copies subscriptions in or out once: `ipx import subs.opml`,
`ipx export subs.opml`, or Settings → Subscriptions in the UI.
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
other feed; every scan re-reads it and keeps your list in step. The feeds inside are not written to
config.toml -- the OPML is the source of truth, so they are re-derived each scan and held in the
database. They show as a folder, download into one nested folder, and inherit the subscription's
settings until you change one, which gives it its own entry.
When a feed drops out of the OPML upstream, it is unsubscribed and removed -- unless it has
downloads, in which case it is kept and flagged in the UI as no longer listed. A downloaded file is
never left behind with nothing explaining where it came from.
## Docker
```sh
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
docker compose pull ipodderx && docker compose up -d ipodderx
docker compose logs -f ipodderx # the first start prints the default admin password
```
`docker-compose.yml` runs the image from the registry above rather than building it, so build and
push first; change the tag in both places to use another registry. It mounts `/config` (config.toml),
`/data` (state.db) and `/downloads` from this install's host paths, which you will want to change for
yours. It publishes 8099 for the
UI and 6881 (TCP **and** UDP -- DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
land owned the way Unraid shares expect. The healthcheck runs `ipx status` through the control
socket, so it catches a daemon that is alive but wedged rather than merely one that has died.
## Running it as a service
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
rather run periodic scans with no daemon -- in which case there is no socket for a UI to attach to.
## Tests ## Tests
```sh ```sh
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
node tests/page-smoke.js # the page script loads without throwing node tests/page-smoke.js # the page script loads without throwing
npx playwright test # a real browser against a real daemon npx playwright test # a real browser against a real daemon on fixture feeds
``` ```
`npm install` gets the test runner; the browser comes from `npm install` gets the test runner, and `npx playwright install --with-deps chromium` gets the
`npx playwright install --with-deps chromium` (in `install.sh`). browser.
## License ## License
MIT. See [LICENSE](LICENSE). MIT, see [LICENSE](LICENSE). The icons are [Font Awesome Free](https://fontawesome.com) 7.3.1 by
@fontawesome, under [CC BY 4.0](https://fontawesome.com/license/free), embedded as SVG.

30
TODO.md Normal file
View File

@@ -0,0 +1,30 @@
# To do
## Cut what is no longer needed
From a whole-repo audit for over-engineering on 2026-09-12. Biggest cut first.
- [x] **Pre-accounts adoption and the dead `entries` columns.** The copy of the old read state into
`entry_state` and the `entries.read`, `flagged` and `position` columns are gone. Its other half,
subscribing the first admin to the catalogue, was not dead and stays as `adopt_catalogue`.
(`src/db.rs`, `src/main.rs`)
- [x] **`contrib/` systemd units.** From before the container; nothing points at them.
- [x] **`migrate_opml_children`.** A one-time move of OPML children out of `config.toml` that has
run. Delete it and its call. (`src/main.rs`)
- [x] **The legacy `interval_mins` key.** Production uses `schedule`. Delete the field, the fallback
in `General::interval` and its test. (`src/config.rs`)
- [x] **`Db::entries` and `Db::count_entries`.** One-line wrappers only the tests call; the tests
call `entries_in` and `count_in` instead. (`src/db.rs`)
- [x] **`web::generate_token`.** Repeats `auth::new_session_token`. Use that. (`src/web.rs`)
- [x] **Page leftovers.** `globalEvery`, `S.busy`, `S.limit`, `unitOptions`' `firstLabel`, `--r`,
`.ep.open`, the phone `.ep .art`, the duplicate phone `.fhead.slim{flex-wrap}`, the second
`#sidebar{z-index}`, and the `on()` helper. (`web/index.html`)
- [x] **`logbuf` visitors.** `record_i64`, `record_u64` and `record_bool` repeat what `Visit`'s
defaults already do through `record_debug`. (`src/logbuf.rs`)
- [x] **The `infer` dependency.** Its torrent check is the `d8:announce` test on the next line.
- [x] **The `dirs` dependency.** `XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME` from `std::env`.
- [x] **The `tokio-stream` dependency.** `futures_util::stream::unfold` over the broadcast receiver.
- [x] **The icon inlined four times.** About 94 KB of base64 across both pages; serve it once as
`/icon.png` from `include_bytes!`, open without signing in like `/login`.
After these: `cargo test`, `node tests/page-smoke.js`, `npx playwright test`.

View File

@@ -1,7 +0,0 @@
[Unit]
Description=ipx feed scan (one shot)
[Service]
Type=oneshot
ExecStart=%h/.cargo/bin/ipx fetch
Environment=IPX_LOG=ipx=info

View File

@@ -1,18 +0,0 @@
# User unit: install to ~/.config/systemd/user/ipx.service, then
# systemctl --user enable --now ipx
# The socket lands in $XDG_RUNTIME_DIR/ipx.sock by default, so a UI running as the
# same user can attach without extra configuration.
[Unit]
Description=ipx podcatcher
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=%h/.cargo/bin/ipx daemon
Restart=on-failure
RestartSec=30
Environment=IPX_LOG=ipx=info
[Install]
WantedBy=default.target

View File

@@ -1,16 +0,0 @@
# Alternative to the daemon: a periodic one-shot scan, closer to how the original
# iPodderX agent was driven. Use this OR ipx.service, not both -- with no daemon
# running there is no socket, so a UI cannot attach.
#
# Install ipx-scan.service and ipx.timer to ~/.config/systemd/user/, then
# systemctl --user enable --now ipx.timer
[Unit]
Description=Periodic ipx feed scan
[Timer]
OnBootSec=5min
OnUnitActiveSec=1h
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -28,12 +28,15 @@ The page is compiled in, so **editing `web/index.html` needs a rebuild**.
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this). 1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there. 2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are 3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
re-derived into the database each scan, never written to config.toml. re-derived into the database each scan, never written to config.toml. A Patreon creator link
(a token, no `show=`) with more than one show is treated the same way, before any fetch: its
shows come from Patreon's web API and each becomes a derived feed.
4. Record entries. A changed title or description flips the item back to unread. 4. Record entries. A changed title or description flips the item back to unread.
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the 5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice. original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with 6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
a reason. a reason. What a filter skipped is judged again every scan, so a change of settings brings it
back. A feed in a group takes your settings on the group for anything you have not set on it.
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to 7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
or an error page — and is deleted. or an error page — and is deleted.
@@ -55,9 +58,9 @@ entry_state user_id, feed_id, guid, read, flagged, position
PK (user_id, feed_id, guid) PK (user_id, feed_id, guid)
``` ```
`entries` still has `read`, `flagged` and `position` columns from before accounts existed. They are Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position` columns from
**dead** — the migration copied them into `entry_state` and nothing reads them now. Anything found before accounts; two bugs came from queries still reading them, and `migrate()` drops them from an
querying them is a bug; two were. older database.
Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in
`migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the `migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the
@@ -104,13 +107,15 @@ else a `401`.
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in | | `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe | | `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe | | `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable | | `GET /api/feeds/{id}/entries` | paged, filtered, searchable, sortable (`sort` = kept, title, feed, type, size or published; `dir` = asc or desc) |
| `GET /api/entries` | the same, across every feed you subscribe to (All Subscriptions) |
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | | | `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, starred, position | | `POST /api/read-all` | everything read in every feed you subscribe to (All Subscriptions) |
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, kept, position |
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning | | `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
| `POST /api/fetch` | | | `POST /api/fetch` | |
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML | | `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
| `GET /api/popular`, `POST /api/popular/{id}` | what others here subscribe to (id, title, art, count; never a URL, never a private feed); subscribe by id | | `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write | | `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed | | `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
| `GET /api/events` | SSE, the same broadcast the socket carries | | `GET /api/events` | SSE, the same broadcast the socket carries |

View File

@@ -60,7 +60,7 @@ ipx reap # actually delete
``` ```
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days` Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
with no file are pruned from the database. **Starred by anyone keeps a file**, and one only counts with no file are pruned from the database. **An item anyone kept keeps its file**, and one only counts
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
stops the next scan fetching it again. stops the next scan fetching it again.

View File

@@ -32,10 +32,10 @@ media_types = ["audio", "video"]
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI. be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`. * **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB * **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
pad. Starred items are never deleted, and a file only counts as read once every subscriber has pad. Kept items are never deleted, and a file only counts as read once every subscriber has
read it. `0` disables it entirely. read it. `0` disables it entirely.
* **`max_age_days`** — items older than this with no file on disk are pruned from the database. * **`max_age_days`** — items older than this with no file on disk are pruned from the database.
Starred ones stay. `0` disables it. Kept ones stay. `0` disables it.
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue. * **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap `0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
fetched 216 files and 22 GB in one scan. fetched 216 files and 22 GB in one scan.
@@ -43,8 +43,6 @@ media_types = ["audio", "video"]
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
without this the disk fills with artwork. Empty takes everything. without this the disk fills with artwork. Empty takes everything.
`interval_mins` from older configs is still read, and `schedule` supersedes it.
## `[torrent]` ## `[torrent]`
```toml ```toml

View File

@@ -6,6 +6,217 @@ reasoning lives. New write-ups go at the top.
See [README.md](../README.md) for what the thing is. See [README.md](../README.md) for what the thing is.
## 2026-09-12 — Cutting what had outlived its reason
A whole-repo audit for over-engineering listed twelve things to cut, and all of them went.
- **Upgrades from before accounts.** `migrate_opml_children` moved OPML feeds that old versions
wrote into `config.toml` out to the database, and ran at every daemon start to do nothing after
the first. Production ran it in 0.3.0; anything older has to pass through 0.4.0.
- **Half of the adoption, and not the other half.** The audit called `adopt_existing_library` a
one-time migration and it was cut whole. It did two jobs: copy the old read state into
`entry_state`, which was dead, and subscribe the first admin to the whole catalogue while nobody
subscribed to anything, which is how a fresh install's first account gets `config.toml`'s feeds.
The browser suite caught it at once, signing in to an empty sidebar; `cargo test` had no idea.
The second job is back as `adopt_catalogue`, with a unit test of its own.
- **The dead `entries` columns.** `read`, `flagged` and `position` moved to `entry_state` with
accounts. The adoption's copy was their last reader, but `record_entry` still wrote them, and
still reset `read` when a title changed, which nothing looked at. Two bugs came from queries
reading them. `migrate()` now drops them from an existing database (SQLite has had `DROP COLUMN`
since 3.35), and a test builds an old table to prove it. On production each drop rewrote the
66 MB `entries` table, about four seconds apiece, so the first start took thirteen. An
`ipx status` run in that window failed with `migrating schema`: every `ipx` command migrates when
it opens the database, and it collided with the daemon doing the same. A failed `ALTER TABLE`
changes nothing, and the database had been copied to `backup/` first anyway.
- **`interval_mins`**, which `schedule` replaced. An old config that still has the key loads; the
key is ignored, and the config test carries it to keep that true.
- **Three dependencies.** `infer` was only asked whether a file is a torrent, and the check after
it already looked for `d8:announce`, which is what `infer` looks for. `dirs` was three lookups of
`XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME`. `tokio-stream` wrapped the broadcast receiver for
the event stream; `futures_util::stream::unfold` does the same, lagging clients included.
- **Two token generators.** The web token came from a copy of the session-token code, with a
clock fallback on top. It uses `auth::new_session_token` now, and is 64 characters.
- **The icon inlined four times**, 23 KB of base64 each, into both pages. It is `/icon.png` now,
outside the sign-in wall with `/login`, since the sign-in page shows it.
- Also: the `contrib/` systemd units from before the container, `Db::entries` and
`Db::count_entries` that only the tests called, three `logbuf` visitors that repeated the trait's
defaults, and unused state, a helper and dead CSS in the page.
## 2026-09-11 — A design pass on the web UI
A review against screenshots of every view in all three themes found that Dark and Light read as a
generic dark dashboard: one pale blue did every job, most labels were bold, and nothing led. The
list it produced, in `TODO.md`, was worked through in one go. What was worth knowing:
- **Amber means new.** Badges, unread dots and download bars take the icon's EQ amber; blue is left
for the primary action and links. Light's amber was `#b06f10`, which gives white text 4.1:1,
short of AA for 11 px bold. It is `#9a5f0a` now, 5.2:1.
- **EQ bars mark what is playing.** Three `<i>` bars stand at 60, 100 and 40 % and animate only
while `body.playing` is set. The first version left the animation on but paused, expecting each
bar to hold a different frame. The frames it held were within a pixel of each other, and on
screen the bars read as three dots. Under reduced motion one rule drops every animation and
transition, which leaves the bars standing.
- **The focus ring was clipped.** `.tgroup` and `#topbar` both set `overflow:hidden`, so a ring
drawn outside a toolbar button was cut off. Rings inside clipping parents are inset instead.
- **The feed list could not be used from the keyboard at all.** Rows were `<div>`s and the triangle
a `<span>`, so Tab went from the feed filter to Sign out. Rows now take focus, the triangle is a
`<button aria-expanded>`, and `renderFeeds` puts focus back on the same row after redrawing,
since every live update replaces every row. The list's own key handler stops Space and the
arrows from reaching the player's shortcuts on the document.
- **The triangle hangs in the margin.** Every row used to reserve an 18 px slot for it, pushing a
hundred feeds 28 px right of the places above for the sake of two folders. It is now absolutely
placed in the row's left padding, the full height of the row, so a near miss no longer opens
the folder's page.
- **A selected tile vanished** because the initials tile and the selected row were both `--raise`.
Tiles now mix their tint into `--bg`, which no row uses.
- **The Files pane hides itself** with `#split:has(>#files[hidden])`, which collapses its column.
The phone layout already hides the pane, and a zero-width extra track there is harmless.
- **Flagged became Kept** in the tab, and in the server's refusal to delete a file someone else
kept. The filter value and the column stay `flagged`; renaming those buys nothing.
## 2026-09-11 — A Patreon creator is a list of shows
Ray asked whether ipx could sync with Patreon. Not in full. The documented API (v2, the
`identity.memberships` scope) lists the creators you back and whether each has a feed (`has_rss`),
but no resource carries the `auth` token that makes a feed URL work. That token only comes from the
creator's page. It is also one per membership, not one per account: techpod's differs from Glass
Cannon's, so no single token finds everything you back.
What does work is one creator at a time, which is what Ray wanted for Glass Cannon and its 33 shows:
- `patreon.com/rss?auth=<token>`, with no creator named, returns that token's creator. Its self link,
about 660 bytes in, gives the campaign by number (`/rss/369921`). Patreon ignores `Range` here, so
ipx reads the stream until the number appears and hangs up, instead of taking all 2.8 MB.
- A show's `show=` number is a Patreon collection. Asked anonymously, the collection listing
(`/api/collection?filter[campaign_id]=`) and a post's `collections` both hide patron-only ones: you
get "FAQ". `/api/campaigns/<id>?include=shows` lists every show, anonymously, in one response.
- Every spelling works: `rss/glasscannon?auth=…&show=N`, `rss/369921?…` and `rss?auth=…&show=N` all
serve the same 131 items. Enclosure URLs are the same in the creator feed and the show feed, and
stable between fetches.
That last point shaped the design. `enclosures.url` is unique, so whichever feed is scanned first owns
the file. The first cut only asked a creator for its shows while it had no entries of its own, so that a
creator already read as a plain feed, holding every show's episodes, would never be split into shows
that came up empty. Within the hour that was the wrong call: Glass Cannon had gone into production on
the build before this one, been read as one feed of 2,385 items, and the rule kept it that way. Finding
anything in that heap was the problem Ray wanted solved.
So a creator with more than one show is always a group, run through the same sync as an OPML
(`sync_group`, split out of `sync_opml`). When it becomes one, its items are cleared and each show
takes over the enclosures the creator holds as the show lists them (`Db::adopt`), downloaded files and
everyone's read state included. One show leaves it a plain feed, which is what techpod already was. If
the shows cannot be listed, a creator already split fails the scan rather than being read as one heap;
one that never was is read as one feed until they can be. An answer without a `shows` list is an error,
not "no shows".
**Filter verdicts follow the settings.** Ray also reported that turning on Allow explicit and
rescanning brought nothing back. An item was judged once, when first seen, and `skipped` was final. The
2026-09-10 entry below saw it coming ("worth a `ipx retry <feed>` command if this bites"). It bit:
2,166 Glass Cannon items and all 88 of Shadowdark's sat at `skipped: explicit` with the setting on.
Every scan now runs the filters again over what they skipped (not over `torrents disabled`, which is not
a filter's call) and requeues what they now let through. Only that direction: a queued item is never
pulled back, because Download latest and a manual download both work by queueing.
**Two gaps beside it.** Add feed had no explicit box, so every new feed's first scan skipped all its
explicit items; it has one now, stored on your subscription like the feed dialog's. And a feed inside a
group ignored your settings on the group, though the group's dialog said they were inherited: settings
live on each person's subscription, and nothing read the group's. `Db::subscribers` now fills what you
have not set on the feed from your subscription to the group, and the feed list shows the same.
**A name that was already used.** Replaying the split on a copy of the production database left one
show with a Supercast episode in it. "Glass Cannon Live! Ascension | Pathfinder 2E" slugs to
`glass-cannon-live-ascension-pathfinder-2`, the id of a Supercast feed of the same show that had been
removed. Removing a feed keeps its rows on purpose, so that re-adding it does not fetch the back
catalogue again, but choosing a new id only checked config.toml and derived feeds. The Patreon show took
the old id and everything still filed under it. An id is now also taken when the database has a feed by
that id at a different URL; the same URL may still have it back, which is the re-add case.
Shows already added by hand are matched by token and show number, not by exact URL (`same_feed`), so a
bare token does not add Get in the Trunk and Shadowdark a second time under another spelling.
The show listing is Patreon's own undocumented web API. If it changes, only finding new shows stops.
## 2026-09-11 — One meaning per icon, sortable columns, and one player
Ray asked for a pass over the whole UI: consistent icons, and buttons placed next to what they act
on. The pass started from screenshots of every view and dialog, taken against the browser suite's
own daemon and fixtures. Several things did not agree:
- Unsubscribe was a minus in the toolbar but an x in the feed header. There it read as "close this
page", and x also meant Cancel in every dialog.
- Half the dialogs still had word buttons: Log, Add feed, Users, Unsubscribe, OPML, Popular's
Subscribe, the Copy buttons and Sign out.
- The player bar and the folder arrow were font characters, and so was the row's "playing" marker.
Those came out thin and varied from font to font.
- The OPML subscription page laid out its header differently from a feed's, and All Subscriptions
had no buttons at all.
The rule now is one meaning per icon. Minus unsubscribes, x closes or cancels, plus adds, subscribes
or imports, and a dialog's confirm button carries the icon of what it does. The words go in the
tooltip. The toolbar is grouped by what it acts on: add, unsubscribe and scan act on feeds; play,
read and keep act on the selected item. Its read and keep buttons now show that item's state, as
the item's own buttons already did. A browser test opens all eight dialogs and fails on any button
without an icon and a tooltip. It also checks that the toolbar and the page use the same icon for
the same action.
The pass turned up three bugs:
- **Escape ignored inside a dialog's text box.** The keydown handler returned early for any key
typed in a text box, and Escape sat below that check. So Add feed, which opens with the cursor in
its URL box, could not be closed with Escape. It showed up because the screenshot run stalled:
the dialog stayed open, and every later click landed on its overlay until the run timed out.
- **A white password box in the dark theme.** The input rule listed text, search and number
fields, but not password ones.
- **A stray "• •" under an item with no date.** Each detail carried its own trailing dot, whether
or not the next one was there.
All Subscriptions gained Mark everything read, as `POST /api/read-all`. It marks the feeds from the
person's subscriptions, which is the same set the view's query is scoped by. It asks first, since
one click covers every feed.
**Sorting** happens on the server. A list arrives fifty at a time, so sorting in the page would
only have reordered what was loaded. The column name picks one of a fixed set of SQL expressions
(`order_sql`), so nothing the browser sends reaches the query, and an unknown name means newest
first. Ties also fall back to newest first, so "Load more" does not skip or repeat items at a page
boundary. A database test runs every column both ways. Size came out of the File column into its
own, and small files show KB, since whole megabytes made every article image "0 MB".
**The double play.** Ray found that pressing play in the Files pane played the file twice at once.
The pane drew its own `<audio controls>`, and its `onplay` handler also started the player bar on
the same file. That meant two audio elements from one click. The pane now has a play button that
hands that exact file to the player bar, the only player. `play()` now takes the file. Its check
for resuming compares the file as well as the item, so another of an item's files starts from its
beginning instead of carrying on with the first. A test plays from the pane and checks the page
has one `<audio>` element.
---
## 2026-09-11 — The original's layout
Ray pointed at a screenshot of the Mac app (techpp.com, 2012) and asked for its panes, its
toolbar, and a Directory that lives in the feed list rather than behind a button. What it had, and
what ipx now does:
- A toolbar across the window: subscribe and unsubscribe, play, flag, refresh, and a search box
scoped to the feed on show. ipx's has the same groups, acting on the selected feed and item, with
Settings and Log at the right end, for admins only.
- A source list opening with Directory, Playlist Builder and All Subscriptions above the feeds.
ipx has Directory, Popular and All Subscriptions there, opening in the main pane. Playlist
Builder is left out, since nothing here builds playlists.
- The entries as a table, with a Files pane beside it and the entry below. The columns are unread,
kept, the item, its feed, its file and when. Sorting by column is not done yet.
- A status bar with the totals for what is on show.
All Subscriptions needed one new endpoint, `GET /api/entries`. It is the per-feed query with
`feed_id = ?` swapped for the person's subscriptions. The enclosure lookup that follows it used to
filter by feed as well; a page can now span feeds, so each file is matched to its row by feed and
guid instead.
A phone has no room for a pane beside the table, so there the files follow the item's text in the
full-screen reader, and the table drops to title and date.
---
## 2026-09-11 — Popular on this server ## 2026-09-11 — Popular on this server
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
@@ -26,6 +237,13 @@ with everything else, so this is built around not doing that:
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`, the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed `user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
`unlisted` flag is the upgrade if that happens. `unlisted` flag is the upgrade if that happens.
It happened the same day. The first screenshot of the new Directory listed "Glass Cannon Live!
Ascension (for Ray Slakinski)", a Supercast feed at `feeds.supercast.com/feeds/<key>`. Treating
any long path segment as a key would have hidden public feeds too: acast's show ids look the
same. So paid-feed services are named instead (Patreon, Supercast, Supporting Cast, Glow,
Memberful), and any feed from one of them is private whatever its URL looks like. The per-feed
flag is still the upgrade for a service not on that list.
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so - Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
they would bury everything anyone chose on purpose. they would bury everything anyone chose on purpose.

View File

@@ -8,7 +8,7 @@ fetch, one parse and one file.
| Yours alone | The same for everyone | | Yours alone | The same for everyone |
|---|---| |---|---|
| Read, starred, playback position | The feed's URL | | Read, kept, playback position | The feed's URL |
| Which feeds you see at all | Its download folder | | Which feeds you see at all | Its download folder |
| Keywords, auto-download, explicit, per-scan cap | When it is scanned | | Keywords, auto-download, explicit, per-scan cap | When it is scanned |
| | The file on disk | | | The file on disk |
@@ -34,10 +34,10 @@ is shared.
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
for everyone** and names them in the confirmation, and the server has the last word: if anyone else for everyone** and names them in the confirmation, and the server has the last word: if anyone else
has starred the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the has kept the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
reason, and only `?force=true` goes through. reason, and only `?force=true` goes through.
Retention follows the same rule: starred by anyone keeps a file, and it counts as read only once Retention follows the same rule: an item anyone kept keeps its file, and it counts as read only once
every subscriber has read it. every subscriber has read it.
## Signing in ## Signing in
@@ -70,11 +70,14 @@ list with their own read state. Unsubscribing removes it from their list alone;
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
re-subscribing does not pull the back catalogue again. re-subscribing does not pull the back catalogue again.
**Add feed** also lists what other people on this server subscribe to, most subscribers first, as **Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
a place to start. It shows a title, artwork and a count, never a URL or who reads it. Feeds from an shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
included. Directory lists every one of them A to Z. Your own feeds are marked Subscribed.
It shows a title, artwork and a count, never a URL or who reads it. Feeds from an
OPML subscription are left out, since they come with the OPML. So is anything that looks private: a OPML subscription are left out, since they come with the OPML. So is anything that looks private: a
login configured for the feed, credentials in its URL, or a key such as `auth=` or `token=` in the login configured for the feed, credentials in its URL, or a key such as `auth=` or `token=` in the
query. Those are someone's paid subscriptions, and listing them would let anyone here read what they query, or a feed from a paid-feed service such as Patreon or Supercast, which put the key in the
path. Those are someone's paid subscriptions, and listing them would let anyone here read what they
pay for. pay for.
An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none

View File

@@ -26,9 +26,6 @@ pub struct General {
/// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d". /// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d".
/// A feed's own `schedule` overrides this. /// A feed's own `schedule` overrides this.
pub schedule: String, pub schedule: String,
/// Superseded by `schedule`. Still read so existing configs keep working.
#[serde(skip_serializing_if = "Option::is_none")]
pub interval_mins: Option<u64>,
pub organize: Organize, pub organize: Organize,
/// 0 = unlimited. /// 0 = unlimited.
pub max_total_gb: f64, pub max_total_gb: f64,
@@ -153,7 +150,6 @@ impl Default for General {
download_dir: home().join("Podcasts"), download_dir: home().join("Podcasts"),
socket: default_socket(), socket: default_socket(),
schedule: "every 60m".into(), schedule: "every 60m".into(),
interval_mins: None,
organize: Organize::Feed, organize: Organize::Feed,
max_total_gb: 0.0, max_total_gb: 0.0,
max_age_days: 0, max_age_days: 0,
@@ -176,8 +172,8 @@ impl Default for Torrent {
} }
impl General { impl General {
/// Minutes between checks. Falls back to the legacy `interval_mins`, then to an hour. /// Minutes between checks, or an hour when `schedule` is empty or unreadable. A malformed
/// A malformed value warns rather than stopping the daemon. /// value warns rather than stopping the daemon.
pub fn interval(&self) -> u64 { pub fn interval(&self) -> u64 {
if let Some(n) = parse_interval(&self.schedule) { if let Some(n) = parse_interval(&self.schedule) {
return n; return n;
@@ -185,7 +181,7 @@ impl General {
if !self.schedule.trim().is_empty() { if !self.schedule.trim().is_empty() {
tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default"); tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default");
} }
self.interval_mins.filter(|n| *n > 0).unwrap_or(60) 60
} }
} }
@@ -278,9 +274,7 @@ pub fn config_path() -> PathBuf {
if let Ok(p) = std::env::var("IPX_CONFIG") { if let Ok(p) = std::env::var("IPX_CONFIG") {
return PathBuf::from(p); return PathBuf::from(p);
} }
dirs::config_dir() xdg("XDG_CONFIG_HOME", ".config").join("ipx/config.toml")
.unwrap_or_else(|| home().join(".config"))
.join("ipx/config.toml")
} }
/// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`. /// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`.
@@ -288,9 +282,7 @@ pub fn data_dir() -> PathBuf {
if let Ok(p) = std::env::var("IPX_DATA_DIR") { if let Ok(p) = std::env::var("IPX_DATA_DIR") {
return PathBuf::from(p); return PathBuf::from(p);
} }
dirs::data_dir() xdg("XDG_DATA_HOME", ".local/share").join("ipx")
.unwrap_or_else(|| home().join(".local/share"))
.join("ipx")
} }
fn default_socket() -> PathBuf { fn default_socket() -> PathBuf {
@@ -346,8 +338,16 @@ pub fn unique_slug(text: &str, taken: &BTreeMap<String, Feed>) -> String {
(2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap() (2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap()
} }
/// `$var`, or `~/fallback` when it is unset or empty, as the XDG base directory spec says.
fn xdg(var: &str, fallback: &str) -> PathBuf {
std::env::var_os(var)
.filter(|v| !v.is_empty())
.map(PathBuf::from)
.unwrap_or_else(|| home().join(fallback))
}
fn home() -> PathBuf { fn home() -> PathBuf {
dirs::home_dir().unwrap_or_else(|| PathBuf::from(".")) std::env::var_os("HOME").map(PathBuf::from).unwrap_or_else(|| PathBuf::from("."))
} }
fn expand_tilde(p: &Path) -> PathBuf { fn expand_tilde(p: &Path) -> PathBuf {
@@ -367,6 +367,8 @@ mod tests {
r#" r#"
[general] [general]
download_dir = "/tmp/pods" download_dir = "/tmp/pods"
# A key older versions read. An old config that still has it has to load.
interval_mins = 45
[feeds.example] [feeds.example]
url = "https://example.com/feed.xml" url = "https://example.com/feed.xml"
@@ -403,22 +405,17 @@ mod tests {
} }
#[test] #[test]
fn interval_falls_back_through_legacy_then_default() { fn interval_falls_back_to_an_hour() {
let mut g = General::default(); let mut g = General::default();
assert_eq!(g.interval(), 60, "the default schedule"); assert_eq!(g.interval(), 60, "the default schedule");
g.schedule = "every 15m".into(); g.schedule = "every 15m".into();
assert_eq!(g.interval(), 15); assert_eq!(g.interval(), 15);
// A config written before `schedule` existed still works. // Empty or garbage must not stop the daemon.
g.schedule = String::new(); g.schedule = String::new();
g.interval_mins = Some(45); assert_eq!(g.interval(), 60);
assert_eq!(g.interval(), 45);
// Garbage must not stop the daemon.
g.schedule = "whenever".into(); g.schedule = "whenever".into();
assert_eq!(g.interval(), 45);
g.interval_mins = None;
assert_eq!(g.interval(), 60); assert_eq!(g.interval(), 60);
} }

358
src/db.rs
View File

@@ -40,14 +40,10 @@ CREATE TABLE IF NOT EXISTS entries (
published INTEGER, published INTEGER,
description TEXT, description TEXT,
first_seen INTEGER NOT NULL, first_seen INTEGER NOT NULL,
read INTEGER NOT NULL DEFAULT 0,
flagged INTEGER NOT NULL DEFAULT 0,
image TEXT, image TEXT,
duration INTEGER, duration INTEGER,
episode INTEGER, episode INTEGER,
season INTEGER, season INTEGER,
-- Seconds into the audio, so playback resumes where it was left.
position INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (feed_id, guid) PRIMARY KEY (feed_id, guid)
); );
@@ -93,7 +89,7 @@ CREATE TABLE IF NOT EXISTS subscriptions (
PRIMARY KEY (user_id, feed_id) PRIMARY KEY (user_id, feed_id)
); );
-- Read, starred and how far in. One row per person per item, created on first touch; -- Read, kept and how far in. One row per person per item, created on first touch;
-- an item nobody has touched has no row at all, which is what unread means. -- an item nobody has touched has no row at all, which is what unread means.
CREATE TABLE IF NOT EXISTS entry_state ( CREATE TABLE IF NOT EXISTS entry_state (
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
@@ -143,8 +139,8 @@ pub struct Managed {
pub orphaned: bool, pub orphaned: bool,
} }
/// Adds columns that later versions introduced. CREATE TABLE IF NOT EXISTS does nothing to /// Adds the columns later versions introduced and drops the ones they retired. CREATE TABLE IF
/// a table that already exists, so an installed database needs them added explicitly. /// NOT EXISTS does nothing to a table that already exists, so an installed database needs both.
fn migrate(conn: &Connection) -> Result<()> { fn migrate(conn: &Connection) -> Result<()> {
let wanted: &[(&str, &str, &str)] = &[ let wanted: &[(&str, &str, &str)] = &[
("feeds", "image", "TEXT"), ("feeds", "image", "TEXT"),
@@ -155,18 +151,29 @@ fn migrate(conn: &Connection) -> Result<()> {
("entries", "duration", "INTEGER"), ("entries", "duration", "INTEGER"),
("entries", "episode", "INTEGER"), ("entries", "episode", "INTEGER"),
("entries", "season", "INTEGER"), ("entries", "season", "INTEGER"),
("entries", "position", "INTEGER NOT NULL DEFAULT 0"),
]; ];
for (table, column, ty) in wanted { // Read state from before accounts, long since moved to entry_state. Two bugs came from
// queries still reading these after they stopped meaning anything, so they go.
let retired: &[(&str, &str)] = &[("entries", "read"), ("entries", "flagged"), ("entries", "position")];
let has = |table: &str, column: &str| -> Result<bool> {
let mut stmt = conn.prepare(&format!("PRAGMA table_info({table})"))?; let mut stmt = conn.prepare(&format!("PRAGMA table_info({table})"))?;
let existing: Vec<String> = stmt let names = stmt
.query_map([], |r| r.get::<_, String>(1))? .query_map([], |r| r.get::<_, String>(1))?
.collect::<rusqlite::Result<Vec<_>>>()?; .collect::<rusqlite::Result<Vec<_>>>()?;
if !existing.iter().any(|c| c == column) { Ok(names.iter().any(|c| c == column))
};
for (table, column, ty) in wanted {
if !has(table, column)? {
tracing::info!(table, column, "adding column"); tracing::info!(table, column, "adding column");
conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?; conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?;
} }
} }
for (table, column) in retired {
if has(table, column)? {
tracing::info!(table, column, "dropping column");
conn.execute_batch(&format!("ALTER TABLE {table} DROP COLUMN {column}"))?;
}
}
Ok(()) Ok(())
} }
@@ -345,22 +352,19 @@ impl Db {
/// Returns true when this entry had not been seen before. /// Returns true when this entry had not been seen before.
/// ///
/// A changed description or title flips `read` back to 0, which is what the original's
/// textDiff dance was ultimately for -- minus the diff markup, which the UI can do.
pub fn record_entry(&self, feed_id: &str, e: &crate::feed::Entry) -> Result<bool> { pub fn record_entry(&self, feed_id: &str, e: &crate::feed::Entry) -> Result<bool> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let inserted = conn.execute( let inserted = conn.execute(
"INSERT OR IGNORE INTO entries "INSERT OR IGNORE INTO entries
(feed_id, guid, title, link, published, description, first_seen, read, flagged, (feed_id, guid, title, link, published, description, first_seen,
image, duration, episode, season) image, duration, episode, season)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, 0, 0, ?8, ?9, ?10, ?11)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11)",
rusqlite::params![ rusqlite::params![
feed_id, e.guid, e.title, e.link, e.published, e.description, now(), feed_id, e.guid, e.title, e.link, e.published, e.description, now(),
e.image, e.duration, e.episode, e.season e.image, e.duration, e.episode, e.season
], ],
)?; )?;
if inserted == 0 { if inserted == 0 {
// The SET expressions see the pre-update row, so this compares old vs new.
conn.execute( conn.execute(
"UPDATE entries SET "UPDATE entries SET
title = coalesce(?3, title), title = coalesce(?3, title),
@@ -368,8 +372,7 @@ impl Db {
image = coalesce(?5, image), image = coalesce(?5, image),
duration = coalesce(?6, duration), duration = coalesce(?6, duration),
episode = coalesce(?7, episode), episode = coalesce(?7, episode),
season = coalesce(?8, season), season = coalesce(?8, season)
read = CASE WHEN description IS NOT ?4 OR title IS NOT ?3 THEN 0 ELSE read END
WHERE feed_id = ?1 AND guid = ?2", WHERE feed_id = ?1 AND guid = ?2",
rusqlite::params![ rusqlite::params![
feed_id, e.guid, e.title, e.description, feed_id, e.guid, e.title, e.description,
@@ -609,6 +612,36 @@ pub struct EntryRow {
const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2 const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2
OR lower(coalesce(e.description, '')) LIKE ?2)"; OR lower(coalesce(e.description, '')) LIKE ?2)";
/// Which feeds a query covers: one, or every feed the person subscribes to. Both forms
/// mention `?1`, since binding a parameter the statement does not use is an error.
fn scope_sql(feed_id: Option<&str>, user_param: u8) -> String {
match feed_id {
Some(_) => "e.feed_id = ?1".into(),
None => format!(
"?1 IS NULL AND e.feed_id IN (SELECT feed_id FROM subscriptions WHERE user_id = ?{user_param})"
),
}
}
/// The item table's ORDER BY. The column name picks one of these fixed expressions, so nothing
/// the caller sends reaches the query, and anything unrecognised is newest first. Ties fall back
/// to newest first too, so a page boundary is stable across "Load more".
///
/// ponytail: file type and size look at the item's first and largest file. The row shows the file
/// it summarises, which is almost always that one; sort by that one if they ever disagree.
pub fn order_sql(col: &str, dir: &str) -> String {
let expr = match col {
"kept" => "coalesce(s.flagged, 0)",
"title" => "lower(coalesce(e.title, ''))",
"feed" => "(SELECT lower(coalesce(f.title, f.id)) FROM feeds f WHERE f.id = e.feed_id)",
"type" => "(SELECT min(x.mime) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
"size" => "(SELECT max(x.length) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
_ => "coalesce(e.published, e.first_seen)",
};
let dir = if dir == "asc" { "ASC" } else { "DESC" };
format!("{expr} {dir}, coalesce(e.published, e.first_seen) DESC, e.rowid DESC")
}
/// Which slice of a feed the UI is asking for. /// Which slice of a feed the UI is asking for.
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Filter { pub enum Filter {
@@ -657,16 +690,18 @@ pub struct EncRow {
} }
impl Db { impl Db {
/// One page of a feed's entries, newest first, each with its enclosures attached. /// One page of entries, each with its enclosures attached: one feed's, or every feed the
/// `search` matches title and description, case-insensitively. /// person subscribes to when `feed_id` is None (All Subscriptions). `search` matches title
pub fn entries( /// and description, case-insensitively.
pub fn entries_in(
&self, &self,
user_id: i64, user_id: i64,
feed_id: &str, feed_id: Option<&str>,
filter: Filter, filter: Filter,
search: Option<&str>, search: Option<&str>,
offset: i64, offset: i64,
limit: i64, limit: i64,
order: &str,
) -> Result<Vec<EntryRow>> { ) -> Result<Vec<EntryRow>> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let like = search let like = search
@@ -679,9 +714,10 @@ impl Db {
FROM entries e FROM entries e
LEFT JOIN entry_state s LEFT JOIN entry_state s
ON s.user_id = ?5 AND s.feed_id = e.feed_id AND s.guid = e.guid ON s.user_id = ?5 AND s.feed_id = e.feed_id AND s.guid = e.guid
WHERE e.feed_id = ?1 AND {} AND {SEARCH} WHERE {} AND {} AND {SEARCH}
ORDER BY coalesce(e.published, e.first_seen) DESC, e.rowid DESC ORDER BY {order}
LIMIT ?4 OFFSET ?3", LIMIT ?4 OFFSET ?3",
scope_sql(feed_id, 5),
filter.sql() filter.sql()
); );
let mut stmt = conn.prepare(&sql)?; let mut stmt = conn.prepare(&sql)?;
@@ -711,14 +747,14 @@ impl Db {
return Ok(rows); return Ok(rows);
} }
// Only the guids on this page, so a feed with thousands of entries stays cheap. // Only the guids on this page, so a feed with thousands of entries stays cheap. A page
// can span feeds, so each file is matched to its row by feed as well as guid, below.
let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(","); let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(",");
let sql = format!( let sql = format!(
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error "SELECT id, feed_id, guid, url, mime, length, path, state, last_error
FROM enclosures WHERE feed_id = ? AND guid IN ({placeholders}) ORDER BY id" FROM enclosures WHERE guid IN ({placeholders}) ORDER BY id"
); );
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len() + 1); let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len());
params.push(&feed_id);
for row in &rows { for row in &rows {
params.push(&row.guid); params.push(&row.guid);
} }
@@ -740,18 +776,18 @@ impl Db {
.collect::<rusqlite::Result<Vec<_>>>()?; .collect::<rusqlite::Result<Vec<_>>>()?;
for enc in encs { for enc in encs {
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid) { if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid && r.feed_id == enc.feed_id) {
row.enclosures.push(enc); row.enclosures.push(enc);
} }
} }
Ok(rows) Ok(rows)
} }
/// How many entries match, so the UI knows whether there is another page. /// How many entries `entries_in` would page through, so the UI knows whether there is more.
pub fn count_entries( pub fn count_in(
&self, &self,
user_id: i64, user_id: i64,
feed_id: &str, feed_id: Option<&str>,
filter: Filter, filter: Filter,
search: Option<&str>, search: Option<&str>,
) -> Result<i64> { ) -> Result<i64> {
@@ -763,7 +799,8 @@ impl Db {
"SELECT count(*) FROM entries e "SELECT count(*) FROM entries e
LEFT JOIN entry_state s LEFT JOIN entry_state s
ON s.user_id = ?3 AND s.feed_id = e.feed_id AND s.guid = e.guid ON s.user_id = ?3 AND s.feed_id = e.feed_id AND s.guid = e.guid
WHERE e.feed_id = ?1 AND {} AND {SEARCH}", WHERE {} AND {} AND {SEARCH}",
scope_sql(feed_id, 3),
filter.sql() filter.sql()
); );
Ok(conn.query_row(&sql, rusqlite::params![feed_id, like, user_id], |r| r.get(0))?) Ok(conn.query_row(&sql, rusqlite::params![feed_id, like, user_id], |r| r.get(0))?)
@@ -780,24 +817,16 @@ impl Db {
Ok(()) Ok(())
} }
/// Marks every entry in a feed read, for the "mark all read" button. /// The first admin starts subscribed to the whole catalogue: whoever wrote config.toml meant
/// Moves a single-user library onto an account: everything read, starred or part-played /// to read those feeds, and without this a fresh install signs in to an empty sidebar. Runs
/// becomes that person's, and they subscribe to every feed already in the catalogue. /// only while nobody subscribes to anything, so an unsubscribe is never undone.
/// Runs once -- the moment there is a first account and no subscriptions yet. pub fn adopt_catalogue(&self, user_id: i64, catalogue: &[String]) -> Result<usize> {
pub fn adopt_existing_library(&self, user_id: i64, catalogue: &[String]) -> Result<usize> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let already: i64 = let already: i64 =
conn.query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0))?; conn.query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0))?;
if already > 0 { if already > 0 {
return Ok(0); return Ok(0);
} }
let moved = conn.execute(
"INSERT INTO entry_state (user_id, feed_id, guid, read, flagged, position)
SELECT ?1, feed_id, guid, read, flagged, position FROM entries
WHERE read = 1 OR flagged = 1 OR position > 0
ON CONFLICT(user_id, feed_id, guid) DO NOTHING",
[user_id],
)?;
for id in catalogue { for id in catalogue {
conn.execute( conn.execute(
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)", "INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
@@ -810,7 +839,7 @@ impl Db {
SELECT ?1, id, ?2 FROM feeds", SELECT ?1, id, ?2 FROM feeds",
params![user_id, now()], params![user_id, now()],
)?; )?;
Ok(moved) Ok(catalogue.len())
} }
// ---- subscriptions ---- // ---- subscriptions ----
@@ -862,14 +891,20 @@ impl Db {
/// Everyone's settings for one feed. The scanner merges these into what it fetches /// Everyone's settings for one feed. The scanner merges these into what it fetches
/// and downloads, since one file serves the lot. /// and downloads, since one file serves the lot.
pub fn subscribers(&self, feed_id: &str) -> Result<Vec<Sub>> { /// In a group, whatever someone has not set on the feed itself comes from their
/// subscription to the group, as the group's settings dialog has always said it does.
pub fn subscribers(&self, feed_id: &str, group: Option<&str>) -> Result<Vec<Sub>> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare( let mut stmt = conn.prepare(
"SELECT keywords, auto_download, allow_explicit, max_new_per_check "SELECT coalesce(c.keywords, p.keywords), coalesce(c.auto_download, p.auto_download),
FROM subscriptions WHERE feed_id = ?1", coalesce(c.allow_explicit, p.allow_explicit),
coalesce(c.max_new_per_check, p.max_new_per_check)
FROM subscriptions c
LEFT JOIN subscriptions p ON p.user_id = c.user_id AND p.feed_id = ?2
WHERE c.feed_id = ?1",
)?; )?;
let out = stmt let out = stmt
.query_map([feed_id], |r| { .query_map(params![feed_id, group], |r| {
Ok(Sub { Ok(Sub {
feed_id: feed_id.to_string(), feed_id: feed_id.to_string(),
keywords: r keywords: r
@@ -1267,6 +1302,71 @@ impl Db {
Ok(()) Ok(())
} }
/// Empties a feed of its items, leaving its files alone.
pub fn clear_entries(&self, feed_id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("DELETE FROM entries WHERE feed_id = ?1", [feed_id])?;
Ok(())
}
/// Every feed the database holds rows for, as (id, url), removed ones included.
pub fn feed_urls(&self) -> Result<Vec<(String, String)>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare("SELECT id, coalesce(url, '') FROM feeds")?;
let out = stmt
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))?
.collect::<rusqlite::Result<_>>()?;
Ok(out)
}
/// Hands a feed in a group the enclosures its parent holds, as (guid, url), with everyone's
/// read state for them. A Patreon creator read as one feed before it was split into shows
/// owns every show's files, and `enclosures.url` is unique, so without this each show would
/// list its items with nothing to play.
pub fn adopt(&self, parent: &str, child: &str, listed: &[(&str, &str)]) -> Result<()> {
let mut conn = self.conn.lock().unwrap();
let holds: bool = conn.query_row(
"SELECT EXISTS (SELECT 1 FROM enclosures WHERE feed_id = ?1)",
[parent],
|r| r.get(0),
)?;
if !holds {
return Ok(()); // An OPML, or a creator already shared out.
}
let tx = conn.transaction()?;
for &(guid, url) in listed {
let moved = tx.execute(
"UPDATE enclosures SET feed_id = ?3, guid = ?4 WHERE url = ?1 AND feed_id = ?2",
params![url, parent, child, guid],
)?;
if moved == 1 {
// Patreon gives a post the same guid in every feed it appears in.
tx.execute(
"UPDATE OR IGNORE entry_state SET feed_id = ?2 WHERE feed_id = ?1 AND guid = ?3",
params![parent, child, guid],
)?;
}
}
tx.commit()?;
Ok(())
}
/// A feed's enclosures skipped by one of its filters, by URL, with the reason: the verdicts a
/// change of settings can overturn. A torrent held back while torrents are off is not a
/// filter's call.
pub fn skipped_by_filter(&self, feed_id: &str) -> Result<std::collections::HashMap<String, String>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT url, last_error FROM enclosures
WHERE feed_id = ?1 AND state = 'skipped' AND last_error IS NOT NULL
AND last_error != 'torrents disabled'",
)?;
let out = stmt
.query_map([feed_id], |r| Ok((r.get(0)?, r.get(1)?)))?
.collect::<rusqlite::Result<_>>()?;
Ok(out)
}
pub fn set_orphaned(&self, feed_id: &str, on: bool) -> Result<()> { pub fn set_orphaned(&self, feed_id: &str, on: bool) -> Result<()> {
let conn = self.conn.lock().unwrap(); let conn = self.conn.lock().unwrap();
conn.execute( conn.execute(
@@ -1319,6 +1419,40 @@ pub fn now() -> i64 {
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn every_sort_column_runs_and_orders_both_ways() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(1,'g',0);
INSERT INTO feeds (id, url, title) VALUES ('f','u','Zebra'),('g','v','Aardvark');
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
('f','a','banana',100),('g','b','Apple',200),('f','c','cherry',300);
INSERT INTO enclosures (id, feed_id, guid, url, mime, length, state) VALUES
(1,'f','a','u1','audio/mpeg',300,'pending'),(2,'g','b','u2','image/png',10,'pending'),
(3,'f','c','u3','video/mp4',2000,'pending');",
)
.unwrap();
let order = |col: &str, dir: &str| -> Vec<String> {
db.entries_in(1, None, Filter::All, None, 0, 50, &order_sql(col, dir))
.unwrap()
.into_iter()
.map(|e| e.guid)
.collect()
};
assert_eq!(order("title", "asc"), ["b", "a", "c"], "Apple, banana, cherry: case folded");
assert_eq!(order("title", "desc"), ["c", "a", "b"]);
assert_eq!(order("feed", "asc"), ["b", "c", "a"], "Aardvark, then Zebra's newest first");
assert_eq!(order("type", "asc"), ["a", "b", "c"], "audio, image, video");
assert_eq!(order("size", "desc"), ["c", "a", "b"]);
assert_eq!(order("published", "desc"), ["c", "b", "a"]);
db.set_entry_flag(1, "f", "a", EntryFlag::Flagged, true).unwrap();
assert_eq!(order("kept", "desc")[0], "a");
// An unknown column or direction is newest first; the name itself never reaches the SQL.
assert_eq!(order("title; DROP TABLE entries", "sideways"), ["c", "b", "a"]);
assert!(!order_sql("x'; --", "asc").contains("x'"));
}
#[test] #[test]
fn deleting_a_shared_file_asks_about_everyone_else() { fn deleting_a_shared_file_asks_about_everyone_else() {
let db = Db::memory().unwrap(); let db = Db::memory().unwrap();
@@ -1365,8 +1499,9 @@ mod tests {
// Starring and position are just as private. // Starring and position are just as private.
db.set_entry_flag(1, "f", "b", EntryFlag::Flagged, true).unwrap(); db.set_entry_flag(1, "f", "b", EntryFlag::Flagged, true).unwrap();
db.set_position(2, "f", "b", 42).unwrap(); db.set_position(2, "f", "b", 42).unwrap();
let ray = db.entries(1, "f", Filter::All, None, 0, 50).unwrap(); let order = order_sql("published", "desc");
let sam = db.entries(2, "f", Filter::All, None, 0, 50).unwrap(); let page = |user| db.entries_in(user, Some("f"), Filter::All, None, 0, 50, &order).unwrap();
let (ray, sam) = (page(1), page(2));
let ray_b = ray.iter().find(|e| e.guid == "b").unwrap(); let ray_b = ray.iter().find(|e| e.guid == "b").unwrap();
let sam_b = sam.iter().find(|e| e.guid == "b").unwrap(); let sam_b = sam.iter().find(|e| e.guid == "b").unwrap();
assert!(ray_b.flagged && ray_b.position == 0); assert!(ray_b.flagged && ray_b.position == 0);
@@ -1390,6 +1525,49 @@ mod tests {
assert_eq!(sum.downloaded, 0); assert_eq!(sum.downloaded, 0);
} }
#[test]
fn an_old_database_loses_the_retired_read_columns() {
let conn = Connection::open_in_memory().unwrap();
conn.execute_batch(
"CREATE TABLE entries (feed_id TEXT NOT NULL, guid TEXT NOT NULL,
first_seen INTEGER NOT NULL, read INTEGER NOT NULL DEFAULT 0,
flagged INTEGER NOT NULL DEFAULT 0, position INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (feed_id, guid));",
)
.unwrap();
// The same order as open(): the schema leaves the old table alone, migrate() fixes it.
conn.execute_batch(SCHEMA).unwrap();
migrate(&conn).unwrap();
let cols: Vec<String> = conn
.prepare("PRAGMA table_info(entries)")
.unwrap()
.query_map([], |r| r.get(1))
.unwrap()
.collect::<rusqlite::Result<_>>()
.unwrap();
assert!(!cols.iter().any(|c| ["read", "flagged", "position"].contains(&c.as_str())), "{cols:?}");
assert!(cols.iter().any(|c| c == "image"), "and it still gains the newer ones");
}
#[test]
fn the_first_admin_starts_with_the_catalogue_and_only_once() {
// Cutting this along with the dead read columns left the browser suite's admin with an
// empty sidebar: it is how a fresh install's first account gets config.toml's feeds.
let db = Db::memory().unwrap();
db.exec_for_test("INSERT INTO users (id, name, is_admin, created) VALUES (1,'admin',1,0);")
.unwrap();
let subs = || -> i64 {
db.conn.lock().unwrap().query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0)).unwrap()
};
let catalogue = ["a".to_string(), "b".to_string()];
assert_eq!(db.adopt_catalogue(1, &catalogue).unwrap(), 2);
assert_eq!(subs(), 2);
// Once anyone subscribes to anything it never runs again, so an unsubscribe sticks.
db.unsubscribe(1, "a").unwrap();
assert_eq!(db.adopt_catalogue(1, &catalogue).unwrap(), 0);
assert_eq!(subs(), 1);
}
#[test] #[test]
fn every_filter_works_with_and_without_a_search_term() { fn every_filter_works_with_and_without_a_search_term() {
// Regression: the search clause used to be omitted when no term was given, while // Regression: the search clause used to be omitted when no term was given, while
@@ -1413,21 +1591,22 @@ mod tests {
for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] { for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] {
// Both paths must run without erroring, and agree with each other. // Both paths must run without erroring, and agree with each other.
let rows = db.entries(7, "f", f, None, 0, 50).unwrap(); let order = order_sql("published", "desc");
let n = db.count_entries(7, "f", f, None).unwrap(); let rows = db.entries_in(7, Some("f"), f, None, 0, 50, &order).unwrap();
let n = db.count_in(7, Some("f"), f, None).unwrap();
assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page"); assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page");
let rows = db.entries(7, "f", f, Some("dive"), 0, 50).unwrap(); let rows = db.entries_in(7, Some("f"), f, Some("dive"), 0, 50, &order).unwrap();
let n = db.count_entries(7, "f", f, Some("dive")).unwrap(); let n = db.count_in(7, Some("f"), f, Some("dive")).unwrap();
assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees"); assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees");
} }
assert_eq!(db.count_entries(7, "f", Filter::All, None).unwrap(), 3); assert_eq!(db.count_in(7, Some("f"), Filter::All, None).unwrap(), 3);
assert_eq!(db.count_entries(7, "f", Filter::Unread, None).unwrap(), 1); assert_eq!(db.count_in(7, Some("f"), Filter::Unread, None).unwrap(), 1);
assert_eq!(db.count_entries(7, "f", Filter::Downloaded, None).unwrap(), 1); assert_eq!(db.count_in(7, Some("f"), Filter::Downloaded, None).unwrap(), 1);
assert_eq!(db.count_entries(7, "f", Filter::Flagged, None).unwrap(), 1); assert_eq!(db.count_in(7, Some("f"), Filter::Flagged, None).unwrap(), 1);
assert_eq!(db.count_entries(7, "f", Filter::All, Some("dive")).unwrap(), 2); assert_eq!(db.count_in(7, Some("f"), Filter::All, Some("dive")).unwrap(), 2);
assert_eq!(db.count_entries(7, "f", Filter::All, Some("NOTES two")).unwrap(), 1, assert_eq!(db.count_in(7, Some("f"), Filter::All, Some("NOTES two")).unwrap(), 1,
"search is case-insensitive and covers the description"); "search is case-insensitive and covers the description");
} }
@@ -1469,6 +1648,61 @@ mod tests {
assert_eq!(state(4), "done"); assert_eq!(state(4), "done");
} }
#[test]
fn a_show_takes_over_what_its_creator_held() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
INSERT INTO enclosures (id, feed_id, guid, url, state, path, last_error) VALUES
(1,'creator','a','u1','done','/x/a.mp3',NULL),
(2,'creator','b','u2','skipped',NULL,'explicit'),
(3,'creator','c','u3','skipped',NULL,'explicit'),
(4,'other','d','u4','skipped',NULL,'torrents disabled');
INSERT INTO entry_state (user_id, feed_id, guid, read) VALUES (1,'creator','a',1);",
)
.unwrap();
db.adopt("creator", "show", &[("a", "u1"), ("b", "u2"), ("d", "u4")]).unwrap();
{
let conn = db.conn.lock().unwrap();
let owner = |id: i64| -> String {
conn.query_row("SELECT feed_id FROM enclosures WHERE id = ?1", [id], |r| r.get(0)).unwrap()
};
assert_eq!(owner(1), "show", "a downloaded file moves with its item");
assert_eq!(owner(2), "show");
assert_eq!(owner(3), "creator", "this show does not list it");
assert_eq!(owner(4), "other", "only the parent's are taken");
let read: String = conn
.query_row("SELECT feed_id FROM entry_state WHERE user_id = 1 AND guid = 'a'", [], |r| r.get(0))
.unwrap();
assert_eq!(read, "show", "what you had read stays read");
}
// Only a filter's verdict can be overturned by a change of settings.
let skipped = db.skipped_by_filter("show").unwrap();
assert_eq!(skipped.get("u2").map(String::as_str), Some("explicit"));
assert_eq!(skipped.len(), 1);
assert!(db.skipped_by_filter("other").unwrap().is_empty(), "torrents disabled is not a filter");
}
#[test]
fn a_feed_in_a_group_follows_your_settings_on_the_group() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
INSERT INTO subscriptions (user_id, feed_id, allow_explicit, created) VALUES
(1,'group',1,0),(1,'show',NULL,0),(2,'group',1,0),(2,'show',0,0);",
)
.unwrap();
let explicit = |group| -> Vec<Option<bool>> {
let mut v: Vec<_> =
db.subscribers("show", group).unwrap().into_iter().map(|s| s.allow_explicit).collect();
v.sort();
v
};
assert_eq!(explicit(Some("group")), [Some(false), Some(true)], "ray inherits; sam's own choice on the show wins");
assert_eq!(explicit(None), [None, Some(false)], "outside a group nothing is inherited");
}
#[test] #[test]
fn enclosure_url_is_the_dedupe_key() { fn enclosure_url_is_the_dedupe_key() {
let db = Db::memory().unwrap(); let db = Db::memory().unwrap();

View File

@@ -223,7 +223,7 @@ enum Sniffed {
/// 2008 and so always answered 'data'. /// 2008 and so always answered 'data'.
async fn sniff(path: &Path) -> Result<Sniffed> { async fn sniff(path: &Path) -> Result<Sniffed> {
let head = read_head(path, 512).await?; let head = read_head(path, 512).await?;
if infer::is(&head, "torrent") || head.starts_with(b"d8:announce") || head.starts_with(b"d7:") { if head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
return Ok(Sniffed::Torrent); return Ok(Sniffed::Torrent);
} }
let text = String::from_utf8_lossy(&head); let text = String::from_utf8_lossy(&head);

View File

@@ -117,6 +117,103 @@ pub fn opml_title(bytes: &[u8]) -> Option<String> {
.filter(|t| !t.is_empty()) .filter(|t| !t.is_empty())
} }
/// The token and show of a Patreon feed link, or None for any other URL.
///
/// Patreon gives each patron one token per creator. With no show it stands for the creator,
/// whose feed carries every show at once.
fn patreon_parts(url: &str) -> Option<(String, Option<String>)> {
let u = url::Url::parse(url).ok()?;
if !matches!(u.host_str()?, "patreon.com" | "www.patreon.com") || !u.path().starts_with("/rss") {
return None;
}
let param = |name: &str| u.query_pairs().find(|(k, _)| k == name).map(|(_, v)| v.into_owned());
Some((param("auth")?, param("show")))
}
/// A Patreon link naming a creator but no show.
pub fn is_patreon_creator(url: &str) -> bool {
matches!(patreon_parts(url), Some((_, None)))
}
/// What was typed into Add feed, as a URL. A bare Patreon token is taken as its creator's
/// feed, since the token alone says whose it is.
pub fn expand_input(input: &str) -> String {
let s = input.trim();
let token = s.len() >= 20 && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_');
if token { format!("https://www.patreon.com/rss?auth={s}") } else { s.to_owned() }
}
/// Whether two URLs are the same feed. One Patreon show has several spellings -- by the
/// creator's name, by number, or with no creator at all -- and the token and show are what
/// identify it.
pub fn same_feed(a: &str, b: &str) -> bool {
a == b || patreon_parts(a).is_some_and(|p| Some(p) == patreon_parts(b))
}
/// A Patreon creator's name and shows, each show as (title, feed URL).
///
/// ponytail: Patreon's own web API, undocumented, asked without signing in. If it changes,
/// finding shows stops and the show feeds already found keep working. The documented API
/// needs an OAuth client per install and does not list shows.
pub async fn patreon_shows(
client: &reqwest::Client,
url: &str,
) -> Result<(Option<String>, Vec<(String, String)>)> {
// The creator feed names its campaign by number in its self link, a few hundred bytes in.
// The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
let mut resp = client.get(url).send().await.context("connecting")?;
if !resp.status().is_success() {
return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
}
let mut head = Vec::new();
while patreon_campaign(&head).is_none() && head.len() < 64 * 1024 {
let Some(chunk) = resp.chunk().await.context("reading the feed")? else { break };
head.extend_from_slice(&chunk);
}
let campaign = patreon_campaign(&head)
.ok_or_else(|| anyhow!("the Patreon feed does not say whose it is"))?;
let api = format!(
"https://www.patreon.com/api/campaigns/{campaign}\
?include=shows&fields%5Bcampaign%5D=name&fields%5Bcollection%5D=title"
);
let resp = client.get(api).send().await.context("asking Patreon for the shows")?;
if !resp.status().is_success() {
return Err(anyhow!("Patreon would not list the shows: HTTP {}", resp.status()));
}
let (name, shows) = parse_patreon_shows(&resp.bytes().await.context("reading the shows")?)?;
Ok((name, shows.into_iter().map(|(id, title)| (title, format!("{url}&show={id}"))).collect()))
}
/// The campaign number in the start of a Patreon feed.
fn patreon_campaign(head: &[u8]) -> Option<String> {
let text = String::from_utf8_lossy(head);
text.match_indices("patreon.com/rss/").find_map(|(i, m)| {
let id: String = text[i + m.len()..].chars().take_while(char::is_ascii_digit).collect();
(!id.is_empty()).then_some(id)
})
}
/// A campaign's name and its shows as (id, title), from Patreon's JSON:API answer.
fn parse_patreon_shows(json: &[u8]) -> Result<(Option<String>, Vec<(String, String)>)> {
let v: serde_json::Value = serde_json::from_slice(json).context("Patreon's answer is not JSON")?;
// Missing is not the same as none. Read as no shows, the creator feed would be scanned as
// a plain feed, claim every show's files, and leave the shows empty once the list returned.
let ids = v["data"]["relationships"]["shows"]["data"]
.as_array()
.ok_or_else(|| anyhow!("Patreon's answer does not list the shows"))?;
let title = |id: &str| -> Option<String> {
let show = v["included"].as_array()?.iter().find(|x| x["type"] == "collection" && x["id"] == id)?;
show["attributes"]["title"].as_str().map(|t| t.trim().to_owned())
};
let shows = ids
.iter()
.filter_map(|s| s["id"].as_str())
.map(|id| (id.to_owned(), title(id).unwrap_or_else(|| format!("Show {id}"))))
.collect();
Ok((v["data"]["attributes"]["name"].as_str().map(str::to_owned), shows))
}
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`. /// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> { pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
match rss::Channel::read_from(bytes) { match rss::Channel::read_from(bytes) {
@@ -550,6 +647,45 @@ mod tests {
assert!(!is_opml(include_bytes!("../tests/data/atom.xml"))); assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
} }
#[test]
fn a_patreon_creator_is_a_list_of_its_shows() {
let tok = "AbCdEfGhIjKlMnOpQrStUvWxYz012_-9";
assert_eq!(expand_input(&format!(" {tok} ")), format!("https://www.patreon.com/rss?auth={tok}"));
assert_eq!(expand_input("https://example.com/rss"), "https://example.com/rss");
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss/glasscannon?auth={tok}")));
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss?auth={tok}")));
assert!(!is_patreon_creator(&format!("https://www.patreon.com/rss/x?auth={tok}&show=1")), "one show is a feed");
assert!(!is_patreon_creator(&format!("https://example.com/rss?auth={tok}")));
// The show you already have by name is the one a bare token would add by number.
assert!(same_feed(
&format!("https://www.patreon.com/rss/glasscannon?auth={tok}&show=2073588"),
&format!("https://www.patreon.com/rss?auth={tok}&show=2073588"),
));
assert!(!same_feed(
&format!("https://www.patreon.com/rss?auth={tok}&show=1"),
&format!("https://www.patreon.com/rss?auth={tok}&show=2"),
));
// The self link carries the campaign by number, whichever spelling was asked for.
let head = br#"<rss><channel><link>https://www.patreon.com/glasscannon</link>
<atom:link href="https://www.patreon.com/rss/369921?auth=t" rel="self"/>"#;
assert_eq!(patreon_campaign(head).as_deref(), Some("369921"));
assert_eq!(patreon_campaign(b"<rss><channel><title>T"), None);
let json = br#"{"data":{"id":"369921","type":"campaign","attributes":{"name":"The Glass Cannon Network"},
"relationships":{"shows":{"data":[{"id":"2073588","type":"collection"},{"id":"2073636","type":"collection"}]}}},
"included":[{"id":"2073588","type":"collection","attributes":{"title":"Get in the Trunk "}},
{"id":"2073636","type":"collection","attributes":{"title":"Shadowdark"}}]}"#;
let (name, shows) = parse_patreon_shows(json).unwrap();
assert_eq!(name.as_deref(), Some("The Glass Cannon Network"));
assert_eq!(shows, [("2073588".into(), "Get in the Trunk".into()), ("2073636".into(), "Shadowdark".into())]);
// An answer that stops naming the shows is an error, never "this creator has none".
assert!(parse_patreon_shows(br#"{"data":{"attributes":{"name":"X"}}}"#).is_err());
}
#[test] #[test]
fn an_item_may_carry_several_enclosures() { fn an_item_may_carry_several_enclosures() {
// The rss crate keeps only one per item -- the last -- so these come from the XML. // The rss crate keeps only one per item -- the last -- so these come from the XML.

View File

@@ -62,10 +62,10 @@ impl Event {
Event::DownloadDone { path, .. } => format!(" saved {path}"), Event::DownloadDone { path, .. } => format!(" saved {path}"),
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"), Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
Event::Reaped { path, bytes } => { Event::Reaped { path, bytes } => {
format!("reap {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0) format!("deleted {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
} }
Event::ReapDone { files, bytes } => format!( Event::ReapDone { files, bytes } => format!(
"reaped {files} file(s), {:.1} MB", "deleted {files} old file(s), {:.1} MB",
*bytes as f64 / 1_048_576.0 *bytes as f64 / 1_048_576.0
), ),
Event::Status { feeds, pending, downloaded } => { Event::Status { feeds, pending, downloaded } => {

View File

@@ -111,18 +111,11 @@ impl Visit for Collect {
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) { fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
self.add(field, format!("{value:?}")); self.add(field, format!("{value:?}"));
} }
// Numbers and bools reach record_debug through the trait's defaults, which prints them the
// same way. A string would print quoted there, hence its own method.
fn record_str(&mut self, field: &Field, value: &str) { fn record_str(&mut self, field: &Field, value: &str) {
self.add(field, value.to_owned()); self.add(field, value.to_owned());
} }
fn record_i64(&mut self, field: &Field, value: i64) {
self.add(field, value.to_string());
}
fn record_u64(&mut self, field: &Field, value: u64) {
self.add(field, value.to_string());
}
fn record_bool(&mut self, field: &Field, value: bool) {
self.add(field, value.to_string());
}
} }
#[cfg(test)] #[cfg(test)]

View File

@@ -334,8 +334,7 @@ async fn daemon(
anyhow::bail!("a daemon is already listening on {}", socket.display()); anyhow::bail!("a daemon is already listening on {}", socket.display());
} }
// A database with nobody in it cannot be signed into, and an install that predates // A database with nobody in it cannot be signed into.
// accounts still has to serve its owner. Both get the same starting point.
if ctx.db.users()?.is_empty() { if ctx.db.users()?.is_empty() {
ctx.db.create_user("admin", Some(&crate::auth::hash_password(DEFAULT_PASSWORD)?), true)?; ctx.db.create_user("admin", Some(&crate::auth::hash_password(DEFAULT_PASSWORD)?), true)?;
tracing::warn!( tracing::warn!(
@@ -344,22 +343,15 @@ async fn daemon(
); );
} }
// A library that predates accounts belongs to whoever was using it: the admin.
if let Some(admin) = ctx.db.users()?.into_iter().find(|u| u.is_admin) { if let Some(admin) = ctx.db.users()?.into_iter().find(|u| u.is_admin) {
let catalogue: Vec<String> = ctx.cfg().feeds.keys().cloned().collect(); let catalogue: Vec<String> = ctx.cfg().feeds.keys().cloned().collect();
match ctx.db.adopt_existing_library(admin.id, &catalogue) { match ctx.db.adopt_catalogue(admin.id, &catalogue) {
Ok(0) => {} Ok(0) => {}
Ok(n) => tracing::info!(user = %admin.name, entries = n, "adopted the existing library"), Ok(n) => tracing::info!(user = %admin.name, feeds = n, "subscribed the first admin to the catalogue"),
Err(e) => tracing::error!(error = %e, "could not adopt the existing library"), Err(e) => tracing::error!(error = %e, "could not subscribe the first admin to the catalogue"),
} }
} }
match migrate_opml_children(&ctx) {
Ok(n) if n > 0 => tracing::info!(count = n, "moved OPML feeds out of config.toml into the database"),
Ok(_) => {}
Err(e) => tracing::warn!(error = ?e, "could not tidy OPML feeds out of the config"),
}
match ctx.db.requeue_interrupted() { match ctx.db.requeue_interrupted() {
Ok(n) if n > 0 => tracing::info!(count = n, "requeued downloads interrupted by a restart"), Ok(n) if n > 0 => tracing::info!(count = n, "requeued downloads interrupted by a restart"),
Ok(_) => {} Ok(_) => {}
@@ -468,7 +460,7 @@ async fn start_web(
let mut fresh = (*cfg).clone(); let mut fresh = (*cfg).clone();
fresh.web.enabled = true; fresh.web.enabled = true;
fresh.web.bind = bind.clone(); fresh.web.bind = bind.clone();
fresh.web.token = web::generate_token(); fresh.web.token = crate::auth::new_session_token();
fresh.save(config_path)?; fresh.save(config_path)?;
ctx.reload_cfg(config_path)?; ctx.reload_cfg(config_path)?;
println!("web ui token generated. Open:\n http://{bind}/?token={}", fresh.web.token); println!("web ui token generated. Open:\n http://{bind}/?token={}", fresh.web.token);
@@ -517,8 +509,9 @@ async fn add(
keywords: Vec<String>, keywords: Vec<String>,
) -> Result<()> { ) -> Result<()> {
let mut cfg = (*ctx.cfg()).clone(); let mut cfg = (*ctx.cfg()).clone();
let url = &feed::expand_input(url);
// Includes feeds derived from an OPML, or the same show could be added twice. // Includes feeds derived from an OPML, or the same show could be added twice.
if let Some(existing) = subscriptions(ctx)?.iter().find(|s| s.cfg.url == url) { if let Some(existing) = subscriptions(ctx)?.iter().find(|s| feed::same_feed(&s.cfg.url, url)) {
anyhow::bail!("already subscribed as {:?}", existing.id); anyhow::bail!("already subscribed as {:?}", existing.id);
} }
let id = add_one(ctx, &mut cfg, url, folder, keywords).await?; let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
@@ -569,10 +562,17 @@ pub async fn add_one(
// Slugs must be unique across derived feeds too, or a new feed can collide with one // Slugs must be unique across derived feeds too, or a new feed can collide with one
// an OPML already introduced. // an OPML already introduced.
let taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)? let mut taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)?
.into_iter() .into_iter()
.map(|s| (s.id, s.cfg)) .map(|s| (s.id, s.cfg))
.collect(); .collect();
// A removed feed keeps its rows, so its id is only free again for the same feed: re-adding
// it gets its history back, and a different feed does not inherit someone else's.
for (id, other) in ctx.db.feed_urls()? {
if !feed::same_feed(&other, url) {
taken.entry(id).or_insert_with(|| probe.clone());
}
}
let id = config::unique_slug(&title, &taken); let id = config::unique_slug(&title, &taken);
cfg.feeds.insert(id.clone(), probe); cfg.feeds.insert(id.clone(), probe);
Ok(id) Ok(id)
@@ -807,7 +807,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool) -> Result<()> {
ctx.out.emit(Event::FeedSkip { ctx.out.emit(Event::FeedSkip {
feed: id.clone(), feed: id.clone(),
reason: format!( reason: format!(
"OPML: {total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing", "{total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing",
added.len() added.len()
), ),
}); });
@@ -908,36 +908,6 @@ pub fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
Ok(out) Ok(out)
} }
/// Moves OPML children that older versions wrote into config.toml over to the database.
/// They were never yours to edit, and 80-odd of them made the file unreadable.
fn migrate_opml_children(ctx: &Ctx) -> Result<usize> {
let cfg = (*ctx.cfg()).clone();
let children: Vec<(String, config::Feed)> = cfg
.feeds
.iter()
.filter(|(_, f)| f.group.is_some())
.map(|(id, f)| (id.clone(), f.clone()))
.collect();
if children.is_empty() {
return Ok(0);
}
let mut fresh = cfg.clone();
for (id, f) in &children {
let group = f.group.clone().unwrap_or_default();
let title = ctx
.db
.feed_summary(id)
.ok()
.and_then(|s| s.title)
.unwrap_or_else(|| id.clone());
ctx.db.upsert_managed(id, &f.url, &title, &group)?;
fresh.feeds.remove(id);
}
fresh.save(&ctx.config_path)?;
ctx.reload_cfg(&ctx.config_path)?;
Ok(children.len())
}
/// Seconds to wait before re-checking a feed. /// Seconds to wait before re-checking a feed.
/// ///
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the /// A per-feed schedule is an explicit instruction and wins outright. Without one, the
@@ -963,7 +933,7 @@ struct Scan {
enum Outcome { enum Outcome {
NotModified, NotModified,
Feed(Scan), Feed(Scan),
/// The URL served an OPML document, so it is a subscription list rather than a feed. /// The URL is a list of feeds rather than a feed: an OPML, or a Patreon creator's shows.
Opml { added: Vec<String>, removed: usize, kept: usize, total: usize }, Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
} }
@@ -973,6 +943,31 @@ async fn scan_one(
feed_cfg: &config::Feed, feed_cfg: &config::Feed,
state: &db::HttpState, state: &db::HttpState,
) -> Result<Outcome> { ) -> Result<Outcome> {
// A Patreon creator with more than one show is a list of feeds, like an OPML.
if feed::is_patreon_creator(&feed_cfg.url) {
match feed::patreon_shows(&ctx.client, &feed_cfg.url).await {
Ok((name, shows)) if shows.len() > 1 => {
ctx.db.touch_feed(id, &feed_cfg.url)?;
if let Some(name) = name {
ctx.db.set_title(id, &name)?;
}
// Read as one feed before it was split, it listed every show's items in one
// heap. The items go; its files and read state move to each show as the show
// lists them (`Db::adopt`), so no show comes up empty for want of a URL.
ctx.db.clear_entries(id)?;
return sync_group(ctx, id, feed_cfg, &shows).await;
}
Ok(_) => {} // One show: the creator's feed is that show.
// Already split: keep the shows it has rather than read the creator as one heap.
Err(e) if ctx.db.managed_feeds()?.iter().any(|m| m.group_id == id) => return Err(e),
Err(e) => tracing::warn!(
feed = id,
error = %format!("{e:#}"),
"could not list the Patreon shows; reading it as one feed"
),
}
}
let mut fetched = feed::fetch( let mut fetched = feed::fetch(
&ctx.client, &ctx.client,
feed_cfg, feed_cfg,
@@ -1018,19 +1013,38 @@ async fn scan_one(
)?; )?;
let policy = policy_for(ctx, id, feed_cfg)?; let policy = policy_for(ctx, id, feed_cfg)?;
if let Some(parent) = &feed_cfg.group {
let listed: Vec<(&str, &str)> = parsed
.entries
.iter()
.flat_map(|e| e.enclosures.iter().map(move |x| (e.guid.as_str(), x.url.as_str())))
.collect();
ctx.db.adopt(parent, id, &listed)?;
}
// Verdicts are recorded in `state`, so the download queue below is just "everything still
// pending". A filter's verdict is looked at again on every scan, though: made once, at
// discovery, it outlived the setting behind it, and allowing explicit items afterwards
// changed nothing however often the feed was scanned.
let skipped = ctx.db.skipped_by_filter(id)?;
let mut scan = Scan::default(); let mut scan = Scan::default();
for entry in &parsed.entries { for entry in &parsed.entries {
if ctx.db.record_entry(id, entry)? { if ctx.db.record_entry(id, entry)? {
scan.new_entries += 1; scan.new_entries += 1;
} }
for enc in &entry.enclosures { for enc in &entry.enclosures {
if !ctx.db.record_enclosure(id, &entry.guid, enc)? { let was = if ctx.db.record_enclosure(id, &entry.guid, enc)? {
continue; // Seen before: downloaded, skipped or deliberately reaped. None
} } else if let Some(reason) = skipped.get(&enc.url) {
// Filters run once, at discovery, and are recorded in `state`. The download Some(reason.as_str())
// queue below is then just "everything still pending". } else {
if let Some(reason) = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc) { continue; // Settled: queued, downloaded, reaped, or another feed's file.
ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?; };
let now = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc);
if now != was {
match now {
Some(reason) => ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?,
None => ctx.db.mark_enclosure(&enc.url, "pending", None)?,
}
} }
} }
} }
@@ -1114,12 +1128,6 @@ async fn scan_one(
Ok(Outcome::Feed(scan)) Ok(Outcome::Feed(scan))
} }
/// Brings the feed list in step with a subscribed OPML.
///
/// New entries are added under the OPML's group and folder. An entry that has gone from
/// the OPML is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
/// the UI to explain them.
async fn sync_opml( async fn sync_opml(
ctx: &Arc<Ctx>, ctx: &Arc<Ctx>,
parent_id: &str, parent_id: &str,
@@ -1130,25 +1138,44 @@ async fn sync_opml(
if let Some(title) = feed::opml_title(bytes) { if let Some(title) = feed::opml_title(bytes) {
ctx.db.set_title(parent_id, &title)?; ctx.db.set_title(parent_id, &title)?;
} }
sync_group(ctx, parent_id, parent, &listed).await
}
/// Brings the feed list in step with a list of feeds: a subscribed OPML, or a Patreon
/// creator's shows.
///
/// New entries are added under the list's group and folder. An entry that has gone from
/// the list is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
/// the UI to explain them.
async fn sync_group(
ctx: &Arc<Ctx>,
parent_id: &str,
parent: &config::Feed,
listed: &[(String, String)],
) -> Result<Outcome> {
let cfg = ctx.cfg(); let cfg = ctx.cfg();
let existing = ctx.db.managed_feeds()?; let existing = ctx.db.managed_feeds()?;
let mut added = vec![]; let mut added = vec![];
for (title, url) in &listed { for (title, url) in listed {
// Already known, whether derived or promoted into the config. // Already known, whether derived or promoted into the config.
if let Some(m) = existing.iter().find(|m| &m.url == url) { if let Some(m) = existing.iter().find(|m| &m.url == url) {
ctx.db.upsert_managed(&m.id, url, title, parent_id)?; ctx.db.upsert_managed(&m.id, url, title, parent_id)?;
continue; continue;
} }
if cfg.feeds.values().any(|f| &f.url == url) { // A Patreon show you added by hand may be spelled differently from the one listed.
if cfg.feeds.values().any(|f| feed::same_feed(&f.url, url)) {
continue; continue;
} }
// A removed feed keeps its rows, so its id is only free again for the same feed.
let known = ctx.db.feed_urls()?;
let taken: std::collections::BTreeMap<String, config::Feed> = cfg let taken: std::collections::BTreeMap<String, config::Feed> = cfg
.feeds .feeds
.keys() .keys()
.chain(existing.iter().map(|m| &m.id)) .chain(existing.iter().map(|m| &m.id))
.chain(added.iter()) .chain(added.iter())
.chain(known.iter().filter(|(_, u)| !feed::same_feed(u, url)).map(|(id, _)| id))
.map(|id| (id.clone(), parent.clone())) .map(|id| (id.clone(), parent.clone()))
.collect(); .collect();
let id = config::unique_slug(title, &taken); let id = config::unique_slug(title, &taken);
@@ -1255,7 +1282,7 @@ pub struct Policy {
fn policy_for(ctx: &Ctx, id: &str, feed_cfg: &config::Feed) -> Result<Policy> { fn policy_for(ctx: &Ctx, id: &str, feed_cfg: &config::Feed) -> Result<Policy> {
let global = ctx.cfg().general.max_new_per_check; let global = ctx.cfg().general.max_new_per_check;
Ok(merge_policy(&ctx.db.subscribers(id)?, feed_cfg, global)) Ok(merge_policy(&ctx.db.subscribers(id, feed_cfg.group.as_deref())?, feed_cfg, global))
} }
fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Policy { fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Policy {

View File

@@ -12,9 +12,7 @@ use axum::{
}, },
routing::{delete, get, patch, post}, routing::{delete, get, patch, post},
}; };
use futures_util::StreamExt;
use serde::Deserialize; use serde::Deserialize;
use tokio_stream::wrappers::BroadcastStream;
use tower::ServiceExt; use tower::ServiceExt;
use tower_http::services::ServeFile; use tower_http::services::ServeFile;
use serde::Serialize; use serde::Serialize;
@@ -35,28 +33,6 @@ pub struct WebState {
pub events: broadcast::Sender<Event>, pub events: broadcast::Sender<Event>,
} }
/// A 32-hex-character shared secret, generated when config.toml has none.
///
/// ponytail: /dev/urandom rather than a CSPRNG crate -- 16 bytes, once, on a Unix-only
/// binary. Falls back to the clock only if urandom is somehow unreadable, which would be a
/// weak token, so that case is logged loudly.
pub fn generate_token() -> String {
use std::io::Read;
let mut bytes = [0u8; 16];
match std::fs::File::open("/dev/urandom").and_then(|mut f| f.read_exact(&mut bytes)) {
Ok(()) => {}
Err(e) => {
tracing::error!(error = %e, "could not read /dev/urandom; token is NOT secure");
let n = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0);
bytes[..8].copy_from_slice(&n.to_le_bytes());
}
}
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
pub fn router(state: WebState) -> Router { pub fn router(state: WebState) -> Router {
Router::new() Router::new()
.route("/", get(index)) .route("/", get(index))
@@ -65,9 +41,11 @@ pub fn router(state: WebState) -> Router {
.route("/api/feeds", get(feeds).post(add_feed)) .route("/api/feeds", get(feeds).post(add_feed))
.route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed)) .route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
.route("/api/feeds/{id}/entries", get(entries)) .route("/api/feeds/{id}/entries", get(entries))
.route("/api/entries", get(all_entries))
.route("/api/entries/{feed_id}/{guid}/flags", post(set_flags)) .route("/api/entries/{feed_id}/{guid}/flags", post(set_flags))
.route("/api/entries/{feed_id}/{guid}/position", post(set_position)) .route("/api/entries/{feed_id}/{guid}/position", post(set_position))
.route("/api/feeds/{id}/read-all", post(read_all)) .route("/api/feeds/{id}/read-all", post(read_all))
.route("/api/read-all", post(read_all_mine))
.route("/api/feeds/{id}/download-latest", post(download_latest)) .route("/api/feeds/{id}/download-latest", post(download_latest))
.route("/api/enclosures/{id}/download", post(download_now)) .route("/api/enclosures/{id}/download", post(download_now))
.route("/api/enclosures/{id}", delete(delete_file)) .route("/api/enclosures/{id}", delete(delete_file))
@@ -75,6 +53,7 @@ pub fn router(state: WebState) -> Router {
.route("/api/opml", get(export_opml).post(import_opml)) .route("/api/opml", get(export_opml).post(import_opml))
.route("/api/settings", get(get_settings).patch(patch_settings)) .route("/api/settings", get(get_settings).patch(patch_settings))
.route("/api/popular", get(get_popular)) .route("/api/popular", get(get_popular))
.route("/api/directory", get(get_directory))
.route("/api/popular/{id}", post(subscribe_popular)) .route("/api/popular/{id}", post(subscribe_popular))
.route("/api/users", get(list_users).post(add_user)) .route("/api/users", get(list_users).post(add_user))
.route("/api/users/{id}", patch(patch_user).delete(remove_user)) .route("/api/users/{id}", patch(patch_user).delete(remove_user))
@@ -85,6 +64,7 @@ pub fn router(state: WebState) -> Router {
// Signing in cannot require being signed in, so these sit outside the auth layer. // Signing in cannot require being signed in, so these sit outside the auth layer.
.route("/login", get(login_page)) .route("/login", get(login_page))
.route("/api/login", post(login)) .route("/api/login", post(login))
.route("/icon.png", get(icon))
.layer(middleware::from_fn(access_log)) .layer(middleware::from_fn(access_log))
.with_state(state) .with_state(state)
} }
@@ -430,6 +410,15 @@ async fn login_page() -> Html<&'static str> {
Html(include_str!("../web/login.html")) Html(include_str!("../web/login.html"))
} }
/// The 2004 icon, served once for both pages rather than inlined as base64 into each. The
/// sign-in page shows it, so it sits outside the auth layer with /login.
async fn icon() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "image/png"), (header::CACHE_CONTROL, "max-age=86400")],
include_bytes!("../web/ipodderx-icon.png").as_slice(),
)
}
fn constant_time_eq(a: &str, b: &str) -> bool { fn constant_time_eq(a: &str, b: &str) -> bool {
let (a, b) = (a.as_bytes(), b.as_bytes()); let (a, b) = (a.as_bytes(), b.as_bytes());
if a.len() != b.len() { if a.len() != b.len() {
@@ -492,6 +481,9 @@ async fn feeds(
let mut out = Vec::with_capacity(mine.len()); let mut out = Vec::with_capacity(mine.len());
for sub in &subs { for sub in &subs {
let (id, feed) = (&sub.id, &sub.cfg); let (id, feed) = (&sub.id, &sub.cfg);
// In a group, what you have not set on the feed comes from your settings on the group,
// the same fallback the scanner uses (`Db::subscribers`).
let up = feed.group.as_deref().and_then(|g| mine.get(g));
let Some(mine) = mine.get(id) else { continue }; let Some(mine) = mine.get(id) else { continue };
let s = state.ctx.db.feed_summary(id)?; let s = state.ctx.db.feed_summary(id)?;
let st = state.ctx.db.http_state(id)?; let st = state.ctx.db.http_state(id)?;
@@ -501,11 +493,22 @@ async fn feeds(
title: s.title, title: s.title,
image: s.image, image: s.image,
folder: feed.folder.clone(), folder: feed.folder.clone(),
keywords: mine.keywords.clone().unwrap_or_else(|| feed.keywords.clone()), keywords: mine
allow_explicit: mine.allow_explicit.unwrap_or(feed.allow_explicit), .keywords
auto_download: mine.auto_download.unwrap_or(feed.auto_download), .clone()
.or_else(|| up.and_then(|u| u.keywords.clone()))
.unwrap_or_else(|| feed.keywords.clone()),
allow_explicit: mine
.allow_explicit
.or(up.and_then(|u| u.allow_explicit))
.unwrap_or(feed.allow_explicit),
auto_download: mine
.auto_download
.or(up.and_then(|u| u.auto_download))
.unwrap_or(feed.auto_download),
max_new_per_check: mine max_new_per_check: mine
.max_new_per_check .max_new_per_check
.or(up.and_then(|u| u.max_new_per_check))
.map(|n| n as usize) .map(|n| n as usize)
.or(feed.max_new_per_check), .or(feed.max_new_per_check),
group: feed.group.clone(), group: feed.group.clone(),
@@ -537,14 +540,20 @@ async fn feeds(
/// A feed that carries a credential is someone's paid or private subscription. Listing it /// A feed that carries a credential is someone's paid or private subscription. Listing it
/// would let anyone signed in subscribe to it and read what they pay for. /// would let anyone signed in subscribe to it and read what they pay for.
/// ///
/// ponytail: a heuristic. A token hidden in the URL's path gets through; a per-feed /// ponytail: a heuristic. A key hidden in the path of a host not in `PAID_HOSTS` gets
/// `unlisted` flag is the upgrade if that ever happens. /// through; a per-feed `unlisted` flag is the upgrade if that happens again.
fn looks_private(feed: &crate::config::Feed) -> bool { fn looks_private(feed: &crate::config::Feed) -> bool {
if feed.username.is_some() || feed.password.is_some() || feed.password_env.is_some() { if feed.username.is_some() || feed.password.is_some() || feed.password_env.is_some() {
return true; return true;
} }
let Ok(u) = url::Url::parse(&feed.url) else { return true }; let Ok(u) = url::Url::parse(&feed.url) else { return true };
!u.username().is_empty() // Paid-feed services put the subscriber's key in the path as often as in the query, and a
// long path segment alone proves nothing (acast's public show ids look the same). So a
// feed from one of them is private whatever its URL looks like. Supercast is why this
// exists: `feeds.supercast.com/feeds/<key>` reached the directory before it did.
let host = u.host_str().unwrap_or("");
PAID_HOSTS.iter().any(|h| host == *h || host.ends_with(&format!(".{h}")))
|| !u.username().is_empty()
|| u.password().is_some() || u.password().is_some()
|| u.query_pairs().any(|(k, _)| { || u.query_pairs().any(|(k, _)| {
let k = k.to_ascii_lowercase(); let k = k.to_ascii_lowercase();
@@ -554,6 +563,10 @@ fn looks_private(feed: &crate::config::Feed) -> bool {
}) })
} }
/// Services whose feeds are always one subscriber's own.
const PAID_HOSTS: &[&str] =
&["patreon.com", "supercast.com", "supportingcast.fm", "glow.fm", "memberful.com"];
/// Only an id, a title, artwork and a count: never a URL, which is where a key would be. /// Only an id, a title, artwork and a count: never a URL, which is where a key would be.
#[derive(Serialize)] #[derive(Serialize)]
struct PopularRow { struct PopularRow {
@@ -561,10 +574,13 @@ struct PopularRow {
title: Option<String>, title: Option<String>,
image: Option<String>, image: Option<String>,
subscribers: i64, subscribers: i64,
/// Yours already. Everyone counts, you included, so your own feeds are listed too.
subscribed: bool,
} }
/// What other people here subscribe to that you don't, most subscribers first. What the /// Every feed that may be listed, with everyone counted, you included, most subscribers
/// Add feed screen offers, and all that `subscribe_popular` will subscribe you to. /// first. Popular is the top of it, the directory is all of it, and it is all that
/// `subscribe_popular` will subscribe you to.
fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> { fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
let db = &state.ctx.db; let db = &state.ctx.db;
let mine: std::collections::HashSet<String> = let mine: std::collections::HashSet<String> =
@@ -576,14 +592,14 @@ fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
// A feed from an OPML rides on the OPML: everyone subscribed to it counts every feed // A feed from an OPML rides on the OPML: everyone subscribed to it counts every feed
// inside, which would bury everything anyone chose on purpose. // inside, which would bury everything anyone chose on purpose.
let from_opml = s.managed || s.cfg.group.is_some(); let from_opml = s.managed || s.cfg.group.is_some();
if n == 0 || from_opml || mine.contains(&s.id) || looks_private(&s.cfg) { if n == 0 || from_opml || looks_private(&s.cfg) {
continue; continue;
} }
let sum = db.feed_summary(&s.id)?; let sum = db.feed_summary(&s.id)?;
out.push(PopularRow { id: s.id, title: sum.title, image: sum.image, subscribers: n }); let subscribed = mine.contains(&s.id);
out.push(PopularRow { id: s.id, title: sum.title, image: sum.image, subscribers: n, subscribed });
} }
let name = |p: &PopularRow| p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase(); out.sort_by(|a, b| b.subscribers.cmp(&a.subscribers).then_with(|| sort_name(a).cmp(&sort_name(b))));
out.sort_by(|a, b| b.subscribers.cmp(&a.subscribers).then_with(|| name(a).cmp(&name(b))));
Ok(out) Ok(out)
} }
@@ -592,10 +608,24 @@ async fn get_popular(
user: crate::db::User, user: crate::db::User,
) -> Result<Json<Vec<PopularRow>>, ApiError> { ) -> Result<Json<Vec<PopularRow>>, ApiError> {
let mut rows = popular(&state, user.id)?; let mut rows = popular(&state, user.id)?;
rows.truncate(20); rows.truncate(10);
Ok(Json(rows)) Ok(Json(rows))
} }
/// Every feed that may be listed, A to Z.
async fn get_directory(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<Vec<PopularRow>>, ApiError> {
let mut rows = popular(&state, user.id)?;
rows.sort_by_key(sort_name);
Ok(Json(rows))
}
fn sort_name(p: &PopularRow) -> String {
p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase()
}
/// Subscribes by id, because the list never shows a URL. Checked against the same list, so /// Subscribes by id, because the list never shows a URL. Checked against the same list, so
/// a guessed id cannot reach a private feed. /// a guessed id cannot reach a private feed.
async fn subscribe_popular( async fn subscribe_popular(
@@ -695,6 +725,11 @@ mod tests {
// Patreon's shape: the key is a query parameter. // Patreon's shape: the key is a query parameter.
assert!(looks_private(&f("https://www.patreon.com/rss/x?auth=abc123&show=2073588"))); assert!(looks_private(&f("https://www.patreon.com/rss/x?auth=abc123&show=2073588")));
assert!(looks_private(&f("https://example.com/rss?api_key=abc"))); assert!(looks_private(&f("https://example.com/rss?api_key=abc")));
// Paid-feed services put the key in the path; the host gives them away.
assert!(looks_private(&f("https://feeds.supercast.com/feeds/abcdefghijklmnopqrstuvwx")));
assert!(looks_private(&f("https://someshow.supportingcast.fm/content/abc123.rss")));
// A long path segment alone is not a key: acast's public show ids look the same.
assert!(!looks_private(&f("https://feeds.acast.com/public/shows/0123456789abcdef01234567")));
assert!(looks_private(&f("https://ray:hunter2@example.com/rss"))); assert!(looks_private(&f("https://ray:hunter2@example.com/rss")));
assert!(looks_private(&f("not a url")), "unparseable is not safe to list"); assert!(looks_private(&f("not a url")), "unparseable is not safe to list");
let mut basic = f("https://example.com/rss"); let mut basic = f("https://example.com/rss");
@@ -768,15 +803,6 @@ mod tests {
"another feed already has that URL" "another feed already has that URL"
); );
} }
#[test]
fn generated_tokens_are_32_hex_chars_and_not_repeated() {
let a = generate_token();
let b = generate_token();
assert_eq!(a.len(), 32);
assert!(a.chars().all(|c| c.is_ascii_hexdigit()));
assert_ne!(a, b);
}
} }
#[derive(Deserialize)] #[derive(Deserialize)]
@@ -789,6 +815,12 @@ struct Page {
filter: Option<String>, filter: Option<String>,
#[serde(default)] #[serde(default)]
q: Option<String>, q: Option<String>,
/// A column name and asc or desc. Anything unrecognised is newest first: the name picks a
/// fixed expression in the query and never reaches it itself.
#[serde(default)]
sort: Option<String>,
#[serde(default)]
dir: Option<String>,
} }
fn fifty() -> i64 { fn fifty() -> i64 {
@@ -806,20 +838,42 @@ async fn entries(
Path(id): Path<String>, Path(id): Path<String>,
user: crate::db::User, user: crate::db::User,
Query(page): Query<Page>, Query(page): Query<Page>,
) -> Result<Json<EntryPage>, ApiError> {
entry_page(&state, user.id, Some(&id), &page)
}
/// Every subscribed feed's items together, newest first: All Subscriptions.
async fn all_entries(
State(state): State<WebState>,
user: crate::db::User,
Query(page): Query<Page>,
) -> Result<Json<EntryPage>, ApiError> {
entry_page(&state, user.id, None, &page)
}
/// One feed's page of items, or every subscribed feed's when `feed` is None.
fn entry_page(
state: &WebState,
user_id: i64,
feed: Option<&str>,
page: &Page,
) -> Result<Json<EntryPage>, ApiError> { ) -> Result<Json<EntryPage>, ApiError> {
let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all")); let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all"));
let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty()); let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty());
let mut rows = state let db = &state.ctx.db;
.ctx let order = crate::db::order_sql(
.db page.sort.as_deref().unwrap_or("published"),
.entries(user.id, &id, filter, search, page.offset, page.limit.clamp(1, 200))?; page.dir.as_deref().unwrap_or("desc"),
);
let mut rows =
db.entries_in(user_id, feed, filter, search, page.offset, page.limit.clamp(1, 200), &order)?;
// Feed HTML is untrusted: it reaches the page only after ammonia has been through it. // Feed HTML is untrusted: it reaches the page only after ammonia has been through it.
for row in &mut rows { for row in &mut rows {
if let Some(d) = &row.description { if let Some(d) = &row.description {
row.description = Some(ammonia::clean(d)); row.description = Some(ammonia::clean(d));
} }
} }
let total = state.ctx.db.count_entries(user.id, &id, filter, search)?; let total = db.count_in(user_id, feed, filter, search)?;
Ok(Json(EntryPage { total, entries: rows })) Ok(Json(EntryPage { total, entries: rows }))
} }
@@ -830,6 +884,19 @@ struct NewFeed {
folder: Option<String>, folder: Option<String>,
#[serde(default)] #[serde(default)]
keywords: Vec<String>, keywords: Vec<String>,
#[serde(default)]
allow_explicit: bool,
}
/// The Add feed dialog's explicit box. Like everything on a feed's own dialog it is yours, so it
/// goes on your subscription, and before the first scan, which would otherwise skip every
/// explicit item.
fn explicit_on_add(state: &WebState, user_id: i64, feed_id: &str, allow: bool) -> Result<(), ApiError> {
if allow {
let sub = crate::db::Sub { feed_id: feed_id.to_owned(), allow_explicit: Some(true), ..Default::default() };
state.ctx.db.set_subscription(user_id, &sub)?;
}
Ok(())
} }
async fn add_feed( async fn add_feed(
@@ -838,25 +905,42 @@ async fn add_feed(
Json(body): Json<NewFeed>, Json(body): Json<NewFeed>,
) -> Result<Json<serde_json::Value>, ApiError> { ) -> Result<Json<serde_json::Value>, ApiError> {
let mut cfg = (*state.ctx.cfg()).clone(); let mut cfg = (*state.ctx.cfg()).clone();
let url = crate::feed::expand_input(&body.url);
// Someone else may already have it. Then adding costs nothing: no second fetch, no // Someone else may already have it. Then adding costs nothing: no second fetch, no
// second copy on disk, just another name against the same feed. // second copy on disk, just another name against the same feed.
if let Some(existing) = crate::subscriptions(&state.ctx)? if let Some(existing) = crate::subscriptions(&state.ctx)?
.into_iter() .into_iter()
.find(|s| s.cfg.url == body.url) .find(|s| crate::feed::same_feed(&s.cfg.url, &url))
{ {
let already = state.ctx.db.subscription(user.id, &existing.id)?.is_some(); let already = state.ctx.db.subscription(user.id, &existing.id)?.is_some();
state.ctx.db.subscribe(user.id, &existing.id)?; state.ctx.db.subscribe(user.id, &existing.id)?;
if !already {
explicit_on_add(&state, user.id, &existing.id, body.allow_explicit)?;
}
scan_soon(&state, Some(existing.id.clone())).await;
return Ok(Json( return Ok(Json(
serde_json::json!({ "id": existing.id, "existing": already }), serde_json::json!({ "id": existing.id, "existing": already }),
)); ));
} }
let id = crate::add_one(&state.ctx, &mut cfg, &body.url, body.folder, body.keywords).await?; let id = crate::add_one(&state.ctx, &mut cfg, &url, body.folder, body.keywords).await?;
cfg.save(&state.config_path)?; cfg.save(&state.config_path)?;
state.ctx.reload_cfg(&state.config_path)?; state.ctx.reload_cfg(&state.config_path)?;
state.ctx.db.subscribe(user.id, &id)?; state.ctx.db.subscribe(user.id, &id)?;
explicit_on_add(&state, user.id, &id, body.allow_explicit)?;
scan_soon(&state, Some(id.clone())).await;
Ok(Json(serde_json::json!({ "id": id, "existing": false }))) Ok(Json(serde_json::json!({ "id": id, "existing": false })))
} }
/// Queues a scan, so a feed just added shows its items without anyone pressing Scan now.
/// `None` scans whatever is due, which a feed never checked always is. The add has
/// succeeded either way, so a daemon not taking commands is only logged.
async fn scan_soon(state: &WebState, feed: Option<String>) {
let force = feed.is_some();
if state.cmds.send(Command::Fetch { feed, force }).await.is_err() {
tracing::warn!("could not queue a scan: the daemon is not accepting commands");
}
}
/// Absent means "leave alone"; JSON `null` means "clear this". /// Absent means "leave alone"; JSON `null` means "clear this".
/// ///
/// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a /// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a
@@ -1089,9 +1173,9 @@ async fn delete_file(
let complaint = match (starred, unread) { let complaint = match (starred, unread) {
(0, 0) => None, (0, 0) => None,
(0, u) => Some(format!("{} subscribed to this feed {} not played it yet", people(u), if u == 1 { "has" } else { "have" })), (0, u) => Some(format!("{} subscribed to this feed {} not played it yet", people(u), if u == 1 { "has" } else { "have" })),
(st, 0) => Some(format!("another {} starred it to keep", people(st))), (st, 0) => Some(format!("another {} kept it", people(st))),
(st, u) => Some(format!( (st, u) => Some(format!(
"another {} starred it to keep, and {} not played it yet", "another {} kept it, and {} not played it yet",
people(st), people(st),
if u == 1 { "one person has".to_string() } else { format!("{u} have") } if u == 1 { "one person has".to_string() } else { format!("{u} have") }
)), )),
@@ -1140,9 +1224,20 @@ async fn fetch_now(
/// The same broadcast the socket clients read, as server-sent events. /// The same broadcast the socket clients read, as server-sent events.
async fn events(State(state): State<WebState>) -> Sse<impl futures_util::Stream<Item = Result<SseEvent, std::convert::Infallible>>> { async fn events(State(state): State<WebState>) -> Sse<impl futures_util::Stream<Item = Result<SseEvent, std::convert::Infallible>>> {
let stream = BroadcastStream::new(state.events.subscribe()).filter_map(|ev| async move { // A client that falls behind skips what it missed rather than being cut off.
let ev = ev.ok()?; let stream = futures_util::stream::unfold(state.events.subscribe(), |mut rx| async move {
Some(Ok(SseEvent::default().data(serde_json::to_string(&ev).ok()?))) loop {
match rx.recv().await {
Ok(ev) => {
if let Ok(data) = serde_json::to_string(&ev) {
let ev = Ok::<_, std::convert::Infallible>(SseEvent::default().data(data));
return Some((ev, rx));
}
}
Err(broadcast::error::RecvError::Lagged(_)) => {}
Err(broadcast::error::RecvError::Closed) => return None,
}
}
}); });
Sse::new(stream).keep_alive(axum::response::sse::KeepAlive::default()) Sse::new(stream).keep_alive(axum::response::sse::KeepAlive::default())
} }
@@ -1197,6 +1292,18 @@ async fn read_all(
Ok(Json(serde_json::json!({ "marked": n }))) Ok(Json(serde_json::json!({ "marked": n })))
} }
/// Everything read in every feed you subscribe to: exactly what All Subscriptions lists, since
/// that view is scoped by the same subscriptions.
async fn read_all_mine(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<serde_json::Value>, ApiError> {
let ids: Vec<String> =
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
let n = state.ctx.db.mark_all_read(user.id, &ids)?;
Ok(Json(serde_json::json!({ "marked": n })))
}
#[derive(Deserialize)] #[derive(Deserialize)]
struct HowMany { struct HowMany {
#[serde(default = "five")] #[serde(default = "five")]
@@ -1285,6 +1392,9 @@ async fn import_opml(
let doc = opml::OPML::from_str(&body.xml) let doc = opml::OPML::from_str(&body.xml)
.map_err(|e| ApiError::bad_request(format!("that is not an OPML file: {e}")))?; .map_err(|e| ApiError::bad_request(format!("that is not an OPML file: {e}")))?;
let (added, already) = crate::subscribe_opml(&state.ctx, &state.config_path, &doc, user.id)?; let (added, already) = crate::subscribe_opml(&state.ctx, &state.config_path, &doc, user.id)?;
if added > 0 {
scan_soon(&state, None).await;
}
Ok(Json(serde_json::json!({ "added": added, "already": already }))) Ok(Json(serde_json::json!({ "added": added, "already": already })))
} }
@@ -1338,8 +1448,6 @@ async fn patch_settings(
))); )));
} }
cfg.general.schedule = sched; cfg.general.schedule = sched;
// The legacy key would otherwise keep shadowing intent in the file.
cfg.general.interval_mins = None;
} }
if let Some(v) = body.max_new_per_check { if let Some(v) = body.max_new_per_check {
cfg.general.max_new_per_check = v; cfg.general.max_new_per_check = v;

View File

@@ -15,7 +15,7 @@ const script = html.split('<script>')[1].split('</script>')[0];
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1])); const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
const missing = []; const missing = [];
const el = (name) => new Proxy({ style: {}, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } }, const el = (name) => new Proxy({ style: { setProperty(){}, getPropertyValue(){ return ''; } }, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null, value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){}, appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){}, setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
@@ -46,7 +46,10 @@ const ctx = {
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 } ? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
: String(url).includes('/api/users') : String(url).includes('/api/users')
? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }] ? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }]
: []), : /\/api\/(popular|directory)/.test(String(url))
? [{ id: 'f', title: 'A Feed', image: null, subscribers: 2, subscribed: true },
{ id: 'g', title: null, image: null, subscribers: 1, subscribed: false }]
: /entries/.test(String(url)) ? { total: 0, entries: [] } : []),
}), }),
EventSource: function () { this.close = () => {}; }, EventSource: function () { this.close = () => {}; },
MediaMetadata: function () {}, MediaMetadata: function () {},
@@ -83,6 +86,9 @@ const drive = [
['prefsModal', () => ctx.prefsModal()], ['prefsModal', () => ctx.prefsModal()],
['usersModal', () => ctx.usersModal()], ['usersModal', () => ctx.usersModal()],
['opmlModal', () => ctx.opmlModal()], ['opmlModal', () => ctx.opmlModal()],
['selectFeed (directory)', () => ctx.selectFeed(':directory')],
['selectFeed (popular)', () => ctx.selectFeed(':popular')],
['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
['logsModal', () => ctx.logsModal()], ['logsModal', () => ctx.logsModal()],
// `const S` is not reachable from here: top-level const/let do not become properties // `const S` is not reachable from here: top-level const/let do not become properties
// of a vm context the way var and function declarations do. // of a vm context the way var and function declarations do.

View File

@@ -29,6 +29,18 @@ test('the theme toggle actually changes the theme', async ({ page }) => {
.not.toBe(before); .not.toBe(before);
}); });
test('the theme button steps through dark, light and classic, and remembers', async ({ page }) => {
const theme = () => page.evaluate(() => document.documentElement.dataset.theme);
for (let i = 0; i < 3 && (await theme()) !== 'classic'; i++) await page.locator('#theme').click();
expect(await theme()).toBe('classic');
await expect(page.locator('#theme')).toHaveAttribute('title', /Classic.*Click for Dark/);
await page.reload();
await expect.poll(theme).toBe('classic');
// The 2004 Mac app set its type in Lucida Grande.
expect(await page.evaluate(() => getComputedStyle(document.body).fontFamily)).toContain('Lucida Grande');
});
test('settings opens and saves the global schedule', async ({ page }) => { test('settings opens and saves the global schedule', async ({ page }) => {
await page.locator('#prefs').click(); await page.locator('#prefs').click();
await expect(page.locator('#modal.on')).toBeVisible(); await expect(page.locator('#modal.on')).toBeVisible();
@@ -69,20 +81,21 @@ test('the three panes are there and the item text lands in the bottom one', asyn
await page.locator('.ep', { hasText: 'First Episode' }).click(); await page.locator('.ep', { hasText: 'First Episode' }).click();
await expect(page.locator('#detail .dt')).toHaveText('First Episode'); await expect(page.locator('#detail .dt')).toHaveText('First Episode');
// The enclosure travels with the item, into the same pane. // The enclosure goes to the Files pane beside the list, as the original's did.
await expect(page.locator('#detail .encbox')).toHaveCount(1); await expect(page.locator('#files')).toBeVisible();
await expect(page.locator('#files .encbox')).toHaveCount(1);
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by // Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
// its chip rather than assuming which episode the daemon happened to fetch. // its chip rather than assuming which episode the daemon happened to fetch.
const downloaded = page.locator('.ep', { hasText: 'downloaded' }).first(); const downloaded = page.locator('.ep', { has: page.locator('.kind.here') }).first();
await downloaded.click(); await downloaded.click();
await expect(page.locator('#detail audio')).toBeVisible(); await expect(page.locator('#files [data-a="play"]')).toBeVisible();
await expect(page.locator('#detail .encbox .btn', { hasText: 'Save' })).toBeVisible(); await expect(page.locator('#files .encbox [title="Save to this computer"]')).toBeVisible();
// Selecting another item replaces the pane rather than stacking. // Selecting another item replaces the pane rather than stacking.
await page.locator('.ep', { hasText: 'First Episode' }).click(); await page.locator('.ep', { hasText: 'First Episode' }).click();
await expect(page.locator('#detail .dt')).toHaveText('First Episode'); await expect(page.locator('#detail .dt')).toHaveText('First Episode');
await expect(page.locator('#detail audio')).toHaveCount(0); await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
}); });
test('a downloaded file that is not audio gets no player', async ({ page }) => { test('a downloaded file that is not audio gets no player', async ({ page }) => {
@@ -96,12 +109,12 @@ test('a downloaded file that is not audio gets no player', async ({ page }) => {
await row.click(); await row.click();
await expect(page.locator('#detail .dt')).toHaveText('An Article'); await expect(page.locator('#detail .dt')).toHaveText('An Article');
await expect(page.locator('#detail audio')).toHaveCount(0); await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
await expect(page.locator('#detail .encbox')).toContainText('image'); // What it is and that it is here: one icon, green, with the words in its tooltip.
await expect(page.locator('#detail .encbox')).toContainText('downloaded'); await expect(page.locator('#files .encbox .kind.here')).toHaveAttribute('title', 'image, downloaded');
// Still offered as a file, just not as an episode: viewable and keepable. // Still offered as a file, just not as an episode: viewable and keepable.
await expect(page.locator('#detail .btn', { hasText: 'Save' })).toBeVisible(); await expect(page.locator('#files [title="Save to this computer"]')).toBeVisible();
const view = page.locator('#detail a', { hasText: 'View' }); const view = page.locator('#files a[title="View in a new tab"]');
await expect(view).toHaveAttribute('target', '_blank'); await expect(view).toHaveAttribute('target', '_blank');
await expect(view).toHaveAttribute('rel', /noopener/); await expect(view).toHaveAttribute('rel', /noopener/);
await expect(view).toHaveAttribute('href', /\/media\/\d+/); await expect(view).toHaveAttribute('href', /\/media\/\d+/);
@@ -115,9 +128,9 @@ test('an item with several enclosures lists them all', async ({ page }) => {
await expect(row).toContainText('+1 more file'); await expect(row).toContainText('+1 more file');
await row.click(); await row.click();
// The pane below lists every one: the audio and the image. // The Files pane lists every one: the audio and the image.
await expect(page.locator('#detail .encbox')).toHaveCount(2); await expect(page.locator('#files .encbox')).toHaveCount(2);
await expect(page.locator('#detail .encbox').nth(1)).toContainText('image'); await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
}); });
test('the filter tabs change what is listed', async ({ page }) => { test('the filter tabs change what is listed', async ({ page }) => {
@@ -129,13 +142,13 @@ test('the filter tabs change what is listed', async ({ page }) => {
await page.locator('.tabs button', { hasText: 'Unread' }).first().click(); await page.locator('.tabs button', { hasText: 'Unread' }).first().click();
expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all); expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all);
await page.locator('.tabs button', { hasText: 'Flagged' }).first().click(); await page.locator('.tabs button', { hasText: 'Kept' }).first().click();
await expect(page.locator('#count')).toContainText('0 items'); await expect(page.locator('#count')).toContainText('0 items');
}); });
test('a feed URL is editable and has a copy button', async ({ page }) => { test('a feed URL is editable and has a copy button', async ({ page }) => {
await page.getByText('Test Show').click(); await page.getByText('Test Show').click();
await page.locator('.btn', { hasText: 'Settings' }).first().click(); await page.locator('#content .acts [data-a="settings"]').click();
await expect(page.locator('#surl')).toHaveValue(/show\.xml/); await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
await expect(page.locator('#scopy')).toBeVisible(); await expect(page.locator('#scopy')).toBeVisible();
@@ -174,9 +187,11 @@ test('an OPML subscription is a collapsible folder', async ({ page }) => {
body: JSON.stringify({ feed: 'test-subscriptions', force: true }), body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
})); }));
// Every row reserves the chevron slot for alignment; only a folder's is clickable. // Only a folder has a triangle, and it is a button that says whether the folder is open.
const chev = page.locator('.feed.group .chev'); const chev = page.locator('.feed.group .chev');
await expect(chev).toBeVisible({ timeout: 20_000 }); await expect(chev).toBeVisible({ timeout: 20_000 });
await expect(page.locator('.feed:not(.group) .chev')).toHaveCount(0);
await expect(chev).toHaveAttribute('aria-expanded', 'false');
// Closed by default: the children are not listed until the folder is opened. // Closed by default: the children are not listed until the folder is opened.
const before = await page.locator('.feed').count(); const before = await page.locator('.feed').count();
@@ -212,7 +227,7 @@ test('inside an OPML, feeds with unread items are listed first', async ({ page }
await side.getByText('Grouped Show').click(); await side.getByText('Grouped Show').click();
const gs = page.locator('.ep', { hasText: 'Grouped Ep' }); const gs = page.locator('.ep', { hasText: 'Grouped Ep' });
await gs.click(); await gs.click();
await page.locator('#detail button', { hasText: 'Mark unread' }).click(); await page.locator('#detail button[title="Mark unread"]').click();
await expect(gs).not.toHaveClass(/read/); await expect(gs).not.toHaveClass(/read/);
// Aardvark comes first alphabetically and in the OPML, so only the unread sort puts // Aardvark comes first alphabetically and in the OPML, so only the unread sort puts
@@ -238,7 +253,7 @@ test('marking an OPML subscription read covers the feeds inside it', async ({ pa
await expect(folder.locator('.badge')).not.toHaveText('0'); await expect(folder.locator('.badge')).not.toHaveText('0');
await folder.click(); await folder.click();
await page.locator('#content .acts button', { hasText: 'Mark all read' }).click(); await page.locator('#content .acts [data-a="read"]').click();
await expect(folder.locator('.badge')).toHaveText('0'); await expect(folder.locator('.badge')).toHaveText('0');
}); });
@@ -280,9 +295,9 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
// Another test may have opened this item already, so start from a known state: the // Another test may have opened this item already, so start from a known state: the
// toggle in the text below flips it back -- which used to recurse until the stack blew. // toggle in the text below flips it back -- which used to recurse until the stack blew.
await row().click(); await row().click();
await page.locator('#detail button', { hasText: 'Mark unread' }).click(); await page.locator('#detail button[title="Mark unread"]').click();
await expect(row()).not.toHaveClass(/read/); await expect(row()).not.toHaveClass(/read/);
await expect(page.locator('#detail button', { hasText: 'Mark read' })).toBeVisible(); await expect(page.locator('#detail button[title="Mark read"]')).toBeVisible();
// Opening it is reading it. // Opening it is reading it.
await row().click(); await row().click();
@@ -291,6 +306,30 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
expect(errors).toEqual([]); expect(errors).toEqual([]);
}); });
test('the toolbar acts on the selected item', async ({ page }) => {
await page.getByText('Test Show').click();
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
await expect(row()).toBeVisible({ timeout: 20_000 });
// Nothing selected, nothing to act on.
await expect(page.locator('#tbRead')).toBeDisabled();
await row().click(); // opening it reads it
await expect(row()).toHaveClass(/read/);
await page.locator('#tbRead').click();
await expect(row()).not.toHaveClass(/read/);
await page.locator('#tbFlag').click();
await expect(row().locator('.fl')).toHaveClass(/on/);
await page.locator('#tbFlag').click(); // and back, so later tests see it unkept
await expect(row().locator('.fl')).not.toHaveClass(/on/);
// Second Episode is the one the daemon downloaded, so it plays from the toolbar.
await expect(page.locator('#tbPlay')).toBeEnabled();
await page.locator('#tbPlay').click();
await expect(page.locator('#player')).toBeVisible();
await page.locator('#pclose').click();
});
test('a second person has their own feeds and their own read state', async ({ browser }) => { test('a second person has their own feeds and their own read state', async ({ browser }) => {
const { execFileSync } = require('child_process'); const { execFileSync } = require('child_process');
const setup = require('./global-setup'); const setup = require('./global-setup');
@@ -309,6 +348,10 @@ test('a second person has their own feeds and their own read state', async ({ br
const ctx = await browser.newContext(); const ctx = await browser.newContext();
const page = await ctx.newPage(); const page = await ctx.newPage();
await page.goto('/login'); await page.goto('/login');
// The sign-in page shows the icon, so it has to load before anyone has signed in.
const icon = await page.request.get('/icon.png');
expect(icon.status()).toBe(200);
expect(icon.headers()['content-type']).toBe('image/png');
await page.locator('#name').fill('sam'); await page.locator('#name').fill('sam');
await page.locator('#pw').fill('sampassword'); await page.locator('#pw').fill('sampassword');
await page.locator('button[type=submit]').click(); await page.locator('button[type=submit]').click();
@@ -350,8 +393,9 @@ test('deleting a shared file warns that it is everyone\'s copy', async ({ page }
await expect(row).toBeVisible({ timeout: 20_000 }); await expect(row).toBeVisible({ timeout: 20_000 });
await row.click(); await row.click();
const del = page.locator('#detail button', { hasText: 'Delete' }); // An icon now; what it does, and to whom, is in its tooltip.
await expect(del).toHaveText('Delete for everyone'); const del = page.locator('#files button[data-a="del"]');
await expect(del).toHaveAttribute('title', /^Delete for everyone \(shared with 1 other person/);
// Two prompts: the page's own, then the server's, because someone else has not played // Two prompts: the page's own, then the server's, because someone else has not played
// it. Accept the first, decline the second, and the file must survive. // it. Accept the first, decline the second, and the file must survive.
@@ -417,7 +461,7 @@ test('Settings exports your OPML and imports a pasted one', async ({ page }) =>
await page.locator('#prefs').click(); await page.locator('#prefs').click();
const [dl] = await Promise.all([ const [dl] = await Promise.all([
page.waitForEvent('download'), page.waitForEvent('download'),
page.locator('#modalCard a', { hasText: 'Export OPML' }).click(), page.locator('#modalCard a[title="Export OPML"]').click(),
]); ]);
expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml'); expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml');
const out = require('fs').readFileSync(await dl.path(), 'utf8'); const out = require('fs').readFileSync(await dl.path(), 'utf8');
@@ -545,7 +589,7 @@ test('ipx import subscribes the admin, and ipx export writes the feeds out', asy
expect(xml).toContain('http://127.0.0.1:8792/two.xml'); expect(xml).toContain('http://127.0.0.1:8792/two.xml');
}); });
test('Add feed offers what other people here read, but never a private feed', async ({ browser }) => { test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
const { execFileSync } = require('child_process'); const { execFileSync } = require('child_process');
const setup = require('./global-setup'); const setup = require('./global-setup');
const env = { const env = {
@@ -566,7 +610,7 @@ test('Add feed offers what other people here read, but never a private feed', as
await piper.locator('button[type=submit]').click(); await piper.locator('button[type=submit]').click();
await expect(piper.locator('#feedlist')).toContainText('No feeds.'); await expect(piper.locator('#feedlist')).toContainText('No feeds.');
await piper.locator('#addFeed').click(); await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
const offered = piper.locator('#popular .childrow'); const offered = piper.locator('#popular .childrow');
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 }); await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// An OPML's own feeds ride on the OPML, and a key in a URL marks someone's paid feed. // An OPML's own feeds ride on the OPML, and a key in a URL marks someone's paid feed.
@@ -579,9 +623,164 @@ test('Add feed offers what other people here read, but never a private feed', as
expect(listed).not.toContain('.xml'); expect(listed).not.toContain('.xml');
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400); expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
await offered.filter({ hasText: 'Test Show' }).locator('button', { hasText: 'Subscribe' }).click(); // Popular is the top ten of the directory, and the directory is every listed feed, A to Z.
const dir = await (await piper.request.get('/api/directory')).json();
const top = await (await piper.request.get('/api/popular')).json();
const names = dir.map(p => (p.title || p.id).toLowerCase());
expect(names).toEqual([...names].sort());
expect(top.length).toBe(Math.min(10, dir.length));
expect(top.every(t => dir.some(d => d.id === t.id))).toBe(true);
expect(dir.map(p => p.id)).not.toContain('paid-show');
// Subscribe from the directory this time; the popular list shares the same rows.
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible();
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
const row = async () =>
(await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
const before = await row();
expect(before.subscribed).toBe(false);
await offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 }); await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// Once it is yours, it is no longer offered.
expect(await (await piper.request.get('/api/popular')).text()).not.toContain('"test-show"'); // Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
// All Subscriptions is every item from piper's feeds and only those: the admin's Picture
// Blog is not among them.
await piper.locator('#feedlist .place', { hasText: 'All Subscriptions' }).click();
const first = piper.locator('.ep', { hasText: 'First Episode' });
await expect(first).toBeVisible({ timeout: 20_000 });
await expect(first.locator('.fd')).toHaveText('Test Show');
await expect(piper.locator('.ep', { hasText: 'An Article' })).toHaveCount(0);
await expect(piper.locator('#count')).toContainText('All Subscriptions:');
await ctx.close(); await ctx.close();
}); });
test('adding a feed scans it straight away', async ({ page }) => {
await page.locator('#addFeed').click();
await page.locator('#nurl').fill('http://127.0.0.1:8792/fresh.xml');
await page.locator('#nsave').click();
// Nobody pressed Scan. The scheduler's tick is a minute, so this is the add scanning it.
await expect(page.locator('.ep', { hasText: 'Fresh Ep' })).toBeVisible({ timeout: 10_000 });
});
test('a deleted file looks as if it was never downloaded', async ({ page }) => {
// Other people subscribe to Picture Blog by now, so both prompts come; take them.
page.on('dialog', d => d.accept());
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
const row = page.locator('.ep', { hasText: 'An Article' });
await expect(row).toBeVisible({ timeout: 20_000 });
await row.click();
await page.locator('#files button[data-a="del"]').click();
// No "reaped", no chip at all: just the way to get it again.
await expect(row.locator('.kind.here')).toHaveCount(0);
await expect(row).not.toContainText(/reaped/i);
await row.click();
await expect(page.locator('#files')).not.toContainText(/reaped/i);
await expect(page.locator('#files button[title="Download to the server"]')).toBeVisible();
});
test('one action, one icon: the toolbar, the page and every dialog agree', async ({ page }) => {
const icon = loc => loc.locator('svg path').first().getAttribute('d');
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
// Unsubscribe is a minus in the toolbar and the feed header, never the x that closes things.
expect(await icon(page.locator('#content .acts [data-a="rm"]'))).toBe(await icon(page.locator('#tbRemove')));
// The toolbar's read and keep show the selected item's state, as its own buttons do, and follow
// a change made from the toolbar.
await page.locator('.ep').first().click();
const pair = async a => [await icon(page.locator(a === 'read' ? '#tbRead' : '#tbFlag')),
await icon(page.locator(`#detail [data-a="${a}"]`))];
for (const a of ['read', 'flag']) { const [tb, own] = await pair(a); expect(tb).toBe(own); }
const [kept] = await pair('flag');
await page.locator('#tbFlag').click();
await expect.poll(async () => { const [tb, own] = await pair('flag'); return tb === own && tb !== kept; }).toBe(true);
await page.locator('#tbFlag').click(); // leave it as it was
await expect.poll(async () => (await pair('flag'))[0]).toBe(kept);
// Every button in every dialog is an icon with its words in the tooltip.
const dialogs = [
() => page.locator('#addFeed').click(),
() => page.locator('#prefs').click(),
async () => { await page.locator('#prefs').click(); await page.locator('#gusers').click(); },
async () => { await page.locator('#prefs').click(); await page.locator('#gopml').click(); },
() => page.locator('#logs').click(),
() => page.locator('#content .acts [data-a="settings"]').click(),
() => page.locator('#content .acts [data-a="dl"]').click(),
() => page.locator('#content .acts [data-a="rm"]').click(),
];
for (const open of dialogs) {
await open();
const btns = page.locator('#modalCard .btn');
await expect(btns.first()).toBeVisible();
for (const b of await btns.all()) {
await expect(b.locator('svg')).toHaveCount(1);
await expect(b).toHaveAttribute('title', /\S/);
}
await page.keyboard.press('Escape');
await expect(page.locator('#modal.on')).toBeHidden();
}
});
test('All Subscriptions marks everything read, across every feed', async ({ page }) => {
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
await all.click();
// Earlier tests read things; make sure something is unread. Opening an item reads it, and
// its own button makes it unread again.
await page.locator('.ep').first().click();
await page.locator('#detail [data-a="read"][title="Mark unread"]').click();
await expect(all.locator('.badge')).not.toHaveText('0');
page.once('dialog', d => d.accept());
await page.locator('#content .acts [data-a="readall"]').click();
await expect(all.locator('.badge')).toHaveText('0');
await page.locator('.tabs button', { hasText: 'Unread' }).click();
await expect(page.locator('.ep')).toHaveCount(0);
});
test('the item table sorts by any column, both ways, and remembers', async ({ page }) => {
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
await all.click();
const head = k => page.locator(`#list .ephead [data-sort="${k}"]`);
const titles = () => page.locator('#eps .ep .t').allTextContents();
// Byte order on lower case, which is what SQLite gives for lower(...).
const cmp = (a, b) => (a.toLowerCase() < b.toLowerCase() ? -1 : a.toLowerCase() > b.toLowerCase() ? 1 : 0);
const sorted = (t, dir) => JSON.stringify(t) === JSON.stringify([...t].sort((a, b) => cmp(a, b) * dir));
await expect(page.locator('#eps .ep').nth(2)).toBeVisible({ timeout: 20_000 });
expect(new Set(await titles()).size).toBeGreaterThan(2); // or both orders would prove nothing
await expect(head('title')).toHaveText('Title');
await head('title').click();
await expect.poll(async () => sorted(await titles(), 1)).toBe(true);
await head('title').click();
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
// Kept across a reload.
await page.reload();
await all.click();
await expect(head('title').locator('.arr.desc')).toBeVisible();
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
// Size has its own column; the file column is just what the file is.
await expect(page.locator('#eps .ep .size', { hasText: /\d/ }).first()).toBeVisible();
await expect(page.locator('#eps .ep .file', { hasText: /\d/ })).toHaveCount(0);
});
test('play in the Files pane plays once, in the player bar', async ({ page }) => {
// Regression: the pane had an <audio> of its own, and playing it started the player bar too,
// so the same file played twice at once.
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
await page.locator('.ep', { has: page.locator('.kind.here') }).first().click();
await page.locator('#files [data-a="play"]').click();
await expect(page.locator('#player')).toBeVisible();
await expect(page.locator('audio')).toHaveCount(1); // the player bar's, and nothing else
await page.locator('#pclose').click();
});

View File

@@ -0,0 +1,5 @@
<?xml version="1.0"?>
<rss version="2.0"><channel><title>Fresh Show</title><link>http://127.0.0.1:8792/</link>
<description>Added in the browser suite, and scanned by adding it.</description>
<item><title>Fresh Ep</title><guid>fresh-1</guid><description>x</description></item>
</channel></rss>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long