Compare commits
38 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 9aae3097e7 | |||
| 2ff2074755 | |||
| 1698cf8d1e | |||
| b94a74ef15 | |||
| 9a8a3c696f | |||
| bedf64e645 | |||
| 586d2c07a1 | |||
| 2ba83c3aed | |||
| 1352f0d54d | |||
| a958f7cb37 | |||
| 457a58dcc5 | |||
| 2af57065c6 | |||
| 564b011c7a | |||
| 9eb7aadced | |||
| c6bceaef37 | |||
| dc63d6acaf | |||
| 8937f35f00 | |||
| 0990f2a90d | |||
| 0cc002cdfa | |||
| 2c9e899762 | |||
| 2e416f96cf | |||
| 9269aa99f7 | |||
| ae8123250b | |||
| 2c34a144ba | |||
| 669e8b5124 | |||
| 5362436766 | |||
| 0efc49519c | |||
| 57dcba2d1a | |||
| e95cccc66f | |||
| 7295be8b25 | |||
| f1b0d97b81 | |||
| ae47e31a97 | |||
| 8a309eb652 | |||
| 1a2b0d87c6 | |||
| d7fac2d0e7 | |||
| df9b7645d6 | |||
| f3825cfc57 | |||
| c0f4b0bcb2 |
168
CHANGELOG.md
168
CHANGELOG.md
@@ -10,6 +10,168 @@ The long form, with what was wrong before and how it was found, is in
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
## [0.5.2] - 2026-09-12
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Settings → Users and `ipx user list` show when each account was added and when it last signed
|
||||||
|
in, to the hour.
|
||||||
|
- `ipx user rename <name> <new name>` renames an account and keeps its feeds, read state and admin
|
||||||
|
rights. An account made before the proxy was set up can take the name the proxy signs it in as.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Directory and Popular list the feeds inside an OPML one by one, and no longer the OPML itself,
|
||||||
|
so you can subscribe to just the shows you want.
|
||||||
|
- The database no longer records when subscriptions and sign-in sessions were created. Nothing
|
||||||
|
ever read it, and an existing database drops the columns on its next start.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Show notes that the podcast's host cut off in the middle of a tag no longer open with a scrap of
|
||||||
|
HTML: the item's other copy of its notes is used instead, from the next time the feed changes.
|
||||||
|
Daily Meditation Podcast had 57.
|
||||||
|
- Docker no longer shows ipodderx as starting, or calls it unhealthy, while it scans or downloads:
|
||||||
|
`ipx status` answers at once instead of waiting for the job in progress to finish.
|
||||||
|
- Signing out after signing in through Cloudflare Access no longer lands on ipodderx's own password
|
||||||
|
page. With the new `sign_out_url` set, Sign out ends the Access session, and the password page
|
||||||
|
sends anyone the proxy signs in straight to their feeds.
|
||||||
|
- The sign-in guide, `docs/sso.md`, describes the setup ipodderx.sdf1.net really runs: Authentik as
|
||||||
|
Cloudflare Access's identity provider, and how to find the address ipx has to trust. It had never
|
||||||
|
been checked against a real setup, and pointed at the wrong address.
|
||||||
|
|
||||||
|
## [0.5.1] - 2026-09-12
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- The triangle that opens an OPML or Patreon folder was cramped against the folder's art. It has
|
||||||
|
more room now, and a wider target to click.
|
||||||
|
|
||||||
|
## [0.5.0] - 2026-09-12
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- A Patreon token pasted into Add feed, or a creator's RSS link without `&show=`, becomes a folder
|
||||||
|
of that creator's shows, kept in step on every scan like a subscribed OPML. A creator with only
|
||||||
|
one show stays a plain feed. One already added as a single long feed is split into its shows on
|
||||||
|
its next scan, keeping its files and what you had read.
|
||||||
|
- Add feed has an "Allow items marked explicit" box, so a new feed's first scan no longer skips
|
||||||
|
every explicit item.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Unread counts, unread dots and download progress are amber, the colour of the icon's EQ bars.
|
||||||
|
Blue is kept for the primary action and links, so a count no longer looks like a button.
|
||||||
|
- What is playing is marked by small EQ bars, in its row and in the player. They move only while it
|
||||||
|
plays.
|
||||||
|
- A folder in the sidebar shows its first four shows' art as a mosaic, and its shows sit under its
|
||||||
|
title. Only folders have a triangle, so every feed lines up with Directory and Popular above.
|
||||||
|
- Feeds without art get initials in a colour of their own, instead of all the same grey.
|
||||||
|
- The Flagged tab is called Kept, as the Keep button and Settings already said.
|
||||||
|
- A feed's header is one short line; when it checks next is in its tooltip.
|
||||||
|
- The item list takes more of the window, and the Files pane shows only when the item has files.
|
||||||
|
- Column headings and tags are in sentence case, and fewer things are bold.
|
||||||
|
- Unsubscribe is a round button beside the feed's other actions.
|
||||||
|
- Export and Import in Settings say what they do.
|
||||||
|
- The sign-in page shows the original icon large.
|
||||||
|
- Nothing animates when your system asks for reduced motion.
|
||||||
|
- The pages are about 90 KB smaller: the icon is served once instead of written into each.
|
||||||
|
- A web token generated for a new install is 64 characters instead of 32.
|
||||||
|
- The README is a short overview of what ipx does and how to run it, and points into `docs/` for
|
||||||
|
the rest. It still described the layout from before 0.4.0.
|
||||||
|
|
||||||
|
### Removed
|
||||||
|
|
||||||
|
- The systemd units in `contrib/`. Run ipx with Docker, or point a unit of your own at
|
||||||
|
`ipx daemon`.
|
||||||
|
- Upgrading from before 0.3.0 directly: what was read, kept or part-played before accounts is no
|
||||||
|
longer carried over to the admin, and OPML feeds that old versions wrote into `config.toml` are
|
||||||
|
no longer moved out of it. Upgrade through 0.4.0 first.
|
||||||
|
- `interval_mins` in `config.toml` is ignored; use `schedule`.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- The feed list works from the keyboard: Tab reaches every feed, Enter opens it, and Right and Left
|
||||||
|
open and close a folder. Every button shows where the focus is, including in the toolbar, which
|
||||||
|
used to clip the ring.
|
||||||
|
- "1 items" reads "1 item".
|
||||||
|
- A selected feed without art no longer loses its initials tile in Dark and Light.
|
||||||
|
- The player shows the feed's initials when there is no art, not the episode's.
|
||||||
|
- Turning on Allow explicit, or changing keywords or auto-download, brings back what those settings
|
||||||
|
had skipped on the feed's next scan. Before, an item was judged once, when first seen, and a
|
||||||
|
skipped one stayed skipped whatever you changed.
|
||||||
|
- Feeds inside an OPML or a Patreon creator follow your settings on the folder unless you set their
|
||||||
|
own, as the folder's settings dialog said they did. Before, the folder's settings reached nothing
|
||||||
|
inside it.
|
||||||
|
- A new feed no longer takes the name of one you removed earlier and shows that feed's old items.
|
||||||
|
Re-adding the same feed still gets its old name, and its history, back.
|
||||||
|
|
||||||
|
## [0.4.0] - 2026-09-11
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- A Classic theme after the 2004 Mac app, beside Dark and Light: brushed-metal toolbar, Aqua
|
||||||
|
blue selection, red unread badges, a striped table and Lucida Grande. The theme button steps
|
||||||
|
through all three and remembers the choice.
|
||||||
|
- A toolbar across the top, after the original iPodderX: add and unsubscribe, play, mark read
|
||||||
|
and keep for the selected item, scan, a search box for what is showing, and Settings and Log.
|
||||||
|
- Directory, Popular and All Subscriptions at the top of the feed list, opening in the main pane.
|
||||||
|
Directory lists every feed anyone here subscribes to, A to Z (`GET /api/directory`). All
|
||||||
|
Subscriptions lists every item from every feed you subscribe to (`GET /api/entries`).
|
||||||
|
- Items show as a table (unread, kept, title, feed, file, size, published) with a Files pane beside
|
||||||
|
it, and a status bar with the totals.
|
||||||
|
- Mark everything read from All Subscriptions, across every feed you subscribe to
|
||||||
|
(`POST /api/read-all`). It asks first. All Subscriptions can also check every feed from its header.
|
||||||
|
- Click a column heading in the item table to sort by it (kept, title, feed, file type, size,
|
||||||
|
published); click again to reverse. The server sorts, so it covers the whole list, not just the
|
||||||
|
fifty shown, and the choice is remembered.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Popular shows the top 10, not 20, and counts everyone, you included. Your own feeds stay on it,
|
||||||
|
marked Subscribed, and clicking one opens it.
|
||||||
|
- Adding a feed scans it straight away, and an OPML import that added feeds scans them, so their
|
||||||
|
items show without pressing Scan.
|
||||||
|
- A file deleted to save space, or by hand, looks as if it was never downloaded: no "reaped"
|
||||||
|
label, just the Download button. The retention summary says "deleted", not "reaped".
|
||||||
|
- Buttons are icons, with the words in their tooltips: the Files pane (save, delete, view,
|
||||||
|
download), an item's own buttons (mark read, keep, open the original), the feed header (scan,
|
||||||
|
download latest, mark all read, settings, unsubscribe), and the Settings, feed settings and
|
||||||
|
Download latest dialogs (save, download, cancel). The icons are Font Awesome Free, embedded as
|
||||||
|
SVG: only the ones used, no font to download, and nothing fetched from anyone else. They
|
||||||
|
replace font characters such as ⟳ ⤓ ↗, which came out thin and tiny and differed from font to
|
||||||
|
font. Keep is a flag everywhere, as it was in the original, and mark unread is an envelope.
|
||||||
|
- One meaning per icon. Minus unsubscribes, x closes or cancels, plus adds or subscribes, and a
|
||||||
|
dialog's confirm button carries the icon of what it does. The feed header's unsubscribe was an x
|
||||||
|
and read as closing the page. The remaining word buttons are icons too:
|
||||||
|
- Log, Add feed, Users, Unsubscribe and OPML.
|
||||||
|
- Popular's Subscribe, Copy and Sign out.
|
||||||
|
- The Subscribed label in Popular, the Directory and Add feed, which is now a green check.
|
||||||
|
- The player's back, play, forward and close, which were font characters, and the folder arrow.
|
||||||
|
- The toolbar's read and keep buttons show the selected item's state, with the same icons as the
|
||||||
|
item's own buttons. Play, read and keep sit together, and Scan sits with add and unsubscribe.
|
||||||
|
- An OPML subscription's page has the same header as a feed's, with its buttons in the same places.
|
||||||
|
- The item table's size has its own column, apart from the file's type, and shows KB for small
|
||||||
|
files instead of "0 MB". The Item heading is now Title.
|
||||||
|
- A file's type is an icon (audio, video, image, PDF, torrent, other), green once it is
|
||||||
|
downloaded and red when the download failed, with the details in its tooltip. One icon per
|
||||||
|
row keeps the column lined up. The DOWNLOADED and PENDING labels are gone.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Playing a file from the Files pane played it twice at once, in the pane and in the player bar.
|
||||||
|
The pane has a play button now, and the player bar is the only player.
|
||||||
|
- The password box in Manage users was white in the dark theme.
|
||||||
|
- An item with no date showed a stray dot in its details.
|
||||||
|
- Escape did not close a dialog while the cursor was in one of its boxes, so Add feed, which opens
|
||||||
|
in its URL box, could not be closed with Escape.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- Feeds from paid-feed services (Patreon, Supercast, Supporting Cast, Glow, Memberful) are never
|
||||||
|
listed in Popular or the Directory. A Supercast feed, which keeps its key in the URL's path
|
||||||
|
rather than the query, was being listed.
|
||||||
|
|
||||||
## [0.3.0] - 2026-09-11
|
## [0.3.0] - 2026-09-11
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
@@ -133,7 +295,11 @@ The long form, with what was wrong before and how it was found, is in
|
|||||||
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
|
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
|
||||||
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
|
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
|
||||||
|
|
||||||
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...main
|
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.2...main
|
||||||
|
[0.5.2]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.1...v0.5.2
|
||||||
|
[0.5.1]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.0...v0.5.1
|
||||||
|
[0.5.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.4.0...v0.5.0
|
||||||
|
[0.4.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...v0.4.0
|
||||||
[0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0
|
[0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0
|
||||||
[0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0
|
[0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0
|
||||||
[0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0
|
[0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0
|
||||||
|
|||||||
35
CLAUDE.md
35
CLAUDE.md
@@ -17,6 +17,7 @@ Arcane project `content`: `/mnt/fast/arcane/projects/content/compose.yaml`. That
|
|||||||
| Database | `/mnt/user/ipodderx/state.db` | `/data/state.db` |
|
| Database | `/mnt/user/ipodderx/state.db` | `/data/state.db` |
|
||||||
| Downloads | `/mnt/user/ipodderx/downloads` | `/downloads` |
|
| Downloads | `/mnt/user/ipodderx/downloads` | `/downloads` |
|
||||||
| Web UI | `192.168.1.130:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel | `0.0.0.0:8099` |
|
| Web UI | `192.168.1.130:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel | `0.0.0.0:8099` |
|
||||||
|
| Sign-in via the tunnel | Cloudflare Access app `ipodderx`, with Authentik as its identity provider; see [docs/sso.md](docs/sso.md) | trusts `Cf-Access-Authenticated-User-Email` from `192.168.16.1`, the `content_default` gateway |
|
||||||
|
|
||||||
Deploying a change is: build and push the image, then pull it and recreate the container.
|
Deploying a change is: build and push the image, then pull it and recreate the container.
|
||||||
|
|
||||||
@@ -30,8 +31,25 @@ docker logs --tail 20 iPodderX
|
|||||||
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
|
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
|
||||||
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
|
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
|
||||||
|
|
||||||
|
**A build that fails with `429 Too Many Requests` on a base image** is Docker Hub rate-limiting
|
||||||
|
this host. There is no Docker Hub login here, and the build asks about `debian:bookworm-slim` and
|
||||||
|
`rust:1-slim-bookworm` every time unless they are already stored locally. Pull them from Google's
|
||||||
|
mirror and tag them; the build then uses the local copies without asking Docker Hub:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker pull mirror.gcr.io/library/debian:bookworm-slim
|
||||||
|
docker tag mirror.gcr.io/library/debian:bookworm-slim debian:bookworm-slim
|
||||||
|
docker pull mirror.gcr.io/library/rust:1-slim-bookworm
|
||||||
|
docker tag mirror.gcr.io/library/rust:1-slim-bookworm rust:1-slim-bookworm
|
||||||
|
```
|
||||||
|
|
||||||
|
Run those again now and then, or the local copies go stale.
|
||||||
|
|
||||||
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
|
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
|
||||||
the worker is alive, not just the web port. The container restarts on its own after a reboot.
|
the daemon answers there and can read its database, not just that the web port is up. The socket
|
||||||
|
answers `status` itself instead of queuing it behind the worker's current job, so a long scan or
|
||||||
|
download does not fail the check; it also means a worker stuck on one job would still pass. The
|
||||||
|
container restarts on its own after a reboot.
|
||||||
|
|
||||||
Before the container, ipx ran by hand in code-server, with its files in `/config/.config/ipx/` and
|
Before the container, ipx ran by hand in code-server, with its files in `/config/.config/ipx/` and
|
||||||
`/config/.local/share/ipx/`. Those are still there and the container does not read them. If you run
|
`/config/.local/share/ipx/`. Those are still there and the container does not read them. If you run
|
||||||
@@ -85,9 +103,9 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
|
|||||||
|
|
||||||
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
|
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
|
||||||
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
|
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
|
||||||
* **`entries.read`, `entries.flagged` and `entries.position` are dead columns.** Read state lives in
|
* **Read state lives in `entry_state`, per user, and nowhere else.** `entries` had `read`, `flagged`
|
||||||
`entry_state` per user. Two bugs have already come from queries still reading the old ones
|
and `position` columns from before accounts; two bugs came from queries still reading them
|
||||||
(retention, and the entry pruner) — grep before adding a third.
|
(retention, and the entry pruner), and `migrate()` now drops them.
|
||||||
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
|
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
|
||||||
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
|
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
|
||||||
derived and never written to config.
|
derived and never written to config.
|
||||||
@@ -99,8 +117,13 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
|
|||||||
watch the shutdown channel itself; the daemon ignored SIGTERM for exactly this reason.
|
watch the shutdown channel itself; the daemon ignored SIGTERM for exactly this reason.
|
||||||
* Only one daemon per socket. Removing the socket file defeats the guard and you get two daemons
|
* Only one daemon per socket. Removing the socket file defeats the guard and you get two daemons
|
||||||
fighting over the database, with the stale one still holding the port.
|
fighting over the database, with the stale one still holding the port.
|
||||||
* `/api/settings` answering `200` does **not** mean the worker is alive — it is a different task.
|
* `/api/settings` answering `200` does **not** mean the daemon is well — the web server is a
|
||||||
Probe the control socket (`ipx status`) to check that.
|
different task. `ipx status` checks the control socket and the database; to see the worker
|
||||||
|
getting through its jobs, watch for `scan complete` in the log.
|
||||||
|
* **Every `ipx` command runs `migrate()` when it opens the database**, the healthcheck's
|
||||||
|
`ipx status` included. A migration that rewrites a big table (`DROP COLUMN`) takes seconds on
|
||||||
|
production, and a command run meanwhile fails with `migrating schema`. It changes nothing; wait
|
||||||
|
for `daemon started` in the log. Copy `state.db` aside before deploying one.
|
||||||
|
|
||||||
## House style
|
## House style
|
||||||
|
|
||||||
|
|||||||
34
Cargo.lock
generated
34
Cargo.lock
generated
@@ -436,17 +436,6 @@ dependencies = [
|
|||||||
"shlex",
|
"shlex",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "cfb"
|
|
||||||
version = "0.14.0"
|
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
||||||
checksum = "a347dcabdae9c31b0825fd6a8bed285ec9c2acb89c47827126d52fa4f59cece3"
|
|
||||||
dependencies = [
|
|
||||||
"fnv",
|
|
||||||
"uuid",
|
|
||||||
"web-time",
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "cfg-if"
|
name = "cfg-if"
|
||||||
version = "1.0.4"
|
version = "1.0.4"
|
||||||
@@ -867,15 +856,6 @@ dependencies = [
|
|||||||
"dirs-sys",
|
"dirs-sys",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "dirs"
|
|
||||||
version = "7.0.0"
|
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
||||||
checksum = "8d57d423b3c82e89b9a24ca3091fee61f456a26edbd28d26c65906f4bc1dcd8f"
|
|
||||||
dependencies = [
|
|
||||||
"dirs-sys",
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dirs-sys"
|
name = "dirs-sys"
|
||||||
version = "0.5.0"
|
version = "0.5.0"
|
||||||
@@ -1608,15 +1588,6 @@ dependencies = [
|
|||||||
"serde_core",
|
"serde_core",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "infer"
|
|
||||||
version = "0.22.0"
|
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
||||||
checksum = "f4200d433cbd5178df7797c9c2e75b348b728e39631cf14520d1e2fc424201f4"
|
|
||||||
dependencies = [
|
|
||||||
"cfb",
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "intervaltree"
|
name = "intervaltree"
|
||||||
version = "0.2.7"
|
version = "0.2.7"
|
||||||
@@ -1634,7 +1605,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "ipx"
|
name = "ipx"
|
||||||
version = "0.3.0"
|
version = "0.5.2"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"ammonia",
|
"ammonia",
|
||||||
"anyhow",
|
"anyhow",
|
||||||
@@ -1643,9 +1614,7 @@ dependencies = [
|
|||||||
"axum",
|
"axum",
|
||||||
"chrono",
|
"chrono",
|
||||||
"clap",
|
"clap",
|
||||||
"dirs",
|
|
||||||
"futures-util",
|
"futures-util",
|
||||||
"infer",
|
|
||||||
"librqbit",
|
"librqbit",
|
||||||
"opml",
|
"opml",
|
||||||
"percent-encoding",
|
"percent-encoding",
|
||||||
@@ -1656,7 +1625,6 @@ dependencies = [
|
|||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
"tokio",
|
"tokio",
|
||||||
"tokio-stream",
|
|
||||||
"toml",
|
"toml",
|
||||||
"tower",
|
"tower",
|
||||||
"tower-http 0.7.1",
|
"tower-http 0.7.1",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "ipx"
|
name = "ipx"
|
||||||
version = "0.3.0"
|
version = "0.5.2"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
@@ -11,9 +11,7 @@ atom_syndication = "0.12.10"
|
|||||||
axum = "0.8.9"
|
axum = "0.8.9"
|
||||||
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
|
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
|
||||||
clap = { version = "4.6.6", features = ["derive"] }
|
clap = { version = "4.6.6", features = ["derive"] }
|
||||||
dirs = "7.0.0"
|
|
||||||
futures-util = { version = "0.3.34", default-features = false, features = ["std"] }
|
futures-util = { version = "0.3.34", default-features = false, features = ["std"] }
|
||||||
infer = "0.22.0"
|
|
||||||
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
|
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
|
||||||
opml = "1.1.6"
|
opml = "1.1.6"
|
||||||
percent-encoding = "2.3.2"
|
percent-encoding = "2.3.2"
|
||||||
@@ -24,7 +22,6 @@ rusqlite = { version = "0.40.2", features = ["bundled"] }
|
|||||||
serde = { version = "1.0.229", features = ["derive"] }
|
serde = { version = "1.0.229", features = ["derive"] }
|
||||||
serde_json = "1.0.151"
|
serde_json = "1.0.151"
|
||||||
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
|
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
|
||||||
tokio-stream = { version = "0.1.19", features = ["sync"] }
|
|
||||||
toml = "1.1.5"
|
toml = "1.1.5"
|
||||||
tower = { version = "0.5.3", features = ["util"] }
|
tower = { version = "0.5.3", features = ["util"] }
|
||||||
tower-http = { version = "0.7.1", features = ["fs"] }
|
tower-http = { version = "0.7.1", features = ["fs"] }
|
||||||
|
|||||||
141
README.md
141
README.md
@@ -1,120 +1,83 @@
|
|||||||
# ipodderx-rs
|
# ipodderx-rs
|
||||||
|
|
||||||
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent), files them
|
A self-hosted podcatcher for a household. It checks your feeds, downloads the episodes, and serves
|
||||||
into per-feed folders, and reaps old files to stay under a disk quota. Runs as a one-shot CLI or as
|
a web UI modelled on the 2004 Mac app **iPodderX**, for any number of people sharing one copy of
|
||||||
a daemon with a web UI, serving any number of people from one copy of the data.
|
the files. One Rust binary, `ipx`, is both the daemon and the command line.
|
||||||
|
|
||||||
## Lineage
|
It is a rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python engine
|
||||||
|
behind iPodderX (2004-2008, Ray Slakinski & August Trometer).
|
||||||
|
|
||||||
A modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python 2
|
## What it does
|
||||||
engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), open-sourced under the MIT
|
|
||||||
License in 2010.
|
|
||||||
|
|
||||||
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
|
- **The web UI.** It has a toolbar, and a feed list that opens with Directory, Popular and All
|
||||||
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
|
Subscriptions. Items sit in a sortable table with a Files pane, and there is a player bar. It
|
||||||
oldest-first disk quota reaper.
|
comes in Dark, Light and Classic themes, and works on a phone.
|
||||||
|
- **Several people, one copy.** Each person has their own subscriptions and their own read, kept
|
||||||
|
and playback state. There is one file on disk per episode, however many people want it. People
|
||||||
|
sign in with a password or through a proxy (Cloudflare Zero Trust or Authentik), and admins
|
||||||
|
manage accounts and settings.
|
||||||
|
- **Scanning.** Feeds are checked on a schedule, globally or per feed, and a feed's own TTL is
|
||||||
|
honoured. Keyword, explicit-content and media-type filters decide what is downloaded, with a cap
|
||||||
|
on new downloads per scan.
|
||||||
|
- **Downloads.** Files come over HTTP or BitTorrent and are filed into a folder per feed.
|
||||||
|
Retention deletes the oldest files to stay under a disk quota or an age limit, and never touches
|
||||||
|
an item someone has kept.
|
||||||
|
- **OPML.** You can import and export your own subscriptions. You can also subscribe to an OPML
|
||||||
|
URL, which keeps a whole list in step as a folder.
|
||||||
|
|
||||||
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
|
## Run it
|
||||||
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
|
|
||||||
3DES-encrypted preferences, and the `printMSG` stdout protocol -- replaced by a JSON-lines socket.
|
|
||||||
|
|
||||||
## Quick start
|
With Docker:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker build -t ipodderx .
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
`docker-compose.yml` is set up for the author's own server. Point its `image` and its three volumes
|
||||||
|
(`/config`, `/data` and `/downloads`) at yours first. The UI is on port 8099. BitTorrent uses 6881
|
||||||
|
over TCP and UDP. Files are written as `PUID`/`PGID`, 99:100 by default.
|
||||||
|
|
||||||
|
From source:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cargo build --release
|
cargo build --release
|
||||||
install -m755 target/release/ipx ~/.cargo/bin/
|
./target/release/ipx daemon
|
||||||
|
|
||||||
ipx add https://atp.fm/rss # subscribe
|
|
||||||
ipx fetch # scan and download
|
|
||||||
ipx daemon # scheduler, control socket and web UI
|
|
||||||
```
|
```
|
||||||
|
|
||||||
On first start with `[web] enabled = true` the daemon mints a token, writes it to config.toml and
|
The first start creates **admin / ipodderx**. Sign in at `/login`, then change it:
|
||||||
prints the URL to open. A database with no accounts starts with **admin / ipodderx** at `/login` --
|
|
||||||
change it with `echo -n '<password>' | ipx user passwd admin`.
|
```sh
|
||||||
|
echo -n 'a good password' | ipx user passwd admin
|
||||||
|
```
|
||||||
|
|
||||||
|
The UI is plain HTTP, so put TLS in front of it if it is reachable from outside your network.
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [docs/configuration.md](docs/configuration.md) | Every config key, paths, environment variables |
|
| [docs/configuration.md](docs/configuration.md) | Every config key, path and environment variable |
|
||||||
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
|
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
|
||||||
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
|
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
|
||||||
| [docs/sso.md](docs/sso.md) | Cloudflare Zero Trust or Authentik in front |
|
| [docs/sso.md](docs/sso.md) | Signing in through Cloudflare Zero Trust or Authentik |
|
||||||
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, socket, HTTP API |
|
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, control socket, HTTP API |
|
||||||
| [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
|
| [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
|
||||||
| [docs/history.md](docs/history.md) | How it was built: the long form, with what was wrong and why |
|
| [docs/history.md](docs/history.md) | How it was built, with what was wrong and why |
|
||||||
| [CLAUDE.md](CLAUDE.md) | Notes for anyone (or anything) working on the code |
|
| [CLAUDE.md](CLAUDE.md) | Notes for working on the code, including how production is deployed |
|
||||||
|
|
||||||
## The web UI
|
|
||||||
|
|
||||||
`ipx daemon` serves it in the same process, so it reads SQLite and the event bus directly.
|
|
||||||
|
|
||||||
Feeds down the side; the selected feed's items across the top; the selected item's text and its
|
|
||||||
enclosures below, which is where you play, download or delete them. The divider drags and its
|
|
||||||
position is remembered. Playback serves Range requests, so seeking works. An OPML subscription is a
|
|
||||||
collapsible folder whose page lists the feeds inside it.
|
|
||||||
|
|
||||||
An item may carry several enclosures; all of them appear below, and anything that is not audio or
|
|
||||||
video gets a View link rather than a player -- the publisher's copy until it is downloaded, the
|
|
||||||
local one after. Opening an item marks it read. Show notes are untrusted feed HTML, sanitized with
|
|
||||||
`ammonia` server-side before they reach the page.
|
|
||||||
|
|
||||||
The **Log** button shows the running daemon live in four tabs: *Daemon I/O* is the control protocol
|
|
||||||
itself, every command in and event out; *Scans* is feed and download activity; *HTTP* is web
|
|
||||||
requests; *All* is everything, with level and text filters and a copy button. It reads a ring buffer
|
|
||||||
held in the process, not a file, so it works the same under Docker.
|
|
||||||
|
|
||||||
It is plain HTTP. On a LAN bind the token and everything else cross the network in the clear, and a
|
|
||||||
feed URL can itself carry a credential. Put TLS in front of it if that matters.
|
|
||||||
|
|
||||||
## OPML
|
|
||||||
|
|
||||||
**Importing and exporting** a file copies subscriptions in or out once: `ipx import subs.opml`,
|
|
||||||
`ipx export subs.opml`, or Settings → Subscriptions in the UI.
|
|
||||||
|
|
||||||
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
|
|
||||||
other feed; every scan re-reads it and keeps your list in step. The feeds inside are not written to
|
|
||||||
config.toml -- the OPML is the source of truth, so they are re-derived each scan and held in the
|
|
||||||
database. They show as a folder, download into one nested folder, and inherit the subscription's
|
|
||||||
settings until you change one, which gives it its own entry.
|
|
||||||
|
|
||||||
When a feed drops out of the OPML upstream, it is unsubscribed and removed -- unless it has
|
|
||||||
downloads, in which case it is kept and flagged in the UI as no longer listed. A downloaded file is
|
|
||||||
never left behind with nothing explaining where it came from.
|
|
||||||
|
|
||||||
## Docker
|
|
||||||
|
|
||||||
```sh
|
|
||||||
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
|
|
||||||
docker compose pull ipodderx && docker compose up -d ipodderx
|
|
||||||
docker compose logs -f ipodderx # the first start prints the default admin password
|
|
||||||
```
|
|
||||||
|
|
||||||
`docker-compose.yml` runs the image from the registry above rather than building it, so build and
|
|
||||||
push first; change the tag in both places to use another registry. It mounts `/config` (config.toml),
|
|
||||||
`/data` (state.db) and `/downloads` from this install's host paths, which you will want to change for
|
|
||||||
yours. It publishes 8099 for the
|
|
||||||
UI and 6881 (TCP **and** UDP -- DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
|
|
||||||
land owned the way Unraid shares expect. The healthcheck runs `ipx status` through the control
|
|
||||||
socket, so it catches a daemon that is alive but wedged rather than merely one that has died.
|
|
||||||
|
|
||||||
## Running it as a service
|
|
||||||
|
|
||||||
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
|
|
||||||
rather run periodic scans with no daemon -- in which case there is no socket for a UI to attach to.
|
|
||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
|
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
|
||||||
node tests/page-smoke.js # the page script loads without throwing
|
node tests/page-smoke.js # the page script loads without throwing
|
||||||
npx playwright test # a real browser against a real daemon
|
npx playwright test # a real browser against a real daemon on fixture feeds
|
||||||
```
|
```
|
||||||
|
|
||||||
`npm install` gets the test runner; the browser comes from
|
`npm install` gets the test runner, and `npx playwright install --with-deps chromium` gets the
|
||||||
`npx playwright install --with-deps chromium` (in `install.sh`).
|
browser.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
MIT. See [LICENSE](LICENSE).
|
MIT, see [LICENSE](LICENSE). The icons are [Font Awesome Free](https://fontawesome.com) 7.3.1 by
|
||||||
|
@fontawesome, under [CC BY 4.0](https://fontawesome.com/license/free), embedded as SVG.
|
||||||
|
|||||||
27
TODO.md
Normal file
27
TODO.md
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
# To do
|
||||||
|
|
||||||
|
## Trim the state database
|
||||||
|
|
||||||
|
From an audit of the database layer and a read-only copy of production on 2026-09-12. The data
|
||||||
|
itself was clean: no leftover tables or indexes, 47 free pages, one stray `entry_state` row.
|
||||||
|
Check each against the code before cutting it.
|
||||||
|
|
||||||
|
- [x] **`migrate()`'s add list.** All eight columns arrived in 0.2.0, and 0.5.0 only supports
|
||||||
|
upgrades from 0.3.0 on. Drop the list and its loop; keep the `retired` drop loop, which a
|
||||||
|
database coming from 0.4.0 still needs. (`src/db.rs`)
|
||||||
|
- [x] **`Db::subscribed_feed_ids`.** No callers; its doc says the scanner walks it, and it does not.
|
||||||
|
(`src/db.rs`)
|
||||||
|
- [x] **`Db::subscriber_count`.** One caller, which only asks whether it is above zero:
|
||||||
|
`subscriber_counts()?.contains_key(&id)`. (`src/db.rs`, `src/web.rs`)
|
||||||
|
- [x] **The `created` columns** on `users`, `subscriptions` and `sessions`. Written on every insert,
|
||||||
|
never read. Add them to `retired` and drop them from the inserts. (`src/db.rs`)
|
||||||
|
- [x] **`Managed.orphaned`.** Selected by `managed_feeds()` on every call and never read;
|
||||||
|
`FeedSummary.orphaned` is what the UI uses. (`src/db.rs`)
|
||||||
|
|
||||||
|
## Popular
|
||||||
|
|
||||||
|
- [x] **Popular lists feeds the way Directory does**: the feeds inside an OPML or a Patreon
|
||||||
|
creator, never the collection itself. (`src/web.rs`, `web/index.html`)
|
||||||
|
|
||||||
|
After these: `cargo test`, `node tests/page-smoke.js`, `npx playwright test`. Copy `state.db`
|
||||||
|
aside before deploying: `migrate()` drops columns on the first start.
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=ipx feed scan (one shot)
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=oneshot
|
|
||||||
ExecStart=%h/.cargo/bin/ipx fetch
|
|
||||||
Environment=IPX_LOG=ipx=info
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
# User unit: install to ~/.config/systemd/user/ipx.service, then
|
|
||||||
# systemctl --user enable --now ipx
|
|
||||||
# The socket lands in $XDG_RUNTIME_DIR/ipx.sock by default, so a UI running as the
|
|
||||||
# same user can attach without extra configuration.
|
|
||||||
[Unit]
|
|
||||||
Description=ipx podcatcher
|
|
||||||
After=network-online.target
|
|
||||||
Wants=network-online.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=simple
|
|
||||||
ExecStart=%h/.cargo/bin/ipx daemon
|
|
||||||
Restart=on-failure
|
|
||||||
RestartSec=30
|
|
||||||
Environment=IPX_LOG=ipx=info
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=default.target
|
|
||||||
@@ -1,16 +0,0 @@
|
|||||||
# Alternative to the daemon: a periodic one-shot scan, closer to how the original
|
|
||||||
# iPodderX agent was driven. Use this OR ipx.service, not both -- with no daemon
|
|
||||||
# running there is no socket, so a UI cannot attach.
|
|
||||||
#
|
|
||||||
# Install ipx-scan.service and ipx.timer to ~/.config/systemd/user/, then
|
|
||||||
# systemctl --user enable --now ipx.timer
|
|
||||||
[Unit]
|
|
||||||
Description=Periodic ipx feed scan
|
|
||||||
|
|
||||||
[Timer]
|
|
||||||
OnBootSec=5min
|
|
||||||
OnUnitActiveSec=1h
|
|
||||||
Persistent=true
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=timers.target
|
|
||||||
@@ -28,12 +28,15 @@ The page is compiled in, so **editing `web/index.html` needs a rebuild**.
|
|||||||
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
|
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
|
||||||
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
|
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
|
||||||
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
|
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
|
||||||
re-derived into the database each scan, never written to config.toml.
|
re-derived into the database each scan, never written to config.toml. A Patreon creator link
|
||||||
|
(a token, no `show=`) with more than one show is treated the same way, before any fetch: its
|
||||||
|
shows come from Patreon's web API and each becomes a derived feed.
|
||||||
4. Record entries. A changed title or description flips the item back to unread.
|
4. Record entries. A changed title or description flips the item back to unread.
|
||||||
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
|
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
|
||||||
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
|
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
|
||||||
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
|
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
|
||||||
a reason.
|
a reason. What a filter skipped is judged again every scan, so a change of settings brings it
|
||||||
|
back. A feed in a group takes your settings on the group for anything you have not set on it.
|
||||||
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
|
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
|
||||||
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
|
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
|
||||||
or an error page — and is deleted.
|
or an error page — and is deleted.
|
||||||
@@ -47,21 +50,23 @@ entries feed_id, guid, title, link, published, description, first_seen,
|
|||||||
image, duration, episode, season PK (feed_id, guid)
|
image, duration, episode, season PK (feed_id, guid)
|
||||||
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
|
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
|
||||||
bytes_done, downloaded_at, last_error
|
bytes_done, downloaded_at, last_error
|
||||||
users id, name, pass_hash, is_admin, created
|
users id, name, pass_hash, is_admin, created, last_login
|
||||||
sessions token, user_id, created, seen
|
sessions token, user_id, seen
|
||||||
subscriptions user_id, feed_id, keywords, auto_download, allow_explicit,
|
subscriptions user_id, feed_id, keywords, auto_download, allow_explicit,
|
||||||
max_new_per_check, created PK (user_id, feed_id)
|
max_new_per_check PK (user_id, feed_id)
|
||||||
entry_state user_id, feed_id, guid, read, flagged, position
|
entry_state user_id, feed_id, guid, read, flagged, position
|
||||||
PK (user_id, feed_id, guid)
|
PK (user_id, feed_id, guid)
|
||||||
```
|
```
|
||||||
|
|
||||||
`entries` still has `read`, `flagged` and `position` columns from before accounts existed. They are
|
Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position` columns from
|
||||||
**dead** — the migration copied them into `entry_state` and nothing reads them now. Anything found
|
before accounts; two bugs came from queries still reading them, and `migrate()` drops them from an
|
||||||
querying them is a bug; two were.
|
older database.
|
||||||
|
|
||||||
Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in
|
Schema changes: add the table or column to `SCHEMA`. `CREATE TABLE IF NOT EXISTS` leaves a table
|
||||||
`migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the
|
that already exists alone, so a new column on one also goes in `migrate()`'s `wanted` list, and a
|
||||||
same path as `Db::open`, so a migration-only column cannot pass tests while missing in production.
|
retired one in its `retired` list; both are checked with `PRAGMA table_info`. Columns from before
|
||||||
|
0.3.0, the oldest version an upgrade may start from, need no entry. `Db::memory()` runs the same
|
||||||
|
path as `Db::open`, so a migration cannot pass the tests while missing in production.
|
||||||
|
|
||||||
## Control socket
|
## Control socket
|
||||||
|
|
||||||
@@ -82,7 +87,9 @@ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/
|
|||||||
**Events** — `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
**Events** — `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
||||||
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
||||||
`scan_done`, `reap_done` and `status` are terminal: a client that asked for work stops reading
|
`scan_done`, `reap_done` and `status` are terminal: a client that asked for work stops reading
|
||||||
there.
|
there. Commands run one at a time, in the order they arrive, except `status`: the socket answers it
|
||||||
|
straight away, so the Docker healthcheck is never left waiting behind a scan or a download, and
|
||||||
|
answers only the client that asked, since `status` would end any other client's session.
|
||||||
|
|
||||||
Progress carries the enclosure id, without which a UI cannot tell one download from another and
|
Progress carries the enclosure id, without which a UI cannot tell one download from another and
|
||||||
ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast,
|
ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast,
|
||||||
@@ -104,13 +111,15 @@ else a `401`.
|
|||||||
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
|
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
|
||||||
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
|
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
|
||||||
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
|
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
|
||||||
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable |
|
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable, sortable (`sort` = kept, title, feed, type, size or published; `dir` = asc or desc) |
|
||||||
|
| `GET /api/entries` | the same, across every feed you subscribe to (All Subscriptions) |
|
||||||
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
|
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
|
||||||
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, starred, position |
|
| `POST /api/read-all` | everything read in every feed you subscribe to (All Subscriptions) |
|
||||||
|
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, kept, position |
|
||||||
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
|
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
|
||||||
| `POST /api/fetch` | |
|
| `POST /api/fetch` | |
|
||||||
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
|
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
|
||||||
| `GET /api/popular`, `POST /api/popular/{id}` | what others here subscribe to (id, title, art, count; never a URL, never a private feed); subscribe by id |
|
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with an OPML's feeds in place of the OPML and everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
|
||||||
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
|
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
|
||||||
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
|
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
|
||||||
| `GET /api/events` | SSE, the same broadcast the socket carries |
|
| `GET /api/events` | SSE, the same broadcast the socket carries |
|
||||||
|
|||||||
@@ -60,7 +60,7 @@ ipx reap # actually delete
|
|||||||
```
|
```
|
||||||
|
|
||||||
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
|
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
|
||||||
with no file are pruned from the database. **Starred by anyone keeps a file**, and one only counts
|
with no file are pruned from the database. **An item anyone kept keeps its file**, and one only counts
|
||||||
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
|
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
|
||||||
stops the next scan fetching it again.
|
stops the next scan fetching it again.
|
||||||
|
|
||||||
|
|||||||
@@ -32,10 +32,10 @@ media_types = ["audio", "video"]
|
|||||||
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
|
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
|
||||||
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
|
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
|
||||||
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
|
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
|
||||||
pad. Starred items are never deleted, and a file only counts as read once every subscriber has
|
pad. Kept items are never deleted, and a file only counts as read once every subscriber has
|
||||||
read it. `0` disables it entirely.
|
read it. `0` disables it entirely.
|
||||||
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
|
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
|
||||||
Starred ones stay. `0` disables it.
|
Kept ones stay. `0` disables it.
|
||||||
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
|
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
|
||||||
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
|
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
|
||||||
fetched 216 files and 22 GB in one scan.
|
fetched 216 files and 22 GB in one scan.
|
||||||
@@ -43,8 +43,6 @@ media_types = ["audio", "video"]
|
|||||||
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
|
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
|
||||||
without this the disk fills with artwork. Empty takes everything.
|
without this the disk fills with artwork. Empty takes everything.
|
||||||
|
|
||||||
`interval_mins` from older configs is still read, and `schedule` supersedes it.
|
|
||||||
|
|
||||||
## `[torrent]`
|
## `[torrent]`
|
||||||
|
|
||||||
```toml
|
```toml
|
||||||
@@ -69,6 +67,7 @@ token = "" # generated and saved on first run
|
|||||||
trusted_header = "" # e.g. "Cf-Access-Authenticated-User-Email"
|
trusted_header = "" # e.g. "Cf-Access-Authenticated-User-Email"
|
||||||
trusted_proxies = ["127.0.0.1", "::1"]
|
trusted_proxies = ["127.0.0.1", "::1"]
|
||||||
auto_create_users = true
|
auto_create_users = true
|
||||||
|
sign_out_url = "" # e.g. "/cdn-cgi/access/logout"
|
||||||
session_days = 30
|
session_days = 30
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -79,6 +78,9 @@ session_days = 30
|
|||||||
* **`trusted_proxies`** — addresses allowed to assert that header, and the entire security boundary
|
* **`trusted_proxies`** — addresses allowed to assert that header, and the entire security boundary
|
||||||
for it. Name the proxy, never a subnet.
|
for it. Name the proxy, never a subnet.
|
||||||
* **`auto_create_users`** — create an account the first time the proxy vouches for a new name.
|
* **`auto_create_users`** — create an account the first time the proxy vouches for a new name.
|
||||||
|
* **`sign_out_url`** — where Sign out sends someone the proxy signed in: the proxy's own sign-out,
|
||||||
|
`/cdn-cgi/access/logout` behind Cloudflare Access. Empty sends them to the sign-in page, where
|
||||||
|
the proxy signs them straight back in.
|
||||||
* **`session_days`** — sign a session out after this long without a request.
|
* **`session_days`** — sign a session out after this long without a request.
|
||||||
|
|
||||||
It is plain HTTP. On a LAN bind everything crosses the network in the clear — and a feed URL can
|
It is plain HTTP. On a LAN bind everything crosses the network in the clear — and a feed URL can
|
||||||
|
|||||||
317
docs/history.md
317
docs/history.md
@@ -6,6 +6,316 @@ reasoning lives. New write-ups go at the top.
|
|||||||
|
|
||||||
See [README.md](../README.md) for what the thing is.
|
See [README.md](../README.md) for what the thing is.
|
||||||
|
|
||||||
|
## 2026-09-12 — Healthy while busy
|
||||||
|
|
||||||
|
After a deploy the container sat at "starting" for a minute, and Docker's health log showed two
|
||||||
|
`ipx status` probes exceeding their 5-second timeout. The daemon's own log explained it. The first
|
||||||
|
scan after the start fetched 23 feeds, from 14:10:41 to 14:11:35, and both probes' `status`
|
||||||
|
commands waited in the job queue behind it; they were answered together at 14:11:35, straight after
|
||||||
|
`scan_done`. The worker runs one job at a time and `status` was one of its jobs, so any scan or
|
||||||
|
download longer than about a minute and a half, three 30-second probes, would have had Docker call
|
||||||
|
a working daemon unhealthy.
|
||||||
|
|
||||||
|
The socket now answers `status` itself, from two short queries, and only real work goes through the
|
||||||
|
queue. The trade is that healthy now means the daemon answers on its socket and can read its
|
||||||
|
database; a worker stuck on one job would still pass. Asking a daemon that downloads hour-long
|
||||||
|
podcasts to be idle within five seconds was never a fair test of whether it was alive. A test holds
|
||||||
|
the queue full and checks `status` still comes back.
|
||||||
|
|
||||||
|
The first version broadcast the answer, as the queued one had been. Timing `status` during a forced
|
||||||
|
scan in production showed the catch: `status` is a terminal event, so the `ipx fetch` watching that
|
||||||
|
scan stopped reading at the first probe and printed the status line as its last, while the scan
|
||||||
|
carried on. When `status` waited behind the scan it could never arrive first, so this had never
|
||||||
|
shown. The answer now goes only to the client that asked, and the test checks that another client
|
||||||
|
hears nothing.
|
||||||
|
|
||||||
|
## 2026-09-12 — Signing in through Authentik, for real
|
||||||
|
|
||||||
|
Ray could not get Authentik's sign-in to reach ipx, following `docs/sso.md`, which had been written
|
||||||
|
without ever being tried. Looking at the Cloudflare account through its API showed that side was
|
||||||
|
already complete. Authentik is Zero Trust's OpenID Connect identity provider; the Access application
|
||||||
|
`ipodderx` allows only it and a list of five addresses; the tunnel `rays-unraid` routes
|
||||||
|
`ipodderx.sdf1.net` to `192.168.1.130:8099`; DNS is a proxied CNAME to the tunnel. Access's log
|
||||||
|
showed `rays@sdf1.net` signing in through it. Nothing on Cloudflare was changed, so no other site
|
||||||
|
was touched.
|
||||||
|
|
||||||
|
The gaps were all at ipx's end: `trusted_header` was empty, `trusted_proxies` held only loopback,
|
||||||
|
and the account was called `rays` while the header carries `rays@sdf1.net`.
|
||||||
|
|
||||||
|
Finding the address to trust took the most time. The page said `127.0.0.1`, but `cloudflared` runs in
|
||||||
|
its own container and reaches ipx through the host's published port. ipx logs no peer addresses, so
|
||||||
|
the address was read from `/proc/net/tcp` inside the ipx container: `192.168.16.1`, the gateway of
|
||||||
|
`content_default`, where Docker's masquerade puts traffic crossing from another bridge. A request
|
||||||
|
from Tower's own shell arrived as `192.168.1.130` instead, and a throwaway `busybox` on the default
|
||||||
|
bridge as `192.168.16.1`: the first was refused with the header, the second believed. LAN machines
|
||||||
|
keep their own addresses, since Docker forwards published ports with iptables (the userland proxy
|
||||||
|
only handles loopback).
|
||||||
|
|
||||||
|
Every change, in order, with how to undo it:
|
||||||
|
|
||||||
|
1. **Code**, commit `586d2c0`: `ipx user rename`, deployed. Revert the commit and redeploy to
|
||||||
|
remove it; nothing depends on it once used.
|
||||||
|
2. **Account**: `docker exec iPodderX ipx user rename rays rays@sdf1.net`. Same id, so its feeds,
|
||||||
|
read state, password and admin rights stayed. Undo: `docker exec iPodderX ipx user rename
|
||||||
|
rays@sdf1.net rays`. Signing in at `/login` now takes the new name.
|
||||||
|
3. **Config**, `/mnt/fast/appdata/ipodderx/config.toml`, `[web]`: `trusted_header` from `""` to
|
||||||
|
`"Cf-Access-Authenticated-User-Email"`, and `"192.168.16.1"` added to `trusted_proxies`. The
|
||||||
|
file as it was is `config.toml.2026-09-12-sso.bak` beside it. Undo: copy the backup back and
|
||||||
|
`docker compose -f /mnt/fast/arcane/projects/content/compose.yaml restart ipodderx`.
|
||||||
|
4. **Cloudflare, Docker networks and other containers**: unchanged. The `busybox` test container
|
||||||
|
was removed when it exited, and its image afterwards.
|
||||||
|
5. **Authentik**, later the same day, because ipodderx had no tile in its library while Outline
|
||||||
|
did: a bookmark application `ipodderx` (pk `5854a98e-816a-4c4f-9f27-63e69dc29d1d`), made
|
||||||
|
through the API with a token of Ray's. No provider and no policy bindings, like Outline's, the
|
||||||
|
iPodderX icon, and a link to `https://ipodderx.sdf1.net`. It changes nothing about who can sign
|
||||||
|
in. Undo: delete it under Applications → Applications, or
|
||||||
|
`DELETE /api/v3/core/applications/ipodderx/`.
|
||||||
|
6. **Signing out**, later again. Sign out landed on ipx's password page while Access still vouched
|
||||||
|
for Ray, so it signed nothing out, and the page looked like the wrong login. Cloudflare's
|
||||||
|
`/cdn-cgi/access/logout` ends the Access session for every Access application at once (there is
|
||||||
|
no per-application sign-out, and it takes no redirect), and Authentik's end-session only ends
|
||||||
|
one application's session unless single logout is set up there. Ray chose Access's sign-out. New
|
||||||
|
`[web] sign_out_url`, set to `/cdn-cgi/access/logout` in production (the file as it was is
|
||||||
|
`config.toml.2026-09-12-signout.bak`), and `/login` now sends anyone the proxy vouches for on to
|
||||||
|
`/`. Undo: take the key out and restart; the code does nothing without it.
|
||||||
|
|
||||||
|
What the address trusts is any container on Tower that connects through the host's port, not only
|
||||||
|
`cloudflared`. Verifying Cloudflare's signed `Cf-Access-Jwt-Assertion` would remove that, and is
|
||||||
|
the upgrade if it matters.
|
||||||
|
|
||||||
|
## 2026-09-12 — Trimming the state database
|
||||||
|
|
||||||
|
An audit of the database layer, with a read-only copy of production to check it against. The
|
||||||
|
data was already clean: no tables or indexes left from older versions, 47 free pages after the
|
||||||
|
column drops earlier the same day, and one stray `entry_state` row. The code had five things:
|
||||||
|
|
||||||
|
- `migrate()` still added eight columns to any table missing them. All eight shipped in 0.2.0 and
|
||||||
|
upgrades now start from 0.3.0 at the oldest, so the list and its loop went; the `retired` drop
|
||||||
|
list stays, since a database coming from 0.4.0 still has the old read columns.
|
||||||
|
- `created` on `users`, `subscriptions` and `sessions` was written by every insert and read by
|
||||||
|
nothing. They joined `retired`. The old-database test now builds all three tables, foreign keys
|
||||||
|
included, since `DROP COLUMN` on a table that references another was the part worth proving.
|
||||||
|
- `Db::subscribed_feed_ids` had no callers, though its doc said the scanner walked it.
|
||||||
|
`Db::subscriber_count` had one caller asking whether it was above zero, which
|
||||||
|
`subscriber_counts().contains_key` answers. `Managed.orphaned` was selected and never read.
|
||||||
|
- `users.created` came back the same afternoon, with `last_login` beside it. Nothing read it, but
|
||||||
|
when an account was made and when it last signed in is what you want to know when tidying
|
||||||
|
accounts, and it cannot be recovered later. Both existing accounts got their creation times back
|
||||||
|
from the backup taken before the drop, and a last sign-in from their newest session in it.
|
||||||
|
`last_login` is kept to the hour, because the proxy vouches for every request and that would
|
||||||
|
otherwise be a write each time.
|
||||||
|
|
||||||
|
## 2026-09-12 — Cutting what had outlived its reason
|
||||||
|
|
||||||
|
A whole-repo audit for over-engineering listed twelve things to cut, and all of them went.
|
||||||
|
|
||||||
|
- **Upgrades from before accounts.** `migrate_opml_children` moved OPML feeds that old versions
|
||||||
|
wrote into `config.toml` out to the database, and ran at every daemon start to do nothing after
|
||||||
|
the first. Production ran it in 0.3.0; anything older has to pass through 0.4.0.
|
||||||
|
- **Half of the adoption, and not the other half.** The audit called `adopt_existing_library` a
|
||||||
|
one-time migration and it was cut whole. It did two jobs: copy the old read state into
|
||||||
|
`entry_state`, which was dead, and subscribe the first admin to the whole catalogue while nobody
|
||||||
|
subscribed to anything, which is how a fresh install's first account gets `config.toml`'s feeds.
|
||||||
|
The browser suite caught it at once, signing in to an empty sidebar; `cargo test` had no idea.
|
||||||
|
The second job is back as `adopt_catalogue`, with a unit test of its own.
|
||||||
|
- **The dead `entries` columns.** `read`, `flagged` and `position` moved to `entry_state` with
|
||||||
|
accounts. The adoption's copy was their last reader, but `record_entry` still wrote them, and
|
||||||
|
still reset `read` when a title changed, which nothing looked at. Two bugs came from queries
|
||||||
|
reading them. `migrate()` now drops them from an existing database (SQLite has had `DROP COLUMN`
|
||||||
|
since 3.35), and a test builds an old table to prove it. On production each drop rewrote the
|
||||||
|
66 MB `entries` table, about four seconds apiece, so the first start took thirteen. An
|
||||||
|
`ipx status` run in that window failed with `migrating schema`: every `ipx` command migrates when
|
||||||
|
it opens the database, and it collided with the daemon doing the same. A failed `ALTER TABLE`
|
||||||
|
changes nothing, and the database had been copied to `backup/` first anyway.
|
||||||
|
- **`interval_mins`**, which `schedule` replaced. An old config that still has the key loads; the
|
||||||
|
key is ignored, and the config test carries it to keep that true.
|
||||||
|
- **Three dependencies.** `infer` was only asked whether a file is a torrent, and the check after
|
||||||
|
it already looked for `d8:announce`, which is what `infer` looks for. `dirs` was three lookups of
|
||||||
|
`XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME`. `tokio-stream` wrapped the broadcast receiver for
|
||||||
|
the event stream; `futures_util::stream::unfold` does the same, lagging clients included.
|
||||||
|
- **Two token generators.** The web token came from a copy of the session-token code, with a
|
||||||
|
clock fallback on top. It uses `auth::new_session_token` now, and is 64 characters.
|
||||||
|
- **The icon inlined four times**, 23 KB of base64 each, into both pages. It is `/icon.png` now,
|
||||||
|
outside the sign-in wall with `/login`, since the sign-in page shows it.
|
||||||
|
- Also: the `contrib/` systemd units from before the container, `Db::entries` and
|
||||||
|
`Db::count_entries` that only the tests called, three `logbuf` visitors that repeated the trait's
|
||||||
|
defaults, and unused state, a helper and dead CSS in the page.
|
||||||
|
|
||||||
|
## 2026-09-11 — A design pass on the web UI
|
||||||
|
|
||||||
|
A review against screenshots of every view in all three themes found that Dark and Light read as a
|
||||||
|
generic dark dashboard: one pale blue did every job, most labels were bold, and nothing led. The
|
||||||
|
list it produced, in `TODO.md`, was worked through in one go. What was worth knowing:
|
||||||
|
|
||||||
|
- **Amber means new.** Badges, unread dots and download bars take the icon's EQ amber; blue is left
|
||||||
|
for the primary action and links. Light's amber was `#b06f10`, which gives white text 4.1:1,
|
||||||
|
short of AA for 11 px bold. It is `#9a5f0a` now, 5.2:1.
|
||||||
|
- **EQ bars mark what is playing.** Three `<i>` bars stand at 60, 100 and 40 % and animate only
|
||||||
|
while `body.playing` is set. The first version left the animation on but paused, expecting each
|
||||||
|
bar to hold a different frame. The frames it held were within a pixel of each other, and on
|
||||||
|
screen the bars read as three dots. Under reduced motion one rule drops every animation and
|
||||||
|
transition, which leaves the bars standing.
|
||||||
|
- **The focus ring was clipped.** `.tgroup` and `#topbar` both set `overflow:hidden`, so a ring
|
||||||
|
drawn outside a toolbar button was cut off. Rings inside clipping parents are inset instead.
|
||||||
|
- **The feed list could not be used from the keyboard at all.** Rows were `<div>`s and the triangle
|
||||||
|
a `<span>`, so Tab went from the feed filter to Sign out. Rows now take focus, the triangle is a
|
||||||
|
`<button aria-expanded>`, and `renderFeeds` puts focus back on the same row after redrawing,
|
||||||
|
since every live update replaces every row. The list's own key handler stops Space and the
|
||||||
|
arrows from reaching the player's shortcuts on the document.
|
||||||
|
- **The triangle hangs in the margin.** Every row used to reserve an 18 px slot for it, pushing a
|
||||||
|
hundred feeds 28 px right of the places above for the sake of two folders. It is now absolutely
|
||||||
|
placed in the row's left padding, the full height of the row, so a near miss no longer opens
|
||||||
|
the folder's page.
|
||||||
|
- **A selected tile vanished** because the initials tile and the selected row were both `--raise`.
|
||||||
|
Tiles now mix their tint into `--bg`, which no row uses.
|
||||||
|
- **The Files pane hides itself** with `#split:has(>#files[hidden])`, which collapses its column.
|
||||||
|
The phone layout already hides the pane, and a zero-width extra track there is harmless.
|
||||||
|
- **Flagged became Kept** in the tab, and in the server's refusal to delete a file someone else
|
||||||
|
kept. The filter value and the column stay `flagged`; renaming those buys nothing.
|
||||||
|
|
||||||
|
## 2026-09-11 — A Patreon creator is a list of shows
|
||||||
|
|
||||||
|
Ray asked whether ipx could sync with Patreon. Not in full. The documented API (v2, the
|
||||||
|
`identity.memberships` scope) lists the creators you back and whether each has a feed (`has_rss`),
|
||||||
|
but no resource carries the `auth` token that makes a feed URL work. That token only comes from the
|
||||||
|
creator's page. It is also one per membership, not one per account: techpod's differs from Glass
|
||||||
|
Cannon's, so no single token finds everything you back.
|
||||||
|
|
||||||
|
What does work is one creator at a time, which is what Ray wanted for Glass Cannon and its 33 shows:
|
||||||
|
|
||||||
|
- `patreon.com/rss?auth=<token>`, with no creator named, returns that token's creator. Its self link,
|
||||||
|
about 660 bytes in, gives the campaign by number (`/rss/369921`). Patreon ignores `Range` here, so
|
||||||
|
ipx reads the stream until the number appears and hangs up, instead of taking all 2.8 MB.
|
||||||
|
- A show's `show=` number is a Patreon collection. Asked anonymously, the collection listing
|
||||||
|
(`/api/collection?filter[campaign_id]=`) and a post's `collections` both hide patron-only ones: you
|
||||||
|
get "FAQ". `/api/campaigns/<id>?include=shows` lists every show, anonymously, in one response.
|
||||||
|
- Every spelling works: `rss/glasscannon?auth=…&show=N`, `rss/369921?…` and `rss?auth=…&show=N` all
|
||||||
|
serve the same 131 items. Enclosure URLs are the same in the creator feed and the show feed, and
|
||||||
|
stable between fetches.
|
||||||
|
|
||||||
|
That last point shaped the design. `enclosures.url` is unique, so whichever feed is scanned first owns
|
||||||
|
the file. The first cut only asked a creator for its shows while it had no entries of its own, so that a
|
||||||
|
creator already read as a plain feed, holding every show's episodes, would never be split into shows
|
||||||
|
that came up empty. Within the hour that was the wrong call: Glass Cannon had gone into production on
|
||||||
|
the build before this one, been read as one feed of 2,385 items, and the rule kept it that way. Finding
|
||||||
|
anything in that heap was the problem Ray wanted solved.
|
||||||
|
|
||||||
|
So a creator with more than one show is always a group, run through the same sync as an OPML
|
||||||
|
(`sync_group`, split out of `sync_opml`). When it becomes one, its items are cleared and each show
|
||||||
|
takes over the enclosures the creator holds as the show lists them (`Db::adopt`), downloaded files and
|
||||||
|
everyone's read state included. One show leaves it a plain feed, which is what techpod already was. If
|
||||||
|
the shows cannot be listed, a creator already split fails the scan rather than being read as one heap;
|
||||||
|
one that never was is read as one feed until they can be. An answer without a `shows` list is an error,
|
||||||
|
not "no shows".
|
||||||
|
|
||||||
|
**Filter verdicts follow the settings.** Ray also reported that turning on Allow explicit and
|
||||||
|
rescanning brought nothing back. An item was judged once, when first seen, and `skipped` was final. The
|
||||||
|
2026-09-10 entry below saw it coming ("worth a `ipx retry <feed>` command if this bites"). It bit:
|
||||||
|
2,166 Glass Cannon items and all 88 of Shadowdark's sat at `skipped: explicit` with the setting on.
|
||||||
|
Every scan now runs the filters again over what they skipped (not over `torrents disabled`, which is not
|
||||||
|
a filter's call) and requeues what they now let through. Only that direction: a queued item is never
|
||||||
|
pulled back, because Download latest and a manual download both work by queueing.
|
||||||
|
|
||||||
|
**Two gaps beside it.** Add feed had no explicit box, so every new feed's first scan skipped all its
|
||||||
|
explicit items; it has one now, stored on your subscription like the feed dialog's. And a feed inside a
|
||||||
|
group ignored your settings on the group, though the group's dialog said they were inherited: settings
|
||||||
|
live on each person's subscription, and nothing read the group's. `Db::subscribers` now fills what you
|
||||||
|
have not set on the feed from your subscription to the group, and the feed list shows the same.
|
||||||
|
|
||||||
|
**A name that was already used.** Replaying the split on a copy of the production database left one
|
||||||
|
show with a Supercast episode in it. "Glass Cannon Live! Ascension | Pathfinder 2E" slugs to
|
||||||
|
`glass-cannon-live-ascension-pathfinder-2`, the id of a Supercast feed of the same show that had been
|
||||||
|
removed. Removing a feed keeps its rows on purpose, so that re-adding it does not fetch the back
|
||||||
|
catalogue again, but choosing a new id only checked config.toml and derived feeds. The Patreon show took
|
||||||
|
the old id and everything still filed under it. An id is now also taken when the database has a feed by
|
||||||
|
that id at a different URL; the same URL may still have it back, which is the re-add case.
|
||||||
|
|
||||||
|
Shows already added by hand are matched by token and show number, not by exact URL (`same_feed`), so a
|
||||||
|
bare token does not add Get in the Trunk and Shadowdark a second time under another spelling.
|
||||||
|
|
||||||
|
The show listing is Patreon's own undocumented web API. If it changes, only finding new shows stops.
|
||||||
|
|
||||||
|
## 2026-09-11 — One meaning per icon, sortable columns, and one player
|
||||||
|
|
||||||
|
Ray asked for a pass over the whole UI: consistent icons, and buttons placed next to what they act
|
||||||
|
on. The pass started from screenshots of every view and dialog, taken against the browser suite's
|
||||||
|
own daemon and fixtures. Several things did not agree:
|
||||||
|
|
||||||
|
- Unsubscribe was a minus in the toolbar but an x in the feed header. There it read as "close this
|
||||||
|
page", and x also meant Cancel in every dialog.
|
||||||
|
- Half the dialogs still had word buttons: Log, Add feed, Users, Unsubscribe, OPML, Popular's
|
||||||
|
Subscribe, the Copy buttons and Sign out.
|
||||||
|
- The player bar and the folder arrow were font characters, and so was the row's "playing" marker.
|
||||||
|
Those came out thin and varied from font to font.
|
||||||
|
- The OPML subscription page laid out its header differently from a feed's, and All Subscriptions
|
||||||
|
had no buttons at all.
|
||||||
|
|
||||||
|
The rule now is one meaning per icon. Minus unsubscribes, x closes or cancels, plus adds, subscribes
|
||||||
|
or imports, and a dialog's confirm button carries the icon of what it does. The words go in the
|
||||||
|
tooltip. The toolbar is grouped by what it acts on: add, unsubscribe and scan act on feeds; play,
|
||||||
|
read and keep act on the selected item. Its read and keep buttons now show that item's state, as
|
||||||
|
the item's own buttons already did. A browser test opens all eight dialogs and fails on any button
|
||||||
|
without an icon and a tooltip. It also checks that the toolbar and the page use the same icon for
|
||||||
|
the same action.
|
||||||
|
|
||||||
|
The pass turned up three bugs:
|
||||||
|
|
||||||
|
- **Escape ignored inside a dialog's text box.** The keydown handler returned early for any key
|
||||||
|
typed in a text box, and Escape sat below that check. So Add feed, which opens with the cursor in
|
||||||
|
its URL box, could not be closed with Escape. It showed up because the screenshot run stalled:
|
||||||
|
the dialog stayed open, and every later click landed on its overlay until the run timed out.
|
||||||
|
- **A white password box in the dark theme.** The input rule listed text, search and number
|
||||||
|
fields, but not password ones.
|
||||||
|
- **A stray "• •" under an item with no date.** Each detail carried its own trailing dot, whether
|
||||||
|
or not the next one was there.
|
||||||
|
|
||||||
|
All Subscriptions gained Mark everything read, as `POST /api/read-all`. It marks the feeds from the
|
||||||
|
person's subscriptions, which is the same set the view's query is scoped by. It asks first, since
|
||||||
|
one click covers every feed.
|
||||||
|
|
||||||
|
**Sorting** happens on the server. A list arrives fifty at a time, so sorting in the page would
|
||||||
|
only have reordered what was loaded. The column name picks one of a fixed set of SQL expressions
|
||||||
|
(`order_sql`), so nothing the browser sends reaches the query, and an unknown name means newest
|
||||||
|
first. Ties also fall back to newest first, so "Load more" does not skip or repeat items at a page
|
||||||
|
boundary. A database test runs every column both ways. Size came out of the File column into its
|
||||||
|
own, and small files show KB, since whole megabytes made every article image "0 MB".
|
||||||
|
|
||||||
|
**The double play.** Ray found that pressing play in the Files pane played the file twice at once.
|
||||||
|
The pane drew its own `<audio controls>`, and its `onplay` handler also started the player bar on
|
||||||
|
the same file. That meant two audio elements from one click. The pane now has a play button that
|
||||||
|
hands that exact file to the player bar, the only player. `play()` now takes the file. Its check
|
||||||
|
for resuming compares the file as well as the item, so another of an item's files starts from its
|
||||||
|
beginning instead of carrying on with the first. A test plays from the pane and checks the page
|
||||||
|
has one `<audio>` element.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — The original's layout
|
||||||
|
|
||||||
|
Ray pointed at a screenshot of the Mac app (techpp.com, 2012) and asked for its panes, its
|
||||||
|
toolbar, and a Directory that lives in the feed list rather than behind a button. What it had, and
|
||||||
|
what ipx now does:
|
||||||
|
|
||||||
|
- A toolbar across the window: subscribe and unsubscribe, play, flag, refresh, and a search box
|
||||||
|
scoped to the feed on show. ipx's has the same groups, acting on the selected feed and item, with
|
||||||
|
Settings and Log at the right end, for admins only.
|
||||||
|
- A source list opening with Directory, Playlist Builder and All Subscriptions above the feeds.
|
||||||
|
ipx has Directory, Popular and All Subscriptions there, opening in the main pane. Playlist
|
||||||
|
Builder is left out, since nothing here builds playlists.
|
||||||
|
- The entries as a table, with a Files pane beside it and the entry below. The columns are unread,
|
||||||
|
kept, the item, its feed, its file and when. Sorting by column is not done yet.
|
||||||
|
- A status bar with the totals for what is on show.
|
||||||
|
|
||||||
|
All Subscriptions needed one new endpoint, `GET /api/entries`. It is the per-feed query with
|
||||||
|
`feed_id = ?` swapped for the person's subscriptions. The enclosure lookup that follows it used to
|
||||||
|
filter by feed as well; a page can now span feeds, so each file is matched to its row by feed and
|
||||||
|
guid instead.
|
||||||
|
|
||||||
|
A phone has no room for a pane beside the table, so there the files follow the item's text in the
|
||||||
|
full-screen reader, and the table drops to title and date.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 2026-09-11 — Popular on this server
|
## 2026-09-11 — Popular on this server
|
||||||
|
|
||||||
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
|
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
|
||||||
@@ -26,6 +336,13 @@ with everything else, so this is built around not doing that:
|
|||||||
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
|
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
|
||||||
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
|
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
|
||||||
`unlisted` flag is the upgrade if that happens.
|
`unlisted` flag is the upgrade if that happens.
|
||||||
|
|
||||||
|
It happened the same day. The first screenshot of the new Directory listed "Glass Cannon Live!
|
||||||
|
Ascension (for Ray Slakinski)", a Supercast feed at `feeds.supercast.com/feeds/<key>`. Treating
|
||||||
|
any long path segment as a key would have hidden public feeds too: acast's show ids look the
|
||||||
|
same. So paid-feed services are named instead (Patreon, Supercast, Supporting Cast, Glow,
|
||||||
|
Memberful), and any feed from one of them is private whatever its URL looks like. The per-feed
|
||||||
|
flag is still the upgrade for a service not on that list.
|
||||||
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
|
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
|
||||||
they would bury everything anyone chose on purpose.
|
they would bury everything anyone chose on purpose.
|
||||||
|
|
||||||
|
|||||||
307
docs/sso.md
307
docs/sso.md
@@ -1,9 +1,8 @@
|
|||||||
# Signing in through Cloudflare Zero Trust or Authentik
|
# Signing in through Cloudflare Access and Authentik
|
||||||
|
|
||||||
ipx can take the signed-in identity from whatever sits in front of it, instead of asking for a
|
ipx can take the signed-in identity from whatever sits in front of it, instead of asking for a
|
||||||
password itself. Both products below do the same thing in the end: they authenticate the person and
|
password itself. The proxy authenticates the person and passes the result to ipx in a **header**;
|
||||||
pass the result to the origin in a **header**. ipx reads that header, finds (or creates) the
|
ipx reads it, finds (or creates) the matching account, and gets on with it.
|
||||||
matching account, and gets on with it.
|
|
||||||
|
|
||||||
Read [How this is secured](#how-this-is-secured) before exposing anything. The short version: a
|
Read [How this is secured](#how-this-is-secured) before exposing anything. The short version: a
|
||||||
header is worth exactly as much as the hop that set it, so ipx only believes one from an address you
|
header is worth exactly as much as the hop that set it, so ipx only believes one from an address you
|
||||||
@@ -11,195 +10,173 @@ list.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## The ipx side (both setups)
|
## How ipodderx.sdf1.net does it
|
||||||
|
|
||||||
|
Checked end to end on 2026-09-12. An earlier version of this page had never been tried against a
|
||||||
|
real setup and pointed at the wrong address.
|
||||||
|
|
||||||
|
```
|
||||||
|
browser ─► Cloudflare Access, app "ipodderx" ─── sign in ───► Authentik (OpenID Connect)
|
||||||
|
─► tunnel "rays-unraid" (the cloudflared container on Tower)
|
||||||
|
─► http://192.168.1.130:8099 ─► ipx
|
||||||
|
```
|
||||||
|
|
||||||
|
Authentik is not in the request path. It is the identity provider Cloudflare Access asks. Access
|
||||||
|
then adds `Cf-Access-Authenticated-User-Email`, the email address Authentik gave it, to every
|
||||||
|
request it forwards through the tunnel, and ipx signs that person in.
|
||||||
|
|
||||||
|
| Piece | Where | Setting |
|
||||||
|
|---|---|---|
|
||||||
|
| Identity provider | Zero Trust → Settings → Authentication | `Authentik`, OpenID Connect; scopes `openid email profile` |
|
||||||
|
| Access application | Zero Trust → Access → Applications → `ipodderx` | Domain `ipodderx.sdf1.net`; identity providers: Authentik only, with instant auth; session 730h; policy *Require Login* allows a list of email addresses |
|
||||||
|
| Tunnel route | Zero Trust → Networks → Tunnels → `rays-unraid` → Public hostnames | `ipodderx.sdf1.net` → HTTP `192.168.1.130:8099` |
|
||||||
|
| DNS | `sdf1.net` | `ipodderx` CNAME to the tunnel, proxied |
|
||||||
|
| ipx | `/mnt/fast/appdata/ipodderx/config.toml`, `[web]` | below |
|
||||||
|
|
||||||
```toml
|
```toml
|
||||||
[web]
|
[web]
|
||||||
enabled = true
|
enabled = true
|
||||||
bind = "0.0.0.0:8099"
|
bind = "0.0.0.0:8099"
|
||||||
token = "…" # keep it: it is the admin, used by the healthcheck
|
trusted_header = "Cf-Access-Authenticated-User-Email"
|
||||||
|
trusted_proxies = ["127.0.0.1", "::1", "192.168.16.1"]
|
||||||
# The header your proxy sets. Empty (the default) disables this whole path.
|
|
||||||
trusted_header = "Cf-Access-Authenticated-User-Email" # Authentik: "X-authentik-username"
|
|
||||||
|
|
||||||
# Addresses allowed to assert that header -- the proxy, and nothing else.
|
|
||||||
trusted_proxies = ["127.0.0.1", "::1"]
|
|
||||||
|
|
||||||
# Create an account the first time the proxy vouches for a name ipx has not seen.
|
|
||||||
auto_create_users = true
|
auto_create_users = true
|
||||||
|
sign_out_url = "/cdn-cgi/access/logout"
|
||||||
session_days = 30
|
session_days = 30
|
||||||
```
|
```
|
||||||
|
|
||||||
Restart the daemon after editing. Accounts made this way have **no password**: they can only ever
|
Restart ipx after editing it: `docker compose -f /mnt/fast/arcane/projects/content/compose.yaml
|
||||||
arrive through the proxy. `ipx user list` marks them `proxy only`.
|
restart ipodderx`.
|
||||||
|
|
||||||
The first account created is an admin. Every later one is an ordinary user, and an ordinary user
|
### What was missing
|
||||||
cannot change global settings, a feed's URL or folder, or how often feeds are scanned: the API
|
|
||||||
refuses those with a `403`, not just the UI. Everything else about a feed (which items they want,
|
|
||||||
whether to fetch them, how many at a time) is theirs alone; see [users.md](users.md).
|
|
||||||
|
|
||||||
Somebody arriving through the proxy for the first time starts with **no feeds**, because
|
Cloudflare and Authentik were already right. Three things on the ipx side were not:
|
||||||
subscriptions are per person. Adding a feed someone else already reads costs no second fetch and no
|
|
||||||
second copy on disk.
|
|
||||||
|
|
||||||
Promote someone with:
|
1. **`trusted_header` was empty**, which switches the whole proxy path off. ipx ignored the header
|
||||||
|
and asked for a password.
|
||||||
|
2. **`trusted_proxies` listed only `127.0.0.1`.** The tunnel's requests do not come from there;
|
||||||
|
see the next section.
|
||||||
|
3. **The account had the wrong name.** It was made by hand as `rays`, but the header carries
|
||||||
|
`rays@sdf1.net`. With `auto_create_users` on, the first visit would have made a second, empty
|
||||||
|
account. `ipx user rename rays rays@sdf1.net` fixed that without losing anything.
|
||||||
|
|
||||||
|
### The address to trust, and why it is 192.168.16.1
|
||||||
|
|
||||||
|
`cloudflared` runs in its own container and reaches ipx through the host's published port. Docker
|
||||||
|
(iptables firewall backend) masquerades traffic between its bridge networks, so the tunnel's
|
||||||
|
requests arrive from the **gateway of ipx's own network**, `content_default`:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
ipx user list
|
docker network inspect content_default -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}'
|
||||||
echo -n 'a good password' | ipx user passwd <name> # optional: also lets them sign in directly
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN
|
That was measured, not assumed. ipx does not log where a request came from, so the addresses were
|
||||||
when the tunnel is down. So does the shared `[web] token`, which signs in as the admin: that is
|
read from the kernel's connection table inside the container while the site was open. (`/proc/net/tcp`
|
||||||
what the Docker healthcheck uses, and the way back in if you lock yourself out. A brand new database
|
lists them in hex.)
|
||||||
starts with **admin / ipodderx** — change it.
|
|
||||||
|
If the `content` project's network is ever recreated, its gateway can change. Check it again, and
|
||||||
|
update `trusted_proxies` to match.
|
||||||
|
|
||||||
|
### Names
|
||||||
|
|
||||||
|
The username is the email address, lower-cased: `rays@sdf1.net`. To sign in at `/login` with a
|
||||||
|
password from the LAN, use that name too.
|
||||||
|
|
||||||
|
To let someone else in, add their address to the Access policy; they need an Authentik account with
|
||||||
|
that email. With `auto_create_users = true` they get an ipx account on their first visit, as an
|
||||||
|
ordinary user with no feeds. An account made before the proxy can be given the name the proxy will
|
||||||
|
send:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker exec iPodderX ipx user rename <old name> <email address>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Signing out
|
||||||
|
|
||||||
|
**Sign out** sends someone the proxy signed in to `sign_out_url`, here Cloudflare's
|
||||||
|
`/cdn-cgi/access/logout`. That ends your Access session for **every** Access application,
|
||||||
|
`code.sdf1.net` included: Cloudflare has no way to end just one, and its sign-out page does not send
|
||||||
|
you anywhere afterwards. The next visit goes back through Authentik, which lets you straight in if
|
||||||
|
you are still signed in there. Signing out of Authentik itself is Authentik's own sign-out.
|
||||||
|
|
||||||
|
ipx never shows its password page to someone the proxy vouches for: `/login` sends them on to their
|
||||||
|
feeds.
|
||||||
|
|
||||||
|
### The tile in Authentik's library
|
||||||
|
|
||||||
|
Authentik's library lists Authentik's own applications, and ipodderx signs in through the one
|
||||||
|
called `Cloudflare Access`, so ipodderx needs a bookmark of its own to show up there. It is
|
||||||
|
Applications → Applications → `ipodderx`: no provider, launch URL `https://ipodderx.sdf1.net`, and
|
||||||
|
the iPodderX icon. Like Outline's, it has no policy bindings, so everyone in Authentik sees the
|
||||||
|
tile. Who actually gets in is still up to the Access policy.
|
||||||
|
|
||||||
|
### Check it
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# From Tower itself: not a trusted address, so the header is ignored.
|
||||||
|
curl -s -H 'Accept: application/json' -H 'Cf-Access-Authenticated-User-Email: rays@sdf1.net' \
|
||||||
|
http://192.168.1.130:8099/api/me # -> sign in
|
||||||
|
|
||||||
|
# From a container on a Docker bridge, as cloudflared is: believed.
|
||||||
|
docker run --rm --network bridge mirror.gcr.io/library/busybox wget -qO- \
|
||||||
|
--header 'Accept: application/json' --header 'Cf-Access-Authenticated-User-Email: rays@sdf1.net' \
|
||||||
|
http://192.168.1.130:8099/api/me # -> {"admin":true,"name":"rays@sdf1.net"}
|
||||||
|
```
|
||||||
|
|
||||||
|
Then open `https://ipodderx.sdf1.net` in a private window. Authentik should ask who you are, and
|
||||||
|
ipx should show `rays@sdf1.net` in the sidebar footer without asking for a password.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Cloudflare Zero Trust
|
## The ipx settings
|
||||||
|
|
||||||
This is what runs `ipodderx.sdf1.net`: a `cloudflared` tunnel to the origin, with an Access
|
| Key | What it does |
|
||||||
application in front of it. Cloudflare authenticates the visitor and adds
|
|
||||||
`Cf-Access-Authenticated-User-Email` to every request it forwards.
|
|
||||||
|
|
||||||
### 1. The tunnel
|
|
||||||
|
|
||||||
In **Zero Trust → Networks → Tunnels**, either use the existing tunnel or create one, then add a
|
|
||||||
public hostname:
|
|
||||||
|
|
||||||
| Field | Value |
|
|
||||||
|---|---|
|
|---|---|
|
||||||
| Subdomain / domain | `ipodderx` / `sdf1.net` |
|
| `trusted_header` | The header the proxy sets. Empty, the default, turns the proxy path off. |
|
||||||
| Type | HTTP |
|
| `trusted_proxies` | The addresses allowed to set it. Nothing else is believed. |
|
||||||
| URL | `localhost:8099` (or the LAN address of the box) |
|
| `auto_create_users` | Make an account the first time the proxy vouches for a name ipx has not seen. |
|
||||||
|
| `sign_out_url` | Where Sign out sends someone the proxy signed in: the proxy's own sign-out. Empty sends them to the sign-in page, where the proxy signs them straight back in. |
|
||||||
|
| `session_days` | How long a password sign-in lasts without use. |
|
||||||
|
|
||||||
Use `localhost` when `cloudflared` runs on the same machine as ipx — that keeps the origin request
|
The first account ever created is an admin. Every later one is an ordinary user, who cannot change
|
||||||
coming from `127.0.0.1`, which is already in `trusted_proxies`. If `cloudflared` runs elsewhere (its
|
global settings, a feed's URL or folder, or how often feeds are scanned: the API refuses those with
|
||||||
own container, another host), put **its** address in `trusted_proxies` instead, and make sure
|
a `403`, not just the UI. Everything else about a feed is theirs alone; see [users.md](users.md).
|
||||||
nothing else can reach port 8099.
|
|
||||||
|
|
||||||
### 2. The Access application
|
Local sign-in at `/login` keeps working alongside the proxy, which is how you get in from the LAN
|
||||||
|
when the tunnel is down. So does the shared `[web] token`, which signs in as the admin and is the
|
||||||
**Zero Trust → Access → Applications → Add an application → Self-hosted**:
|
way back in if you lock yourself out. A brand new database starts with **admin / ipodderx**;
|
||||||
|
change it.
|
||||||
- Application domain: `ipodderx.sdf1.net`
|
|
||||||
- Session duration: whatever suits; ipx keeps its own 30-day session on top.
|
|
||||||
- Add a policy — *Allow*, with a rule such as `Emails` → your address, or `Emails ending in` →
|
|
||||||
your domain. Anyone this policy admits gets an ipx account when `auto_create_users` is on, so keep
|
|
||||||
the policy as narrow as the people you actually want reading your feeds.
|
|
||||||
|
|
||||||
### 3. Point ipx at the header
|
|
||||||
|
|
||||||
```toml
|
|
||||||
trusted_header = "Cf-Access-Authenticated-User-Email"
|
|
||||||
trusted_proxies = ["127.0.0.1", "::1"]
|
|
||||||
```
|
|
||||||
|
|
||||||
The username becomes the email address, lower-cased (`ray@example.com`). That is what shows in the
|
|
||||||
sidebar and what `ipx user list` prints.
|
|
||||||
|
|
||||||
### 4. Check it
|
|
||||||
|
|
||||||
```sh
|
|
||||||
# From the box itself: no header, no session -> the sign-in page.
|
|
||||||
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' -H 'Accept: text/html' http://127.0.0.1:8099/
|
|
||||||
|
|
||||||
# Pretending to be the tunnel (only works because 127.0.0.1 is trusted):
|
|
||||||
curl -s -H 'Cf-Access-Authenticated-User-Email: you@example.com' http://127.0.0.1:8099/api/me
|
|
||||||
```
|
|
||||||
|
|
||||||
Then load `https://ipodderx.sdf1.net` in a browser: Cloudflare should ask who you are, and ipx
|
|
||||||
should show your address in the sidebar footer without ever asking for a password.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Authentik
|
## Authentik in the request path instead
|
||||||
|
|
||||||
Authentik does this with a **Proxy Provider** plus an **outpost**, which sits in the request path and
|
Not what ipodderx.sdf1.net uses, and **not verified**. Authentik can also sit in front of ipx
|
||||||
adds `X-authentik-username` (also `X-authentik-email`, `X-authentik-name`, `X-authentik-groups`).
|
itself, with a **Proxy Provider** and an **outpost** that adds `X-authentik-username`:
|
||||||
|
|
||||||
### 1. Provider
|
- Applications → Providers → Create → Proxy Provider; mode **Proxy** (the outpost talks to ipx) or
|
||||||
|
**Forward auth** (an existing reverse proxy asks the outpost).
|
||||||
**Applications → Providers → Create → Proxy Provider**:
|
- Applications → Create, bound to that provider, with a policy; add the provider to an outpost.
|
||||||
|
- In ipx: `trusted_header = "X-authentik-username"`, and the outpost's or reverse proxy's address
|
||||||
- Name: `ipx`
|
in `trusted_proxies`. Measure that address as above rather than guessing it.
|
||||||
- Authorization flow: your usual (`default-provider-authorization-implicit-consent`)
|
|
||||||
- Mode: **Forward auth (single application)** if an existing reverse proxy fronts ipx, or
|
|
||||||
**Proxy** to let the outpost talk to ipx directly.
|
|
||||||
- External host: `https://ipodderx.example.net`
|
|
||||||
- Internal host (Proxy mode): `http://<ip of the ipx box>:8099`
|
|
||||||
|
|
||||||
### 2. Application and outpost
|
|
||||||
|
|
||||||
**Applications → Create**, bind it to that provider, and give it a policy so only the people you
|
|
||||||
mean are let through. Then add the provider to an outpost (**Applications → Outposts**, the embedded
|
|
||||||
one is fine).
|
|
||||||
|
|
||||||
### 3. Forward auth, if you use nginx/SWAG in front
|
|
||||||
|
|
||||||
In the server block for ipx:
|
|
||||||
|
|
||||||
```nginx
|
|
||||||
location /outpost.goauthentik.io {
|
|
||||||
proxy_pass http://authentik-server:9000/outpost.goauthentik.io;
|
|
||||||
proxy_set_header Host $host;
|
|
||||||
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
|
|
||||||
add_header Set-Cookie $auth_cookie;
|
|
||||||
auth_request_set $auth_cookie $upstream_http_set_cookie;
|
|
||||||
}
|
|
||||||
|
|
||||||
location / {
|
|
||||||
auth_request /outpost.goauthentik.io/auth/nginx;
|
|
||||||
error_page 401 = @goauthentik_proxy_signin;
|
|
||||||
auth_request_set $auth_cookie $upstream_http_set_cookie;
|
|
||||||
add_header Set-Cookie $auth_cookie;
|
|
||||||
|
|
||||||
# This is the line that matters to ipx.
|
|
||||||
auth_request_set $authentik_username $upstream_http_x_authentik_username;
|
|
||||||
proxy_set_header X-authentik-username $authentik_username;
|
|
||||||
|
|
||||||
proxy_pass http://ipx:8099;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Point ipx at the header
|
|
||||||
|
|
||||||
```toml
|
|
||||||
trusted_header = "X-authentik-username"
|
|
||||||
trusted_proxies = ["172.18.0.5"] # the outpost or nginx container, NOT a whole subnet
|
|
||||||
```
|
|
||||||
|
|
||||||
Usernames arrive as Authentik knows them (`ray`), lower-cased.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## How this is secured
|
## How this is secured
|
||||||
|
|
||||||
**The header is only believed from `trusted_proxies`.** Every other source is ignored, and the
|
**The header is only believed from `trusted_proxies`.** Every other source is ignored, and the
|
||||||
request falls through to a session cookie or the shared token. This is the whole security boundary,
|
request falls through to a session cookie or the shared token. That is the whole security boundary.
|
||||||
so:
|
|
||||||
|
|
||||||
- List the **proxy's own address**, not a range. `["127.0.0.1"]` when the tunnel runs beside ipx;
|
With the tunnel reaching ipx through the host's port, `192.168.16.1` means **any container on Tower
|
||||||
the container's IP when it does not.
|
that connects to `192.168.1.130:8099`**, not only `cloudflared`. Machines on the LAN, and Tower
|
||||||
- Never list a LAN subnet. Anyone on your network could then send
|
itself, arrive under their own addresses and cannot set the header; the checks above show both
|
||||||
`Cf-Access-Authenticated-User-Email: admin@…` and be your admin.
|
sides. Never list a LAN address or range: anyone there could then send
|
||||||
- Make sure the origin port is not reachable *around* the proxy by anyone you would not admit
|
`Cf-Access-Authenticated-User-Email: rays@sdf1.net` and be you.
|
||||||
through it. If it is, bind ipx to `127.0.0.1` and let only the proxy reach it.
|
|
||||||
|
|
||||||
Verify the refusal, don't assume it — set `trusted_proxies = ["10.9.9.9"]` briefly and confirm a
|
**What ipx does not do:** it does not verify Cloudflare's signed `Cf-Access-Jwt-Assertion`. It
|
||||||
header from your machine gets a `401`:
|
trusts the hop. Verifying the signature would make the containers on Tower irrelevant to the
|
||||||
|
boundary, and is the upgrade if that ever matters.
|
||||||
|
|
||||||
```sh
|
**Turning it off:** clear `trusted_header` and restart. Proxy-made accounts stay, but nobody can sign
|
||||||
curl -s -o /dev/null -w '%{http_code}\n' \
|
|
||||||
-H 'Cf-Access-Authenticated-User-Email: someone@example.com' http://127.0.0.1:8099/api/me
|
|
||||||
```
|
|
||||||
|
|
||||||
**What ipx does not do:** it does not verify Cloudflare's `Cf-Access-Jwt-Assertion` signature or
|
|
||||||
Authentik's session. It trusts the hop. That is a deliberate trade — it keeps the configuration to
|
|
||||||
three lines — and it is sound exactly as long as the point above holds.
|
|
||||||
|
|
||||||
**Turning it off:** clear `trusted_header`. Existing proxy-only accounts stay, but nobody can sign
|
|
||||||
in with them until they are given a password (`ipx user passwd <name>`).
|
in with them until they are given a password (`ipx user passwd <name>`).
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -207,22 +184,20 @@ in with them until they are given a password (`ipx user passwd <name>`).
|
|||||||
## Everyday administration
|
## Everyday administration
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
ipx user list # who exists, and how each one signs in
|
ipx user list # who exists, how each signs in, and when
|
||||||
echo -n 'secret123' | ipx user add sam # local account, password on stdin
|
echo -n 'secret123' | ipx user add sam # local account, password on stdin
|
||||||
ipx user add sam --no-password # proxy-only account, created ahead of time
|
ipx user add sam@example.com --no-password # proxy-only account, made ahead of time
|
||||||
|
ipx user rename sam sam@example.com # give an account the name the proxy sends
|
||||||
echo -n 'newsecret' | ipx user passwd sam # change a password
|
echo -n 'newsecret' | ipx user passwd sam # change a password
|
||||||
ipx user rm sam # remove the account
|
ipx user rm sam # remove the account
|
||||||
```
|
```
|
||||||
|
|
||||||
Set `auto_create_users = false` once everyone who should have an account has one. After that the
|
In the container, put `docker exec iPodderX` in front, and `docker exec -i iPodderX` for the ones
|
||||||
proxy vouching for an unknown name is logged and refused, rather than quietly making an account.
|
that read a password.
|
||||||
Pre-create people instead with `ipx user add <name> --no-password`, using exactly the name the
|
|
||||||
header will carry (Cloudflare sends the email address, lower-cased).
|
|
||||||
|
|
||||||
Scanning intervals, the disk quota, retention, the download folder and a feed's URL are
|
Set `auto_create_users = false` once everyone who should have an account has one. After that the
|
||||||
**admin-only**: the Settings button is hidden for everyone else, and the API refuses the change even
|
proxy vouching for an unknown name is logged and refused. Make people ahead of time instead, with
|
||||||
if the request is made by hand. Everyone controls their own keywords, auto-download, explicit
|
the exact name the header will carry.
|
||||||
setting and per-scan cap, along with their own read state and which feeds they see.
|
|
||||||
|
|
||||||
See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
|
See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
|
||||||
for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.
|
for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ fetch, one parse and one file.
|
|||||||
|
|
||||||
| Yours alone | The same for everyone |
|
| Yours alone | The same for everyone |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Read, starred, playback position | The feed's URL |
|
| Read, kept, playback position | The feed's URL |
|
||||||
| Which feeds you see at all | Its download folder |
|
| Which feeds you see at all | Its download folder |
|
||||||
| Keywords, auto-download, explicit, per-scan cap | When it is scanned |
|
| Keywords, auto-download, explicit, per-scan cap | When it is scanned |
|
||||||
| | The file on disk |
|
| | The file on disk |
|
||||||
@@ -34,10 +34,10 @@ is shared.
|
|||||||
|
|
||||||
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
|
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
|
||||||
for everyone** and names them in the confirmation, and the server has the last word: if anyone else
|
for everyone** and names them in the confirmation, and the server has the last word: if anyone else
|
||||||
has starred the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
|
has kept the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
|
||||||
reason, and only `?force=true` goes through.
|
reason, and only `?force=true` goes through.
|
||||||
|
|
||||||
Retention follows the same rule: starred by anyone keeps a file, and it counts as read only once
|
Retention follows the same rule: an item anyone kept keeps its file, and it counts as read only once
|
||||||
every subscriber has read it.
|
every subscriber has read it.
|
||||||
|
|
||||||
## Signing in
|
## Signing in
|
||||||
@@ -70,12 +70,16 @@ list with their own read state. Unsubscribing removes it from their list alone;
|
|||||||
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
|
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
|
||||||
re-subscribing does not pull the back catalogue again.
|
re-subscribing does not pull the back catalogue again.
|
||||||
|
|
||||||
**Add feed** also lists what other people on this server subscribe to, most subscribers first, as
|
**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
|
||||||
a place to start. It shows a title, artwork and a count, never a URL or who reads it. Feeds from an
|
shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
|
||||||
OPML subscription are left out, since they come with the OPML. So is anything that looks private: a
|
included. Directory lists every one of them A to Z. Your own feeds are marked Subscribed.
|
||||||
login configured for the feed, credentials in its URL, or a key such as `auth=` or `token=` in the
|
It shows a title, artwork and a count, never a URL or who reads it. An OPML subscription is listed
|
||||||
query. Those are someone's paid subscriptions, and listing them would let anyone here read what they
|
as the feeds inside it, one by one, and never the OPML itself, so you can take just the shows you
|
||||||
pay for.
|
want. Anything that looks private is left out: a login configured for the feed, credentials in its URL,
|
||||||
|
or a key such as `auth=` or `token=` in the query, or a feed from a paid-feed service such as
|
||||||
|
Patreon or Supercast, which put the key in the path, and any feed inside an OPML that looks private
|
||||||
|
itself. Those are someone's paid subscriptions, and listing them would let anyone here read what
|
||||||
|
they pay for.
|
||||||
|
|
||||||
An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none
|
An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none
|
||||||
for someone the proxy signs in), tick or untick Admin, or remove an account. Removing one takes its
|
for someone the proxy signs in), tick or untick Admin, or remove an account. Removing one takes its
|
||||||
|
|||||||
@@ -26,9 +26,6 @@ pub struct General {
|
|||||||
/// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d".
|
/// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d".
|
||||||
/// A feed's own `schedule` overrides this.
|
/// A feed's own `schedule` overrides this.
|
||||||
pub schedule: String,
|
pub schedule: String,
|
||||||
/// Superseded by `schedule`. Still read so existing configs keep working.
|
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
|
||||||
pub interval_mins: Option<u64>,
|
|
||||||
pub organize: Organize,
|
pub organize: Organize,
|
||||||
/// 0 = unlimited.
|
/// 0 = unlimited.
|
||||||
pub max_total_gb: f64,
|
pub max_total_gb: f64,
|
||||||
@@ -83,6 +80,10 @@ pub struct Web {
|
|||||||
pub trusted_proxies: Vec<String>,
|
pub trusted_proxies: Vec<String>,
|
||||||
/// Create an account the first time the proxy vouches for a name it has not seen.
|
/// Create an account the first time the proxy vouches for a name it has not seen.
|
||||||
pub auto_create_users: bool,
|
pub auto_create_users: bool,
|
||||||
|
/// Where Sign out sends someone the proxy signed in. Signing out of ipx alone cannot stick
|
||||||
|
/// while the proxy still vouches for them, so this is the proxy's own sign-out:
|
||||||
|
/// `/cdn-cgi/access/logout` behind Cloudflare Access. Empty sends them to /login.
|
||||||
|
pub sign_out_url: String,
|
||||||
/// Sign a session out after this long without a request.
|
/// Sign a session out after this long without a request.
|
||||||
pub session_days: i64,
|
pub session_days: i64,
|
||||||
}
|
}
|
||||||
@@ -96,6 +97,7 @@ impl Default for Web {
|
|||||||
trusted_header: String::new(),
|
trusted_header: String::new(),
|
||||||
trusted_proxies: vec!["127.0.0.1".into(), "::1".into()],
|
trusted_proxies: vec!["127.0.0.1".into(), "::1".into()],
|
||||||
auto_create_users: true,
|
auto_create_users: true,
|
||||||
|
sign_out_url: String::new(),
|
||||||
session_days: 30,
|
session_days: 30,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -153,7 +155,6 @@ impl Default for General {
|
|||||||
download_dir: home().join("Podcasts"),
|
download_dir: home().join("Podcasts"),
|
||||||
socket: default_socket(),
|
socket: default_socket(),
|
||||||
schedule: "every 60m".into(),
|
schedule: "every 60m".into(),
|
||||||
interval_mins: None,
|
|
||||||
organize: Organize::Feed,
|
organize: Organize::Feed,
|
||||||
max_total_gb: 0.0,
|
max_total_gb: 0.0,
|
||||||
max_age_days: 0,
|
max_age_days: 0,
|
||||||
@@ -176,8 +177,8 @@ impl Default for Torrent {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl General {
|
impl General {
|
||||||
/// Minutes between checks. Falls back to the legacy `interval_mins`, then to an hour.
|
/// Minutes between checks, or an hour when `schedule` is empty or unreadable. A malformed
|
||||||
/// A malformed value warns rather than stopping the daemon.
|
/// value warns rather than stopping the daemon.
|
||||||
pub fn interval(&self) -> u64 {
|
pub fn interval(&self) -> u64 {
|
||||||
if let Some(n) = parse_interval(&self.schedule) {
|
if let Some(n) = parse_interval(&self.schedule) {
|
||||||
return n;
|
return n;
|
||||||
@@ -185,7 +186,7 @@ impl General {
|
|||||||
if !self.schedule.trim().is_empty() {
|
if !self.schedule.trim().is_empty() {
|
||||||
tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default");
|
tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default");
|
||||||
}
|
}
|
||||||
self.interval_mins.filter(|n| *n > 0).unwrap_or(60)
|
60
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -278,9 +279,7 @@ pub fn config_path() -> PathBuf {
|
|||||||
if let Ok(p) = std::env::var("IPX_CONFIG") {
|
if let Ok(p) = std::env::var("IPX_CONFIG") {
|
||||||
return PathBuf::from(p);
|
return PathBuf::from(p);
|
||||||
}
|
}
|
||||||
dirs::config_dir()
|
xdg("XDG_CONFIG_HOME", ".config").join("ipx/config.toml")
|
||||||
.unwrap_or_else(|| home().join(".config"))
|
|
||||||
.join("ipx/config.toml")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`.
|
/// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`.
|
||||||
@@ -288,9 +287,7 @@ pub fn data_dir() -> PathBuf {
|
|||||||
if let Ok(p) = std::env::var("IPX_DATA_DIR") {
|
if let Ok(p) = std::env::var("IPX_DATA_DIR") {
|
||||||
return PathBuf::from(p);
|
return PathBuf::from(p);
|
||||||
}
|
}
|
||||||
dirs::data_dir()
|
xdg("XDG_DATA_HOME", ".local/share").join("ipx")
|
||||||
.unwrap_or_else(|| home().join(".local/share"))
|
|
||||||
.join("ipx")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fn default_socket() -> PathBuf {
|
fn default_socket() -> PathBuf {
|
||||||
@@ -346,8 +343,16 @@ pub fn unique_slug(text: &str, taken: &BTreeMap<String, Feed>) -> String {
|
|||||||
(2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap()
|
(2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// `$var`, or `~/fallback` when it is unset or empty, as the XDG base directory spec says.
|
||||||
|
fn xdg(var: &str, fallback: &str) -> PathBuf {
|
||||||
|
std::env::var_os(var)
|
||||||
|
.filter(|v| !v.is_empty())
|
||||||
|
.map(PathBuf::from)
|
||||||
|
.unwrap_or_else(|| home().join(fallback))
|
||||||
|
}
|
||||||
|
|
||||||
fn home() -> PathBuf {
|
fn home() -> PathBuf {
|
||||||
dirs::home_dir().unwrap_or_else(|| PathBuf::from("."))
|
std::env::var_os("HOME").map(PathBuf::from).unwrap_or_else(|| PathBuf::from("."))
|
||||||
}
|
}
|
||||||
|
|
||||||
fn expand_tilde(p: &Path) -> PathBuf {
|
fn expand_tilde(p: &Path) -> PathBuf {
|
||||||
@@ -367,6 +372,8 @@ mod tests {
|
|||||||
r#"
|
r#"
|
||||||
[general]
|
[general]
|
||||||
download_dir = "/tmp/pods"
|
download_dir = "/tmp/pods"
|
||||||
|
# A key older versions read. An old config that still has it has to load.
|
||||||
|
interval_mins = 45
|
||||||
|
|
||||||
[feeds.example]
|
[feeds.example]
|
||||||
url = "https://example.com/feed.xml"
|
url = "https://example.com/feed.xml"
|
||||||
@@ -403,22 +410,17 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn interval_falls_back_through_legacy_then_default() {
|
fn interval_falls_back_to_an_hour() {
|
||||||
let mut g = General::default();
|
let mut g = General::default();
|
||||||
assert_eq!(g.interval(), 60, "the default schedule");
|
assert_eq!(g.interval(), 60, "the default schedule");
|
||||||
|
|
||||||
g.schedule = "every 15m".into();
|
g.schedule = "every 15m".into();
|
||||||
assert_eq!(g.interval(), 15);
|
assert_eq!(g.interval(), 15);
|
||||||
|
|
||||||
// A config written before `schedule` existed still works.
|
// Empty or garbage must not stop the daemon.
|
||||||
g.schedule = String::new();
|
g.schedule = String::new();
|
||||||
g.interval_mins = Some(45);
|
assert_eq!(g.interval(), 60);
|
||||||
assert_eq!(g.interval(), 45);
|
|
||||||
|
|
||||||
// Garbage must not stop the daemon.
|
|
||||||
g.schedule = "whenever".into();
|
g.schedule = "whenever".into();
|
||||||
assert_eq!(g.interval(), 45);
|
|
||||||
g.interval_mins = None;
|
|
||||||
assert_eq!(g.interval(), 60);
|
assert_eq!(g.interval(), 60);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
573
src/db.rs
573
src/db.rs
@@ -40,14 +40,10 @@ CREATE TABLE IF NOT EXISTS entries (
|
|||||||
published INTEGER,
|
published INTEGER,
|
||||||
description TEXT,
|
description TEXT,
|
||||||
first_seen INTEGER NOT NULL,
|
first_seen INTEGER NOT NULL,
|
||||||
read INTEGER NOT NULL DEFAULT 0,
|
|
||||||
flagged INTEGER NOT NULL DEFAULT 0,
|
|
||||||
image TEXT,
|
image TEXT,
|
||||||
duration INTEGER,
|
duration INTEGER,
|
||||||
episode INTEGER,
|
episode INTEGER,
|
||||||
season INTEGER,
|
season INTEGER,
|
||||||
-- Seconds into the audio, so playback resumes where it was left.
|
|
||||||
position INTEGER NOT NULL DEFAULT 0,
|
|
||||||
PRIMARY KEY (feed_id, guid)
|
PRIMARY KEY (feed_id, guid)
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -77,7 +73,9 @@ CREATE TABLE IF NOT EXISTS users (
|
|||||||
name TEXT NOT NULL UNIQUE COLLATE NOCASE,
|
name TEXT NOT NULL UNIQUE COLLATE NOCASE,
|
||||||
pass_hash TEXT,
|
pass_hash TEXT,
|
||||||
is_admin INTEGER NOT NULL DEFAULT 0,
|
is_admin INTEGER NOT NULL DEFAULT 0,
|
||||||
created INTEGER NOT NULL
|
-- For whoever maintains the server. NULL where it is not known.
|
||||||
|
created INTEGER,
|
||||||
|
last_login INTEGER
|
||||||
);
|
);
|
||||||
|
|
||||||
-- What one person wants from a feed. The feed, its items and its files are shared; this
|
-- What one person wants from a feed. The feed, its items and its files are shared; this
|
||||||
@@ -89,11 +87,10 @@ CREATE TABLE IF NOT EXISTS subscriptions (
|
|||||||
auto_download INTEGER,
|
auto_download INTEGER,
|
||||||
allow_explicit INTEGER,
|
allow_explicit INTEGER,
|
||||||
max_new_per_check INTEGER,
|
max_new_per_check INTEGER,
|
||||||
created INTEGER NOT NULL,
|
|
||||||
PRIMARY KEY (user_id, feed_id)
|
PRIMARY KEY (user_id, feed_id)
|
||||||
);
|
);
|
||||||
|
|
||||||
-- Read, starred and how far in. One row per person per item, created on first touch;
|
-- Read, kept and how far in. One row per person per item, created on first touch;
|
||||||
-- an item nobody has touched has no row at all, which is what unread means.
|
-- an item nobody has touched has no row at all, which is what unread means.
|
||||||
CREATE TABLE IF NOT EXISTS entry_state (
|
CREATE TABLE IF NOT EXISTS entry_state (
|
||||||
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
@@ -108,7 +105,6 @@ CREATE TABLE IF NOT EXISTS entry_state (
|
|||||||
CREATE TABLE IF NOT EXISTS sessions (
|
CREATE TABLE IF NOT EXISTS sessions (
|
||||||
token TEXT PRIMARY KEY,
|
token TEXT PRIMARY KEY,
|
||||||
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
created INTEGER NOT NULL,
|
|
||||||
seen INTEGER NOT NULL
|
seen INTEGER NOT NULL
|
||||||
);
|
);
|
||||||
";
|
";
|
||||||
@@ -131,6 +127,23 @@ pub struct User {
|
|||||||
pub name: String,
|
pub name: String,
|
||||||
pub pass_hash: Option<String>,
|
pub pass_hash: Option<String>,
|
||||||
pub is_admin: bool,
|
pub is_admin: bool,
|
||||||
|
/// When the account was made and when it last signed in, for whoever maintains the server.
|
||||||
|
pub created: Option<i64>,
|
||||||
|
pub last_login: Option<i64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The columns `user_row` reads, in its order.
|
||||||
|
const USER_COLS: &str = "id, name, pass_hash, is_admin, created, last_login";
|
||||||
|
|
||||||
|
fn user_row(r: &rusqlite::Row<'_>) -> rusqlite::Result<User> {
|
||||||
|
Ok(User {
|
||||||
|
id: r.get(0)?,
|
||||||
|
name: r.get(1)?,
|
||||||
|
pass_hash: r.get(2)?,
|
||||||
|
is_admin: r.get::<_, i64>(3)? != 0,
|
||||||
|
created: r.get(4)?,
|
||||||
|
last_login: r.get(5)?,
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A feed derived from an OPML subscription rather than written into the config.
|
/// A feed derived from an OPML subscription rather than written into the config.
|
||||||
@@ -140,33 +153,48 @@ pub struct Managed {
|
|||||||
pub url: String,
|
pub url: String,
|
||||||
pub title: Option<String>,
|
pub title: Option<String>,
|
||||||
pub group_id: String,
|
pub group_id: String,
|
||||||
pub orphaned: bool,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Adds columns that later versions introduced. CREATE TABLE IF NOT EXISTS does nothing to
|
/// Adds the columns later versions introduced and drops the ones they retired. CREATE TABLE IF
|
||||||
/// a table that already exists, so an installed database needs them added explicitly.
|
/// NOT EXISTS leaves a table that already exists alone, so an installed database needs both done
|
||||||
|
/// explicitly. Columns from before 0.3.0, the oldest version an upgrade may start from, need no
|
||||||
|
/// entry.
|
||||||
fn migrate(conn: &Connection) -> Result<()> {
|
fn migrate(conn: &Connection) -> Result<()> {
|
||||||
let wanted: &[(&str, &str, &str)] = &[
|
let wanted: &[(&str, &str, &str)] = &[
|
||||||
("feeds", "image", "TEXT"),
|
// For whoever maintains the server. An audit dropped `created` as unread on 2026-09-12,
|
||||||
("feeds", "orphaned", "INTEGER NOT NULL DEFAULT 0"),
|
// and it came back the same day with `last_login` beside it.
|
||||||
("feeds", "group_id", "TEXT"),
|
("users", "created", "INTEGER"),
|
||||||
("feeds", "managed", "INTEGER NOT NULL DEFAULT 0"),
|
("users", "last_login", "INTEGER"),
|
||||||
("entries", "image", "TEXT"),
|
|
||||||
("entries", "duration", "INTEGER"),
|
|
||||||
("entries", "episode", "INTEGER"),
|
|
||||||
("entries", "season", "INTEGER"),
|
|
||||||
("entries", "position", "INTEGER NOT NULL DEFAULT 0"),
|
|
||||||
];
|
];
|
||||||
|
let retired: &[(&str, &str)] = &[
|
||||||
|
// Read state from before accounts, long since moved to entry_state. Two bugs came from
|
||||||
|
// queries still reading these after they stopped meaning anything.
|
||||||
|
("entries", "read"),
|
||||||
|
("entries", "flagged"),
|
||||||
|
("entries", "position"),
|
||||||
|
// Written by every insert and read by nothing.
|
||||||
|
("subscriptions", "created"),
|
||||||
|
("sessions", "created"),
|
||||||
|
];
|
||||||
|
let has = |table: &str, column: &str| -> Result<bool> {
|
||||||
|
let names: Vec<String> = conn
|
||||||
|
.prepare(&format!("PRAGMA table_info({table})"))?
|
||||||
|
.query_map([], |r| r.get(1))?
|
||||||
|
.collect::<rusqlite::Result<_>>()?;
|
||||||
|
Ok(names.iter().any(|c| c == column))
|
||||||
|
};
|
||||||
for (table, column, ty) in wanted {
|
for (table, column, ty) in wanted {
|
||||||
let mut stmt = conn.prepare(&format!("PRAGMA table_info({table})"))?;
|
if !has(table, column)? {
|
||||||
let existing: Vec<String> = stmt
|
|
||||||
.query_map([], |r| r.get::<_, String>(1))?
|
|
||||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
|
||||||
if !existing.iter().any(|c| c == column) {
|
|
||||||
tracing::info!(table, column, "adding column");
|
tracing::info!(table, column, "adding column");
|
||||||
conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?;
|
conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
for (table, column) in retired {
|
||||||
|
if has(table, column)? {
|
||||||
|
tracing::info!(table, column, "dropping column");
|
||||||
|
conn.execute_batch(&format!("ALTER TABLE {table} DROP COLUMN {column}"))?;
|
||||||
|
}
|
||||||
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -345,22 +373,19 @@ impl Db {
|
|||||||
|
|
||||||
/// Returns true when this entry had not been seen before.
|
/// Returns true when this entry had not been seen before.
|
||||||
///
|
///
|
||||||
/// A changed description or title flips `read` back to 0, which is what the original's
|
|
||||||
/// textDiff dance was ultimately for -- minus the diff markup, which the UI can do.
|
|
||||||
pub fn record_entry(&self, feed_id: &str, e: &crate::feed::Entry) -> Result<bool> {
|
pub fn record_entry(&self, feed_id: &str, e: &crate::feed::Entry) -> Result<bool> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let inserted = conn.execute(
|
let inserted = conn.execute(
|
||||||
"INSERT OR IGNORE INTO entries
|
"INSERT OR IGNORE INTO entries
|
||||||
(feed_id, guid, title, link, published, description, first_seen, read, flagged,
|
(feed_id, guid, title, link, published, description, first_seen,
|
||||||
image, duration, episode, season)
|
image, duration, episode, season)
|
||||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, 0, 0, ?8, ?9, ?10, ?11)",
|
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11)",
|
||||||
rusqlite::params![
|
rusqlite::params![
|
||||||
feed_id, e.guid, e.title, e.link, e.published, e.description, now(),
|
feed_id, e.guid, e.title, e.link, e.published, e.description, now(),
|
||||||
e.image, e.duration, e.episode, e.season
|
e.image, e.duration, e.episode, e.season
|
||||||
],
|
],
|
||||||
)?;
|
)?;
|
||||||
if inserted == 0 {
|
if inserted == 0 {
|
||||||
// The SET expressions see the pre-update row, so this compares old vs new.
|
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE entries SET
|
"UPDATE entries SET
|
||||||
title = coalesce(?3, title),
|
title = coalesce(?3, title),
|
||||||
@@ -368,8 +393,7 @@ impl Db {
|
|||||||
image = coalesce(?5, image),
|
image = coalesce(?5, image),
|
||||||
duration = coalesce(?6, duration),
|
duration = coalesce(?6, duration),
|
||||||
episode = coalesce(?7, episode),
|
episode = coalesce(?7, episode),
|
||||||
season = coalesce(?8, season),
|
season = coalesce(?8, season)
|
||||||
read = CASE WHEN description IS NOT ?4 OR title IS NOT ?3 THEN 0 ELSE read END
|
|
||||||
WHERE feed_id = ?1 AND guid = ?2",
|
WHERE feed_id = ?1 AND guid = ?2",
|
||||||
rusqlite::params![
|
rusqlite::params![
|
||||||
feed_id, e.guid, e.title, e.description,
|
feed_id, e.guid, e.title, e.description,
|
||||||
@@ -609,6 +633,36 @@ pub struct EntryRow {
|
|||||||
const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2
|
const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2
|
||||||
OR lower(coalesce(e.description, '')) LIKE ?2)";
|
OR lower(coalesce(e.description, '')) LIKE ?2)";
|
||||||
|
|
||||||
|
/// Which feeds a query covers: one, or every feed the person subscribes to. Both forms
|
||||||
|
/// mention `?1`, since binding a parameter the statement does not use is an error.
|
||||||
|
fn scope_sql(feed_id: Option<&str>, user_param: u8) -> String {
|
||||||
|
match feed_id {
|
||||||
|
Some(_) => "e.feed_id = ?1".into(),
|
||||||
|
None => format!(
|
||||||
|
"?1 IS NULL AND e.feed_id IN (SELECT feed_id FROM subscriptions WHERE user_id = ?{user_param})"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The item table's ORDER BY. The column name picks one of these fixed expressions, so nothing
|
||||||
|
/// the caller sends reaches the query, and anything unrecognised is newest first. Ties fall back
|
||||||
|
/// to newest first too, so a page boundary is stable across "Load more".
|
||||||
|
///
|
||||||
|
/// ponytail: file type and size look at the item's first and largest file. The row shows the file
|
||||||
|
/// it summarises, which is almost always that one; sort by that one if they ever disagree.
|
||||||
|
pub fn order_sql(col: &str, dir: &str) -> String {
|
||||||
|
let expr = match col {
|
||||||
|
"kept" => "coalesce(s.flagged, 0)",
|
||||||
|
"title" => "lower(coalesce(e.title, ''))",
|
||||||
|
"feed" => "(SELECT lower(coalesce(f.title, f.id)) FROM feeds f WHERE f.id = e.feed_id)",
|
||||||
|
"type" => "(SELECT min(x.mime) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
|
||||||
|
"size" => "(SELECT max(x.length) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
|
||||||
|
_ => "coalesce(e.published, e.first_seen)",
|
||||||
|
};
|
||||||
|
let dir = if dir == "asc" { "ASC" } else { "DESC" };
|
||||||
|
format!("{expr} {dir}, coalesce(e.published, e.first_seen) DESC, e.rowid DESC")
|
||||||
|
}
|
||||||
|
|
||||||
/// Which slice of a feed the UI is asking for.
|
/// Which slice of a feed the UI is asking for.
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub enum Filter {
|
pub enum Filter {
|
||||||
@@ -657,16 +711,18 @@ pub struct EncRow {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl Db {
|
impl Db {
|
||||||
/// One page of a feed's entries, newest first, each with its enclosures attached.
|
/// One page of entries, each with its enclosures attached: one feed's, or every feed the
|
||||||
/// `search` matches title and description, case-insensitively.
|
/// person subscribes to when `feed_id` is None (All Subscriptions). `search` matches title
|
||||||
pub fn entries(
|
/// and description, case-insensitively.
|
||||||
|
pub fn entries_in(
|
||||||
&self,
|
&self,
|
||||||
user_id: i64,
|
user_id: i64,
|
||||||
feed_id: &str,
|
feed_id: Option<&str>,
|
||||||
filter: Filter,
|
filter: Filter,
|
||||||
search: Option<&str>,
|
search: Option<&str>,
|
||||||
offset: i64,
|
offset: i64,
|
||||||
limit: i64,
|
limit: i64,
|
||||||
|
order: &str,
|
||||||
) -> Result<Vec<EntryRow>> {
|
) -> Result<Vec<EntryRow>> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let like = search
|
let like = search
|
||||||
@@ -679,9 +735,10 @@ impl Db {
|
|||||||
FROM entries e
|
FROM entries e
|
||||||
LEFT JOIN entry_state s
|
LEFT JOIN entry_state s
|
||||||
ON s.user_id = ?5 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
ON s.user_id = ?5 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
||||||
WHERE e.feed_id = ?1 AND {} AND {SEARCH}
|
WHERE {} AND {} AND {SEARCH}
|
||||||
ORDER BY coalesce(e.published, e.first_seen) DESC, e.rowid DESC
|
ORDER BY {order}
|
||||||
LIMIT ?4 OFFSET ?3",
|
LIMIT ?4 OFFSET ?3",
|
||||||
|
scope_sql(feed_id, 5),
|
||||||
filter.sql()
|
filter.sql()
|
||||||
);
|
);
|
||||||
let mut stmt = conn.prepare(&sql)?;
|
let mut stmt = conn.prepare(&sql)?;
|
||||||
@@ -711,14 +768,14 @@ impl Db {
|
|||||||
return Ok(rows);
|
return Ok(rows);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Only the guids on this page, so a feed with thousands of entries stays cheap.
|
// Only the guids on this page, so a feed with thousands of entries stays cheap. A page
|
||||||
|
// can span feeds, so each file is matched to its row by feed as well as guid, below.
|
||||||
let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(",");
|
let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(",");
|
||||||
let sql = format!(
|
let sql = format!(
|
||||||
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error
|
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error
|
||||||
FROM enclosures WHERE feed_id = ? AND guid IN ({placeholders}) ORDER BY id"
|
FROM enclosures WHERE guid IN ({placeholders}) ORDER BY id"
|
||||||
);
|
);
|
||||||
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len() + 1);
|
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len());
|
||||||
params.push(&feed_id);
|
|
||||||
for row in &rows {
|
for row in &rows {
|
||||||
params.push(&row.guid);
|
params.push(&row.guid);
|
||||||
}
|
}
|
||||||
@@ -740,18 +797,18 @@ impl Db {
|
|||||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||||
|
|
||||||
for enc in encs {
|
for enc in encs {
|
||||||
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid) {
|
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid && r.feed_id == enc.feed_id) {
|
||||||
row.enclosures.push(enc);
|
row.enclosures.push(enc);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Ok(rows)
|
Ok(rows)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// How many entries match, so the UI knows whether there is another page.
|
/// How many entries `entries_in` would page through, so the UI knows whether there is more.
|
||||||
pub fn count_entries(
|
pub fn count_in(
|
||||||
&self,
|
&self,
|
||||||
user_id: i64,
|
user_id: i64,
|
||||||
feed_id: &str,
|
feed_id: Option<&str>,
|
||||||
filter: Filter,
|
filter: Filter,
|
||||||
search: Option<&str>,
|
search: Option<&str>,
|
||||||
) -> Result<i64> {
|
) -> Result<i64> {
|
||||||
@@ -763,7 +820,8 @@ impl Db {
|
|||||||
"SELECT count(*) FROM entries e
|
"SELECT count(*) FROM entries e
|
||||||
LEFT JOIN entry_state s
|
LEFT JOIN entry_state s
|
||||||
ON s.user_id = ?3 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
ON s.user_id = ?3 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
||||||
WHERE e.feed_id = ?1 AND {} AND {SEARCH}",
|
WHERE {} AND {} AND {SEARCH}",
|
||||||
|
scope_sql(feed_id, 3),
|
||||||
filter.sql()
|
filter.sql()
|
||||||
);
|
);
|
||||||
Ok(conn.query_row(&sql, rusqlite::params![feed_id, like, user_id], |r| r.get(0))?)
|
Ok(conn.query_row(&sql, rusqlite::params![feed_id, like, user_id], |r| r.get(0))?)
|
||||||
@@ -780,37 +838,28 @@ impl Db {
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Marks every entry in a feed read, for the "mark all read" button.
|
/// The first admin starts subscribed to the whole catalogue: whoever wrote config.toml meant
|
||||||
/// Moves a single-user library onto an account: everything read, starred or part-played
|
/// to read those feeds, and without this a fresh install signs in to an empty sidebar. Runs
|
||||||
/// becomes that person's, and they subscribe to every feed already in the catalogue.
|
/// only while nobody subscribes to anything, so an unsubscribe is never undone.
|
||||||
/// Runs once -- the moment there is a first account and no subscriptions yet.
|
pub fn adopt_catalogue(&self, user_id: i64, catalogue: &[String]) -> Result<usize> {
|
||||||
pub fn adopt_existing_library(&self, user_id: i64, catalogue: &[String]) -> Result<usize> {
|
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let already: i64 =
|
let already: i64 =
|
||||||
conn.query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0))?;
|
conn.query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0))?;
|
||||||
if already > 0 {
|
if already > 0 {
|
||||||
return Ok(0);
|
return Ok(0);
|
||||||
}
|
}
|
||||||
let moved = conn.execute(
|
|
||||||
"INSERT INTO entry_state (user_id, feed_id, guid, read, flagged, position)
|
|
||||||
SELECT ?1, feed_id, guid, read, flagged, position FROM entries
|
|
||||||
WHERE read = 1 OR flagged = 1 OR position > 0
|
|
||||||
ON CONFLICT(user_id, feed_id, guid) DO NOTHING",
|
|
||||||
[user_id],
|
|
||||||
)?;
|
|
||||||
for id in catalogue {
|
for id in catalogue {
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
|
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id) VALUES (?1, ?2)",
|
||||||
params![user_id, id, now()],
|
params![user_id, id],
|
||||||
)?;
|
)?;
|
||||||
}
|
}
|
||||||
// Feeds that exist only in the database (OPML children) count too.
|
// Feeds that exist only in the database (OPML children) count too.
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created)
|
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id) SELECT ?1, id FROM feeds",
|
||||||
SELECT ?1, id, ?2 FROM feeds",
|
params![user_id],
|
||||||
params![user_id, now()],
|
|
||||||
)?;
|
)?;
|
||||||
Ok(moved)
|
Ok(catalogue.len())
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---- subscriptions ----
|
// ---- subscriptions ----
|
||||||
@@ -862,14 +911,20 @@ impl Db {
|
|||||||
|
|
||||||
/// Everyone's settings for one feed. The scanner merges these into what it fetches
|
/// Everyone's settings for one feed. The scanner merges these into what it fetches
|
||||||
/// and downloads, since one file serves the lot.
|
/// and downloads, since one file serves the lot.
|
||||||
pub fn subscribers(&self, feed_id: &str) -> Result<Vec<Sub>> {
|
/// In a group, whatever someone has not set on the feed itself comes from their
|
||||||
|
/// subscription to the group, as the group's settings dialog has always said it does.
|
||||||
|
pub fn subscribers(&self, feed_id: &str, group: Option<&str>) -> Result<Vec<Sub>> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let mut stmt = conn.prepare(
|
let mut stmt = conn.prepare(
|
||||||
"SELECT keywords, auto_download, allow_explicit, max_new_per_check
|
"SELECT coalesce(c.keywords, p.keywords), coalesce(c.auto_download, p.auto_download),
|
||||||
FROM subscriptions WHERE feed_id = ?1",
|
coalesce(c.allow_explicit, p.allow_explicit),
|
||||||
|
coalesce(c.max_new_per_check, p.max_new_per_check)
|
||||||
|
FROM subscriptions c
|
||||||
|
LEFT JOIN subscriptions p ON p.user_id = c.user_id AND p.feed_id = ?2
|
||||||
|
WHERE c.feed_id = ?1",
|
||||||
)?;
|
)?;
|
||||||
let out = stmt
|
let out = stmt
|
||||||
.query_map([feed_id], |r| {
|
.query_map(params![feed_id, group], |r| {
|
||||||
Ok(Sub {
|
Ok(Sub {
|
||||||
feed_id: feed_id.to_string(),
|
feed_id: feed_id.to_string(),
|
||||||
keywords: r
|
keywords: r
|
||||||
@@ -919,8 +974,8 @@ impl Db {
|
|||||||
pub fn subscribe(&self, user_id: i64, feed_id: &str) -> Result<()> {
|
pub fn subscribe(&self, user_id: i64, feed_id: &str) -> Result<()> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
|
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id) VALUES (?1, ?2)",
|
||||||
params![user_id, feed_id, now()],
|
params![user_id, feed_id],
|
||||||
)?;
|
)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
@@ -934,16 +989,6 @@ impl Db {
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// How many people want this feed. Nobody means it stops being scanned.
|
|
||||||
pub fn subscriber_count(&self, feed_id: &str) -> Result<i64> {
|
|
||||||
let conn = self.conn.lock().unwrap();
|
|
||||||
Ok(conn.query_row(
|
|
||||||
"SELECT count(*) FROM subscriptions WHERE feed_id = ?1",
|
|
||||||
[feed_id],
|
|
||||||
|r| r.get(0),
|
|
||||||
)?)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Overwrites one person's settings for a feed. A None field means: follow the feed.
|
/// Overwrites one person's settings for a feed. A None field means: follow the feed.
|
||||||
pub fn set_subscription(&self, user_id: i64, sub: &Sub) -> Result<()> {
|
pub fn set_subscription(&self, user_id: i64, sub: &Sub) -> Result<()> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
@@ -954,8 +999,8 @@ impl Db {
|
|||||||
.transpose()?;
|
.transpose()?;
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO subscriptions
|
"INSERT INTO subscriptions
|
||||||
(user_id, feed_id, keywords, auto_download, allow_explicit, max_new_per_check, created)
|
(user_id, feed_id, keywords, auto_download, allow_explicit, max_new_per_check)
|
||||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
|
VALUES (?1, ?2, ?3, ?4, ?5, ?6)
|
||||||
ON CONFLICT(user_id, feed_id) DO UPDATE SET
|
ON CONFLICT(user_id, feed_id) DO UPDATE SET
|
||||||
keywords = excluded.keywords,
|
keywords = excluded.keywords,
|
||||||
auto_download = excluded.auto_download,
|
auto_download = excluded.auto_download,
|
||||||
@@ -968,22 +1013,11 @@ impl Db {
|
|||||||
sub.auto_download.map(|v| v as i64),
|
sub.auto_download.map(|v| v as i64),
|
||||||
sub.allow_explicit.map(|v| v as i64),
|
sub.allow_explicit.map(|v| v as i64),
|
||||||
sub.max_new_per_check,
|
sub.max_new_per_check,
|
||||||
now()
|
|
||||||
],
|
],
|
||||||
)?;
|
)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Feeds with at least one subscriber. What the scanner walks.
|
|
||||||
pub fn subscribed_feed_ids(&self) -> Result<Vec<String>> {
|
|
||||||
let conn = self.conn.lock().unwrap();
|
|
||||||
let mut stmt = conn.prepare("SELECT DISTINCT feed_id FROM subscriptions")?;
|
|
||||||
let out = stmt
|
|
||||||
.query_map([], |r| r.get::<_, String>(0))?
|
|
||||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
|
||||||
Ok(out)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---- users and sessions ----
|
// ---- users and sessions ----
|
||||||
|
|
||||||
pub fn create_user(&self, name: &str, pass_hash: Option<&str>, admin: bool) -> Result<i64> {
|
pub fn create_user(&self, name: &str, pass_hash: Option<&str>, admin: bool) -> Result<i64> {
|
||||||
@@ -996,11 +1030,11 @@ impl Db {
|
|||||||
}
|
}
|
||||||
|
|
||||||
pub fn user_by_name(&self, name: &str) -> Result<Option<User>> {
|
pub fn user_by_name(&self, name: &str) -> Result<Option<User>> {
|
||||||
self.one_user("SELECT id, name, pass_hash, is_admin FROM users WHERE name = ?1", name)
|
self.one_user(&format!("SELECT {USER_COLS} FROM users WHERE name = ?1"), name)
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn user_by_id(&self, id: i64) -> Result<Option<User>> {
|
pub fn user_by_id(&self, id: i64) -> Result<Option<User>> {
|
||||||
self.one_user("SELECT id, name, pass_hash, is_admin FROM users WHERE id = ?1", id)
|
self.one_user(&format!("SELECT {USER_COLS} FROM users WHERE id = ?1"), id)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn one_user<P: rusqlite::ToSql>(&self, sql: &str, key: P) -> Result<Option<User>> {
|
fn one_user<P: rusqlite::ToSql>(&self, sql: &str, key: P) -> Result<Option<User>> {
|
||||||
@@ -1008,12 +1042,7 @@ impl Db {
|
|||||||
let mut stmt = conn.prepare(sql)?;
|
let mut stmt = conn.prepare(sql)?;
|
||||||
let mut rows = stmt.query(params![key])?;
|
let mut rows = stmt.query(params![key])?;
|
||||||
Ok(match rows.next()? {
|
Ok(match rows.next()? {
|
||||||
Some(r) => Some(User {
|
Some(r) => Some(user_row(r)?),
|
||||||
id: r.get(0)?,
|
|
||||||
name: r.get(1)?,
|
|
||||||
pass_hash: r.get(2)?,
|
|
||||||
is_admin: r.get::<_, i64>(3)? != 0,
|
|
||||||
}),
|
|
||||||
None => None,
|
None => None,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -1021,16 +1050,9 @@ impl Db {
|
|||||||
pub fn users(&self) -> Result<Vec<User>> {
|
pub fn users(&self) -> Result<Vec<User>> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let mut stmt =
|
let mut stmt =
|
||||||
conn.prepare("SELECT id, name, pass_hash, is_admin FROM users ORDER BY name")?;
|
conn.prepare(&format!("SELECT {USER_COLS} FROM users ORDER BY name"))?;
|
||||||
let out = stmt
|
let out = stmt
|
||||||
.query_map([], |r| {
|
.query_map([], user_row)?
|
||||||
Ok(User {
|
|
||||||
id: r.get(0)?,
|
|
||||||
name: r.get(1)?,
|
|
||||||
pass_hash: r.get(2)?,
|
|
||||||
is_admin: r.get::<_, i64>(3)? != 0,
|
|
||||||
})
|
|
||||||
})?
|
|
||||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||||
Ok(out)
|
Ok(out)
|
||||||
}
|
}
|
||||||
@@ -1047,6 +1069,26 @@ impl Db {
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The proxy signs people in by the name it vouches for, so an account made before the proxy
|
||||||
|
/// was set up has to take that name to be found by it. The name is UNIQUE, so a taken one is
|
||||||
|
/// refused here as well as by the caller.
|
||||||
|
pub fn rename_user(&self, id: i64, name: &str) -> Result<()> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
conn.execute("UPDATE users SET name = ?2 WHERE id = ?1", params![id, name])?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Records a sign-in, to the hour: the proxy vouches for every request, and writing each one
|
||||||
|
/// would buy nothing.
|
||||||
|
pub fn signed_in(&self, id: i64) -> Result<()> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE users SET last_login = ?2 WHERE id = ?1 AND coalesce(last_login, 0) <= ?2 - 3600",
|
||||||
|
params![id, now()],
|
||||||
|
)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
/// Sessions go with the user: a deleted account must not leave a usable cookie behind.
|
/// Sessions go with the user: a deleted account must not leave a usable cookie behind.
|
||||||
pub fn delete_user(&self, id: i64) -> Result<()> {
|
pub fn delete_user(&self, id: i64) -> Result<()> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
@@ -1058,7 +1100,7 @@ impl Db {
|
|||||||
pub fn create_session(&self, user_id: i64, token: &str) -> Result<()> {
|
pub fn create_session(&self, user_id: i64, token: &str) -> Result<()> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO sessions (token, user_id, created, seen) VALUES (?1, ?2, ?3, ?3)",
|
"INSERT INTO sessions (token, user_id, seen) VALUES (?1, ?2, ?3)",
|
||||||
params![token, user_id, now()],
|
params![token, user_id, now()],
|
||||||
)?;
|
)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
@@ -1070,18 +1112,13 @@ impl Db {
|
|||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let cutoff = now() - max_idle_secs;
|
let cutoff = now() - max_idle_secs;
|
||||||
let mut stmt = conn.prepare(
|
let mut stmt = conn.prepare(
|
||||||
"SELECT u.id, u.name, u.pass_hash, u.is_admin
|
"SELECT u.id, u.name, u.pass_hash, u.is_admin, u.created, u.last_login
|
||||||
FROM sessions s JOIN users u ON u.id = s.user_id
|
FROM sessions s JOIN users u ON u.id = s.user_id
|
||||||
WHERE s.token = ?1 AND s.seen >= ?2",
|
WHERE s.token = ?1 AND s.seen >= ?2",
|
||||||
)?;
|
)?;
|
||||||
let mut rows = stmt.query(params![token, cutoff])?;
|
let mut rows = stmt.query(params![token, cutoff])?;
|
||||||
let found = match rows.next()? {
|
let found = match rows.next()? {
|
||||||
Some(r) => Some(User {
|
Some(r) => Some(user_row(r)?),
|
||||||
id: r.get(0)?,
|
|
||||||
name: r.get(1)?,
|
|
||||||
pass_hash: r.get(2)?,
|
|
||||||
is_admin: r.get::<_, i64>(3)? != 0,
|
|
||||||
}),
|
|
||||||
None => None,
|
None => None,
|
||||||
};
|
};
|
||||||
drop(rows);
|
drop(rows);
|
||||||
@@ -1160,7 +1197,7 @@ impl Db {
|
|||||||
.optional()?)
|
.optional()?)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Read and starred, per person. The row is created on first touch.
|
/// Read and kept, per person. The row is created on first touch.
|
||||||
pub fn set_entry_flag(
|
pub fn set_entry_flag(
|
||||||
&self,
|
&self,
|
||||||
user_id: i64,
|
user_id: i64,
|
||||||
@@ -1235,7 +1272,7 @@ impl Db {
|
|||||||
pub fn managed_feeds(&self) -> Result<Vec<Managed>> {
|
pub fn managed_feeds(&self) -> Result<Vec<Managed>> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let mut stmt = conn.prepare(
|
let mut stmt = conn.prepare(
|
||||||
"SELECT id, url, title, group_id, orphaned FROM feeds
|
"SELECT id, url, title, group_id FROM feeds
|
||||||
WHERE managed = 1 AND group_id IS NOT NULL ORDER BY coalesce(title, id)",
|
WHERE managed = 1 AND group_id IS NOT NULL ORDER BY coalesce(title, id)",
|
||||||
)?;
|
)?;
|
||||||
Ok(stmt
|
Ok(stmt
|
||||||
@@ -1245,7 +1282,6 @@ impl Db {
|
|||||||
url: r.get(1)?,
|
url: r.get(1)?,
|
||||||
title: r.get(2)?,
|
title: r.get(2)?,
|
||||||
group_id: r.get(3)?,
|
group_id: r.get(3)?,
|
||||||
orphaned: r.get::<_, i64>(4)? != 0,
|
|
||||||
})
|
})
|
||||||
})?
|
})?
|
||||||
.collect::<rusqlite::Result<Vec<_>>>()?)
|
.collect::<rusqlite::Result<Vec<_>>>()?)
|
||||||
@@ -1267,6 +1303,71 @@ impl Db {
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Empties a feed of its items, leaving its files alone.
|
||||||
|
pub fn clear_entries(&self, feed_id: &str) -> Result<()> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
conn.execute("DELETE FROM entries WHERE feed_id = ?1", [feed_id])?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every feed the database holds rows for, as (id, url), removed ones included.
|
||||||
|
pub fn feed_urls(&self) -> Result<Vec<(String, String)>> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let mut stmt = conn.prepare("SELECT id, coalesce(url, '') FROM feeds")?;
|
||||||
|
let out = stmt
|
||||||
|
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))?
|
||||||
|
.collect::<rusqlite::Result<_>>()?;
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Hands a feed in a group the enclosures its parent holds, as (guid, url), with everyone's
|
||||||
|
/// read state for them. A Patreon creator read as one feed before it was split into shows
|
||||||
|
/// owns every show's files, and `enclosures.url` is unique, so without this each show would
|
||||||
|
/// list its items with nothing to play.
|
||||||
|
pub fn adopt(&self, parent: &str, child: &str, listed: &[(&str, &str)]) -> Result<()> {
|
||||||
|
let mut conn = self.conn.lock().unwrap();
|
||||||
|
let holds: bool = conn.query_row(
|
||||||
|
"SELECT EXISTS (SELECT 1 FROM enclosures WHERE feed_id = ?1)",
|
||||||
|
[parent],
|
||||||
|
|r| r.get(0),
|
||||||
|
)?;
|
||||||
|
if !holds {
|
||||||
|
return Ok(()); // An OPML, or a creator already shared out.
|
||||||
|
}
|
||||||
|
let tx = conn.transaction()?;
|
||||||
|
for &(guid, url) in listed {
|
||||||
|
let moved = tx.execute(
|
||||||
|
"UPDATE enclosures SET feed_id = ?3, guid = ?4 WHERE url = ?1 AND feed_id = ?2",
|
||||||
|
params![url, parent, child, guid],
|
||||||
|
)?;
|
||||||
|
if moved == 1 {
|
||||||
|
// Patreon gives a post the same guid in every feed it appears in.
|
||||||
|
tx.execute(
|
||||||
|
"UPDATE OR IGNORE entry_state SET feed_id = ?2 WHERE feed_id = ?1 AND guid = ?3",
|
||||||
|
params![parent, child, guid],
|
||||||
|
)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tx.commit()?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A feed's enclosures skipped by one of its filters, by URL, with the reason: the verdicts a
|
||||||
|
/// change of settings can overturn. A torrent held back while torrents are off is not a
|
||||||
|
/// filter's call.
|
||||||
|
pub fn skipped_by_filter(&self, feed_id: &str) -> Result<std::collections::HashMap<String, String>> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let mut stmt = conn.prepare(
|
||||||
|
"SELECT url, last_error FROM enclosures
|
||||||
|
WHERE feed_id = ?1 AND state = 'skipped' AND last_error IS NOT NULL
|
||||||
|
AND last_error != 'torrents disabled'",
|
||||||
|
)?;
|
||||||
|
let out = stmt
|
||||||
|
.query_map([feed_id], |r| Ok((r.get(0)?, r.get(1)?)))?
|
||||||
|
.collect::<rusqlite::Result<_>>()?;
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
pub fn set_orphaned(&self, feed_id: &str, on: bool) -> Result<()> {
|
pub fn set_orphaned(&self, feed_id: &str, on: bool) -> Result<()> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
conn.execute(
|
conn.execute(
|
||||||
@@ -1319,12 +1420,46 @@ pub fn now() -> i64 {
|
|||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn every_sort_column_runs_and_orders_both_ways() {
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
db.exec_for_test(
|
||||||
|
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1);
|
||||||
|
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'f'),(1,'g');
|
||||||
|
INSERT INTO feeds (id, url, title) VALUES ('f','u','Zebra'),('g','v','Aardvark');
|
||||||
|
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
|
||||||
|
('f','a','banana',100),('g','b','Apple',200),('f','c','cherry',300);
|
||||||
|
INSERT INTO enclosures (id, feed_id, guid, url, mime, length, state) VALUES
|
||||||
|
(1,'f','a','u1','audio/mpeg',300,'pending'),(2,'g','b','u2','image/png',10,'pending'),
|
||||||
|
(3,'f','c','u3','video/mp4',2000,'pending');",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let order = |col: &str, dir: &str| -> Vec<String> {
|
||||||
|
db.entries_in(1, None, Filter::All, None, 0, 50, &order_sql(col, dir))
|
||||||
|
.unwrap()
|
||||||
|
.into_iter()
|
||||||
|
.map(|e| e.guid)
|
||||||
|
.collect()
|
||||||
|
};
|
||||||
|
assert_eq!(order("title", "asc"), ["b", "a", "c"], "Apple, banana, cherry: case folded");
|
||||||
|
assert_eq!(order("title", "desc"), ["c", "a", "b"]);
|
||||||
|
assert_eq!(order("feed", "asc"), ["b", "c", "a"], "Aardvark, then Zebra's newest first");
|
||||||
|
assert_eq!(order("type", "asc"), ["a", "b", "c"], "audio, image, video");
|
||||||
|
assert_eq!(order("size", "desc"), ["c", "a", "b"]);
|
||||||
|
assert_eq!(order("published", "desc"), ["c", "b", "a"]);
|
||||||
|
db.set_entry_flag(1, "f", "a", EntryFlag::Flagged, true).unwrap();
|
||||||
|
assert_eq!(order("kept", "desc")[0], "a");
|
||||||
|
// An unknown column or direction is newest first; the name itself never reaches the SQL.
|
||||||
|
assert_eq!(order("title; DROP TABLE entries", "sideways"), ["c", "b", "a"]);
|
||||||
|
assert!(!order_sql("x'; --", "asc").contains("x'"));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn deleting_a_shared_file_asks_about_everyone_else() {
|
fn deleting_a_shared_file_asks_about_everyone_else() {
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0),(3,'kit',0,0);
|
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0),(3,'kit',0);
|
||||||
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0),(3,'f',0);
|
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'f'),(2,'f'),(3,'f');
|
||||||
INSERT INTO entries (feed_id, guid, first_seen) VALUES ('f','a',0);
|
INSERT INTO entries (feed_id, guid, first_seen) VALUES ('f','a',0);
|
||||||
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
||||||
(1,'f','a','u1','/tmp/a','done');",
|
(1,'f','a','u1','/tmp/a','done');",
|
||||||
@@ -1349,7 +1484,7 @@ mod tests {
|
|||||||
fn read_state_belongs_to_one_person() {
|
fn read_state_belongs_to_one_person() {
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
|
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0);
|
||||||
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
|
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
|
||||||
('f','a','One',100),('f','b','Two',200);",
|
('f','a','One',100),('f','b','Two',200);",
|
||||||
)
|
)
|
||||||
@@ -1365,8 +1500,9 @@ mod tests {
|
|||||||
// Starring and position are just as private.
|
// Starring and position are just as private.
|
||||||
db.set_entry_flag(1, "f", "b", EntryFlag::Flagged, true).unwrap();
|
db.set_entry_flag(1, "f", "b", EntryFlag::Flagged, true).unwrap();
|
||||||
db.set_position(2, "f", "b", 42).unwrap();
|
db.set_position(2, "f", "b", 42).unwrap();
|
||||||
let ray = db.entries(1, "f", Filter::All, None, 0, 50).unwrap();
|
let order = order_sql("published", "desc");
|
||||||
let sam = db.entries(2, "f", Filter::All, None, 0, 50).unwrap();
|
let page = |user| db.entries_in(user, Some("f"), Filter::All, None, 0, 50, &order).unwrap();
|
||||||
|
let (ray, sam) = (page(1), page(2));
|
||||||
let ray_b = ray.iter().find(|e| e.guid == "b").unwrap();
|
let ray_b = ray.iter().find(|e| e.guid == "b").unwrap();
|
||||||
let sam_b = sam.iter().find(|e| e.guid == "b").unwrap();
|
let sam_b = sam.iter().find(|e| e.guid == "b").unwrap();
|
||||||
assert!(ray_b.flagged && ray_b.position == 0);
|
assert!(ray_b.flagged && ray_b.position == 0);
|
||||||
@@ -1390,6 +1526,111 @@ mod tests {
|
|||||||
assert_eq!(sum.downloaded, 0);
|
assert_eq!(sum.downloaded, 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_old_database_loses_its_retired_columns() {
|
||||||
|
let conn = Connection::open_in_memory().unwrap();
|
||||||
|
// As open() has it: a DROP COLUMN on a table that references another is the part worth
|
||||||
|
// proving, and it has to work with the foreign keys switched on.
|
||||||
|
conn.pragma_update(None, "foreign_keys", "ON").unwrap();
|
||||||
|
conn.execute_batch(
|
||||||
|
"CREATE TABLE entries (feed_id TEXT NOT NULL, guid TEXT NOT NULL,
|
||||||
|
first_seen INTEGER NOT NULL, read INTEGER NOT NULL DEFAULT 0,
|
||||||
|
flagged INTEGER NOT NULL DEFAULT 0, position INTEGER NOT NULL DEFAULT 0,
|
||||||
|
PRIMARY KEY (feed_id, guid));
|
||||||
|
CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE COLLATE NOCASE,
|
||||||
|
pass_hash TEXT, is_admin INTEGER NOT NULL DEFAULT 0, created INTEGER NOT NULL);
|
||||||
|
CREATE TABLE subscriptions (
|
||||||
|
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
feed_id TEXT NOT NULL, created INTEGER NOT NULL, PRIMARY KEY (user_id, feed_id));
|
||||||
|
CREATE TABLE sessions (token TEXT PRIMARY KEY,
|
||||||
|
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
created INTEGER NOT NULL, seen INTEGER NOT NULL);
|
||||||
|
INSERT INTO users VALUES (1, 'ray', NULL, 1, 0);
|
||||||
|
INSERT INTO subscriptions VALUES (1, 'f', 0);
|
||||||
|
INSERT INTO sessions VALUES ('t', 1, 0, 0);",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
// The same order as open(): the schema leaves the old tables alone, migrate() fixes them.
|
||||||
|
conn.execute_batch(SCHEMA).unwrap();
|
||||||
|
migrate(&conn).unwrap();
|
||||||
|
let cols = |table: &str| -> Vec<String> {
|
||||||
|
conn.prepare(&format!("PRAGMA table_info({table})"))
|
||||||
|
.unwrap()
|
||||||
|
.query_map([], |r| r.get(1))
|
||||||
|
.unwrap()
|
||||||
|
.collect::<rusqlite::Result<_>>()
|
||||||
|
.unwrap()
|
||||||
|
};
|
||||||
|
for (table, gone) in [
|
||||||
|
("entries", &["read", "flagged", "position"][..]),
|
||||||
|
("subscriptions", &["created"][..]),
|
||||||
|
("sessions", &["created"][..]),
|
||||||
|
] {
|
||||||
|
let cols = cols(table);
|
||||||
|
assert!(!cols.iter().any(|c| gone.contains(&c.as_str())), "{table}: {cols:?}");
|
||||||
|
}
|
||||||
|
// users.created is not retired: it keeps what it held, and last_login joins it.
|
||||||
|
let users = cols("users");
|
||||||
|
assert!(users.iter().any(|c| c == "last_login"), "{users:?}");
|
||||||
|
assert_eq!(conn.query_row("SELECT created FROM users", [], |r| r.get::<_, i64>(0)).unwrap(), 0);
|
||||||
|
// And the rows come through it.
|
||||||
|
let kept: i64 = conn
|
||||||
|
.query_row("SELECT count(*) FROM subscriptions JOIN sessions USING (user_id)", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(kept, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_renamed_account_keeps_everything_but_its_name() {
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
let ray = db.create_user("rays", None, true).unwrap();
|
||||||
|
db.create_user("sam", None, false).unwrap();
|
||||||
|
db.subscribe(ray, "f").unwrap();
|
||||||
|
db.rename_user(ray, "rays@sdf1.net").unwrap();
|
||||||
|
assert!(db.user_by_name("rays").unwrap().is_none());
|
||||||
|
let renamed = db.user_by_name("RAYS@sdf1.net").unwrap().unwrap();
|
||||||
|
assert_eq!((renamed.id, renamed.is_admin), (ray, true), "same account, still the admin");
|
||||||
|
assert_eq!(db.subscriptions_for(ray).unwrap().len(), 1, "and still subscribed");
|
||||||
|
assert!(db.rename_user(ray, "sam").is_err(), "a taken name is refused");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_account_knows_when_it_was_made_and_last_signed_in() {
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
let id = db.create_user("ray", None, true).unwrap();
|
||||||
|
let get = || db.user_by_id(id).unwrap().unwrap();
|
||||||
|
assert!(get().created.is_some_and(|t| t > 0));
|
||||||
|
assert_eq!(get().last_login, None, "made, but never signed in");
|
||||||
|
db.signed_in(id).unwrap();
|
||||||
|
let first = get().last_login.unwrap();
|
||||||
|
// Within the hour, the proxy vouching again writes nothing; after it, it does.
|
||||||
|
db.exec_for_test(&format!("UPDATE users SET last_login = {} WHERE id = {id}", first - 60)).unwrap();
|
||||||
|
db.signed_in(id).unwrap();
|
||||||
|
assert_eq!(get().last_login, Some(first - 60));
|
||||||
|
db.exec_for_test(&format!("UPDATE users SET last_login = {} WHERE id = {id}", first - 7200)).unwrap();
|
||||||
|
db.signed_in(id).unwrap();
|
||||||
|
assert!(get().last_login.unwrap() >= first);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_first_admin_starts_with_the_catalogue_and_only_once() {
|
||||||
|
// Cutting this along with the dead read columns left the browser suite's admin with an
|
||||||
|
// empty sidebar: it is how a fresh install's first account gets config.toml's feeds.
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
db.exec_for_test("INSERT INTO users (id, name, is_admin) VALUES (1,'admin',1);")
|
||||||
|
.unwrap();
|
||||||
|
let subs = || -> i64 {
|
||||||
|
db.conn.lock().unwrap().query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0)).unwrap()
|
||||||
|
};
|
||||||
|
let catalogue = ["a".to_string(), "b".to_string()];
|
||||||
|
assert_eq!(db.adopt_catalogue(1, &catalogue).unwrap(), 2);
|
||||||
|
assert_eq!(subs(), 2);
|
||||||
|
// Once anyone subscribes to anything it never runs again, so an unsubscribe sticks.
|
||||||
|
db.unsubscribe(1, "a").unwrap();
|
||||||
|
assert_eq!(db.adopt_catalogue(1, &catalogue).unwrap(), 0);
|
||||||
|
assert_eq!(subs(), 1);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn every_filter_works_with_and_without_a_search_term() {
|
fn every_filter_works_with_and_without_a_search_term() {
|
||||||
// Regression: the search clause used to be omitted when no term was given, while
|
// Regression: the search clause used to be omitted when no term was given, while
|
||||||
@@ -1404,7 +1645,7 @@ mod tests {
|
|||||||
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
||||||
(1,'f','b','u1','/tmp/b','done');
|
(1,'f','b','u1','/tmp/b','done');
|
||||||
-- Read and starred belong to a person now, so say which one.
|
-- Read and starred belong to a person now, so say which one.
|
||||||
INSERT INTO users (id, name, is_admin, created) VALUES (7,'reader',1,0);
|
INSERT INTO users (id, name, is_admin) VALUES (7,'reader',1);
|
||||||
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
|
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
|
||||||
(7,'f','b',1,0),
|
(7,'f','b',1,0),
|
||||||
(7,'f','c',1,1);",
|
(7,'f','c',1,1);",
|
||||||
@@ -1413,21 +1654,22 @@ mod tests {
|
|||||||
|
|
||||||
for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] {
|
for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] {
|
||||||
// Both paths must run without erroring, and agree with each other.
|
// Both paths must run without erroring, and agree with each other.
|
||||||
let rows = db.entries(7, "f", f, None, 0, 50).unwrap();
|
let order = order_sql("published", "desc");
|
||||||
let n = db.count_entries(7, "f", f, None).unwrap();
|
let rows = db.entries_in(7, Some("f"), f, None, 0, 50, &order).unwrap();
|
||||||
|
let n = db.count_in(7, Some("f"), f, None).unwrap();
|
||||||
assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page");
|
assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page");
|
||||||
|
|
||||||
let rows = db.entries(7, "f", f, Some("dive"), 0, 50).unwrap();
|
let rows = db.entries_in(7, Some("f"), f, Some("dive"), 0, 50, &order).unwrap();
|
||||||
let n = db.count_entries(7, "f", f, Some("dive")).unwrap();
|
let n = db.count_in(7, Some("f"), f, Some("dive")).unwrap();
|
||||||
assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees");
|
assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees");
|
||||||
}
|
}
|
||||||
|
|
||||||
assert_eq!(db.count_entries(7, "f", Filter::All, None).unwrap(), 3);
|
assert_eq!(db.count_in(7, Some("f"), Filter::All, None).unwrap(), 3);
|
||||||
assert_eq!(db.count_entries(7, "f", Filter::Unread, None).unwrap(), 1);
|
assert_eq!(db.count_in(7, Some("f"), Filter::Unread, None).unwrap(), 1);
|
||||||
assert_eq!(db.count_entries(7, "f", Filter::Downloaded, None).unwrap(), 1);
|
assert_eq!(db.count_in(7, Some("f"), Filter::Downloaded, None).unwrap(), 1);
|
||||||
assert_eq!(db.count_entries(7, "f", Filter::Flagged, None).unwrap(), 1);
|
assert_eq!(db.count_in(7, Some("f"), Filter::Flagged, None).unwrap(), 1);
|
||||||
assert_eq!(db.count_entries(7, "f", Filter::All, Some("dive")).unwrap(), 2);
|
assert_eq!(db.count_in(7, Some("f"), Filter::All, Some("dive")).unwrap(), 2);
|
||||||
assert_eq!(db.count_entries(7, "f", Filter::All, Some("NOTES two")).unwrap(), 1,
|
assert_eq!(db.count_in(7, Some("f"), Filter::All, Some("NOTES two")).unwrap(), 1,
|
||||||
"search is case-insensitive and covers the description");
|
"search is case-insensitive and covers the description");
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1469,6 +1711,61 @@ mod tests {
|
|||||||
assert_eq!(state(4), "done");
|
assert_eq!(state(4), "done");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_show_takes_over_what_its_creator_held() {
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
db.exec_for_test(
|
||||||
|
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1);
|
||||||
|
INSERT INTO enclosures (id, feed_id, guid, url, state, path, last_error) VALUES
|
||||||
|
(1,'creator','a','u1','done','/x/a.mp3',NULL),
|
||||||
|
(2,'creator','b','u2','skipped',NULL,'explicit'),
|
||||||
|
(3,'creator','c','u3','skipped',NULL,'explicit'),
|
||||||
|
(4,'other','d','u4','skipped',NULL,'torrents disabled');
|
||||||
|
INSERT INTO entry_state (user_id, feed_id, guid, read) VALUES (1,'creator','a',1);",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
db.adopt("creator", "show", &[("a", "u1"), ("b", "u2"), ("d", "u4")]).unwrap();
|
||||||
|
{
|
||||||
|
let conn = db.conn.lock().unwrap();
|
||||||
|
let owner = |id: i64| -> String {
|
||||||
|
conn.query_row("SELECT feed_id FROM enclosures WHERE id = ?1", [id], |r| r.get(0)).unwrap()
|
||||||
|
};
|
||||||
|
assert_eq!(owner(1), "show", "a downloaded file moves with its item");
|
||||||
|
assert_eq!(owner(2), "show");
|
||||||
|
assert_eq!(owner(3), "creator", "this show does not list it");
|
||||||
|
assert_eq!(owner(4), "other", "only the parent's are taken");
|
||||||
|
let read: String = conn
|
||||||
|
.query_row("SELECT feed_id FROM entry_state WHERE user_id = 1 AND guid = 'a'", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(read, "show", "what you had read stays read");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only a filter's verdict can be overturned by a change of settings.
|
||||||
|
let skipped = db.skipped_by_filter("show").unwrap();
|
||||||
|
assert_eq!(skipped.get("u2").map(String::as_str), Some("explicit"));
|
||||||
|
assert_eq!(skipped.len(), 1);
|
||||||
|
assert!(db.skipped_by_filter("other").unwrap().is_empty(), "torrents disabled is not a filter");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_feed_in_a_group_follows_your_settings_on_the_group() {
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
db.exec_for_test(
|
||||||
|
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0);
|
||||||
|
INSERT INTO subscriptions (user_id, feed_id, allow_explicit) VALUES
|
||||||
|
(1,'group',1),(1,'show',NULL),(2,'group',1),(2,'show',0);",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let explicit = |group| -> Vec<Option<bool>> {
|
||||||
|
let mut v: Vec<_> =
|
||||||
|
db.subscribers("show", group).unwrap().into_iter().map(|s| s.allow_explicit).collect();
|
||||||
|
v.sort();
|
||||||
|
v
|
||||||
|
};
|
||||||
|
assert_eq!(explicit(Some("group")), [Some(false), Some(true)], "ray inherits; sam's own choice on the show wins");
|
||||||
|
assert_eq!(explicit(None), [None, Some(false)], "outside a group nothing is inherited");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn enclosure_url_is_the_dedupe_key() {
|
fn enclosure_url_is_the_dedupe_key() {
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
|
|||||||
@@ -223,7 +223,7 @@ enum Sniffed {
|
|||||||
/// 2008 and so always answered 'data'.
|
/// 2008 and so always answered 'data'.
|
||||||
async fn sniff(path: &Path) -> Result<Sniffed> {
|
async fn sniff(path: &Path) -> Result<Sniffed> {
|
||||||
let head = read_head(path, 512).await?;
|
let head = read_head(path, 512).await?;
|
||||||
if infer::is(&head, "torrent") || head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
|
if head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
|
||||||
return Ok(Sniffed::Torrent);
|
return Ok(Sniffed::Torrent);
|
||||||
}
|
}
|
||||||
let text = String::from_utf8_lossy(&head);
|
let text = String::from_utf8_lossy(&head);
|
||||||
|
|||||||
175
src/feed.rs
175
src/feed.rs
@@ -117,6 +117,103 @@ pub fn opml_title(bytes: &[u8]) -> Option<String> {
|
|||||||
.filter(|t| !t.is_empty())
|
.filter(|t| !t.is_empty())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The token and show of a Patreon feed link, or None for any other URL.
|
||||||
|
///
|
||||||
|
/// Patreon gives each patron one token per creator. With no show it stands for the creator,
|
||||||
|
/// whose feed carries every show at once.
|
||||||
|
fn patreon_parts(url: &str) -> Option<(String, Option<String>)> {
|
||||||
|
let u = url::Url::parse(url).ok()?;
|
||||||
|
if !matches!(u.host_str()?, "patreon.com" | "www.patreon.com") || !u.path().starts_with("/rss") {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let param = |name: &str| u.query_pairs().find(|(k, _)| k == name).map(|(_, v)| v.into_owned());
|
||||||
|
Some((param("auth")?, param("show")))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A Patreon link naming a creator but no show.
|
||||||
|
pub fn is_patreon_creator(url: &str) -> bool {
|
||||||
|
matches!(patreon_parts(url), Some((_, None)))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What was typed into Add feed, as a URL. A bare Patreon token is taken as its creator's
|
||||||
|
/// feed, since the token alone says whose it is.
|
||||||
|
pub fn expand_input(input: &str) -> String {
|
||||||
|
let s = input.trim();
|
||||||
|
let token = s.len() >= 20 && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_');
|
||||||
|
if token { format!("https://www.patreon.com/rss?auth={s}") } else { s.to_owned() }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether two URLs are the same feed. One Patreon show has several spellings -- by the
|
||||||
|
/// creator's name, by number, or with no creator at all -- and the token and show are what
|
||||||
|
/// identify it.
|
||||||
|
pub fn same_feed(a: &str, b: &str) -> bool {
|
||||||
|
a == b || patreon_parts(a).is_some_and(|p| Some(p) == patreon_parts(b))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A Patreon creator's name and shows, each show as (title, feed URL).
|
||||||
|
///
|
||||||
|
/// ponytail: Patreon's own web API, undocumented, asked without signing in. If it changes,
|
||||||
|
/// finding shows stops and the show feeds already found keep working. The documented API
|
||||||
|
/// needs an OAuth client per install and does not list shows.
|
||||||
|
pub async fn patreon_shows(
|
||||||
|
client: &reqwest::Client,
|
||||||
|
url: &str,
|
||||||
|
) -> Result<(Option<String>, Vec<(String, String)>)> {
|
||||||
|
// The creator feed names its campaign by number in its self link, a few hundred bytes in.
|
||||||
|
// The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
|
||||||
|
let mut resp = client.get(url).send().await.context("connecting")?;
|
||||||
|
if !resp.status().is_success() {
|
||||||
|
return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
|
||||||
|
}
|
||||||
|
let mut head = Vec::new();
|
||||||
|
while patreon_campaign(&head).is_none() && head.len() < 64 * 1024 {
|
||||||
|
let Some(chunk) = resp.chunk().await.context("reading the feed")? else { break };
|
||||||
|
head.extend_from_slice(&chunk);
|
||||||
|
}
|
||||||
|
let campaign = patreon_campaign(&head)
|
||||||
|
.ok_or_else(|| anyhow!("the Patreon feed does not say whose it is"))?;
|
||||||
|
|
||||||
|
let api = format!(
|
||||||
|
"https://www.patreon.com/api/campaigns/{campaign}\
|
||||||
|
?include=shows&fields%5Bcampaign%5D=name&fields%5Bcollection%5D=title"
|
||||||
|
);
|
||||||
|
let resp = client.get(api).send().await.context("asking Patreon for the shows")?;
|
||||||
|
if !resp.status().is_success() {
|
||||||
|
return Err(anyhow!("Patreon would not list the shows: HTTP {}", resp.status()));
|
||||||
|
}
|
||||||
|
let (name, shows) = parse_patreon_shows(&resp.bytes().await.context("reading the shows")?)?;
|
||||||
|
Ok((name, shows.into_iter().map(|(id, title)| (title, format!("{url}&show={id}"))).collect()))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The campaign number in the start of a Patreon feed.
|
||||||
|
fn patreon_campaign(head: &[u8]) -> Option<String> {
|
||||||
|
let text = String::from_utf8_lossy(head);
|
||||||
|
text.match_indices("patreon.com/rss/").find_map(|(i, m)| {
|
||||||
|
let id: String = text[i + m.len()..].chars().take_while(char::is_ascii_digit).collect();
|
||||||
|
(!id.is_empty()).then_some(id)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A campaign's name and its shows as (id, title), from Patreon's JSON:API answer.
|
||||||
|
fn parse_patreon_shows(json: &[u8]) -> Result<(Option<String>, Vec<(String, String)>)> {
|
||||||
|
let v: serde_json::Value = serde_json::from_slice(json).context("Patreon's answer is not JSON")?;
|
||||||
|
// Missing is not the same as none. Read as no shows, the creator feed would be scanned as
|
||||||
|
// a plain feed, claim every show's files, and leave the shows empty once the list returned.
|
||||||
|
let ids = v["data"]["relationships"]["shows"]["data"]
|
||||||
|
.as_array()
|
||||||
|
.ok_or_else(|| anyhow!("Patreon's answer does not list the shows"))?;
|
||||||
|
let title = |id: &str| -> Option<String> {
|
||||||
|
let show = v["included"].as_array()?.iter().find(|x| x["type"] == "collection" && x["id"] == id)?;
|
||||||
|
show["attributes"]["title"].as_str().map(|t| t.trim().to_owned())
|
||||||
|
};
|
||||||
|
let shows = ids
|
||||||
|
.iter()
|
||||||
|
.filter_map(|s| s["id"].as_str())
|
||||||
|
.map(|id| (id.to_owned(), title(id).unwrap_or_else(|| format!("Show {id}"))))
|
||||||
|
.collect();
|
||||||
|
Ok((v["data"]["attributes"]["name"].as_str().map(str::to_owned), shows))
|
||||||
|
}
|
||||||
|
|
||||||
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
|
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
|
||||||
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
|
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
|
||||||
match rss::Channel::read_from(bytes) {
|
match rss::Channel::read_from(bytes) {
|
||||||
@@ -243,7 +340,7 @@ fn from_rss(ch: rss::Channel, bytes: &[u8]) -> ParsedFeed {
|
|||||||
link: non_empty(item.link()),
|
link: non_empty(item.link()),
|
||||||
published: item.pub_date().and_then(parse_date),
|
published: item.pub_date().and_then(parse_date),
|
||||||
// Content wins over description, as __getEntries preferred entry.content.
|
// Content wins over description, as __getEntries preferred entry.content.
|
||||||
description: non_empty(item.content()).or_else(|| non_empty(item.description())),
|
description: body(item.content(), item.description()),
|
||||||
categories: item
|
categories: item
|
||||||
.categories()
|
.categories()
|
||||||
.iter()
|
.iter()
|
||||||
@@ -309,11 +406,7 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
|
|||||||
title: non_empty(Some(e.title().as_str())),
|
title: non_empty(Some(e.title().as_str())),
|
||||||
link: alt.map(str::to_owned),
|
link: alt.map(str::to_owned),
|
||||||
published: e.published().or(Some(e.updated())).map(|d| d.timestamp()),
|
published: e.published().or(Some(e.updated())).map(|d| d.timestamp()),
|
||||||
description: e
|
description: body(e.content().and_then(|c| c.value()), e.summary().map(|s| s.as_str())),
|
||||||
.content()
|
|
||||||
.and_then(|c| c.value())
|
|
||||||
.or_else(|| e.summary().map(|s| s.as_str()))
|
|
||||||
.map(str::to_owned),
|
|
||||||
categories: e.categories().iter().map(|c| c.term().to_owned()).collect(),
|
categories: e.categories().iter().map(|c| c.term().to_owned()).collect(),
|
||||||
explicit: false,
|
explicit: false,
|
||||||
image: None,
|
image: None,
|
||||||
@@ -358,6 +451,25 @@ fn non_empty(s: Option<&str>) -> Option<String> {
|
|||||||
s.map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned)
|
s.map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// An item's show notes: its full body when that is whole, else its description.
|
||||||
|
///
|
||||||
|
/// libsyn served Daily Meditation Podcast's `content:encoded` cut at the `>` inside a class name
|
||||||
|
/// pasted from a web app (`[&:has([data-writing-block])>*]:pointer-events-auto`), so the body
|
||||||
|
/// began halfway through a tag and the page showed the rest of the tag as text. The same item's
|
||||||
|
/// `description` was whole. With no description to fall back on, a damaged body beats none.
|
||||||
|
fn body(content: Option<&str>, description: Option<&str>) -> Option<String> {
|
||||||
|
non_empty(content)
|
||||||
|
.filter(|c| !starts_mid_tag(c))
|
||||||
|
.or_else(|| non_empty(description))
|
||||||
|
.or_else(|| non_empty(content))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Text that closes an attribute list (`">`) before any tag has opened is the tail of a tag whose
|
||||||
|
/// start was cut off.
|
||||||
|
fn starts_mid_tag(html: &str) -> bool {
|
||||||
|
html[..html.find('<').unwrap_or(html.len())].contains("\">")
|
||||||
|
}
|
||||||
|
|
||||||
/// The picture to show beside an item, in order of how deliberate it is:
|
/// The picture to show beside an item, in order of how deliberate it is:
|
||||||
/// `itunes:image`, then Media RSS `media:thumbnail`, then a `media:content` that is an
|
/// `itunes:image`, then Media RSS `media:thumbnail`, then a `media:content` that is an
|
||||||
/// image, and finally an image enclosure -- which is how a blog's article picture arrives
|
/// image, and finally an image enclosure -- which is how a blog's article picture arrives
|
||||||
@@ -451,6 +563,18 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_body_cut_off_mid_tag_gives_way_to_the_description() {
|
||||||
|
// How libsyn served Daily Meditation Podcast #3477: content:encoded began inside a tag.
|
||||||
|
let cut = r#"*]:pointer-events-auto R6Vx5W_threadScrollVars" dir="auto" data-turn="assistant"> <p>What if</p>"#;
|
||||||
|
let whole = r#"<div class="[&:has([data-writing-block])>*]:pointer-events-auto"><p>What if</p></div>"#;
|
||||||
|
assert_eq!(body(Some(cut), Some(whole)).as_deref(), Some(whole));
|
||||||
|
assert_eq!(body(Some("<p>Notes</p>"), Some("Summary")).as_deref(), Some("<p>Notes</p>"), "a whole body wins");
|
||||||
|
assert_eq!(body(Some("Plain notes, no tags."), Some("Summary")).as_deref(), Some("Plain notes, no tags."));
|
||||||
|
assert_eq!(body(Some(cut), None).as_deref(), Some(cut), "a damaged body beats none");
|
||||||
|
assert_eq!(body(None, Some("Summary")).as_deref(), Some("Summary"));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn feed_level_explicit_overrides_entries() {
|
fn feed_level_explicit_overrides_entries() {
|
||||||
let xml = br#"<?xml version="1.0"?>
|
let xml = br#"<?xml version="1.0"?>
|
||||||
@@ -550,6 +674,45 @@ mod tests {
|
|||||||
assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
|
assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_patreon_creator_is_a_list_of_its_shows() {
|
||||||
|
let tok = "AbCdEfGhIjKlMnOpQrStUvWxYz012_-9";
|
||||||
|
assert_eq!(expand_input(&format!(" {tok} ")), format!("https://www.patreon.com/rss?auth={tok}"));
|
||||||
|
assert_eq!(expand_input("https://example.com/rss"), "https://example.com/rss");
|
||||||
|
|
||||||
|
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss/glasscannon?auth={tok}")));
|
||||||
|
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss?auth={tok}")));
|
||||||
|
assert!(!is_patreon_creator(&format!("https://www.patreon.com/rss/x?auth={tok}&show=1")), "one show is a feed");
|
||||||
|
assert!(!is_patreon_creator(&format!("https://example.com/rss?auth={tok}")));
|
||||||
|
|
||||||
|
// The show you already have by name is the one a bare token would add by number.
|
||||||
|
assert!(same_feed(
|
||||||
|
&format!("https://www.patreon.com/rss/glasscannon?auth={tok}&show=2073588"),
|
||||||
|
&format!("https://www.patreon.com/rss?auth={tok}&show=2073588"),
|
||||||
|
));
|
||||||
|
assert!(!same_feed(
|
||||||
|
&format!("https://www.patreon.com/rss?auth={tok}&show=1"),
|
||||||
|
&format!("https://www.patreon.com/rss?auth={tok}&show=2"),
|
||||||
|
));
|
||||||
|
|
||||||
|
// The self link carries the campaign by number, whichever spelling was asked for.
|
||||||
|
let head = br#"<rss><channel><link>https://www.patreon.com/glasscannon</link>
|
||||||
|
<atom:link href="https://www.patreon.com/rss/369921?auth=t" rel="self"/>"#;
|
||||||
|
assert_eq!(patreon_campaign(head).as_deref(), Some("369921"));
|
||||||
|
assert_eq!(patreon_campaign(b"<rss><channel><title>T"), None);
|
||||||
|
|
||||||
|
let json = br#"{"data":{"id":"369921","type":"campaign","attributes":{"name":"The Glass Cannon Network"},
|
||||||
|
"relationships":{"shows":{"data":[{"id":"2073588","type":"collection"},{"id":"2073636","type":"collection"}]}}},
|
||||||
|
"included":[{"id":"2073588","type":"collection","attributes":{"title":"Get in the Trunk "}},
|
||||||
|
{"id":"2073636","type":"collection","attributes":{"title":"Shadowdark"}}]}"#;
|
||||||
|
let (name, shows) = parse_patreon_shows(json).unwrap();
|
||||||
|
assert_eq!(name.as_deref(), Some("The Glass Cannon Network"));
|
||||||
|
assert_eq!(shows, [("2073588".into(), "Get in the Trunk".into()), ("2073636".into(), "Shadowdark".into())]);
|
||||||
|
|
||||||
|
// An answer that stops naming the shows is an error, never "this creator has none".
|
||||||
|
assert!(parse_patreon_shows(br#"{"data":{"attributes":{"name":"X"}}}"#).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn an_item_may_carry_several_enclosures() {
|
fn an_item_may_carry_several_enclosures() {
|
||||||
// The rss crate keeps only one per item -- the last -- so these come from the XML.
|
// The rss crate keeps only one per item -- the last -- so these come from the XML.
|
||||||
|
|||||||
61
src/ipc.rs
61
src/ipc.rs
@@ -62,10 +62,10 @@ impl Event {
|
|||||||
Event::DownloadDone { path, .. } => format!(" saved {path}"),
|
Event::DownloadDone { path, .. } => format!(" saved {path}"),
|
||||||
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
|
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
|
||||||
Event::Reaped { path, bytes } => {
|
Event::Reaped { path, bytes } => {
|
||||||
format!("reap {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
|
format!("deleted {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
|
||||||
}
|
}
|
||||||
Event::ReapDone { files, bytes } => format!(
|
Event::ReapDone { files, bytes } => format!(
|
||||||
"reaped {files} file(s), {:.1} MB",
|
"deleted {files} old file(s), {:.1} MB",
|
||||||
*bytes as f64 / 1_048_576.0
|
*bytes as f64 / 1_048_576.0
|
||||||
),
|
),
|
||||||
Event::Status { feeds, pending, downloaded } => {
|
Event::Status { feeds, pending, downloaded } => {
|
||||||
@@ -172,11 +172,18 @@ pub async fn daemon_is_live(path: &Path) -> bool {
|
|||||||
UnixStream::connect(path).await.is_ok()
|
UnixStream::connect(path).await.is_ok()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Answers `status` for the socket, without the worker. The worker runs one job at a time, and a
|
||||||
|
/// healthcheck left waiting behind a scan or a long download timed out and called a busy daemon
|
||||||
|
/// dead. The answer goes to the client that asked and no one else: broadcast, it ended any
|
||||||
|
/// `ipx fetch` that was watching a scan, since `status` is a terminal event.
|
||||||
|
pub type StatusFn = std::sync::Arc<dyn Fn() -> Event + Send + Sync>;
|
||||||
|
|
||||||
/// Accepts connections, feeding commands to `cmds` and events from `events` back out.
|
/// Accepts connections, feeding commands to `cmds` and events from `events` back out.
|
||||||
pub async fn serve(
|
pub async fn serve(
|
||||||
path: PathBuf,
|
path: PathBuf,
|
||||||
events: broadcast::Sender<Event>,
|
events: broadcast::Sender<Event>,
|
||||||
cmds: mpsc::Sender<Command>,
|
cmds: mpsc::Sender<Command>,
|
||||||
|
status: StatusFn,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
// A socket file left by a crashed daemon would block the bind; a live one was already
|
// A socket file left by a crashed daemon would block the bind; a live one was already
|
||||||
// rejected by the caller's daemon_is_live() check.
|
// rejected by the caller's daemon_is_live() check.
|
||||||
@@ -195,8 +202,9 @@ pub async fn serve(
|
|||||||
let (stream, _) = listener.accept().await?;
|
let (stream, _) = listener.accept().await?;
|
||||||
let rx = events.subscribe();
|
let rx = events.subscribe();
|
||||||
let cmds = cmds.clone();
|
let cmds = cmds.clone();
|
||||||
|
let status = status.clone();
|
||||||
tokio::spawn(async move {
|
tokio::spawn(async move {
|
||||||
if let Err(e) = handle(stream, rx, cmds).await {
|
if let Err(e) = handle(stream, rx, cmds, status).await {
|
||||||
tracing::debug!(error = %e, "client gone");
|
tracing::debug!(error = %e, "client gone");
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -207,12 +215,21 @@ async fn handle(
|
|||||||
stream: UnixStream,
|
stream: UnixStream,
|
||||||
mut rx: broadcast::Receiver<Event>,
|
mut rx: broadcast::Receiver<Event>,
|
||||||
cmds: mpsc::Sender<Command>,
|
cmds: mpsc::Sender<Command>,
|
||||||
|
status: StatusFn,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
let (read, mut write) = stream.into_split();
|
let (read, mut write) = stream.into_split();
|
||||||
|
|
||||||
// Events out.
|
// Events out: everything broadcast, and the answers meant for this client alone.
|
||||||
|
let (reply, mut replies) = mpsc::channel::<Event>(4);
|
||||||
let writer = tokio::spawn(async move {
|
let writer = tokio::spawn(async move {
|
||||||
while let Ok(ev) = rx.recv().await {
|
loop {
|
||||||
|
let ev = tokio::select! {
|
||||||
|
Some(ev) = replies.recv() => ev,
|
||||||
|
got = rx.recv() => match got {
|
||||||
|
Ok(ev) => ev,
|
||||||
|
Err(_) => break,
|
||||||
|
},
|
||||||
|
};
|
||||||
let mut line = serde_json::to_string(&ev).unwrap_or_default();
|
let mut line = serde_json::to_string(&ev).unwrap_or_default();
|
||||||
line.push('\n');
|
line.push('\n');
|
||||||
if write.write_all(line.as_bytes()).await.is_err() {
|
if write.write_all(line.as_bytes()).await.is_err() {
|
||||||
@@ -229,6 +246,15 @@ async fn handle(
|
|||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
match serde_json::from_str::<Command>(line) {
|
match serde_json::from_str::<Command>(line) {
|
||||||
|
// Answered here, not queued behind whatever the worker is on: see StatusFn.
|
||||||
|
Ok(Command::Status) => {
|
||||||
|
tracing::info!(target: "ipx::io", "-> {line}");
|
||||||
|
let ev = status();
|
||||||
|
if let Ok(json) = serde_json::to_string(&ev) {
|
||||||
|
tracing::info!(target: "ipx::io", "<- {json}");
|
||||||
|
}
|
||||||
|
let _ = reply.send(ev).await;
|
||||||
|
}
|
||||||
Ok(cmd) => {
|
Ok(cmd) => {
|
||||||
if cmds.send(cmd).await.is_err() {
|
if cmds.send(cmd).await.is_err() {
|
||||||
break; // Worker is gone; so are we.
|
break; // Worker is gone; so are we.
|
||||||
@@ -329,4 +355,29 @@ mod tests {
|
|||||||
}
|
}
|
||||||
.is_terminal());
|
.is_terminal());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn status_is_answered_while_the_worker_is_busy() {
|
||||||
|
// The queue is full and nobody drains it, as when the worker is deep in a long download:
|
||||||
|
// anything sent to it would wait for ever.
|
||||||
|
let (cmds, _worker) = mpsc::channel::<Command>(1);
|
||||||
|
cmds.send(Command::Reap { dry_run: true }).await.unwrap();
|
||||||
|
let (events, _) = broadcast::channel::<Event>(8);
|
||||||
|
// Another client, watching a scan: it must not be handed someone else's answer, which
|
||||||
|
// would end its session.
|
||||||
|
let mut watcher = events.subscribe();
|
||||||
|
let status: StatusFn = std::sync::Arc::new(|| Event::Status { feeds: 1, pending: 2, downloaded: 3 });
|
||||||
|
let (client, server) = UnixStream::pair().unwrap();
|
||||||
|
tokio::spawn(handle(server, events.subscribe(), cmds, status));
|
||||||
|
|
||||||
|
let (read, mut write) = client.into_split();
|
||||||
|
write.write_all(b"{\"cmd\":\"status\"}\n").await.unwrap();
|
||||||
|
let line = tokio::time::timeout(std::time::Duration::from_secs(2), BufReader::new(read).lines().next_line())
|
||||||
|
.await
|
||||||
|
.expect("status waited behind the worker")
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
assert!(line.contains(r#""ev":"status""#), "{line}");
|
||||||
|
assert!(watcher.try_recv().is_err(), "the answer went to every client, not just the one asking");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -111,18 +111,11 @@ impl Visit for Collect {
|
|||||||
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
|
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
|
||||||
self.add(field, format!("{value:?}"));
|
self.add(field, format!("{value:?}"));
|
||||||
}
|
}
|
||||||
|
// Numbers and bools reach record_debug through the trait's defaults, which prints them the
|
||||||
|
// same way. A string would print quoted there, hence its own method.
|
||||||
fn record_str(&mut self, field: &Field, value: &str) {
|
fn record_str(&mut self, field: &Field, value: &str) {
|
||||||
self.add(field, value.to_owned());
|
self.add(field, value.to_owned());
|
||||||
}
|
}
|
||||||
fn record_i64(&mut self, field: &Field, value: i64) {
|
|
||||||
self.add(field, value.to_string());
|
|
||||||
}
|
|
||||||
fn record_u64(&mut self, field: &Field, value: u64) {
|
|
||||||
self.add(field, value.to_string());
|
|
||||||
}
|
|
||||||
fn record_bool(&mut self, field: &Field, value: bool) {
|
|
||||||
self.add(field, value.to_string());
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
|
|||||||
202
src/main.rs
202
src/main.rs
@@ -99,6 +99,9 @@ enum UserCmd {
|
|||||||
Passwd { name: String },
|
Passwd { name: String },
|
||||||
/// Delete an account and everything it knows: its subscriptions and read state
|
/// Delete an account and everything it knows: its subscriptions and read state
|
||||||
Rm { name: String },
|
Rm { name: String },
|
||||||
|
/// Rename an account, keeping its feeds, read state and admin rights. This is how an
|
||||||
|
/// account made before the proxy takes the name the proxy signs it in as
|
||||||
|
Rename { name: String, new_name: String },
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What a brand new database starts with, so there is always a way in. Announced loudly
|
/// What a brand new database starts with, so there is always a way in. Announced loudly
|
||||||
@@ -273,11 +276,16 @@ fn user_cmd(ctx: &Arc<Ctx>, cmd: UserCmd) -> Result<()> {
|
|||||||
println!("no accounts yet: ipx user add <name>");
|
println!("no accounts yet: ipx user add <name>");
|
||||||
}
|
}
|
||||||
for u in users {
|
for u in users {
|
||||||
|
let added = u
|
||||||
|
.created
|
||||||
|
.and_then(|t| chrono::DateTime::from_timestamp(t, 0))
|
||||||
|
.map_or("?".into(), |d| d.format("%Y-%m-%d").to_string());
|
||||||
|
let seen = u.last_login.map_or("never signed in".into(), |t| format!("signed in {}", ago(Some(t))));
|
||||||
println!(
|
println!(
|
||||||
"{:<20} {:<8} {}",
|
"{:<20} {:<6} {:<11} added {added} {seen}",
|
||||||
u.name,
|
u.name,
|
||||||
if u.is_admin { "admin" } else { "" },
|
if u.is_admin { "admin" } else { "" },
|
||||||
if u.pass_hash.is_some() { "password" } else { "proxy only" }
|
if u.pass_hash.is_some() { "password" } else { "proxy only" },
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
@@ -292,6 +300,22 @@ fn user_cmd(ctx: &Arc<Ctx>, cmd: UserCmd) -> Result<()> {
|
|||||||
println!("password changed for {name}");
|
println!("password changed for {name}");
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
UserCmd::Rename { name, new_name } => {
|
||||||
|
let name = name.trim().to_ascii_lowercase();
|
||||||
|
// The same rules as a name the proxy vouches for, or the proxy would never find it.
|
||||||
|
let new_name = crate::auth::name_from_header(&new_name)
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("not a usable name: no commas, semicolons or line breaks"))?;
|
||||||
|
let user = ctx
|
||||||
|
.db
|
||||||
|
.user_by_name(&name)?
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("no such account: {name}"))?;
|
||||||
|
if ctx.db.user_by_name(&new_name)?.is_some() {
|
||||||
|
anyhow::bail!("{new_name} already exists");
|
||||||
|
}
|
||||||
|
ctx.db.rename_user(user.id, &new_name)?;
|
||||||
|
println!("renamed {name} to {new_name}");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
UserCmd::Rm { name } => {
|
UserCmd::Rm { name } => {
|
||||||
let name = name.trim().to_ascii_lowercase();
|
let name = name.trim().to_ascii_lowercase();
|
||||||
let user = ctx
|
let user = ctx
|
||||||
@@ -315,14 +339,24 @@ async fn run(ctx: &Arc<Ctx>, cmd: Cmd) -> Result<()> {
|
|||||||
Cmd::Reap { dry_run } => reap(ctx, dry_run, true),
|
Cmd::Reap { dry_run } => reap(ctx, dry_run, true),
|
||||||
Cmd::Download { enclosure } => download_one(ctx, enclosure).await,
|
Cmd::Download { enclosure } => download_one(ctx, enclosure).await,
|
||||||
Cmd::Status => {
|
Cmd::Status => {
|
||||||
let (pending, downloaded) = ctx.db.counts()?;
|
ctx.out.emit(status(ctx));
|
||||||
let feeds = subscriptions(ctx).map(|s| s.len()).unwrap_or(0);
|
|
||||||
ctx.out.emit(Event::Status { feeds, pending, downloaded });
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The counts `ipx status` prints. A running daemon's socket answers with this directly rather
|
||||||
|
/// than through the job queue.
|
||||||
|
fn status(ctx: &Ctx) -> Event {
|
||||||
|
match ctx.db.counts() {
|
||||||
|
Ok((pending, downloaded)) => {
|
||||||
|
let feeds = subscriptions(ctx).map(|s| s.len()).unwrap_or(0);
|
||||||
|
Event::Status { feeds, pending, downloaded }
|
||||||
|
}
|
||||||
|
Err(e) => Event::Error { msg: format!("{e:#}") },
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async fn daemon(
|
async fn daemon(
|
||||||
ctx: Arc<Ctx>,
|
ctx: Arc<Ctx>,
|
||||||
config_path: PathBuf,
|
config_path: PathBuf,
|
||||||
@@ -334,8 +368,7 @@ async fn daemon(
|
|||||||
anyhow::bail!("a daemon is already listening on {}", socket.display());
|
anyhow::bail!("a daemon is already listening on {}", socket.display());
|
||||||
}
|
}
|
||||||
|
|
||||||
// A database with nobody in it cannot be signed into, and an install that predates
|
// A database with nobody in it cannot be signed into.
|
||||||
// accounts still has to serve its owner. Both get the same starting point.
|
|
||||||
if ctx.db.users()?.is_empty() {
|
if ctx.db.users()?.is_empty() {
|
||||||
ctx.db.create_user("admin", Some(&crate::auth::hash_password(DEFAULT_PASSWORD)?), true)?;
|
ctx.db.create_user("admin", Some(&crate::auth::hash_password(DEFAULT_PASSWORD)?), true)?;
|
||||||
tracing::warn!(
|
tracing::warn!(
|
||||||
@@ -344,22 +377,15 @@ async fn daemon(
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
// A library that predates accounts belongs to whoever was using it: the admin.
|
|
||||||
if let Some(admin) = ctx.db.users()?.into_iter().find(|u| u.is_admin) {
|
if let Some(admin) = ctx.db.users()?.into_iter().find(|u| u.is_admin) {
|
||||||
let catalogue: Vec<String> = ctx.cfg().feeds.keys().cloned().collect();
|
let catalogue: Vec<String> = ctx.cfg().feeds.keys().cloned().collect();
|
||||||
match ctx.db.adopt_existing_library(admin.id, &catalogue) {
|
match ctx.db.adopt_catalogue(admin.id, &catalogue) {
|
||||||
Ok(0) => {}
|
Ok(0) => {}
|
||||||
Ok(n) => tracing::info!(user = %admin.name, entries = n, "adopted the existing library"),
|
Ok(n) => tracing::info!(user = %admin.name, feeds = n, "subscribed the first admin to the catalogue"),
|
||||||
Err(e) => tracing::error!(error = %e, "could not adopt the existing library"),
|
Err(e) => tracing::error!(error = %e, "could not subscribe the first admin to the catalogue"),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
match migrate_opml_children(&ctx) {
|
|
||||||
Ok(n) if n > 0 => tracing::info!(count = n, "moved OPML feeds out of config.toml into the database"),
|
|
||||||
Ok(_) => {}
|
|
||||||
Err(e) => tracing::warn!(error = ?e, "could not tidy OPML feeds out of the config"),
|
|
||||||
}
|
|
||||||
|
|
||||||
match ctx.db.requeue_interrupted() {
|
match ctx.db.requeue_interrupted() {
|
||||||
Ok(n) if n > 0 => tracing::info!(count = n, "requeued downloads interrupted by a restart"),
|
Ok(n) if n > 0 => tracing::info!(count = n, "requeued downloads interrupted by a restart"),
|
||||||
Ok(_) => {}
|
Ok(_) => {}
|
||||||
@@ -369,7 +395,12 @@ async fn daemon(
|
|||||||
let (tx_cmd, mut rx_cmd) = mpsc::channel::<Cmd>(64);
|
let (tx_cmd, mut rx_cmd) = mpsc::channel::<Cmd>(64);
|
||||||
|
|
||||||
let web = start_web(&ctx, &config_path, web_addr, &tx_cmd, &events).await?;
|
let web = start_web(&ctx, &config_path, web_addr, &tx_cmd, &events).await?;
|
||||||
let server = tokio::spawn(ipc::serve(socket.clone(), events.clone(), tx_cmd));
|
// status is answered by the socket itself; everything else waits its turn in the queue.
|
||||||
|
let answer: ipc::StatusFn = {
|
||||||
|
let ctx = ctx.clone();
|
||||||
|
Arc::new(move || status(&ctx))
|
||||||
|
};
|
||||||
|
let server = tokio::spawn(ipc::serve(socket.clone(), events.clone(), tx_cmd, answer));
|
||||||
|
|
||||||
// One command at a time: the queue is what keeps two scans from overlapping.
|
// One command at a time: the queue is what keeps two scans from overlapping.
|
||||||
let mut ticker = tokio::time::interval(std::time::Duration::from_secs(60));
|
let mut ticker = tokio::time::interval(std::time::Duration::from_secs(60));
|
||||||
@@ -468,7 +499,7 @@ async fn start_web(
|
|||||||
let mut fresh = (*cfg).clone();
|
let mut fresh = (*cfg).clone();
|
||||||
fresh.web.enabled = true;
|
fresh.web.enabled = true;
|
||||||
fresh.web.bind = bind.clone();
|
fresh.web.bind = bind.clone();
|
||||||
fresh.web.token = web::generate_token();
|
fresh.web.token = crate::auth::new_session_token();
|
||||||
fresh.save(config_path)?;
|
fresh.save(config_path)?;
|
||||||
ctx.reload_cfg(config_path)?;
|
ctx.reload_cfg(config_path)?;
|
||||||
println!("web ui token generated. Open:\n http://{bind}/?token={}", fresh.web.token);
|
println!("web ui token generated. Open:\n http://{bind}/?token={}", fresh.web.token);
|
||||||
@@ -517,8 +548,9 @@ async fn add(
|
|||||||
keywords: Vec<String>,
|
keywords: Vec<String>,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
let mut cfg = (*ctx.cfg()).clone();
|
let mut cfg = (*ctx.cfg()).clone();
|
||||||
|
let url = &feed::expand_input(url);
|
||||||
// Includes feeds derived from an OPML, or the same show could be added twice.
|
// Includes feeds derived from an OPML, or the same show could be added twice.
|
||||||
if let Some(existing) = subscriptions(ctx)?.iter().find(|s| s.cfg.url == url) {
|
if let Some(existing) = subscriptions(ctx)?.iter().find(|s| feed::same_feed(&s.cfg.url, url)) {
|
||||||
anyhow::bail!("already subscribed as {:?}", existing.id);
|
anyhow::bail!("already subscribed as {:?}", existing.id);
|
||||||
}
|
}
|
||||||
let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
|
let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
|
||||||
@@ -569,10 +601,17 @@ pub async fn add_one(
|
|||||||
|
|
||||||
// Slugs must be unique across derived feeds too, or a new feed can collide with one
|
// Slugs must be unique across derived feeds too, or a new feed can collide with one
|
||||||
// an OPML already introduced.
|
// an OPML already introduced.
|
||||||
let taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)?
|
let mut taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)?
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.map(|s| (s.id, s.cfg))
|
.map(|s| (s.id, s.cfg))
|
||||||
.collect();
|
.collect();
|
||||||
|
// A removed feed keeps its rows, so its id is only free again for the same feed: re-adding
|
||||||
|
// it gets its history back, and a different feed does not inherit someone else's.
|
||||||
|
for (id, other) in ctx.db.feed_urls()? {
|
||||||
|
if !feed::same_feed(&other, url) {
|
||||||
|
taken.entry(id).or_insert_with(|| probe.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
let id = config::unique_slug(&title, &taken);
|
let id = config::unique_slug(&title, &taken);
|
||||||
cfg.feeds.insert(id.clone(), probe);
|
cfg.feeds.insert(id.clone(), probe);
|
||||||
Ok(id)
|
Ok(id)
|
||||||
@@ -807,7 +846,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool) -> Result<()> {
|
|||||||
ctx.out.emit(Event::FeedSkip {
|
ctx.out.emit(Event::FeedSkip {
|
||||||
feed: id.clone(),
|
feed: id.clone(),
|
||||||
reason: format!(
|
reason: format!(
|
||||||
"OPML: {total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing",
|
"{total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing",
|
||||||
added.len()
|
added.len()
|
||||||
),
|
),
|
||||||
});
|
});
|
||||||
@@ -908,36 +947,6 @@ pub fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
|
|||||||
Ok(out)
|
Ok(out)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Moves OPML children that older versions wrote into config.toml over to the database.
|
|
||||||
/// They were never yours to edit, and 80-odd of them made the file unreadable.
|
|
||||||
fn migrate_opml_children(ctx: &Ctx) -> Result<usize> {
|
|
||||||
let cfg = (*ctx.cfg()).clone();
|
|
||||||
let children: Vec<(String, config::Feed)> = cfg
|
|
||||||
.feeds
|
|
||||||
.iter()
|
|
||||||
.filter(|(_, f)| f.group.is_some())
|
|
||||||
.map(|(id, f)| (id.clone(), f.clone()))
|
|
||||||
.collect();
|
|
||||||
if children.is_empty() {
|
|
||||||
return Ok(0);
|
|
||||||
}
|
|
||||||
let mut fresh = cfg.clone();
|
|
||||||
for (id, f) in &children {
|
|
||||||
let group = f.group.clone().unwrap_or_default();
|
|
||||||
let title = ctx
|
|
||||||
.db
|
|
||||||
.feed_summary(id)
|
|
||||||
.ok()
|
|
||||||
.and_then(|s| s.title)
|
|
||||||
.unwrap_or_else(|| id.clone());
|
|
||||||
ctx.db.upsert_managed(id, &f.url, &title, &group)?;
|
|
||||||
fresh.feeds.remove(id);
|
|
||||||
}
|
|
||||||
fresh.save(&ctx.config_path)?;
|
|
||||||
ctx.reload_cfg(&ctx.config_path)?;
|
|
||||||
Ok(children.len())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Seconds to wait before re-checking a feed.
|
/// Seconds to wait before re-checking a feed.
|
||||||
///
|
///
|
||||||
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the
|
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the
|
||||||
@@ -963,7 +972,7 @@ struct Scan {
|
|||||||
enum Outcome {
|
enum Outcome {
|
||||||
NotModified,
|
NotModified,
|
||||||
Feed(Scan),
|
Feed(Scan),
|
||||||
/// The URL served an OPML document, so it is a subscription list rather than a feed.
|
/// The URL is a list of feeds rather than a feed: an OPML, or a Patreon creator's shows.
|
||||||
Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
|
Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -973,6 +982,31 @@ async fn scan_one(
|
|||||||
feed_cfg: &config::Feed,
|
feed_cfg: &config::Feed,
|
||||||
state: &db::HttpState,
|
state: &db::HttpState,
|
||||||
) -> Result<Outcome> {
|
) -> Result<Outcome> {
|
||||||
|
// A Patreon creator with more than one show is a list of feeds, like an OPML.
|
||||||
|
if feed::is_patreon_creator(&feed_cfg.url) {
|
||||||
|
match feed::patreon_shows(&ctx.client, &feed_cfg.url).await {
|
||||||
|
Ok((name, shows)) if shows.len() > 1 => {
|
||||||
|
ctx.db.touch_feed(id, &feed_cfg.url)?;
|
||||||
|
if let Some(name) = name {
|
||||||
|
ctx.db.set_title(id, &name)?;
|
||||||
|
}
|
||||||
|
// Read as one feed before it was split, it listed every show's items in one
|
||||||
|
// heap. The items go; its files and read state move to each show as the show
|
||||||
|
// lists them (`Db::adopt`), so no show comes up empty for want of a URL.
|
||||||
|
ctx.db.clear_entries(id)?;
|
||||||
|
return sync_group(ctx, id, feed_cfg, &shows).await;
|
||||||
|
}
|
||||||
|
Ok(_) => {} // One show: the creator's feed is that show.
|
||||||
|
// Already split: keep the shows it has rather than read the creator as one heap.
|
||||||
|
Err(e) if ctx.db.managed_feeds()?.iter().any(|m| m.group_id == id) => return Err(e),
|
||||||
|
Err(e) => tracing::warn!(
|
||||||
|
feed = id,
|
||||||
|
error = %format!("{e:#}"),
|
||||||
|
"could not list the Patreon shows; reading it as one feed"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
let mut fetched = feed::fetch(
|
let mut fetched = feed::fetch(
|
||||||
&ctx.client,
|
&ctx.client,
|
||||||
feed_cfg,
|
feed_cfg,
|
||||||
@@ -1018,19 +1052,38 @@ async fn scan_one(
|
|||||||
)?;
|
)?;
|
||||||
|
|
||||||
let policy = policy_for(ctx, id, feed_cfg)?;
|
let policy = policy_for(ctx, id, feed_cfg)?;
|
||||||
|
if let Some(parent) = &feed_cfg.group {
|
||||||
|
let listed: Vec<(&str, &str)> = parsed
|
||||||
|
.entries
|
||||||
|
.iter()
|
||||||
|
.flat_map(|e| e.enclosures.iter().map(move |x| (e.guid.as_str(), x.url.as_str())))
|
||||||
|
.collect();
|
||||||
|
ctx.db.adopt(parent, id, &listed)?;
|
||||||
|
}
|
||||||
|
// Verdicts are recorded in `state`, so the download queue below is just "everything still
|
||||||
|
// pending". A filter's verdict is looked at again on every scan, though: made once, at
|
||||||
|
// discovery, it outlived the setting behind it, and allowing explicit items afterwards
|
||||||
|
// changed nothing however often the feed was scanned.
|
||||||
|
let skipped = ctx.db.skipped_by_filter(id)?;
|
||||||
let mut scan = Scan::default();
|
let mut scan = Scan::default();
|
||||||
for entry in &parsed.entries {
|
for entry in &parsed.entries {
|
||||||
if ctx.db.record_entry(id, entry)? {
|
if ctx.db.record_entry(id, entry)? {
|
||||||
scan.new_entries += 1;
|
scan.new_entries += 1;
|
||||||
}
|
}
|
||||||
for enc in &entry.enclosures {
|
for enc in &entry.enclosures {
|
||||||
if !ctx.db.record_enclosure(id, &entry.guid, enc)? {
|
let was = if ctx.db.record_enclosure(id, &entry.guid, enc)? {
|
||||||
continue; // Seen before: downloaded, skipped or deliberately reaped.
|
None
|
||||||
|
} else if let Some(reason) = skipped.get(&enc.url) {
|
||||||
|
Some(reason.as_str())
|
||||||
|
} else {
|
||||||
|
continue; // Settled: queued, downloaded, reaped, or another feed's file.
|
||||||
|
};
|
||||||
|
let now = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc);
|
||||||
|
if now != was {
|
||||||
|
match now {
|
||||||
|
Some(reason) => ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?,
|
||||||
|
None => ctx.db.mark_enclosure(&enc.url, "pending", None)?,
|
||||||
}
|
}
|
||||||
// Filters run once, at discovery, and are recorded in `state`. The download
|
|
||||||
// queue below is then just "everything still pending".
|
|
||||||
if let Some(reason) = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc) {
|
|
||||||
ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1114,12 +1167,6 @@ async fn scan_one(
|
|||||||
Ok(Outcome::Feed(scan))
|
Ok(Outcome::Feed(scan))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Brings the feed list in step with a subscribed OPML.
|
|
||||||
///
|
|
||||||
/// New entries are added under the OPML's group and folder. An entry that has gone from
|
|
||||||
/// the OPML is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
|
|
||||||
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
|
|
||||||
/// the UI to explain them.
|
|
||||||
async fn sync_opml(
|
async fn sync_opml(
|
||||||
ctx: &Arc<Ctx>,
|
ctx: &Arc<Ctx>,
|
||||||
parent_id: &str,
|
parent_id: &str,
|
||||||
@@ -1130,25 +1177,44 @@ async fn sync_opml(
|
|||||||
if let Some(title) = feed::opml_title(bytes) {
|
if let Some(title) = feed::opml_title(bytes) {
|
||||||
ctx.db.set_title(parent_id, &title)?;
|
ctx.db.set_title(parent_id, &title)?;
|
||||||
}
|
}
|
||||||
|
sync_group(ctx, parent_id, parent, &listed).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Brings the feed list in step with a list of feeds: a subscribed OPML, or a Patreon
|
||||||
|
/// creator's shows.
|
||||||
|
///
|
||||||
|
/// New entries are added under the list's group and folder. An entry that has gone from
|
||||||
|
/// the list is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
|
||||||
|
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
|
||||||
|
/// the UI to explain them.
|
||||||
|
async fn sync_group(
|
||||||
|
ctx: &Arc<Ctx>,
|
||||||
|
parent_id: &str,
|
||||||
|
parent: &config::Feed,
|
||||||
|
listed: &[(String, String)],
|
||||||
|
) -> Result<Outcome> {
|
||||||
let cfg = ctx.cfg();
|
let cfg = ctx.cfg();
|
||||||
let existing = ctx.db.managed_feeds()?;
|
let existing = ctx.db.managed_feeds()?;
|
||||||
let mut added = vec![];
|
let mut added = vec![];
|
||||||
|
|
||||||
for (title, url) in &listed {
|
for (title, url) in listed {
|
||||||
// Already known, whether derived or promoted into the config.
|
// Already known, whether derived or promoted into the config.
|
||||||
if let Some(m) = existing.iter().find(|m| &m.url == url) {
|
if let Some(m) = existing.iter().find(|m| &m.url == url) {
|
||||||
ctx.db.upsert_managed(&m.id, url, title, parent_id)?;
|
ctx.db.upsert_managed(&m.id, url, title, parent_id)?;
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if cfg.feeds.values().any(|f| &f.url == url) {
|
// A Patreon show you added by hand may be spelled differently from the one listed.
|
||||||
|
if cfg.feeds.values().any(|f| feed::same_feed(&f.url, url)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
// A removed feed keeps its rows, so its id is only free again for the same feed.
|
||||||
|
let known = ctx.db.feed_urls()?;
|
||||||
let taken: std::collections::BTreeMap<String, config::Feed> = cfg
|
let taken: std::collections::BTreeMap<String, config::Feed> = cfg
|
||||||
.feeds
|
.feeds
|
||||||
.keys()
|
.keys()
|
||||||
.chain(existing.iter().map(|m| &m.id))
|
.chain(existing.iter().map(|m| &m.id))
|
||||||
.chain(added.iter())
|
.chain(added.iter())
|
||||||
|
.chain(known.iter().filter(|(_, u)| !feed::same_feed(u, url)).map(|(id, _)| id))
|
||||||
.map(|id| (id.clone(), parent.clone()))
|
.map(|id| (id.clone(), parent.clone()))
|
||||||
.collect();
|
.collect();
|
||||||
let id = config::unique_slug(title, &taken);
|
let id = config::unique_slug(title, &taken);
|
||||||
@@ -1255,7 +1321,7 @@ pub struct Policy {
|
|||||||
|
|
||||||
fn policy_for(ctx: &Ctx, id: &str, feed_cfg: &config::Feed) -> Result<Policy> {
|
fn policy_for(ctx: &Ctx, id: &str, feed_cfg: &config::Feed) -> Result<Policy> {
|
||||||
let global = ctx.cfg().general.max_new_per_check;
|
let global = ctx.cfg().general.max_new_per_check;
|
||||||
Ok(merge_policy(&ctx.db.subscribers(id)?, feed_cfg, global))
|
Ok(merge_policy(&ctx.db.subscribers(id, feed_cfg.group.as_deref())?, feed_cfg, global))
|
||||||
}
|
}
|
||||||
|
|
||||||
fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Policy {
|
fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Policy {
|
||||||
|
|||||||
@@ -154,8 +154,8 @@ mod tests {
|
|||||||
// One file serves both subscribers, so it takes both of them to release it.
|
// One file serves both subscribers, so it takes both of them to release it.
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
|
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0);
|
||||||
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0);
|
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'f'),(2,'f');
|
||||||
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
||||||
('f', 'keep', 0),
|
('f', 'keep', 0),
|
||||||
('f', 'half', 0),
|
('f', 'half', 0),
|
||||||
@@ -189,7 +189,7 @@ mod tests {
|
|||||||
fn prune_keeps_entries_that_still_have_a_file() {
|
fn prune_keeps_entries_that_still_have_a_file() {
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
|
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1);
|
||||||
INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',1);
|
INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',1);
|
||||||
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
||||||
('f', 'has-file', 100),
|
('f', 'has-file', 100),
|
||||||
|
|||||||
333
src/web.rs
333
src/web.rs
@@ -12,9 +12,7 @@ use axum::{
|
|||||||
},
|
},
|
||||||
routing::{delete, get, patch, post},
|
routing::{delete, get, patch, post},
|
||||||
};
|
};
|
||||||
use futures_util::StreamExt;
|
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use tokio_stream::wrappers::BroadcastStream;
|
|
||||||
use tower::ServiceExt;
|
use tower::ServiceExt;
|
||||||
use tower_http::services::ServeFile;
|
use tower_http::services::ServeFile;
|
||||||
use serde::Serialize;
|
use serde::Serialize;
|
||||||
@@ -35,28 +33,6 @@ pub struct WebState {
|
|||||||
pub events: broadcast::Sender<Event>,
|
pub events: broadcast::Sender<Event>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A 32-hex-character shared secret, generated when config.toml has none.
|
|
||||||
///
|
|
||||||
/// ponytail: /dev/urandom rather than a CSPRNG crate -- 16 bytes, once, on a Unix-only
|
|
||||||
/// binary. Falls back to the clock only if urandom is somehow unreadable, which would be a
|
|
||||||
/// weak token, so that case is logged loudly.
|
|
||||||
pub fn generate_token() -> String {
|
|
||||||
use std::io::Read;
|
|
||||||
let mut bytes = [0u8; 16];
|
|
||||||
match std::fs::File::open("/dev/urandom").and_then(|mut f| f.read_exact(&mut bytes)) {
|
|
||||||
Ok(()) => {}
|
|
||||||
Err(e) => {
|
|
||||||
tracing::error!(error = %e, "could not read /dev/urandom; token is NOT secure");
|
|
||||||
let n = std::time::SystemTime::now()
|
|
||||||
.duration_since(std::time::UNIX_EPOCH)
|
|
||||||
.map(|d| d.as_nanos() as u64)
|
|
||||||
.unwrap_or(0);
|
|
||||||
bytes[..8].copy_from_slice(&n.to_le_bytes());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
bytes.iter().map(|b| format!("{b:02x}")).collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn router(state: WebState) -> Router {
|
pub fn router(state: WebState) -> Router {
|
||||||
Router::new()
|
Router::new()
|
||||||
.route("/", get(index))
|
.route("/", get(index))
|
||||||
@@ -65,9 +41,11 @@ pub fn router(state: WebState) -> Router {
|
|||||||
.route("/api/feeds", get(feeds).post(add_feed))
|
.route("/api/feeds", get(feeds).post(add_feed))
|
||||||
.route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
|
.route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
|
||||||
.route("/api/feeds/{id}/entries", get(entries))
|
.route("/api/feeds/{id}/entries", get(entries))
|
||||||
|
.route("/api/entries", get(all_entries))
|
||||||
.route("/api/entries/{feed_id}/{guid}/flags", post(set_flags))
|
.route("/api/entries/{feed_id}/{guid}/flags", post(set_flags))
|
||||||
.route("/api/entries/{feed_id}/{guid}/position", post(set_position))
|
.route("/api/entries/{feed_id}/{guid}/position", post(set_position))
|
||||||
.route("/api/feeds/{id}/read-all", post(read_all))
|
.route("/api/feeds/{id}/read-all", post(read_all))
|
||||||
|
.route("/api/read-all", post(read_all_mine))
|
||||||
.route("/api/feeds/{id}/download-latest", post(download_latest))
|
.route("/api/feeds/{id}/download-latest", post(download_latest))
|
||||||
.route("/api/enclosures/{id}/download", post(download_now))
|
.route("/api/enclosures/{id}/download", post(download_now))
|
||||||
.route("/api/enclosures/{id}", delete(delete_file))
|
.route("/api/enclosures/{id}", delete(delete_file))
|
||||||
@@ -75,6 +53,7 @@ pub fn router(state: WebState) -> Router {
|
|||||||
.route("/api/opml", get(export_opml).post(import_opml))
|
.route("/api/opml", get(export_opml).post(import_opml))
|
||||||
.route("/api/settings", get(get_settings).patch(patch_settings))
|
.route("/api/settings", get(get_settings).patch(patch_settings))
|
||||||
.route("/api/popular", get(get_popular))
|
.route("/api/popular", get(get_popular))
|
||||||
|
.route("/api/directory", get(get_directory))
|
||||||
.route("/api/popular/{id}", post(subscribe_popular))
|
.route("/api/popular/{id}", post(subscribe_popular))
|
||||||
.route("/api/users", get(list_users).post(add_user))
|
.route("/api/users", get(list_users).post(add_user))
|
||||||
.route("/api/users/{id}", patch(patch_user).delete(remove_user))
|
.route("/api/users/{id}", patch(patch_user).delete(remove_user))
|
||||||
@@ -85,6 +64,7 @@ pub fn router(state: WebState) -> Router {
|
|||||||
// Signing in cannot require being signed in, so these sit outside the auth layer.
|
// Signing in cannot require being signed in, so these sit outside the auth layer.
|
||||||
.route("/login", get(login_page))
|
.route("/login", get(login_page))
|
||||||
.route("/api/login", post(login))
|
.route("/api/login", post(login))
|
||||||
|
.route("/icon.png", get(icon))
|
||||||
.layer(middleware::from_fn(access_log))
|
.layer(middleware::from_fn(access_log))
|
||||||
.with_state(state)
|
.with_state(state)
|
||||||
}
|
}
|
||||||
@@ -111,23 +91,8 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
|||||||
let cfg = state.ctx.cfg();
|
let cfg = state.ctx.cfg();
|
||||||
let token = cfg.web.token.clone();
|
let token = cfg.web.token.clone();
|
||||||
|
|
||||||
let peer = req
|
// 1. A header, but only from a hop we were told to believe.
|
||||||
.extensions()
|
let vouched = vouched_name(&cfg, &req);
|
||||||
.get::<axum::extract::ConnectInfo<std::net::SocketAddr>>()
|
|
||||||
.map(|c| c.0.ip().to_string())
|
|
||||||
.unwrap_or_default();
|
|
||||||
|
|
||||||
// 1. A header, but only from a hop we were told to believe. Anyone able to reach the
|
|
||||||
// port could otherwise send it and be whoever they liked.
|
|
||||||
let vouched = (!cfg.web.trusted_header.is_empty()
|
|
||||||
&& cfg.web.trusted_proxies.iter().any(|p| p == &peer))
|
|
||||||
.then(|| {
|
|
||||||
req.headers()
|
|
||||||
.get(&cfg.web.trusted_header)
|
|
||||||
.and_then(|v| v.to_str().ok())
|
|
||||||
.and_then(crate::auth::name_from_header)
|
|
||||||
})
|
|
||||||
.flatten();
|
|
||||||
|
|
||||||
let mut set_cookie: Option<String> = None;
|
let mut set_cookie: Option<String> = None;
|
||||||
let mut user = None;
|
let mut user = None;
|
||||||
@@ -153,7 +118,13 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
|||||||
None
|
None
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
// Every request comes vouched for; signed_in keeps one an hour. Failing to note the time
|
||||||
|
// must not turn anyone away, so its error goes unanswered.
|
||||||
|
if let Some(u) = &user {
|
||||||
|
let _ = state.ctx.db.signed_in(u.id);
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
let by_proxy = user.is_some();
|
||||||
|
|
||||||
// 2. A session cookie from signing in here.
|
// 2. A session cookie from signing in here.
|
||||||
if user.is_none() {
|
if user.is_none() {
|
||||||
@@ -177,6 +148,10 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
|||||||
if supplied.is_some_and(|t| constant_time_eq(&t, &token)) {
|
if supplied.is_some_and(|t| constant_time_eq(&t, &token)) {
|
||||||
user = admin_user(&state);
|
user = admin_user(&state);
|
||||||
if from_query.is_some() {
|
if from_query.is_some() {
|
||||||
|
// The token link is a sign-in; the cookie it leaves behind is not one each time.
|
||||||
|
if let Some(u) = &user {
|
||||||
|
let _ = state.ctx.db.signed_in(u.id);
|
||||||
|
}
|
||||||
set_cookie = Some(format!(
|
set_cookie = Some(format!(
|
||||||
"{COOKIE}={token}; Path=/; HttpOnly; SameSite=Lax; Max-Age=31536000"
|
"{COOKIE}={token}; Path=/; HttpOnly; SameSite=Lax; Max-Age=31536000"
|
||||||
));
|
));
|
||||||
@@ -200,6 +175,7 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
|||||||
};
|
};
|
||||||
|
|
||||||
req.extensions_mut().insert(user);
|
req.extensions_mut().insert(user);
|
||||||
|
req.extensions_mut().insert(Proxied(by_proxy));
|
||||||
let mut resp = next.run(req).await;
|
let mut resp = next.run(req).await;
|
||||||
if let Some(c) = set_cookie {
|
if let Some(c) = set_cookie {
|
||||||
if let Ok(v) = header::HeaderValue::from_str(&c) {
|
if let Ok(v) = header::HeaderValue::from_str(&c) {
|
||||||
@@ -211,6 +187,29 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
|||||||
|
|
||||||
const SESSION_COOKIE: &str = "ipx_session";
|
const SESSION_COOKIE: &str = "ipx_session";
|
||||||
|
|
||||||
|
/// Whether the proxy signed this request in, rather than a session or the token: signing out
|
||||||
|
/// has to go through the proxy then, or its next request signs the person straight back in.
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
struct Proxied(bool);
|
||||||
|
|
||||||
|
/// The name the proxy vouches for, when this request came from one of `trusted_proxies` and
|
||||||
|
/// carries `trusted_header`. Anyone able to reach the port could otherwise send the header and
|
||||||
|
/// be whoever they liked.
|
||||||
|
fn vouched_name(cfg: &crate::config::Config, req: &Request) -> Option<String> {
|
||||||
|
let peer = req
|
||||||
|
.extensions()
|
||||||
|
.get::<axum::extract::ConnectInfo<std::net::SocketAddr>>()
|
||||||
|
.map(|c| c.0.ip().to_string())
|
||||||
|
.unwrap_or_default();
|
||||||
|
if cfg.web.trusted_header.is_empty() || !cfg.web.trusted_proxies.iter().any(|p| p == &peer) {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
req.headers()
|
||||||
|
.get(&cfg.web.trusted_header)
|
||||||
|
.and_then(|v| v.to_str().ok())
|
||||||
|
.and_then(crate::auth::name_from_header)
|
||||||
|
}
|
||||||
|
|
||||||
/// Handlers take `User` to say they need one; the auth layer put it there, and nothing
|
/// Handlers take `User` to say they need one; the auth layer put it there, and nothing
|
||||||
/// reaches a handler without passing through it.
|
/// reaches a handler without passing through it.
|
||||||
impl<S: Send + Sync> axum::extract::FromRequestParts<S> for crate::db::User {
|
impl<S: Send + Sync> axum::extract::FromRequestParts<S> for crate::db::User {
|
||||||
@@ -273,6 +272,7 @@ async fn login(
|
|||||||
let user = user.expect("verified above");
|
let user = user.expect("verified above");
|
||||||
let token = crate::auth::new_session_token();
|
let token = crate::auth::new_session_token();
|
||||||
state.ctx.db.create_session(user.id, &token)?;
|
state.ctx.db.create_session(user.id, &token)?;
|
||||||
|
state.ctx.db.signed_in(user.id)?;
|
||||||
tracing::info!(user = %user.name, "signed in");
|
tracing::info!(user = %user.name, "signed in");
|
||||||
|
|
||||||
let days = state.ctx.cfg().web.session_days.max(1);
|
let days = state.ctx.cfg().web.session_days.max(1);
|
||||||
@@ -303,8 +303,15 @@ async fn logout(State(state): State<WebState>, req: Request) -> Response {
|
|||||||
resp
|
resp
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn me(user: crate::db::User) -> Json<serde_json::Value> {
|
/// Who is signed in, and, for someone the proxy signed in, where Sign out should send them.
|
||||||
Json(serde_json::json!({ "name": user.name, "admin": user.is_admin }))
|
async fn me(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
axum::Extension(Proxied(by_proxy)): axum::Extension<Proxied>,
|
||||||
|
) -> Json<serde_json::Value> {
|
||||||
|
let url = state.ctx.cfg().web.sign_out_url.clone();
|
||||||
|
let sign_out = (by_proxy && !url.is_empty()).then_some(url);
|
||||||
|
Json(serde_json::json!({ "name": user.name, "admin": user.is_admin, "sign_out": sign_out }))
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---- accounts: admin only ----
|
// ---- accounts: admin only ----
|
||||||
@@ -337,6 +344,7 @@ async fn list_users(
|
|||||||
.map(|u| {
|
.map(|u| {
|
||||||
serde_json::json!({
|
serde_json::json!({
|
||||||
"id": u.id, "name": u.name, "admin": u.is_admin, "password": u.pass_hash.is_some(),
|
"id": u.id, "name": u.name, "admin": u.is_admin, "password": u.pass_hash.is_some(),
|
||||||
|
"created": u.created, "last_login": u.last_login,
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
.collect();
|
.collect();
|
||||||
@@ -426,8 +434,22 @@ async fn remove_user(
|
|||||||
Ok(StatusCode::NO_CONTENT)
|
Ok(StatusCode::NO_CONTENT)
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn login_page() -> Html<&'static str> {
|
/// The password form, except for someone the proxy vouches for: they are signed in already, and
|
||||||
Html(include_str!("../web/login.html"))
|
/// the form only made it look as if they were not.
|
||||||
|
async fn login_page(State(state): State<WebState>, req: Request) -> Response {
|
||||||
|
if vouched_name(&state.ctx.cfg(), &req).is_some() {
|
||||||
|
return Redirect::to("/").into_response();
|
||||||
|
}
|
||||||
|
Html(include_str!("../web/login.html")).into_response()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The 2004 icon, served once for both pages rather than inlined as base64 into each. The
|
||||||
|
/// sign-in page shows it, so it sits outside the auth layer with /login.
|
||||||
|
async fn icon() -> impl IntoResponse {
|
||||||
|
(
|
||||||
|
[(header::CONTENT_TYPE, "image/png"), (header::CACHE_CONTROL, "max-age=86400")],
|
||||||
|
include_bytes!("../web/ipodderx-icon.png").as_slice(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn constant_time_eq(a: &str, b: &str) -> bool {
|
fn constant_time_eq(a: &str, b: &str) -> bool {
|
||||||
@@ -492,6 +514,9 @@ async fn feeds(
|
|||||||
let mut out = Vec::with_capacity(mine.len());
|
let mut out = Vec::with_capacity(mine.len());
|
||||||
for sub in &subs {
|
for sub in &subs {
|
||||||
let (id, feed) = (&sub.id, &sub.cfg);
|
let (id, feed) = (&sub.id, &sub.cfg);
|
||||||
|
// In a group, what you have not set on the feed comes from your settings on the group,
|
||||||
|
// the same fallback the scanner uses (`Db::subscribers`).
|
||||||
|
let up = feed.group.as_deref().and_then(|g| mine.get(g));
|
||||||
let Some(mine) = mine.get(id) else { continue };
|
let Some(mine) = mine.get(id) else { continue };
|
||||||
let s = state.ctx.db.feed_summary(id)?;
|
let s = state.ctx.db.feed_summary(id)?;
|
||||||
let st = state.ctx.db.http_state(id)?;
|
let st = state.ctx.db.http_state(id)?;
|
||||||
@@ -501,11 +526,22 @@ async fn feeds(
|
|||||||
title: s.title,
|
title: s.title,
|
||||||
image: s.image,
|
image: s.image,
|
||||||
folder: feed.folder.clone(),
|
folder: feed.folder.clone(),
|
||||||
keywords: mine.keywords.clone().unwrap_or_else(|| feed.keywords.clone()),
|
keywords: mine
|
||||||
allow_explicit: mine.allow_explicit.unwrap_or(feed.allow_explicit),
|
.keywords
|
||||||
auto_download: mine.auto_download.unwrap_or(feed.auto_download),
|
.clone()
|
||||||
|
.or_else(|| up.and_then(|u| u.keywords.clone()))
|
||||||
|
.unwrap_or_else(|| feed.keywords.clone()),
|
||||||
|
allow_explicit: mine
|
||||||
|
.allow_explicit
|
||||||
|
.or(up.and_then(|u| u.allow_explicit))
|
||||||
|
.unwrap_or(feed.allow_explicit),
|
||||||
|
auto_download: mine
|
||||||
|
.auto_download
|
||||||
|
.or(up.and_then(|u| u.auto_download))
|
||||||
|
.unwrap_or(feed.auto_download),
|
||||||
max_new_per_check: mine
|
max_new_per_check: mine
|
||||||
.max_new_per_check
|
.max_new_per_check
|
||||||
|
.or(up.and_then(|u| u.max_new_per_check))
|
||||||
.map(|n| n as usize)
|
.map(|n| n as usize)
|
||||||
.or(feed.max_new_per_check),
|
.or(feed.max_new_per_check),
|
||||||
group: feed.group.clone(),
|
group: feed.group.clone(),
|
||||||
@@ -537,14 +573,20 @@ async fn feeds(
|
|||||||
/// A feed that carries a credential is someone's paid or private subscription. Listing it
|
/// A feed that carries a credential is someone's paid or private subscription. Listing it
|
||||||
/// would let anyone signed in subscribe to it and read what they pay for.
|
/// would let anyone signed in subscribe to it and read what they pay for.
|
||||||
///
|
///
|
||||||
/// ponytail: a heuristic. A token hidden in the URL's path gets through; a per-feed
|
/// ponytail: a heuristic. A key hidden in the path of a host not in `PAID_HOSTS` gets
|
||||||
/// `unlisted` flag is the upgrade if that ever happens.
|
/// through; a per-feed `unlisted` flag is the upgrade if that happens again.
|
||||||
fn looks_private(feed: &crate::config::Feed) -> bool {
|
fn looks_private(feed: &crate::config::Feed) -> bool {
|
||||||
if feed.username.is_some() || feed.password.is_some() || feed.password_env.is_some() {
|
if feed.username.is_some() || feed.password.is_some() || feed.password_env.is_some() {
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
let Ok(u) = url::Url::parse(&feed.url) else { return true };
|
let Ok(u) = url::Url::parse(&feed.url) else { return true };
|
||||||
!u.username().is_empty()
|
// Paid-feed services put the subscriber's key in the path as often as in the query, and a
|
||||||
|
// long path segment alone proves nothing (acast's public show ids look the same). So a
|
||||||
|
// feed from one of them is private whatever its URL looks like. Supercast is why this
|
||||||
|
// exists: `feeds.supercast.com/feeds/<key>` reached the directory before it did.
|
||||||
|
let host = u.host_str().unwrap_or("");
|
||||||
|
PAID_HOSTS.iter().any(|h| host == *h || host.ends_with(&format!(".{h}")))
|
||||||
|
|| !u.username().is_empty()
|
||||||
|| u.password().is_some()
|
|| u.password().is_some()
|
||||||
|| u.query_pairs().any(|(k, _)| {
|
|| u.query_pairs().any(|(k, _)| {
|
||||||
let k = k.to_ascii_lowercase();
|
let k = k.to_ascii_lowercase();
|
||||||
@@ -554,6 +596,10 @@ fn looks_private(feed: &crate::config::Feed) -> bool {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Services whose feeds are always one subscriber's own.
|
||||||
|
const PAID_HOSTS: &[&str] =
|
||||||
|
&["patreon.com", "supercast.com", "supportingcast.fm", "glow.fm", "memberful.com"];
|
||||||
|
|
||||||
/// Only an id, a title, artwork and a count: never a URL, which is where a key would be.
|
/// Only an id, a title, artwork and a count: never a URL, which is where a key would be.
|
||||||
#[derive(Serialize)]
|
#[derive(Serialize)]
|
||||||
struct PopularRow {
|
struct PopularRow {
|
||||||
@@ -561,29 +607,41 @@ struct PopularRow {
|
|||||||
title: Option<String>,
|
title: Option<String>,
|
||||||
image: Option<String>,
|
image: Option<String>,
|
||||||
subscribers: i64,
|
subscribers: i64,
|
||||||
|
/// Yours already. Everyone counts, you included, so your own feeds are listed too.
|
||||||
|
subscribed: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What other people here subscribe to that you don't, most subscribers first. What the
|
/// Every feed that may be listed, with everyone counted, you included, most subscribers
|
||||||
/// Add feed screen offers, and all that `subscribe_popular` will subscribe you to.
|
/// first. Popular is the top of it, the directory is all of it, and it is all that
|
||||||
|
/// `subscribe_popular` will subscribe you to. An OPML or a Patreon creator is listed as the
|
||||||
|
/// feeds inside it and never itself: both lists are for finding a show.
|
||||||
fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||||
let db = &state.ctx.db;
|
let db = &state.ctx.db;
|
||||||
let mine: std::collections::HashSet<String> =
|
let mine: std::collections::HashSet<String> =
|
||||||
db.subscriptions_for(user_id)?.into_iter().map(|s| s.feed_id).collect();
|
db.subscriptions_for(user_id)?.into_iter().map(|s| s.feed_id).collect();
|
||||||
let counts = db.subscriber_counts()?;
|
let counts = db.subscriber_counts()?;
|
||||||
|
let catalogue = crate::subscriptions(&state.ctx)?;
|
||||||
|
let by_id: std::collections::HashMap<&str, &crate::config::Feed> =
|
||||||
|
catalogue.iter().map(|s| (s.id.as_str(), &s.cfg)).collect();
|
||||||
|
let is_folder: std::collections::HashSet<&str> =
|
||||||
|
catalogue.iter().filter_map(|s| s.cfg.group.as_deref()).collect();
|
||||||
let mut out = vec![];
|
let mut out = vec![];
|
||||||
for s in crate::subscriptions(&state.ctx)? {
|
for s in &catalogue {
|
||||||
let n = counts.get(&s.id).copied().unwrap_or(0);
|
let n = counts.get(&s.id).copied().unwrap_or(0);
|
||||||
// A feed from an OPML rides on the OPML: everyone subscribed to it counts every feed
|
// A feed inside an OPML that looks private is as private as the OPML.
|
||||||
// inside, which would bury everything anyone chose on purpose.
|
let folder = s.cfg.group.as_deref().and_then(|g| by_id.get(g));
|
||||||
let from_opml = s.managed || s.cfg.group.is_some();
|
if n == 0
|
||||||
if n == 0 || from_opml || mine.contains(&s.id) || looks_private(&s.cfg) {
|
|| is_folder.contains(s.id.as_str())
|
||||||
|
|| looks_private(&s.cfg)
|
||||||
|
|| folder.is_some_and(|f| looks_private(f))
|
||||||
|
{
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
let sum = db.feed_summary(&s.id)?;
|
let sum = db.feed_summary(&s.id)?;
|
||||||
out.push(PopularRow { id: s.id, title: sum.title, image: sum.image, subscribers: n });
|
let subscribed = mine.contains(&s.id);
|
||||||
|
out.push(PopularRow { id: s.id.clone(), title: sum.title, image: sum.image, subscribers: n, subscribed });
|
||||||
}
|
}
|
||||||
let name = |p: &PopularRow| p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase();
|
out.sort_by(|a, b| b.subscribers.cmp(&a.subscribers).then_with(|| sort_name(a).cmp(&sort_name(b))));
|
||||||
out.sort_by(|a, b| b.subscribers.cmp(&a.subscribers).then_with(|| name(a).cmp(&name(b))));
|
|
||||||
Ok(out)
|
Ok(out)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -592,10 +650,24 @@ async fn get_popular(
|
|||||||
user: crate::db::User,
|
user: crate::db::User,
|
||||||
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||||
let mut rows = popular(&state, user.id)?;
|
let mut rows = popular(&state, user.id)?;
|
||||||
rows.truncate(20);
|
rows.truncate(10);
|
||||||
Ok(Json(rows))
|
Ok(Json(rows))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Every feed that may be listed, A to Z, with the feeds inside an OPML in place of the OPML.
|
||||||
|
async fn get_directory(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||||
|
let mut rows = popular(&state, user.id)?;
|
||||||
|
rows.sort_by_key(sort_name);
|
||||||
|
Ok(Json(rows))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sort_name(p: &PopularRow) -> String {
|
||||||
|
p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase()
|
||||||
|
}
|
||||||
|
|
||||||
/// Subscribes by id, because the list never shows a URL. Checked against the same list, so
|
/// Subscribes by id, because the list never shows a URL. Checked against the same list, so
|
||||||
/// a guessed id cannot reach a private feed.
|
/// a guessed id cannot reach a private feed.
|
||||||
async fn subscribe_popular(
|
async fn subscribe_popular(
|
||||||
@@ -604,7 +676,7 @@ async fn subscribe_popular(
|
|||||||
Path(id): Path<String>,
|
Path(id): Path<String>,
|
||||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
if !popular(&state, user.id)?.iter().any(|p| p.id == id) {
|
if !popular(&state, user.id)?.iter().any(|p| p.id == id) {
|
||||||
return Err(ApiError::bad_request(format!("{id:?} is not on the popular list")));
|
return Err(ApiError::bad_request(format!("{id:?} is not in the directory")));
|
||||||
}
|
}
|
||||||
state.ctx.db.subscribe(user.id, &id)?;
|
state.ctx.db.subscribe(user.id, &id)?;
|
||||||
Ok(Json(serde_json::json!({ "id": id })))
|
Ok(Json(serde_json::json!({ "id": id })))
|
||||||
@@ -695,6 +767,11 @@ mod tests {
|
|||||||
// Patreon's shape: the key is a query parameter.
|
// Patreon's shape: the key is a query parameter.
|
||||||
assert!(looks_private(&f("https://www.patreon.com/rss/x?auth=abc123&show=2073588")));
|
assert!(looks_private(&f("https://www.patreon.com/rss/x?auth=abc123&show=2073588")));
|
||||||
assert!(looks_private(&f("https://example.com/rss?api_key=abc")));
|
assert!(looks_private(&f("https://example.com/rss?api_key=abc")));
|
||||||
|
// Paid-feed services put the key in the path; the host gives them away.
|
||||||
|
assert!(looks_private(&f("https://feeds.supercast.com/feeds/abcdefghijklmnopqrstuvwx")));
|
||||||
|
assert!(looks_private(&f("https://someshow.supportingcast.fm/content/abc123.rss")));
|
||||||
|
// A long path segment alone is not a key: acast's public show ids look the same.
|
||||||
|
assert!(!looks_private(&f("https://feeds.acast.com/public/shows/0123456789abcdef01234567")));
|
||||||
assert!(looks_private(&f("https://ray:hunter2@example.com/rss")));
|
assert!(looks_private(&f("https://ray:hunter2@example.com/rss")));
|
||||||
assert!(looks_private(&f("not a url")), "unparseable is not safe to list");
|
assert!(looks_private(&f("not a url")), "unparseable is not safe to list");
|
||||||
let mut basic = f("https://example.com/rss");
|
let mut basic = f("https://example.com/rss");
|
||||||
@@ -704,7 +781,14 @@ mod tests {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn only_the_last_admin_is_protected() {
|
fn only_the_last_admin_is_protected() {
|
||||||
let u = |id, is_admin| crate::db::User { id, name: format!("u{id}"), pass_hash: None, is_admin };
|
let u = |id, is_admin| crate::db::User {
|
||||||
|
id,
|
||||||
|
name: format!("u{id}"),
|
||||||
|
pass_hash: None,
|
||||||
|
is_admin,
|
||||||
|
created: None,
|
||||||
|
last_login: None,
|
||||||
|
};
|
||||||
assert!(last_admin(&[u(1, true), u(2, false)], 1));
|
assert!(last_admin(&[u(1, true), u(2, false)], 1));
|
||||||
assert!(!last_admin(&[u(1, true), u(2, true)], 1), "another admin remains");
|
assert!(!last_admin(&[u(1, true), u(2, true)], 1), "another admin remains");
|
||||||
assert!(!last_admin(&[u(1, true), u(2, false)], 2), "not an admin at all");
|
assert!(!last_admin(&[u(1, true), u(2, false)], 2), "not an admin at all");
|
||||||
@@ -768,15 +852,6 @@ mod tests {
|
|||||||
"another feed already has that URL"
|
"another feed already has that URL"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn generated_tokens_are_32_hex_chars_and_not_repeated() {
|
|
||||||
let a = generate_token();
|
|
||||||
let b = generate_token();
|
|
||||||
assert_eq!(a.len(), 32);
|
|
||||||
assert!(a.chars().all(|c| c.is_ascii_hexdigit()));
|
|
||||||
assert_ne!(a, b);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Deserialize)]
|
#[derive(Deserialize)]
|
||||||
@@ -789,6 +864,12 @@ struct Page {
|
|||||||
filter: Option<String>,
|
filter: Option<String>,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
q: Option<String>,
|
q: Option<String>,
|
||||||
|
/// A column name and asc or desc. Anything unrecognised is newest first: the name picks a
|
||||||
|
/// fixed expression in the query and never reaches it itself.
|
||||||
|
#[serde(default)]
|
||||||
|
sort: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
dir: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
fn fifty() -> i64 {
|
fn fifty() -> i64 {
|
||||||
@@ -806,20 +887,42 @@ async fn entries(
|
|||||||
Path(id): Path<String>,
|
Path(id): Path<String>,
|
||||||
user: crate::db::User,
|
user: crate::db::User,
|
||||||
Query(page): Query<Page>,
|
Query(page): Query<Page>,
|
||||||
|
) -> Result<Json<EntryPage>, ApiError> {
|
||||||
|
entry_page(&state, user.id, Some(&id), &page)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every subscribed feed's items together, newest first: All Subscriptions.
|
||||||
|
async fn all_entries(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
Query(page): Query<Page>,
|
||||||
|
) -> Result<Json<EntryPage>, ApiError> {
|
||||||
|
entry_page(&state, user.id, None, &page)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One feed's page of items, or every subscribed feed's when `feed` is None.
|
||||||
|
fn entry_page(
|
||||||
|
state: &WebState,
|
||||||
|
user_id: i64,
|
||||||
|
feed: Option<&str>,
|
||||||
|
page: &Page,
|
||||||
) -> Result<Json<EntryPage>, ApiError> {
|
) -> Result<Json<EntryPage>, ApiError> {
|
||||||
let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all"));
|
let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all"));
|
||||||
let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty());
|
let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty());
|
||||||
let mut rows = state
|
let db = &state.ctx.db;
|
||||||
.ctx
|
let order = crate::db::order_sql(
|
||||||
.db
|
page.sort.as_deref().unwrap_or("published"),
|
||||||
.entries(user.id, &id, filter, search, page.offset, page.limit.clamp(1, 200))?;
|
page.dir.as_deref().unwrap_or("desc"),
|
||||||
|
);
|
||||||
|
let mut rows =
|
||||||
|
db.entries_in(user_id, feed, filter, search, page.offset, page.limit.clamp(1, 200), &order)?;
|
||||||
// Feed HTML is untrusted: it reaches the page only after ammonia has been through it.
|
// Feed HTML is untrusted: it reaches the page only after ammonia has been through it.
|
||||||
for row in &mut rows {
|
for row in &mut rows {
|
||||||
if let Some(d) = &row.description {
|
if let Some(d) = &row.description {
|
||||||
row.description = Some(ammonia::clean(d));
|
row.description = Some(ammonia::clean(d));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
let total = state.ctx.db.count_entries(user.id, &id, filter, search)?;
|
let total = db.count_in(user_id, feed, filter, search)?;
|
||||||
Ok(Json(EntryPage { total, entries: rows }))
|
Ok(Json(EntryPage { total, entries: rows }))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -830,6 +933,19 @@ struct NewFeed {
|
|||||||
folder: Option<String>,
|
folder: Option<String>,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
keywords: Vec<String>,
|
keywords: Vec<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
allow_explicit: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The Add feed dialog's explicit box. Like everything on a feed's own dialog it is yours, so it
|
||||||
|
/// goes on your subscription, and before the first scan, which would otherwise skip every
|
||||||
|
/// explicit item.
|
||||||
|
fn explicit_on_add(state: &WebState, user_id: i64, feed_id: &str, allow: bool) -> Result<(), ApiError> {
|
||||||
|
if allow {
|
||||||
|
let sub = crate::db::Sub { feed_id: feed_id.to_owned(), allow_explicit: Some(true), ..Default::default() };
|
||||||
|
state.ctx.db.set_subscription(user_id, &sub)?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn add_feed(
|
async fn add_feed(
|
||||||
@@ -838,25 +954,42 @@ async fn add_feed(
|
|||||||
Json(body): Json<NewFeed>,
|
Json(body): Json<NewFeed>,
|
||||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
let mut cfg = (*state.ctx.cfg()).clone();
|
let mut cfg = (*state.ctx.cfg()).clone();
|
||||||
|
let url = crate::feed::expand_input(&body.url);
|
||||||
// Someone else may already have it. Then adding costs nothing: no second fetch, no
|
// Someone else may already have it. Then adding costs nothing: no second fetch, no
|
||||||
// second copy on disk, just another name against the same feed.
|
// second copy on disk, just another name against the same feed.
|
||||||
if let Some(existing) = crate::subscriptions(&state.ctx)?
|
if let Some(existing) = crate::subscriptions(&state.ctx)?
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.find(|s| s.cfg.url == body.url)
|
.find(|s| crate::feed::same_feed(&s.cfg.url, &url))
|
||||||
{
|
{
|
||||||
let already = state.ctx.db.subscription(user.id, &existing.id)?.is_some();
|
let already = state.ctx.db.subscription(user.id, &existing.id)?.is_some();
|
||||||
state.ctx.db.subscribe(user.id, &existing.id)?;
|
state.ctx.db.subscribe(user.id, &existing.id)?;
|
||||||
|
if !already {
|
||||||
|
explicit_on_add(&state, user.id, &existing.id, body.allow_explicit)?;
|
||||||
|
}
|
||||||
|
scan_soon(&state, Some(existing.id.clone())).await;
|
||||||
return Ok(Json(
|
return Ok(Json(
|
||||||
serde_json::json!({ "id": existing.id, "existing": already }),
|
serde_json::json!({ "id": existing.id, "existing": already }),
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
let id = crate::add_one(&state.ctx, &mut cfg, &body.url, body.folder, body.keywords).await?;
|
let id = crate::add_one(&state.ctx, &mut cfg, &url, body.folder, body.keywords).await?;
|
||||||
cfg.save(&state.config_path)?;
|
cfg.save(&state.config_path)?;
|
||||||
state.ctx.reload_cfg(&state.config_path)?;
|
state.ctx.reload_cfg(&state.config_path)?;
|
||||||
state.ctx.db.subscribe(user.id, &id)?;
|
state.ctx.db.subscribe(user.id, &id)?;
|
||||||
|
explicit_on_add(&state, user.id, &id, body.allow_explicit)?;
|
||||||
|
scan_soon(&state, Some(id.clone())).await;
|
||||||
Ok(Json(serde_json::json!({ "id": id, "existing": false })))
|
Ok(Json(serde_json::json!({ "id": id, "existing": false })))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Queues a scan, so a feed just added shows its items without anyone pressing Scan now.
|
||||||
|
/// `None` scans whatever is due, which a feed never checked always is. The add has
|
||||||
|
/// succeeded either way, so a daemon not taking commands is only logged.
|
||||||
|
async fn scan_soon(state: &WebState, feed: Option<String>) {
|
||||||
|
let force = feed.is_some();
|
||||||
|
if state.cmds.send(Command::Fetch { feed, force }).await.is_err() {
|
||||||
|
tracing::warn!("could not queue a scan: the daemon is not accepting commands");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Absent means "leave alone"; JSON `null` means "clear this".
|
/// Absent means "leave alone"; JSON `null` means "clear this".
|
||||||
///
|
///
|
||||||
/// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a
|
/// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a
|
||||||
@@ -999,7 +1132,7 @@ async fn remove_feed(
|
|||||||
{
|
{
|
||||||
state.ctx.db.unsubscribe(user.id, &child.id)?;
|
state.ctx.db.unsubscribe(user.id, &child.id)?;
|
||||||
}
|
}
|
||||||
if state.ctx.db.subscriber_count(&id)? > 0 {
|
if state.ctx.db.subscriber_counts()?.contains_key(&id) {
|
||||||
return Ok(StatusCode::NO_CONTENT);
|
return Ok(StatusCode::NO_CONTENT);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1089,9 +1222,9 @@ async fn delete_file(
|
|||||||
let complaint = match (starred, unread) {
|
let complaint = match (starred, unread) {
|
||||||
(0, 0) => None,
|
(0, 0) => None,
|
||||||
(0, u) => Some(format!("{} subscribed to this feed {} not played it yet", people(u), if u == 1 { "has" } else { "have" })),
|
(0, u) => Some(format!("{} subscribed to this feed {} not played it yet", people(u), if u == 1 { "has" } else { "have" })),
|
||||||
(st, 0) => Some(format!("another {} starred it to keep", people(st))),
|
(st, 0) => Some(format!("another {} kept it", people(st))),
|
||||||
(st, u) => Some(format!(
|
(st, u) => Some(format!(
|
||||||
"another {} starred it to keep, and {} not played it yet",
|
"another {} kept it, and {} not played it yet",
|
||||||
people(st),
|
people(st),
|
||||||
if u == 1 { "one person has".to_string() } else { format!("{u} have") }
|
if u == 1 { "one person has".to_string() } else { format!("{u} have") }
|
||||||
)),
|
)),
|
||||||
@@ -1140,9 +1273,20 @@ async fn fetch_now(
|
|||||||
|
|
||||||
/// The same broadcast the socket clients read, as server-sent events.
|
/// The same broadcast the socket clients read, as server-sent events.
|
||||||
async fn events(State(state): State<WebState>) -> Sse<impl futures_util::Stream<Item = Result<SseEvent, std::convert::Infallible>>> {
|
async fn events(State(state): State<WebState>) -> Sse<impl futures_util::Stream<Item = Result<SseEvent, std::convert::Infallible>>> {
|
||||||
let stream = BroadcastStream::new(state.events.subscribe()).filter_map(|ev| async move {
|
// A client that falls behind skips what it missed rather than being cut off.
|
||||||
let ev = ev.ok()?;
|
let stream = futures_util::stream::unfold(state.events.subscribe(), |mut rx| async move {
|
||||||
Some(Ok(SseEvent::default().data(serde_json::to_string(&ev).ok()?)))
|
loop {
|
||||||
|
match rx.recv().await {
|
||||||
|
Ok(ev) => {
|
||||||
|
if let Ok(data) = serde_json::to_string(&ev) {
|
||||||
|
let ev = Ok::<_, std::convert::Infallible>(SseEvent::default().data(data));
|
||||||
|
return Some((ev, rx));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(broadcast::error::RecvError::Lagged(_)) => {}
|
||||||
|
Err(broadcast::error::RecvError::Closed) => return None,
|
||||||
|
}
|
||||||
|
}
|
||||||
});
|
});
|
||||||
Sse::new(stream).keep_alive(axum::response::sse::KeepAlive::default())
|
Sse::new(stream).keep_alive(axum::response::sse::KeepAlive::default())
|
||||||
}
|
}
|
||||||
@@ -1197,6 +1341,18 @@ async fn read_all(
|
|||||||
Ok(Json(serde_json::json!({ "marked": n })))
|
Ok(Json(serde_json::json!({ "marked": n })))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Everything read in every feed you subscribe to: exactly what All Subscriptions lists, since
|
||||||
|
/// that view is scoped by the same subscriptions.
|
||||||
|
async fn read_all_mine(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
|
let ids: Vec<String> =
|
||||||
|
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
|
||||||
|
let n = state.ctx.db.mark_all_read(user.id, &ids)?;
|
||||||
|
Ok(Json(serde_json::json!({ "marked": n })))
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Deserialize)]
|
#[derive(Deserialize)]
|
||||||
struct HowMany {
|
struct HowMany {
|
||||||
#[serde(default = "five")]
|
#[serde(default = "five")]
|
||||||
@@ -1285,6 +1441,9 @@ async fn import_opml(
|
|||||||
let doc = opml::OPML::from_str(&body.xml)
|
let doc = opml::OPML::from_str(&body.xml)
|
||||||
.map_err(|e| ApiError::bad_request(format!("that is not an OPML file: {e}")))?;
|
.map_err(|e| ApiError::bad_request(format!("that is not an OPML file: {e}")))?;
|
||||||
let (added, already) = crate::subscribe_opml(&state.ctx, &state.config_path, &doc, user.id)?;
|
let (added, already) = crate::subscribe_opml(&state.ctx, &state.config_path, &doc, user.id)?;
|
||||||
|
if added > 0 {
|
||||||
|
scan_soon(&state, None).await;
|
||||||
|
}
|
||||||
Ok(Json(serde_json::json!({ "added": added, "already": already })))
|
Ok(Json(serde_json::json!({ "added": added, "already": already })))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1338,8 +1497,6 @@ async fn patch_settings(
|
|||||||
)));
|
)));
|
||||||
}
|
}
|
||||||
cfg.general.schedule = sched;
|
cfg.general.schedule = sched;
|
||||||
// The legacy key would otherwise keep shadowing intent in the file.
|
|
||||||
cfg.general.interval_mins = None;
|
|
||||||
}
|
}
|
||||||
if let Some(v) = body.max_new_per_check {
|
if let Some(v) = body.max_new_per_check {
|
||||||
cfg.general.max_new_per_check = v;
|
cfg.general.max_new_per_check = v;
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ const script = html.split('<script>')[1].split('</script>')[0];
|
|||||||
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
|
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
|
||||||
|
|
||||||
const missing = [];
|
const missing = [];
|
||||||
const el = (name) => new Proxy({ style: {}, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
|
const el = (name) => new Proxy({ style: { setProperty(){}, getPropertyValue(){ return ''; } }, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
|
||||||
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
|
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
|
||||||
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
|
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
|
||||||
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
|
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
|
||||||
@@ -46,7 +46,10 @@ const ctx = {
|
|||||||
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
|
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
|
||||||
: String(url).includes('/api/users')
|
: String(url).includes('/api/users')
|
||||||
? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }]
|
? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }]
|
||||||
: []),
|
: /\/api\/(popular|directory)/.test(String(url))
|
||||||
|
? [{ id: 'f', title: 'A Feed', image: null, subscribers: 2, subscribed: true },
|
||||||
|
{ id: 'g', title: null, image: null, subscribers: 1, subscribed: false }]
|
||||||
|
: /entries/.test(String(url)) ? { total: 0, entries: [] } : []),
|
||||||
}),
|
}),
|
||||||
EventSource: function () { this.close = () => {}; },
|
EventSource: function () { this.close = () => {}; },
|
||||||
MediaMetadata: function () {},
|
MediaMetadata: function () {},
|
||||||
@@ -83,6 +86,9 @@ const drive = [
|
|||||||
['prefsModal', () => ctx.prefsModal()],
|
['prefsModal', () => ctx.prefsModal()],
|
||||||
['usersModal', () => ctx.usersModal()],
|
['usersModal', () => ctx.usersModal()],
|
||||||
['opmlModal', () => ctx.opmlModal()],
|
['opmlModal', () => ctx.opmlModal()],
|
||||||
|
['selectFeed (directory)', () => ctx.selectFeed(':directory')],
|
||||||
|
['selectFeed (popular)', () => ctx.selectFeed(':popular')],
|
||||||
|
['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
|
||||||
['logsModal', () => ctx.logsModal()],
|
['logsModal', () => ctx.logsModal()],
|
||||||
// `const S` is not reachable from here: top-level const/let do not become properties
|
// `const S` is not reachable from here: top-level const/let do not become properties
|
||||||
// of a vm context the way var and function declarations do.
|
// of a vm context the way var and function declarations do.
|
||||||
|
|||||||
@@ -29,6 +29,18 @@ test('the theme toggle actually changes the theme', async ({ page }) => {
|
|||||||
.not.toBe(before);
|
.not.toBe(before);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('the theme button steps through dark, light and classic, and remembers', async ({ page }) => {
|
||||||
|
const theme = () => page.evaluate(() => document.documentElement.dataset.theme);
|
||||||
|
for (let i = 0; i < 3 && (await theme()) !== 'classic'; i++) await page.locator('#theme').click();
|
||||||
|
expect(await theme()).toBe('classic');
|
||||||
|
await expect(page.locator('#theme')).toHaveAttribute('title', /Classic.*Click for Dark/);
|
||||||
|
|
||||||
|
await page.reload();
|
||||||
|
await expect.poll(theme).toBe('classic');
|
||||||
|
// The 2004 Mac app set its type in Lucida Grande.
|
||||||
|
expect(await page.evaluate(() => getComputedStyle(document.body).fontFamily)).toContain('Lucida Grande');
|
||||||
|
});
|
||||||
|
|
||||||
test('settings opens and saves the global schedule', async ({ page }) => {
|
test('settings opens and saves the global schedule', async ({ page }) => {
|
||||||
await page.locator('#prefs').click();
|
await page.locator('#prefs').click();
|
||||||
await expect(page.locator('#modal.on')).toBeVisible();
|
await expect(page.locator('#modal.on')).toBeVisible();
|
||||||
@@ -69,20 +81,21 @@ test('the three panes are there and the item text lands in the bottom one', asyn
|
|||||||
|
|
||||||
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
||||||
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||||
// The enclosure travels with the item, into the same pane.
|
// The enclosure goes to the Files pane beside the list, as the original's did.
|
||||||
await expect(page.locator('#detail .encbox')).toHaveCount(1);
|
await expect(page.locator('#files')).toBeVisible();
|
||||||
|
await expect(page.locator('#files .encbox')).toHaveCount(1);
|
||||||
|
|
||||||
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
|
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
|
||||||
// its chip rather than assuming which episode the daemon happened to fetch.
|
// its chip rather than assuming which episode the daemon happened to fetch.
|
||||||
const downloaded = page.locator('.ep', { hasText: 'downloaded' }).first();
|
const downloaded = page.locator('.ep', { has: page.locator('.kind.here') }).first();
|
||||||
await downloaded.click();
|
await downloaded.click();
|
||||||
await expect(page.locator('#detail audio')).toBeVisible();
|
await expect(page.locator('#files [data-a="play"]')).toBeVisible();
|
||||||
await expect(page.locator('#detail .encbox .btn', { hasText: 'Save' })).toBeVisible();
|
await expect(page.locator('#files .encbox [title="Save to this computer"]')).toBeVisible();
|
||||||
|
|
||||||
// Selecting another item replaces the pane rather than stacking.
|
// Selecting another item replaces the pane rather than stacking.
|
||||||
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
||||||
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||||
await expect(page.locator('#detail audio')).toHaveCount(0);
|
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
||||||
@@ -96,12 +109,12 @@ test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
|||||||
await row.click();
|
await row.click();
|
||||||
|
|
||||||
await expect(page.locator('#detail .dt')).toHaveText('An Article');
|
await expect(page.locator('#detail .dt')).toHaveText('An Article');
|
||||||
await expect(page.locator('#detail audio')).toHaveCount(0);
|
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
|
||||||
await expect(page.locator('#detail .encbox')).toContainText('image');
|
// What it is and that it is here: one icon, green, with the words in its tooltip.
|
||||||
await expect(page.locator('#detail .encbox')).toContainText('downloaded');
|
await expect(page.locator('#files .encbox .kind.here')).toHaveAttribute('title', 'image, downloaded');
|
||||||
// Still offered as a file, just not as an episode: viewable and keepable.
|
// Still offered as a file, just not as an episode: viewable and keepable.
|
||||||
await expect(page.locator('#detail .btn', { hasText: 'Save' })).toBeVisible();
|
await expect(page.locator('#files [title="Save to this computer"]')).toBeVisible();
|
||||||
const view = page.locator('#detail a', { hasText: 'View' });
|
const view = page.locator('#files a[title="View in a new tab"]');
|
||||||
await expect(view).toHaveAttribute('target', '_blank');
|
await expect(view).toHaveAttribute('target', '_blank');
|
||||||
await expect(view).toHaveAttribute('rel', /noopener/);
|
await expect(view).toHaveAttribute('rel', /noopener/);
|
||||||
await expect(view).toHaveAttribute('href', /\/media\/\d+/);
|
await expect(view).toHaveAttribute('href', /\/media\/\d+/);
|
||||||
@@ -115,9 +128,9 @@ test('an item with several enclosures lists them all', async ({ page }) => {
|
|||||||
await expect(row).toContainText('+1 more file');
|
await expect(row).toContainText('+1 more file');
|
||||||
|
|
||||||
await row.click();
|
await row.click();
|
||||||
// The pane below lists every one: the audio and the image.
|
// The Files pane lists every one: the audio and the image.
|
||||||
await expect(page.locator('#detail .encbox')).toHaveCount(2);
|
await expect(page.locator('#files .encbox')).toHaveCount(2);
|
||||||
await expect(page.locator('#detail .encbox').nth(1)).toContainText('image');
|
await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
|
||||||
});
|
});
|
||||||
|
|
||||||
test('the filter tabs change what is listed', async ({ page }) => {
|
test('the filter tabs change what is listed', async ({ page }) => {
|
||||||
@@ -129,13 +142,13 @@ test('the filter tabs change what is listed', async ({ page }) => {
|
|||||||
await page.locator('.tabs button', { hasText: 'Unread' }).first().click();
|
await page.locator('.tabs button', { hasText: 'Unread' }).first().click();
|
||||||
expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all);
|
expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all);
|
||||||
|
|
||||||
await page.locator('.tabs button', { hasText: 'Flagged' }).first().click();
|
await page.locator('.tabs button', { hasText: 'Kept' }).first().click();
|
||||||
await expect(page.locator('#count')).toContainText('0 items');
|
await expect(page.locator('#count')).toContainText('0 items');
|
||||||
});
|
});
|
||||||
|
|
||||||
test('a feed URL is editable and has a copy button', async ({ page }) => {
|
test('a feed URL is editable and has a copy button', async ({ page }) => {
|
||||||
await page.getByText('Test Show').click();
|
await page.getByText('Test Show').click();
|
||||||
await page.locator('.btn', { hasText: 'Settings' }).first().click();
|
await page.locator('#content .acts [data-a="settings"]').click();
|
||||||
await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
|
await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
|
||||||
await expect(page.locator('#scopy')).toBeVisible();
|
await expect(page.locator('#scopy')).toBeVisible();
|
||||||
|
|
||||||
@@ -174,9 +187,11 @@ test('an OPML subscription is a collapsible folder', async ({ page }) => {
|
|||||||
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
|
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
|
||||||
}));
|
}));
|
||||||
|
|
||||||
// Every row reserves the chevron slot for alignment; only a folder's is clickable.
|
// Only a folder has a triangle, and it is a button that says whether the folder is open.
|
||||||
const chev = page.locator('.feed.group .chev');
|
const chev = page.locator('.feed.group .chev');
|
||||||
await expect(chev).toBeVisible({ timeout: 20_000 });
|
await expect(chev).toBeVisible({ timeout: 20_000 });
|
||||||
|
await expect(page.locator('.feed:not(.group) .chev')).toHaveCount(0);
|
||||||
|
await expect(chev).toHaveAttribute('aria-expanded', 'false');
|
||||||
|
|
||||||
// Closed by default: the children are not listed until the folder is opened.
|
// Closed by default: the children are not listed until the folder is opened.
|
||||||
const before = await page.locator('.feed').count();
|
const before = await page.locator('.feed').count();
|
||||||
@@ -212,7 +227,7 @@ test('inside an OPML, feeds with unread items are listed first', async ({ page }
|
|||||||
await side.getByText('Grouped Show').click();
|
await side.getByText('Grouped Show').click();
|
||||||
const gs = page.locator('.ep', { hasText: 'Grouped Ep' });
|
const gs = page.locator('.ep', { hasText: 'Grouped Ep' });
|
||||||
await gs.click();
|
await gs.click();
|
||||||
await page.locator('#detail button', { hasText: 'Mark unread' }).click();
|
await page.locator('#detail button[title="Mark unread"]').click();
|
||||||
await expect(gs).not.toHaveClass(/read/);
|
await expect(gs).not.toHaveClass(/read/);
|
||||||
|
|
||||||
// Aardvark comes first alphabetically and in the OPML, so only the unread sort puts
|
// Aardvark comes first alphabetically and in the OPML, so only the unread sort puts
|
||||||
@@ -238,7 +253,7 @@ test('marking an OPML subscription read covers the feeds inside it', async ({ pa
|
|||||||
await expect(folder.locator('.badge')).not.toHaveText('0');
|
await expect(folder.locator('.badge')).not.toHaveText('0');
|
||||||
await folder.click();
|
await folder.click();
|
||||||
|
|
||||||
await page.locator('#content .acts button', { hasText: 'Mark all read' }).click();
|
await page.locator('#content .acts [data-a="read"]').click();
|
||||||
await expect(folder.locator('.badge')).toHaveText('0');
|
await expect(folder.locator('.badge')).toHaveText('0');
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -280,9 +295,9 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
|
|||||||
// Another test may have opened this item already, so start from a known state: the
|
// Another test may have opened this item already, so start from a known state: the
|
||||||
// toggle in the text below flips it back -- which used to recurse until the stack blew.
|
// toggle in the text below flips it back -- which used to recurse until the stack blew.
|
||||||
await row().click();
|
await row().click();
|
||||||
await page.locator('#detail button', { hasText: 'Mark unread' }).click();
|
await page.locator('#detail button[title="Mark unread"]').click();
|
||||||
await expect(row()).not.toHaveClass(/read/);
|
await expect(row()).not.toHaveClass(/read/);
|
||||||
await expect(page.locator('#detail button', { hasText: 'Mark read' })).toBeVisible();
|
await expect(page.locator('#detail button[title="Mark read"]')).toBeVisible();
|
||||||
|
|
||||||
// Opening it is reading it.
|
// Opening it is reading it.
|
||||||
await row().click();
|
await row().click();
|
||||||
@@ -291,6 +306,30 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
|
|||||||
expect(errors).toEqual([]);
|
expect(errors).toEqual([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('the toolbar acts on the selected item', async ({ page }) => {
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
|
||||||
|
await expect(row()).toBeVisible({ timeout: 20_000 });
|
||||||
|
// Nothing selected, nothing to act on.
|
||||||
|
await expect(page.locator('#tbRead')).toBeDisabled();
|
||||||
|
|
||||||
|
await row().click(); // opening it reads it
|
||||||
|
await expect(row()).toHaveClass(/read/);
|
||||||
|
await page.locator('#tbRead').click();
|
||||||
|
await expect(row()).not.toHaveClass(/read/);
|
||||||
|
|
||||||
|
await page.locator('#tbFlag').click();
|
||||||
|
await expect(row().locator('.fl')).toHaveClass(/on/);
|
||||||
|
await page.locator('#tbFlag').click(); // and back, so later tests see it unkept
|
||||||
|
await expect(row().locator('.fl')).not.toHaveClass(/on/);
|
||||||
|
|
||||||
|
// Second Episode is the one the daemon downloaded, so it plays from the toolbar.
|
||||||
|
await expect(page.locator('#tbPlay')).toBeEnabled();
|
||||||
|
await page.locator('#tbPlay').click();
|
||||||
|
await expect(page.locator('#player')).toBeVisible();
|
||||||
|
await page.locator('#pclose').click();
|
||||||
|
});
|
||||||
|
|
||||||
test('a second person has their own feeds and their own read state', async ({ browser }) => {
|
test('a second person has their own feeds and their own read state', async ({ browser }) => {
|
||||||
const { execFileSync } = require('child_process');
|
const { execFileSync } = require('child_process');
|
||||||
const setup = require('./global-setup');
|
const setup = require('./global-setup');
|
||||||
@@ -309,6 +348,10 @@ test('a second person has their own feeds and their own read state', async ({ br
|
|||||||
const ctx = await browser.newContext();
|
const ctx = await browser.newContext();
|
||||||
const page = await ctx.newPage();
|
const page = await ctx.newPage();
|
||||||
await page.goto('/login');
|
await page.goto('/login');
|
||||||
|
// The sign-in page shows the icon, so it has to load before anyone has signed in.
|
||||||
|
const icon = await page.request.get('/icon.png');
|
||||||
|
expect(icon.status()).toBe(200);
|
||||||
|
expect(icon.headers()['content-type']).toBe('image/png');
|
||||||
await page.locator('#name').fill('sam');
|
await page.locator('#name').fill('sam');
|
||||||
await page.locator('#pw').fill('sampassword');
|
await page.locator('#pw').fill('sampassword');
|
||||||
await page.locator('button[type=submit]').click();
|
await page.locator('button[type=submit]').click();
|
||||||
@@ -350,8 +393,9 @@ test('deleting a shared file warns that it is everyone\'s copy', async ({ page }
|
|||||||
await expect(row).toBeVisible({ timeout: 20_000 });
|
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||||
await row.click();
|
await row.click();
|
||||||
|
|
||||||
const del = page.locator('#detail button', { hasText: 'Delete' });
|
// An icon now; what it does, and to whom, is in its tooltip.
|
||||||
await expect(del).toHaveText('Delete for everyone');
|
const del = page.locator('#files button[data-a="del"]');
|
||||||
|
await expect(del).toHaveAttribute('title', /^Delete for everyone \(shared with 1 other person/);
|
||||||
|
|
||||||
// Two prompts: the page's own, then the server's, because someone else has not played
|
// Two prompts: the page's own, then the server's, because someone else has not played
|
||||||
// it. Accept the first, decline the second, and the file must survive.
|
// it. Accept the first, decline the second, and the file must survive.
|
||||||
@@ -389,6 +433,9 @@ test('an admin adds someone, makes them an admin, and removes them', async ({ pa
|
|||||||
await page.locator('#uadd').click();
|
await page.locator('#uadd').click();
|
||||||
const row = userRow(page, 'pat');
|
const row = userRow(page, 'pat');
|
||||||
await expect(row).toBeVisible();
|
await expect(row).toBeVisible();
|
||||||
|
// When each account was added and last signed in; the admin signed in with the token link.
|
||||||
|
await expect(row).toContainText(/Added .* never signed in/);
|
||||||
|
await expect(userRow(page, 'admin')).toContainText(/signed in \d+m ago/);
|
||||||
await expect(row.locator('[data-a="admin"]')).not.toBeChecked();
|
await expect(row.locator('[data-a="admin"]')).not.toBeChecked();
|
||||||
|
|
||||||
await row.locator('[data-a="admin"]').check();
|
await row.locator('[data-a="admin"]').check();
|
||||||
@@ -417,7 +464,7 @@ test('Settings exports your OPML and imports a pasted one', async ({ page }) =>
|
|||||||
await page.locator('#prefs').click();
|
await page.locator('#prefs').click();
|
||||||
const [dl] = await Promise.all([
|
const [dl] = await Promise.all([
|
||||||
page.waitForEvent('download'),
|
page.waitForEvent('download'),
|
||||||
page.locator('#modalCard a', { hasText: 'Export OPML' }).click(),
|
page.locator('#modalCard a[title="Export OPML"]').click(),
|
||||||
]);
|
]);
|
||||||
expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml');
|
expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml');
|
||||||
const out = require('fs').readFileSync(await dl.path(), 'utf8');
|
const out = require('fs').readFileSync(await dl.path(), 'utf8');
|
||||||
@@ -545,7 +592,7 @@ test('ipx import subscribes the admin, and ipx export writes the feeds out', asy
|
|||||||
expect(xml).toContain('http://127.0.0.1:8792/two.xml');
|
expect(xml).toContain('http://127.0.0.1:8792/two.xml');
|
||||||
});
|
});
|
||||||
|
|
||||||
test('Add feed offers what other people here read, but never a private feed', async ({ browser }) => {
|
test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
|
||||||
const { execFileSync } = require('child_process');
|
const { execFileSync } = require('child_process');
|
||||||
const setup = require('./global-setup');
|
const setup = require('./global-setup');
|
||||||
const env = {
|
const env = {
|
||||||
@@ -566,11 +613,11 @@ test('Add feed offers what other people here read, but never a private feed', as
|
|||||||
await piper.locator('button[type=submit]').click();
|
await piper.locator('button[type=submit]').click();
|
||||||
await expect(piper.locator('#feedlist')).toContainText('No feeds.');
|
await expect(piper.locator('#feedlist')).toContainText('No feeds.');
|
||||||
|
|
||||||
await piper.locator('#addFeed').click();
|
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||||
const offered = piper.locator('#popular .childrow');
|
const offered = piper.locator('#popular .childrow');
|
||||||
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
// An OPML's own feeds ride on the OPML, and a key in a URL marks someone's paid feed.
|
// An OPML is listed as the feeds inside it, and a key in a URL marks someone's paid feed.
|
||||||
await expect(offered.filter({ hasText: /Grouped Show|grouped-show/ })).toHaveCount(0);
|
await expect(offered.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
|
||||||
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||||
|
|
||||||
// No URL reaches the page at all, so neither can a key, and the server holds the same line.
|
// No URL reaches the page at all, so neither can a key, and the server holds the same line.
|
||||||
@@ -579,9 +626,187 @@ test('Add feed offers what other people here read, but never a private feed', as
|
|||||||
expect(listed).not.toContain('.xml');
|
expect(listed).not.toContain('.xml');
|
||||||
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
|
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
|
||||||
|
|
||||||
await offered.filter({ hasText: 'Test Show' }).locator('button', { hasText: 'Subscribe' }).click();
|
// Popular is the top ten of the directory, and the directory is every listed feed A to Z,
|
||||||
|
// with an OPML's feeds in place of the OPML in both.
|
||||||
|
const dir = await (await piper.request.get('/api/directory')).json();
|
||||||
|
const top = await (await piper.request.get('/api/popular')).json();
|
||||||
|
const names = dir.map(p => (p.title || p.id).toLowerCase());
|
||||||
|
expect(names).toEqual([...names].sort());
|
||||||
|
const ids = dir.map(p => p.id);
|
||||||
|
expect(ids).not.toContain('test-subscriptions');
|
||||||
|
expect(ids).toEqual(expect.arrayContaining(['grouped-show', 'aardvark-radio']));
|
||||||
|
expect(top.length).toBe(Math.min(10, dir.length));
|
||||||
|
expect(top.every(t => ids.includes(t.id))).toBe(true);
|
||||||
|
expect(ids).not.toContain('paid-show');
|
||||||
|
|
||||||
|
// Subscribe from the directory this time; the popular list shares the same rows.
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||||
|
await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible();
|
||||||
|
await expect(offered.filter({ hasText: /Grouped Show|grouped-show/ })).toBeVisible();
|
||||||
|
await expect(offered.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
|
||||||
|
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||||
|
|
||||||
|
const row = async () =>
|
||||||
|
(await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
|
||||||
|
const before = await row();
|
||||||
|
expect(before.subscribed).toBe(false);
|
||||||
|
await offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
|
||||||
await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
// Once it is yours, it is no longer offered.
|
|
||||||
expect(await (await piper.request.get('/api/popular')).text()).not.toContain('"test-show"');
|
// Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
|
||||||
|
expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
|
||||||
|
|
||||||
|
// All Subscriptions is every item from piper's feeds and only those: the admin's Picture
|
||||||
|
// Blog is not among them.
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'All Subscriptions' }).click();
|
||||||
|
const first = piper.locator('.ep', { hasText: 'First Episode' });
|
||||||
|
await expect(first).toBeVisible({ timeout: 20_000 });
|
||||||
|
await expect(first.locator('.fd')).toHaveText('Test Show');
|
||||||
|
await expect(piper.locator('.ep', { hasText: 'An Article' })).toHaveCount(0);
|
||||||
|
await expect(piper.locator('#count')).toContainText('All Subscriptions:');
|
||||||
|
await ctx.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('adding a feed scans it straight away', async ({ page }) => {
|
||||||
|
await page.locator('#addFeed').click();
|
||||||
|
await page.locator('#nurl').fill('http://127.0.0.1:8792/fresh.xml');
|
||||||
|
await page.locator('#nsave').click();
|
||||||
|
// Nobody pressed Scan. The scheduler's tick is a minute, so this is the add scanning it.
|
||||||
|
await expect(page.locator('.ep', { hasText: 'Fresh Ep' })).toBeVisible({ timeout: 10_000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a deleted file looks as if it was never downloaded', async ({ page }) => {
|
||||||
|
// Other people subscribe to Picture Blog by now, so both prompts come; take them.
|
||||||
|
page.on('dialog', d => d.accept());
|
||||||
|
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
|
||||||
|
const row = page.locator('.ep', { hasText: 'An Article' });
|
||||||
|
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||||
|
await row.click();
|
||||||
|
await page.locator('#files button[data-a="del"]').click();
|
||||||
|
|
||||||
|
// No "reaped", no chip at all: just the way to get it again.
|
||||||
|
await expect(row.locator('.kind.here')).toHaveCount(0);
|
||||||
|
await expect(row).not.toContainText(/reaped/i);
|
||||||
|
await row.click();
|
||||||
|
await expect(page.locator('#files')).not.toContainText(/reaped/i);
|
||||||
|
await expect(page.locator('#files button[title="Download to the server"]')).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('one action, one icon: the toolbar, the page and every dialog agree', async ({ page }) => {
|
||||||
|
const icon = loc => loc.locator('svg path').first().getAttribute('d');
|
||||||
|
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
|
||||||
|
|
||||||
|
// Unsubscribe is a minus in the toolbar and the feed header, never the x that closes things.
|
||||||
|
expect(await icon(page.locator('#content .acts [data-a="rm"]'))).toBe(await icon(page.locator('#tbRemove')));
|
||||||
|
|
||||||
|
// The toolbar's read and keep show the selected item's state, as its own buttons do, and follow
|
||||||
|
// a change made from the toolbar.
|
||||||
|
await page.locator('.ep').first().click();
|
||||||
|
const pair = async a => [await icon(page.locator(a === 'read' ? '#tbRead' : '#tbFlag')),
|
||||||
|
await icon(page.locator(`#detail [data-a="${a}"]`))];
|
||||||
|
for (const a of ['read', 'flag']) { const [tb, own] = await pair(a); expect(tb).toBe(own); }
|
||||||
|
const [kept] = await pair('flag');
|
||||||
|
await page.locator('#tbFlag').click();
|
||||||
|
await expect.poll(async () => { const [tb, own] = await pair('flag'); return tb === own && tb !== kept; }).toBe(true);
|
||||||
|
await page.locator('#tbFlag').click(); // leave it as it was
|
||||||
|
await expect.poll(async () => (await pair('flag'))[0]).toBe(kept);
|
||||||
|
|
||||||
|
// Every button in every dialog is an icon with its words in the tooltip.
|
||||||
|
const dialogs = [
|
||||||
|
() => page.locator('#addFeed').click(),
|
||||||
|
() => page.locator('#prefs').click(),
|
||||||
|
async () => { await page.locator('#prefs').click(); await page.locator('#gusers').click(); },
|
||||||
|
async () => { await page.locator('#prefs').click(); await page.locator('#gopml').click(); },
|
||||||
|
() => page.locator('#logs').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="settings"]').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="dl"]').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="rm"]').click(),
|
||||||
|
];
|
||||||
|
for (const open of dialogs) {
|
||||||
|
await open();
|
||||||
|
const btns = page.locator('#modalCard .btn');
|
||||||
|
await expect(btns.first()).toBeVisible();
|
||||||
|
for (const b of await btns.all()) {
|
||||||
|
await expect(b.locator('svg')).toHaveCount(1);
|
||||||
|
await expect(b).toHaveAttribute('title', /\S/);
|
||||||
|
}
|
||||||
|
await page.keyboard.press('Escape');
|
||||||
|
await expect(page.locator('#modal.on')).toBeHidden();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('All Subscriptions marks everything read, across every feed', async ({ page }) => {
|
||||||
|
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
|
||||||
|
await all.click();
|
||||||
|
// Earlier tests read things; make sure something is unread. Opening an item reads it, and
|
||||||
|
// its own button makes it unread again.
|
||||||
|
await page.locator('.ep').first().click();
|
||||||
|
await page.locator('#detail [data-a="read"][title="Mark unread"]').click();
|
||||||
|
await expect(all.locator('.badge')).not.toHaveText('0');
|
||||||
|
|
||||||
|
page.once('dialog', d => d.accept());
|
||||||
|
await page.locator('#content .acts [data-a="readall"]').click();
|
||||||
|
await expect(all.locator('.badge')).toHaveText('0');
|
||||||
|
await page.locator('.tabs button', { hasText: 'Unread' }).click();
|
||||||
|
await expect(page.locator('.ep')).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the item table sorts by any column, both ways, and remembers', async ({ page }) => {
|
||||||
|
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
|
||||||
|
await all.click();
|
||||||
|
const head = k => page.locator(`#list .ephead [data-sort="${k}"]`);
|
||||||
|
const titles = () => page.locator('#eps .ep .t').allTextContents();
|
||||||
|
// Byte order on lower case, which is what SQLite gives for lower(...).
|
||||||
|
const cmp = (a, b) => (a.toLowerCase() < b.toLowerCase() ? -1 : a.toLowerCase() > b.toLowerCase() ? 1 : 0);
|
||||||
|
const sorted = (t, dir) => JSON.stringify(t) === JSON.stringify([...t].sort((a, b) => cmp(a, b) * dir));
|
||||||
|
await expect(page.locator('#eps .ep').nth(2)).toBeVisible({ timeout: 20_000 });
|
||||||
|
expect(new Set(await titles()).size).toBeGreaterThan(2); // or both orders would prove nothing
|
||||||
|
|
||||||
|
await expect(head('title')).toHaveText('Title');
|
||||||
|
await head('title').click();
|
||||||
|
await expect.poll(async () => sorted(await titles(), 1)).toBe(true);
|
||||||
|
await head('title').click();
|
||||||
|
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
|
||||||
|
|
||||||
|
// Kept across a reload.
|
||||||
|
await page.reload();
|
||||||
|
await all.click();
|
||||||
|
await expect(head('title').locator('.arr.desc')).toBeVisible();
|
||||||
|
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
|
||||||
|
|
||||||
|
// Size has its own column; the file column is just what the file is.
|
||||||
|
await expect(page.locator('#eps .ep .size', { hasText: /\d/ }).first()).toBeVisible();
|
||||||
|
await expect(page.locator('#eps .ep .file', { hasText: /\d/ })).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('play in the Files pane plays once, in the player bar', async ({ page }) => {
|
||||||
|
// Regression: the pane had an <audio> of its own, and playing it started the player bar too,
|
||||||
|
// so the same file played twice at once.
|
||||||
|
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
|
||||||
|
await page.locator('.ep', { has: page.locator('.kind.here') }).first().click();
|
||||||
|
await page.locator('#files [data-a="play"]').click();
|
||||||
|
await expect(page.locator('#player')).toBeVisible();
|
||||||
|
await expect(page.locator('audio')).toHaveCount(1); // the player bar's, and nothing else
|
||||||
|
await page.locator('#pclose').click();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('someone the proxy signs in never sees the password page, and signs out through the proxy', async ({ page, browser }) => {
|
||||||
|
// Signed in with the token, not by the proxy: Sign out stays ipx's own.
|
||||||
|
expect((await (await page.request.get('/api/me')).json()).sign_out).toBeNull();
|
||||||
|
|
||||||
|
const ctx = await browser.newContext({ extraHTTPHeaders: { 'X-Test-User': 'proxied@example.com' } });
|
||||||
|
const proxied = await ctx.newPage();
|
||||||
|
// Regression: after Sign out, the password form showed to someone the proxy still vouched for.
|
||||||
|
await proxied.goto('/login');
|
||||||
|
await expect(proxied).toHaveURL(/:8791\/$/);
|
||||||
|
await expect(proxied.locator('#who')).toContainText('proxied@example.com');
|
||||||
|
expect(await (await proxied.request.get('/api/me')).json())
|
||||||
|
.toMatchObject({ name: 'proxied@example.com', sign_out: '/signed-out-by-the-proxy' });
|
||||||
|
await proxied.locator('#signout').click();
|
||||||
|
await expect(proxied).toHaveURL(/\/signed-out-by-the-proxy$/);
|
||||||
await ctx.close();
|
await ctx.close();
|
||||||
});
|
});
|
||||||
|
|||||||
5
tests/ui/fixtures/fresh.xml
Normal file
5
tests/ui/fixtures/fresh.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Fresh Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Added in the browser suite, and scanned by adding it.</description>
|
||||||
|
<item><title>Fresh Ep</title><guid>fresh-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
@@ -37,6 +37,10 @@ enabled = false
|
|||||||
enabled = true
|
enabled = true
|
||||||
bind = "127.0.0.1:8791"
|
bind = "127.0.0.1:8791"
|
||||||
token = "${TOKEN}"
|
token = "${TOKEN}"
|
||||||
|
# The proxy path, for tests that send the header themselves: the daemon sees them at 127.0.0.1.
|
||||||
|
trusted_header = "X-Test-User"
|
||||||
|
trusted_proxies = ["127.0.0.1"]
|
||||||
|
sign_out_url = "/signed-out-by-the-proxy"
|
||||||
|
|
||||||
[feeds.test-show]
|
[feeds.test-show]
|
||||||
url = "http://127.0.0.1:8792/show.xml"
|
url = "http://127.0.0.1:8792/show.xml"
|
||||||
|
|||||||
1039
web/index.html
1039
web/index.html
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user