Compare commits
27 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2af57065c6 | |||
| 564b011c7a | |||
| 9eb7aadced | |||
| c6bceaef37 | |||
| dc63d6acaf | |||
| 8937f35f00 | |||
| 0990f2a90d | |||
| 0cc002cdfa | |||
| 2c9e899762 | |||
| 2e416f96cf | |||
| 9269aa99f7 | |||
| ae8123250b | |||
| 2c34a144ba | |||
| 669e8b5124 | |||
| 5362436766 | |||
| 0efc49519c | |||
| 57dcba2d1a | |||
| e95cccc66f | |||
| 7295be8b25 | |||
| f1b0d97b81 | |||
| ae47e31a97 | |||
| 8a309eb652 | |||
| 1a2b0d87c6 | |||
| d7fac2d0e7 | |||
| df9b7645d6 | |||
| f3825cfc57 | |||
| c0f4b0bcb2 |
137
CHANGELOG.md
137
CHANGELOG.md
@@ -10,6 +10,138 @@ The long form, with what was wrong before and how it was found, is in
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.5.1] - 2026-09-12
|
||||
|
||||
### Fixed
|
||||
|
||||
- The triangle that opens an OPML or Patreon folder was cramped against the folder's art. It has
|
||||
more room now, and a wider target to click.
|
||||
|
||||
## [0.5.0] - 2026-09-12
|
||||
|
||||
### Added
|
||||
|
||||
- A Patreon token pasted into Add feed, or a creator's RSS link without `&show=`, becomes a folder
|
||||
of that creator's shows, kept in step on every scan like a subscribed OPML. A creator with only
|
||||
one show stays a plain feed. One already added as a single long feed is split into its shows on
|
||||
its next scan, keeping its files and what you had read.
|
||||
- Add feed has an "Allow items marked explicit" box, so a new feed's first scan no longer skips
|
||||
every explicit item.
|
||||
|
||||
### Changed
|
||||
|
||||
- Unread counts, unread dots and download progress are amber, the colour of the icon's EQ bars.
|
||||
Blue is kept for the primary action and links, so a count no longer looks like a button.
|
||||
- What is playing is marked by small EQ bars, in its row and in the player. They move only while it
|
||||
plays.
|
||||
- A folder in the sidebar shows its first four shows' art as a mosaic, and its shows sit under its
|
||||
title. Only folders have a triangle, so every feed lines up with Directory and Popular above.
|
||||
- Feeds without art get initials in a colour of their own, instead of all the same grey.
|
||||
- The Flagged tab is called Kept, as the Keep button and Settings already said.
|
||||
- A feed's header is one short line; when it checks next is in its tooltip.
|
||||
- The item list takes more of the window, and the Files pane shows only when the item has files.
|
||||
- Column headings and tags are in sentence case, and fewer things are bold.
|
||||
- Unsubscribe is a round button beside the feed's other actions.
|
||||
- Export and Import in Settings say what they do.
|
||||
- The sign-in page shows the original icon large.
|
||||
- Nothing animates when your system asks for reduced motion.
|
||||
- The pages are about 90 KB smaller: the icon is served once instead of written into each.
|
||||
- A web token generated for a new install is 64 characters instead of 32.
|
||||
- The README is a short overview of what ipx does and how to run it, and points into `docs/` for
|
||||
the rest. It still described the layout from before 0.4.0.
|
||||
|
||||
### Removed
|
||||
|
||||
- The systemd units in `contrib/`. Run ipx with Docker, or point a unit of your own at
|
||||
`ipx daemon`.
|
||||
- Upgrading from before 0.3.0 directly: what was read, kept or part-played before accounts is no
|
||||
longer carried over to the admin, and OPML feeds that old versions wrote into `config.toml` are
|
||||
no longer moved out of it. Upgrade through 0.4.0 first.
|
||||
- `interval_mins` in `config.toml` is ignored; use `schedule`.
|
||||
|
||||
### Fixed
|
||||
|
||||
- The feed list works from the keyboard: Tab reaches every feed, Enter opens it, and Right and Left
|
||||
open and close a folder. Every button shows where the focus is, including in the toolbar, which
|
||||
used to clip the ring.
|
||||
- "1 items" reads "1 item".
|
||||
- A selected feed without art no longer loses its initials tile in Dark and Light.
|
||||
- The player shows the feed's initials when there is no art, not the episode's.
|
||||
- Turning on Allow explicit, or changing keywords or auto-download, brings back what those settings
|
||||
had skipped on the feed's next scan. Before, an item was judged once, when first seen, and a
|
||||
skipped one stayed skipped whatever you changed.
|
||||
- Feeds inside an OPML or a Patreon creator follow your settings on the folder unless you set their
|
||||
own, as the folder's settings dialog said they did. Before, the folder's settings reached nothing
|
||||
inside it.
|
||||
- A new feed no longer takes the name of one you removed earlier and shows that feed's old items.
|
||||
Re-adding the same feed still gets its old name, and its history, back.
|
||||
|
||||
## [0.4.0] - 2026-09-11
|
||||
|
||||
### Added
|
||||
|
||||
- A Classic theme after the 2004 Mac app, beside Dark and Light: brushed-metal toolbar, Aqua
|
||||
blue selection, red unread badges, a striped table and Lucida Grande. The theme button steps
|
||||
through all three and remembers the choice.
|
||||
- A toolbar across the top, after the original iPodderX: add and unsubscribe, play, mark read
|
||||
and keep for the selected item, scan, a search box for what is showing, and Settings and Log.
|
||||
- Directory, Popular and All Subscriptions at the top of the feed list, opening in the main pane.
|
||||
Directory lists every feed anyone here subscribes to, A to Z (`GET /api/directory`). All
|
||||
Subscriptions lists every item from every feed you subscribe to (`GET /api/entries`).
|
||||
- Items show as a table (unread, kept, title, feed, file, size, published) with a Files pane beside
|
||||
it, and a status bar with the totals.
|
||||
- Mark everything read from All Subscriptions, across every feed you subscribe to
|
||||
(`POST /api/read-all`). It asks first. All Subscriptions can also check every feed from its header.
|
||||
- Click a column heading in the item table to sort by it (kept, title, feed, file type, size,
|
||||
published); click again to reverse. The server sorts, so it covers the whole list, not just the
|
||||
fifty shown, and the choice is remembered.
|
||||
|
||||
### Changed
|
||||
|
||||
- Popular shows the top 10, not 20, and counts everyone, you included. Your own feeds stay on it,
|
||||
marked Subscribed, and clicking one opens it.
|
||||
- Adding a feed scans it straight away, and an OPML import that added feeds scans them, so their
|
||||
items show without pressing Scan.
|
||||
- A file deleted to save space, or by hand, looks as if it was never downloaded: no "reaped"
|
||||
label, just the Download button. The retention summary says "deleted", not "reaped".
|
||||
- Buttons are icons, with the words in their tooltips: the Files pane (save, delete, view,
|
||||
download), an item's own buttons (mark read, keep, open the original), the feed header (scan,
|
||||
download latest, mark all read, settings, unsubscribe), and the Settings, feed settings and
|
||||
Download latest dialogs (save, download, cancel). The icons are Font Awesome Free, embedded as
|
||||
SVG: only the ones used, no font to download, and nothing fetched from anyone else. They
|
||||
replace font characters such as ⟳ ⤓ ↗, which came out thin and tiny and differed from font to
|
||||
font. Keep is a flag everywhere, as it was in the original, and mark unread is an envelope.
|
||||
- One meaning per icon. Minus unsubscribes, x closes or cancels, plus adds or subscribes, and a
|
||||
dialog's confirm button carries the icon of what it does. The feed header's unsubscribe was an x
|
||||
and read as closing the page. The remaining word buttons are icons too:
|
||||
- Log, Add feed, Users, Unsubscribe and OPML.
|
||||
- Popular's Subscribe, Copy and Sign out.
|
||||
- The Subscribed label in Popular, the Directory and Add feed, which is now a green check.
|
||||
- The player's back, play, forward and close, which were font characters, and the folder arrow.
|
||||
- The toolbar's read and keep buttons show the selected item's state, with the same icons as the
|
||||
item's own buttons. Play, read and keep sit together, and Scan sits with add and unsubscribe.
|
||||
- An OPML subscription's page has the same header as a feed's, with its buttons in the same places.
|
||||
- The item table's size has its own column, apart from the file's type, and shows KB for small
|
||||
files instead of "0 MB". The Item heading is now Title.
|
||||
- A file's type is an icon (audio, video, image, PDF, torrent, other), green once it is
|
||||
downloaded and red when the download failed, with the details in its tooltip. One icon per
|
||||
row keeps the column lined up. The DOWNLOADED and PENDING labels are gone.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Playing a file from the Files pane played it twice at once, in the pane and in the player bar.
|
||||
The pane has a play button now, and the player bar is the only player.
|
||||
- The password box in Manage users was white in the dark theme.
|
||||
- An item with no date showed a stray dot in its details.
|
||||
- Escape did not close a dialog while the cursor was in one of its boxes, so Add feed, which opens
|
||||
in its URL box, could not be closed with Escape.
|
||||
|
||||
### Security
|
||||
|
||||
- Feeds from paid-feed services (Patreon, Supercast, Supporting Cast, Glow, Memberful) are never
|
||||
listed in Popular or the Directory. A Supercast feed, which keeps its key in the URL's path
|
||||
rather than the query, was being listed.
|
||||
|
||||
## [0.3.0] - 2026-09-11
|
||||
|
||||
### Added
|
||||
@@ -133,7 +265,10 @@ The long form, with what was wrong before and how it was found, is in
|
||||
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
|
||||
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
|
||||
|
||||
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...main
|
||||
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.1...main
|
||||
[0.5.1]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.0...v0.5.1
|
||||
[0.5.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.4.0...v0.5.0
|
||||
[0.4.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...v0.4.0
|
||||
[0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0
|
||||
[0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0
|
||||
[0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0
|
||||
|
||||
24
CLAUDE.md
24
CLAUDE.md
@@ -30,6 +30,20 @@ docker logs --tail 20 iPodderX
|
||||
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
|
||||
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
|
||||
|
||||
**A build that fails with `429 Too Many Requests` on a base image** is Docker Hub rate-limiting
|
||||
this host. There is no Docker Hub login here, and the build asks about `debian:bookworm-slim` and
|
||||
`rust:1-slim-bookworm` every time unless they are already stored locally. Pull them from Google's
|
||||
mirror and tag them; the build then uses the local copies without asking Docker Hub:
|
||||
|
||||
```sh
|
||||
docker pull mirror.gcr.io/library/debian:bookworm-slim
|
||||
docker tag mirror.gcr.io/library/debian:bookworm-slim debian:bookworm-slim
|
||||
docker pull mirror.gcr.io/library/rust:1-slim-bookworm
|
||||
docker tag mirror.gcr.io/library/rust:1-slim-bookworm rust:1-slim-bookworm
|
||||
```
|
||||
|
||||
Run those again now and then, or the local copies go stale.
|
||||
|
||||
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
|
||||
the worker is alive, not just the web port. The container restarts on its own after a reboot.
|
||||
|
||||
@@ -85,9 +99,9 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
|
||||
|
||||
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
|
||||
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
|
||||
* **`entries.read`, `entries.flagged` and `entries.position` are dead columns.** Read state lives in
|
||||
`entry_state` per user. Two bugs have already come from queries still reading the old ones
|
||||
(retention, and the entry pruner) — grep before adding a third.
|
||||
* **Read state lives in `entry_state`, per user, and nowhere else.** `entries` had `read`, `flagged`
|
||||
and `position` columns from before accounts; two bugs came from queries still reading them
|
||||
(retention, and the entry pruner), and `migrate()` now drops them.
|
||||
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
|
||||
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
|
||||
derived and never written to config.
|
||||
@@ -101,6 +115,10 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
|
||||
fighting over the database, with the stale one still holding the port.
|
||||
* `/api/settings` answering `200` does **not** mean the worker is alive — it is a different task.
|
||||
Probe the control socket (`ipx status`) to check that.
|
||||
* **Every `ipx` command runs `migrate()` when it opens the database**, the healthcheck's
|
||||
`ipx status` included. A migration that rewrites a big table (`DROP COLUMN`) takes seconds on
|
||||
production, and a command run meanwhile fails with `migrating schema`. It changes nothing; wait
|
||||
for `daemon started` in the log. Copy `state.db` aside before deploying one.
|
||||
|
||||
## House style
|
||||
|
||||
|
||||
34
Cargo.lock
generated
34
Cargo.lock
generated
@@ -436,17 +436,6 @@ dependencies = [
|
||||
"shlex",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cfb"
|
||||
version = "0.14.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a347dcabdae9c31b0825fd6a8bed285ec9c2acb89c47827126d52fa4f59cece3"
|
||||
dependencies = [
|
||||
"fnv",
|
||||
"uuid",
|
||||
"web-time",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cfg-if"
|
||||
version = "1.0.4"
|
||||
@@ -867,15 +856,6 @@ dependencies = [
|
||||
"dirs-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dirs"
|
||||
version = "7.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8d57d423b3c82e89b9a24ca3091fee61f456a26edbd28d26c65906f4bc1dcd8f"
|
||||
dependencies = [
|
||||
"dirs-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dirs-sys"
|
||||
version = "0.5.0"
|
||||
@@ -1608,15 +1588,6 @@ dependencies = [
|
||||
"serde_core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "infer"
|
||||
version = "0.22.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f4200d433cbd5178df7797c9c2e75b348b728e39631cf14520d1e2fc424201f4"
|
||||
dependencies = [
|
||||
"cfb",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "intervaltree"
|
||||
version = "0.2.7"
|
||||
@@ -1634,7 +1605,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
|
||||
|
||||
[[package]]
|
||||
name = "ipx"
|
||||
version = "0.3.0"
|
||||
version = "0.5.1"
|
||||
dependencies = [
|
||||
"ammonia",
|
||||
"anyhow",
|
||||
@@ -1643,9 +1614,7 @@ dependencies = [
|
||||
"axum",
|
||||
"chrono",
|
||||
"clap",
|
||||
"dirs",
|
||||
"futures-util",
|
||||
"infer",
|
||||
"librqbit",
|
||||
"opml",
|
||||
"percent-encoding",
|
||||
@@ -1656,7 +1625,6 @@ dependencies = [
|
||||
"serde",
|
||||
"serde_json",
|
||||
"tokio",
|
||||
"tokio-stream",
|
||||
"toml",
|
||||
"tower",
|
||||
"tower-http 0.7.1",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "ipx"
|
||||
version = "0.3.0"
|
||||
version = "0.5.1"
|
||||
edition = "2024"
|
||||
|
||||
[dependencies]
|
||||
@@ -11,9 +11,7 @@ atom_syndication = "0.12.10"
|
||||
axum = "0.8.9"
|
||||
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
|
||||
clap = { version = "4.6.6", features = ["derive"] }
|
||||
dirs = "7.0.0"
|
||||
futures-util = { version = "0.3.34", default-features = false, features = ["std"] }
|
||||
infer = "0.22.0"
|
||||
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
|
||||
opml = "1.1.6"
|
||||
percent-encoding = "2.3.2"
|
||||
@@ -24,7 +22,6 @@ rusqlite = { version = "0.40.2", features = ["bundled"] }
|
||||
serde = { version = "1.0.229", features = ["derive"] }
|
||||
serde_json = "1.0.151"
|
||||
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
|
||||
tokio-stream = { version = "0.1.19", features = ["sync"] }
|
||||
toml = "1.1.5"
|
||||
tower = { version = "0.5.3", features = ["util"] }
|
||||
tower-http = { version = "0.7.1", features = ["fs"] }
|
||||
|
||||
141
README.md
141
README.md
@@ -1,120 +1,83 @@
|
||||
# ipodderx-rs
|
||||
|
||||
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent), files them
|
||||
into per-feed folders, and reaps old files to stay under a disk quota. Runs as a one-shot CLI or as
|
||||
a daemon with a web UI, serving any number of people from one copy of the data.
|
||||
A self-hosted podcatcher for a household. It checks your feeds, downloads the episodes, and serves
|
||||
a web UI modelled on the 2004 Mac app **iPodderX**, for any number of people sharing one copy of
|
||||
the files. One Rust binary, `ipx`, is both the daemon and the command line.
|
||||
|
||||
## Lineage
|
||||
It is a rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python engine
|
||||
behind iPodderX (2004-2008, Ray Slakinski & August Trometer).
|
||||
|
||||
A modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python 2
|
||||
engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), open-sourced under the MIT
|
||||
License in 2010.
|
||||
## What it does
|
||||
|
||||
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
|
||||
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
|
||||
oldest-first disk quota reaper.
|
||||
- **The web UI.** It has a toolbar, and a feed list that opens with Directory, Popular and All
|
||||
Subscriptions. Items sit in a sortable table with a Files pane, and there is a player bar. It
|
||||
comes in Dark, Light and Classic themes, and works on a phone.
|
||||
- **Several people, one copy.** Each person has their own subscriptions and their own read, kept
|
||||
and playback state. There is one file on disk per episode, however many people want it. People
|
||||
sign in with a password or through a proxy (Cloudflare Zero Trust or Authentik), and admins
|
||||
manage accounts and settings.
|
||||
- **Scanning.** Feeds are checked on a schedule, globally or per feed, and a feed's own TTL is
|
||||
honoured. Keyword, explicit-content and media-type filters decide what is downloaded, with a cap
|
||||
on new downloads per scan.
|
||||
- **Downloads.** Files come over HTTP or BitTorrent and are filed into a folder per feed.
|
||||
Retention deletes the oldest files to stay under a disk quota or an age limit, and never touches
|
||||
an item someone has kept.
|
||||
- **OPML.** You can import and export your own subscriptions. You can also subscribe to an OPML
|
||||
URL, which keeps a whole list in step as a folder.
|
||||
|
||||
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
|
||||
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
|
||||
3DES-encrypted preferences, and the `printMSG` stdout protocol -- replaced by a JSON-lines socket.
|
||||
## Run it
|
||||
|
||||
## Quick start
|
||||
With Docker:
|
||||
|
||||
```sh
|
||||
docker build -t ipodderx .
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
`docker-compose.yml` is set up for the author's own server. Point its `image` and its three volumes
|
||||
(`/config`, `/data` and `/downloads`) at yours first. The UI is on port 8099. BitTorrent uses 6881
|
||||
over TCP and UDP. Files are written as `PUID`/`PGID`, 99:100 by default.
|
||||
|
||||
From source:
|
||||
|
||||
```sh
|
||||
cargo build --release
|
||||
install -m755 target/release/ipx ~/.cargo/bin/
|
||||
|
||||
ipx add https://atp.fm/rss # subscribe
|
||||
ipx fetch # scan and download
|
||||
ipx daemon # scheduler, control socket and web UI
|
||||
./target/release/ipx daemon
|
||||
```
|
||||
|
||||
On first start with `[web] enabled = true` the daemon mints a token, writes it to config.toml and
|
||||
prints the URL to open. A database with no accounts starts with **admin / ipodderx** at `/login` --
|
||||
change it with `echo -n '<password>' | ipx user passwd admin`.
|
||||
The first start creates **admin / ipodderx**. Sign in at `/login`, then change it:
|
||||
|
||||
```sh
|
||||
echo -n 'a good password' | ipx user passwd admin
|
||||
```
|
||||
|
||||
The UI is plain HTTP, so put TLS in front of it if it is reachable from outside your network.
|
||||
|
||||
## Documentation
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| [docs/configuration.md](docs/configuration.md) | Every config key, paths, environment variables |
|
||||
| [docs/configuration.md](docs/configuration.md) | Every config key, path and environment variable |
|
||||
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
|
||||
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
|
||||
| [docs/sso.md](docs/sso.md) | Cloudflare Zero Trust or Authentik in front |
|
||||
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, socket, HTTP API |
|
||||
| [docs/sso.md](docs/sso.md) | Signing in through Cloudflare Zero Trust or Authentik |
|
||||
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, control socket, HTTP API |
|
||||
| [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
|
||||
| [docs/history.md](docs/history.md) | How it was built: the long form, with what was wrong and why |
|
||||
| [CLAUDE.md](CLAUDE.md) | Notes for anyone (or anything) working on the code |
|
||||
|
||||
## The web UI
|
||||
|
||||
`ipx daemon` serves it in the same process, so it reads SQLite and the event bus directly.
|
||||
|
||||
Feeds down the side; the selected feed's items across the top; the selected item's text and its
|
||||
enclosures below, which is where you play, download or delete them. The divider drags and its
|
||||
position is remembered. Playback serves Range requests, so seeking works. An OPML subscription is a
|
||||
collapsible folder whose page lists the feeds inside it.
|
||||
|
||||
An item may carry several enclosures; all of them appear below, and anything that is not audio or
|
||||
video gets a View link rather than a player -- the publisher's copy until it is downloaded, the
|
||||
local one after. Opening an item marks it read. Show notes are untrusted feed HTML, sanitized with
|
||||
`ammonia` server-side before they reach the page.
|
||||
|
||||
The **Log** button shows the running daemon live in four tabs: *Daemon I/O* is the control protocol
|
||||
itself, every command in and event out; *Scans* is feed and download activity; *HTTP* is web
|
||||
requests; *All* is everything, with level and text filters and a copy button. It reads a ring buffer
|
||||
held in the process, not a file, so it works the same under Docker.
|
||||
|
||||
It is plain HTTP. On a LAN bind the token and everything else cross the network in the clear, and a
|
||||
feed URL can itself carry a credential. Put TLS in front of it if that matters.
|
||||
|
||||
## OPML
|
||||
|
||||
**Importing and exporting** a file copies subscriptions in or out once: `ipx import subs.opml`,
|
||||
`ipx export subs.opml`, or Settings → Subscriptions in the UI.
|
||||
|
||||
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
|
||||
other feed; every scan re-reads it and keeps your list in step. The feeds inside are not written to
|
||||
config.toml -- the OPML is the source of truth, so they are re-derived each scan and held in the
|
||||
database. They show as a folder, download into one nested folder, and inherit the subscription's
|
||||
settings until you change one, which gives it its own entry.
|
||||
|
||||
When a feed drops out of the OPML upstream, it is unsubscribed and removed -- unless it has
|
||||
downloads, in which case it is kept and flagged in the UI as no longer listed. A downloaded file is
|
||||
never left behind with nothing explaining where it came from.
|
||||
|
||||
## Docker
|
||||
|
||||
```sh
|
||||
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
|
||||
docker compose pull ipodderx && docker compose up -d ipodderx
|
||||
docker compose logs -f ipodderx # the first start prints the default admin password
|
||||
```
|
||||
|
||||
`docker-compose.yml` runs the image from the registry above rather than building it, so build and
|
||||
push first; change the tag in both places to use another registry. It mounts `/config` (config.toml),
|
||||
`/data` (state.db) and `/downloads` from this install's host paths, which you will want to change for
|
||||
yours. It publishes 8099 for the
|
||||
UI and 6881 (TCP **and** UDP -- DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
|
||||
land owned the way Unraid shares expect. The healthcheck runs `ipx status` through the control
|
||||
socket, so it catches a daemon that is alive but wedged rather than merely one that has died.
|
||||
|
||||
## Running it as a service
|
||||
|
||||
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
|
||||
rather run periodic scans with no daemon -- in which case there is no socket for a UI to attach to.
|
||||
| [docs/history.md](docs/history.md) | How it was built, with what was wrong and why |
|
||||
| [CLAUDE.md](CLAUDE.md) | Notes for working on the code, including how production is deployed |
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
|
||||
node tests/page-smoke.js # the page script loads without throwing
|
||||
npx playwright test # a real browser against a real daemon
|
||||
npx playwright test # a real browser against a real daemon on fixture feeds
|
||||
```
|
||||
|
||||
`npm install` gets the test runner; the browser comes from
|
||||
`npx playwright install --with-deps chromium` (in `install.sh`).
|
||||
`npm install` gets the test runner, and `npx playwright install --with-deps chromium` gets the
|
||||
browser.
|
||||
|
||||
## License
|
||||
|
||||
MIT. See [LICENSE](LICENSE).
|
||||
MIT, see [LICENSE](LICENSE). The icons are [Font Awesome Free](https://fontawesome.com) 7.3.1 by
|
||||
@fontawesome, under [CC BY 4.0](https://fontawesome.com/license/free), embedded as SVG.
|
||||
|
||||
30
TODO.md
Normal file
30
TODO.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# To do
|
||||
|
||||
## Cut what is no longer needed
|
||||
|
||||
From a whole-repo audit for over-engineering on 2026-09-12. Biggest cut first.
|
||||
|
||||
- [x] **Pre-accounts adoption and the dead `entries` columns.** The copy of the old read state into
|
||||
`entry_state` and the `entries.read`, `flagged` and `position` columns are gone. Its other half,
|
||||
subscribing the first admin to the catalogue, was not dead and stays as `adopt_catalogue`.
|
||||
(`src/db.rs`, `src/main.rs`)
|
||||
- [x] **`contrib/` systemd units.** From before the container; nothing points at them.
|
||||
- [x] **`migrate_opml_children`.** A one-time move of OPML children out of `config.toml` that has
|
||||
run. Delete it and its call. (`src/main.rs`)
|
||||
- [x] **The legacy `interval_mins` key.** Production uses `schedule`. Delete the field, the fallback
|
||||
in `General::interval` and its test. (`src/config.rs`)
|
||||
- [x] **`Db::entries` and `Db::count_entries`.** One-line wrappers only the tests call; the tests
|
||||
call `entries_in` and `count_in` instead. (`src/db.rs`)
|
||||
- [x] **`web::generate_token`.** Repeats `auth::new_session_token`. Use that. (`src/web.rs`)
|
||||
- [x] **Page leftovers.** `globalEvery`, `S.busy`, `S.limit`, `unitOptions`' `firstLabel`, `--r`,
|
||||
`.ep.open`, the phone `.ep .art`, the duplicate phone `.fhead.slim{flex-wrap}`, the second
|
||||
`#sidebar{z-index}`, and the `on()` helper. (`web/index.html`)
|
||||
- [x] **`logbuf` visitors.** `record_i64`, `record_u64` and `record_bool` repeat what `Visit`'s
|
||||
defaults already do through `record_debug`. (`src/logbuf.rs`)
|
||||
- [x] **The `infer` dependency.** Its torrent check is the `d8:announce` test on the next line.
|
||||
- [x] **The `dirs` dependency.** `XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME` from `std::env`.
|
||||
- [x] **The `tokio-stream` dependency.** `futures_util::stream::unfold` over the broadcast receiver.
|
||||
- [x] **The icon inlined four times.** About 94 KB of base64 across both pages; serve it once as
|
||||
`/icon.png` from `include_bytes!`, open without signing in like `/login`.
|
||||
|
||||
After these: `cargo test`, `node tests/page-smoke.js`, `npx playwright test`.
|
||||
@@ -1,7 +0,0 @@
|
||||
[Unit]
|
||||
Description=ipx feed scan (one shot)
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=%h/.cargo/bin/ipx fetch
|
||||
Environment=IPX_LOG=ipx=info
|
||||
@@ -1,18 +0,0 @@
|
||||
# User unit: install to ~/.config/systemd/user/ipx.service, then
|
||||
# systemctl --user enable --now ipx
|
||||
# The socket lands in $XDG_RUNTIME_DIR/ipx.sock by default, so a UI running as the
|
||||
# same user can attach without extra configuration.
|
||||
[Unit]
|
||||
Description=ipx podcatcher
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=%h/.cargo/bin/ipx daemon
|
||||
Restart=on-failure
|
||||
RestartSec=30
|
||||
Environment=IPX_LOG=ipx=info
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,16 +0,0 @@
|
||||
# Alternative to the daemon: a periodic one-shot scan, closer to how the original
|
||||
# iPodderX agent was driven. Use this OR ipx.service, not both -- with no daemon
|
||||
# running there is no socket, so a UI cannot attach.
|
||||
#
|
||||
# Install ipx-scan.service and ipx.timer to ~/.config/systemd/user/, then
|
||||
# systemctl --user enable --now ipx.timer
|
||||
[Unit]
|
||||
Description=Periodic ipx feed scan
|
||||
|
||||
[Timer]
|
||||
OnBootSec=5min
|
||||
OnUnitActiveSec=1h
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -28,12 +28,15 @@ The page is compiled in, so **editing `web/index.html` needs a rebuild**.
|
||||
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
|
||||
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
|
||||
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
|
||||
re-derived into the database each scan, never written to config.toml.
|
||||
re-derived into the database each scan, never written to config.toml. A Patreon creator link
|
||||
(a token, no `show=`) with more than one show is treated the same way, before any fetch: its
|
||||
shows come from Patreon's web API and each becomes a derived feed.
|
||||
4. Record entries. A changed title or description flips the item back to unread.
|
||||
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
|
||||
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
|
||||
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
|
||||
a reason.
|
||||
a reason. What a filter skipped is judged again every scan, so a change of settings brings it
|
||||
back. A feed in a group takes your settings on the group for anything you have not set on it.
|
||||
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
|
||||
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
|
||||
or an error page — and is deleted.
|
||||
@@ -55,9 +58,9 @@ entry_state user_id, feed_id, guid, read, flagged, position
|
||||
PK (user_id, feed_id, guid)
|
||||
```
|
||||
|
||||
`entries` still has `read`, `flagged` and `position` columns from before accounts existed. They are
|
||||
**dead** — the migration copied them into `entry_state` and nothing reads them now. Anything found
|
||||
querying them is a bug; two were.
|
||||
Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position` columns from
|
||||
before accounts; two bugs came from queries still reading them, and `migrate()` drops them from an
|
||||
older database.
|
||||
|
||||
Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in
|
||||
`migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the
|
||||
@@ -104,13 +107,15 @@ else a `401`.
|
||||
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
|
||||
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
|
||||
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
|
||||
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable |
|
||||
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable, sortable (`sort` = kept, title, feed, type, size or published; `dir` = asc or desc) |
|
||||
| `GET /api/entries` | the same, across every feed you subscribe to (All Subscriptions) |
|
||||
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
|
||||
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, starred, position |
|
||||
| `POST /api/read-all` | everything read in every feed you subscribe to (All Subscriptions) |
|
||||
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, kept, position |
|
||||
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
|
||||
| `POST /api/fetch` | |
|
||||
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
|
||||
| `GET /api/popular`, `POST /api/popular/{id}` | what others here subscribe to (id, title, art, count; never a URL, never a private feed); subscribe by id |
|
||||
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
|
||||
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
|
||||
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
|
||||
| `GET /api/events` | SSE, the same broadcast the socket carries |
|
||||
|
||||
@@ -60,7 +60,7 @@ ipx reap # actually delete
|
||||
```
|
||||
|
||||
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
|
||||
with no file are pruned from the database. **Starred by anyone keeps a file**, and one only counts
|
||||
with no file are pruned from the database. **An item anyone kept keeps its file**, and one only counts
|
||||
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
|
||||
stops the next scan fetching it again.
|
||||
|
||||
|
||||
@@ -32,10 +32,10 @@ media_types = ["audio", "video"]
|
||||
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
|
||||
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
|
||||
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
|
||||
pad. Starred items are never deleted, and a file only counts as read once every subscriber has
|
||||
pad. Kept items are never deleted, and a file only counts as read once every subscriber has
|
||||
read it. `0` disables it entirely.
|
||||
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
|
||||
Starred ones stay. `0` disables it.
|
||||
Kept ones stay. `0` disables it.
|
||||
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
|
||||
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
|
||||
fetched 216 files and 22 GB in one scan.
|
||||
@@ -43,8 +43,6 @@ media_types = ["audio", "video"]
|
||||
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
|
||||
without this the disk fills with artwork. Empty takes everything.
|
||||
|
||||
`interval_mins` from older configs is still read, and `schedule` supersedes it.
|
||||
|
||||
## `[torrent]`
|
||||
|
||||
```toml
|
||||
|
||||
218
docs/history.md
218
docs/history.md
@@ -6,6 +6,217 @@ reasoning lives. New write-ups go at the top.
|
||||
|
||||
See [README.md](../README.md) for what the thing is.
|
||||
|
||||
## 2026-09-12 — Cutting what had outlived its reason
|
||||
|
||||
A whole-repo audit for over-engineering listed twelve things to cut, and all of them went.
|
||||
|
||||
- **Upgrades from before accounts.** `migrate_opml_children` moved OPML feeds that old versions
|
||||
wrote into `config.toml` out to the database, and ran at every daemon start to do nothing after
|
||||
the first. Production ran it in 0.3.0; anything older has to pass through 0.4.0.
|
||||
- **Half of the adoption, and not the other half.** The audit called `adopt_existing_library` a
|
||||
one-time migration and it was cut whole. It did two jobs: copy the old read state into
|
||||
`entry_state`, which was dead, and subscribe the first admin to the whole catalogue while nobody
|
||||
subscribed to anything, which is how a fresh install's first account gets `config.toml`'s feeds.
|
||||
The browser suite caught it at once, signing in to an empty sidebar; `cargo test` had no idea.
|
||||
The second job is back as `adopt_catalogue`, with a unit test of its own.
|
||||
- **The dead `entries` columns.** `read`, `flagged` and `position` moved to `entry_state` with
|
||||
accounts. The adoption's copy was their last reader, but `record_entry` still wrote them, and
|
||||
still reset `read` when a title changed, which nothing looked at. Two bugs came from queries
|
||||
reading them. `migrate()` now drops them from an existing database (SQLite has had `DROP COLUMN`
|
||||
since 3.35), and a test builds an old table to prove it. On production each drop rewrote the
|
||||
66 MB `entries` table, about four seconds apiece, so the first start took thirteen. An
|
||||
`ipx status` run in that window failed with `migrating schema`: every `ipx` command migrates when
|
||||
it opens the database, and it collided with the daemon doing the same. A failed `ALTER TABLE`
|
||||
changes nothing, and the database had been copied to `backup/` first anyway.
|
||||
- **`interval_mins`**, which `schedule` replaced. An old config that still has the key loads; the
|
||||
key is ignored, and the config test carries it to keep that true.
|
||||
- **Three dependencies.** `infer` was only asked whether a file is a torrent, and the check after
|
||||
it already looked for `d8:announce`, which is what `infer` looks for. `dirs` was three lookups of
|
||||
`XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME`. `tokio-stream` wrapped the broadcast receiver for
|
||||
the event stream; `futures_util::stream::unfold` does the same, lagging clients included.
|
||||
- **Two token generators.** The web token came from a copy of the session-token code, with a
|
||||
clock fallback on top. It uses `auth::new_session_token` now, and is 64 characters.
|
||||
- **The icon inlined four times**, 23 KB of base64 each, into both pages. It is `/icon.png` now,
|
||||
outside the sign-in wall with `/login`, since the sign-in page shows it.
|
||||
- Also: the `contrib/` systemd units from before the container, `Db::entries` and
|
||||
`Db::count_entries` that only the tests called, three `logbuf` visitors that repeated the trait's
|
||||
defaults, and unused state, a helper and dead CSS in the page.
|
||||
|
||||
## 2026-09-11 — A design pass on the web UI
|
||||
|
||||
A review against screenshots of every view in all three themes found that Dark and Light read as a
|
||||
generic dark dashboard: one pale blue did every job, most labels were bold, and nothing led. The
|
||||
list it produced, in `TODO.md`, was worked through in one go. What was worth knowing:
|
||||
|
||||
- **Amber means new.** Badges, unread dots and download bars take the icon's EQ amber; blue is left
|
||||
for the primary action and links. Light's amber was `#b06f10`, which gives white text 4.1:1,
|
||||
short of AA for 11 px bold. It is `#9a5f0a` now, 5.2:1.
|
||||
- **EQ bars mark what is playing.** Three `<i>` bars stand at 60, 100 and 40 % and animate only
|
||||
while `body.playing` is set. The first version left the animation on but paused, expecting each
|
||||
bar to hold a different frame. The frames it held were within a pixel of each other, and on
|
||||
screen the bars read as three dots. Under reduced motion one rule drops every animation and
|
||||
transition, which leaves the bars standing.
|
||||
- **The focus ring was clipped.** `.tgroup` and `#topbar` both set `overflow:hidden`, so a ring
|
||||
drawn outside a toolbar button was cut off. Rings inside clipping parents are inset instead.
|
||||
- **The feed list could not be used from the keyboard at all.** Rows were `<div>`s and the triangle
|
||||
a `<span>`, so Tab went from the feed filter to Sign out. Rows now take focus, the triangle is a
|
||||
`<button aria-expanded>`, and `renderFeeds` puts focus back on the same row after redrawing,
|
||||
since every live update replaces every row. The list's own key handler stops Space and the
|
||||
arrows from reaching the player's shortcuts on the document.
|
||||
- **The triangle hangs in the margin.** Every row used to reserve an 18 px slot for it, pushing a
|
||||
hundred feeds 28 px right of the places above for the sake of two folders. It is now absolutely
|
||||
placed in the row's left padding, the full height of the row, so a near miss no longer opens
|
||||
the folder's page.
|
||||
- **A selected tile vanished** because the initials tile and the selected row were both `--raise`.
|
||||
Tiles now mix their tint into `--bg`, which no row uses.
|
||||
- **The Files pane hides itself** with `#split:has(>#files[hidden])`, which collapses its column.
|
||||
The phone layout already hides the pane, and a zero-width extra track there is harmless.
|
||||
- **Flagged became Kept** in the tab, and in the server's refusal to delete a file someone else
|
||||
kept. The filter value and the column stay `flagged`; renaming those buys nothing.
|
||||
|
||||
## 2026-09-11 — A Patreon creator is a list of shows
|
||||
|
||||
Ray asked whether ipx could sync with Patreon. Not in full. The documented API (v2, the
|
||||
`identity.memberships` scope) lists the creators you back and whether each has a feed (`has_rss`),
|
||||
but no resource carries the `auth` token that makes a feed URL work. That token only comes from the
|
||||
creator's page. It is also one per membership, not one per account: techpod's differs from Glass
|
||||
Cannon's, so no single token finds everything you back.
|
||||
|
||||
What does work is one creator at a time, which is what Ray wanted for Glass Cannon and its 33 shows:
|
||||
|
||||
- `patreon.com/rss?auth=<token>`, with no creator named, returns that token's creator. Its self link,
|
||||
about 660 bytes in, gives the campaign by number (`/rss/369921`). Patreon ignores `Range` here, so
|
||||
ipx reads the stream until the number appears and hangs up, instead of taking all 2.8 MB.
|
||||
- A show's `show=` number is a Patreon collection. Asked anonymously, the collection listing
|
||||
(`/api/collection?filter[campaign_id]=`) and a post's `collections` both hide patron-only ones: you
|
||||
get "FAQ". `/api/campaigns/<id>?include=shows` lists every show, anonymously, in one response.
|
||||
- Every spelling works: `rss/glasscannon?auth=…&show=N`, `rss/369921?…` and `rss?auth=…&show=N` all
|
||||
serve the same 131 items. Enclosure URLs are the same in the creator feed and the show feed, and
|
||||
stable between fetches.
|
||||
|
||||
That last point shaped the design. `enclosures.url` is unique, so whichever feed is scanned first owns
|
||||
the file. The first cut only asked a creator for its shows while it had no entries of its own, so that a
|
||||
creator already read as a plain feed, holding every show's episodes, would never be split into shows
|
||||
that came up empty. Within the hour that was the wrong call: Glass Cannon had gone into production on
|
||||
the build before this one, been read as one feed of 2,385 items, and the rule kept it that way. Finding
|
||||
anything in that heap was the problem Ray wanted solved.
|
||||
|
||||
So a creator with more than one show is always a group, run through the same sync as an OPML
|
||||
(`sync_group`, split out of `sync_opml`). When it becomes one, its items are cleared and each show
|
||||
takes over the enclosures the creator holds as the show lists them (`Db::adopt`), downloaded files and
|
||||
everyone's read state included. One show leaves it a plain feed, which is what techpod already was. If
|
||||
the shows cannot be listed, a creator already split fails the scan rather than being read as one heap;
|
||||
one that never was is read as one feed until they can be. An answer without a `shows` list is an error,
|
||||
not "no shows".
|
||||
|
||||
**Filter verdicts follow the settings.** Ray also reported that turning on Allow explicit and
|
||||
rescanning brought nothing back. An item was judged once, when first seen, and `skipped` was final. The
|
||||
2026-09-10 entry below saw it coming ("worth a `ipx retry <feed>` command if this bites"). It bit:
|
||||
2,166 Glass Cannon items and all 88 of Shadowdark's sat at `skipped: explicit` with the setting on.
|
||||
Every scan now runs the filters again over what they skipped (not over `torrents disabled`, which is not
|
||||
a filter's call) and requeues what they now let through. Only that direction: a queued item is never
|
||||
pulled back, because Download latest and a manual download both work by queueing.
|
||||
|
||||
**Two gaps beside it.** Add feed had no explicit box, so every new feed's first scan skipped all its
|
||||
explicit items; it has one now, stored on your subscription like the feed dialog's. And a feed inside a
|
||||
group ignored your settings on the group, though the group's dialog said they were inherited: settings
|
||||
live on each person's subscription, and nothing read the group's. `Db::subscribers` now fills what you
|
||||
have not set on the feed from your subscription to the group, and the feed list shows the same.
|
||||
|
||||
**A name that was already used.** Replaying the split on a copy of the production database left one
|
||||
show with a Supercast episode in it. "Glass Cannon Live! Ascension | Pathfinder 2E" slugs to
|
||||
`glass-cannon-live-ascension-pathfinder-2`, the id of a Supercast feed of the same show that had been
|
||||
removed. Removing a feed keeps its rows on purpose, so that re-adding it does not fetch the back
|
||||
catalogue again, but choosing a new id only checked config.toml and derived feeds. The Patreon show took
|
||||
the old id and everything still filed under it. An id is now also taken when the database has a feed by
|
||||
that id at a different URL; the same URL may still have it back, which is the re-add case.
|
||||
|
||||
Shows already added by hand are matched by token and show number, not by exact URL (`same_feed`), so a
|
||||
bare token does not add Get in the Trunk and Shadowdark a second time under another spelling.
|
||||
|
||||
The show listing is Patreon's own undocumented web API. If it changes, only finding new shows stops.
|
||||
|
||||
## 2026-09-11 — One meaning per icon, sortable columns, and one player
|
||||
|
||||
Ray asked for a pass over the whole UI: consistent icons, and buttons placed next to what they act
|
||||
on. The pass started from screenshots of every view and dialog, taken against the browser suite's
|
||||
own daemon and fixtures. Several things did not agree:
|
||||
|
||||
- Unsubscribe was a minus in the toolbar but an x in the feed header. There it read as "close this
|
||||
page", and x also meant Cancel in every dialog.
|
||||
- Half the dialogs still had word buttons: Log, Add feed, Users, Unsubscribe, OPML, Popular's
|
||||
Subscribe, the Copy buttons and Sign out.
|
||||
- The player bar and the folder arrow were font characters, and so was the row's "playing" marker.
|
||||
Those came out thin and varied from font to font.
|
||||
- The OPML subscription page laid out its header differently from a feed's, and All Subscriptions
|
||||
had no buttons at all.
|
||||
|
||||
The rule now is one meaning per icon. Minus unsubscribes, x closes or cancels, plus adds, subscribes
|
||||
or imports, and a dialog's confirm button carries the icon of what it does. The words go in the
|
||||
tooltip. The toolbar is grouped by what it acts on: add, unsubscribe and scan act on feeds; play,
|
||||
read and keep act on the selected item. Its read and keep buttons now show that item's state, as
|
||||
the item's own buttons already did. A browser test opens all eight dialogs and fails on any button
|
||||
without an icon and a tooltip. It also checks that the toolbar and the page use the same icon for
|
||||
the same action.
|
||||
|
||||
The pass turned up three bugs:
|
||||
|
||||
- **Escape ignored inside a dialog's text box.** The keydown handler returned early for any key
|
||||
typed in a text box, and Escape sat below that check. So Add feed, which opens with the cursor in
|
||||
its URL box, could not be closed with Escape. It showed up because the screenshot run stalled:
|
||||
the dialog stayed open, and every later click landed on its overlay until the run timed out.
|
||||
- **A white password box in the dark theme.** The input rule listed text, search and number
|
||||
fields, but not password ones.
|
||||
- **A stray "• •" under an item with no date.** Each detail carried its own trailing dot, whether
|
||||
or not the next one was there.
|
||||
|
||||
All Subscriptions gained Mark everything read, as `POST /api/read-all`. It marks the feeds from the
|
||||
person's subscriptions, which is the same set the view's query is scoped by. It asks first, since
|
||||
one click covers every feed.
|
||||
|
||||
**Sorting** happens on the server. A list arrives fifty at a time, so sorting in the page would
|
||||
only have reordered what was loaded. The column name picks one of a fixed set of SQL expressions
|
||||
(`order_sql`), so nothing the browser sends reaches the query, and an unknown name means newest
|
||||
first. Ties also fall back to newest first, so "Load more" does not skip or repeat items at a page
|
||||
boundary. A database test runs every column both ways. Size came out of the File column into its
|
||||
own, and small files show KB, since whole megabytes made every article image "0 MB".
|
||||
|
||||
**The double play.** Ray found that pressing play in the Files pane played the file twice at once.
|
||||
The pane drew its own `<audio controls>`, and its `onplay` handler also started the player bar on
|
||||
the same file. That meant two audio elements from one click. The pane now has a play button that
|
||||
hands that exact file to the player bar, the only player. `play()` now takes the file. Its check
|
||||
for resuming compares the file as well as the item, so another of an item's files starts from its
|
||||
beginning instead of carrying on with the first. A test plays from the pane and checks the page
|
||||
has one `<audio>` element.
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-11 — The original's layout
|
||||
|
||||
Ray pointed at a screenshot of the Mac app (techpp.com, 2012) and asked for its panes, its
|
||||
toolbar, and a Directory that lives in the feed list rather than behind a button. What it had, and
|
||||
what ipx now does:
|
||||
|
||||
- A toolbar across the window: subscribe and unsubscribe, play, flag, refresh, and a search box
|
||||
scoped to the feed on show. ipx's has the same groups, acting on the selected feed and item, with
|
||||
Settings and Log at the right end, for admins only.
|
||||
- A source list opening with Directory, Playlist Builder and All Subscriptions above the feeds.
|
||||
ipx has Directory, Popular and All Subscriptions there, opening in the main pane. Playlist
|
||||
Builder is left out, since nothing here builds playlists.
|
||||
- The entries as a table, with a Files pane beside it and the entry below. The columns are unread,
|
||||
kept, the item, its feed, its file and when. Sorting by column is not done yet.
|
||||
- A status bar with the totals for what is on show.
|
||||
|
||||
All Subscriptions needed one new endpoint, `GET /api/entries`. It is the per-feed query with
|
||||
`feed_id = ?` swapped for the person's subscriptions. The enclosure lookup that follows it used to
|
||||
filter by feed as well; a page can now span feeds, so each file is matched to its row by feed and
|
||||
guid instead.
|
||||
|
||||
A phone has no room for a pane beside the table, so there the files follow the item's text in the
|
||||
full-screen reader, and the table drops to title and date.
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-11 — Popular on this server
|
||||
|
||||
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
|
||||
@@ -26,6 +237,13 @@ with everything else, so this is built around not doing that:
|
||||
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
|
||||
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
|
||||
`unlisted` flag is the upgrade if that happens.
|
||||
|
||||
It happened the same day. The first screenshot of the new Directory listed "Glass Cannon Live!
|
||||
Ascension (for Ray Slakinski)", a Supercast feed at `feeds.supercast.com/feeds/<key>`. Treating
|
||||
any long path segment as a key would have hidden public feeds too: acast's show ids look the
|
||||
same. So paid-feed services are named instead (Patreon, Supercast, Supporting Cast, Glow,
|
||||
Memberful), and any feed from one of them is private whatever its URL looks like. The per-feed
|
||||
flag is still the upgrade for a service not on that list.
|
||||
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
|
||||
they would bury everything anyone chose on purpose.
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ fetch, one parse and one file.
|
||||
|
||||
| Yours alone | The same for everyone |
|
||||
|---|---|
|
||||
| Read, starred, playback position | The feed's URL |
|
||||
| Read, kept, playback position | The feed's URL |
|
||||
| Which feeds you see at all | Its download folder |
|
||||
| Keywords, auto-download, explicit, per-scan cap | When it is scanned |
|
||||
| | The file on disk |
|
||||
@@ -34,10 +34,10 @@ is shared.
|
||||
|
||||
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
|
||||
for everyone** and names them in the confirmation, and the server has the last word: if anyone else
|
||||
has starred the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
|
||||
has kept the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
|
||||
reason, and only `?force=true` goes through.
|
||||
|
||||
Retention follows the same rule: starred by anyone keeps a file, and it counts as read only once
|
||||
Retention follows the same rule: an item anyone kept keeps its file, and it counts as read only once
|
||||
every subscriber has read it.
|
||||
|
||||
## Signing in
|
||||
@@ -70,11 +70,14 @@ list with their own read state. Unsubscribing removes it from their list alone;
|
||||
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
|
||||
re-subscribing does not pull the back catalogue again.
|
||||
|
||||
**Add feed** also lists what other people on this server subscribe to, most subscribers first, as
|
||||
a place to start. It shows a title, artwork and a count, never a URL or who reads it. Feeds from an
|
||||
**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
|
||||
shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
|
||||
included. Directory lists every one of them A to Z. Your own feeds are marked Subscribed.
|
||||
It shows a title, artwork and a count, never a URL or who reads it. Feeds from an
|
||||
OPML subscription are left out, since they come with the OPML. So is anything that looks private: a
|
||||
login configured for the feed, credentials in its URL, or a key such as `auth=` or `token=` in the
|
||||
query. Those are someone's paid subscriptions, and listing them would let anyone here read what they
|
||||
query, or a feed from a paid-feed service such as Patreon or Supercast, which put the key in the
|
||||
path. Those are someone's paid subscriptions, and listing them would let anyone here read what they
|
||||
pay for.
|
||||
|
||||
An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none
|
||||
|
||||
@@ -26,9 +26,6 @@ pub struct General {
|
||||
/// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d".
|
||||
/// A feed's own `schedule` overrides this.
|
||||
pub schedule: String,
|
||||
/// Superseded by `schedule`. Still read so existing configs keep working.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub interval_mins: Option<u64>,
|
||||
pub organize: Organize,
|
||||
/// 0 = unlimited.
|
||||
pub max_total_gb: f64,
|
||||
@@ -153,7 +150,6 @@ impl Default for General {
|
||||
download_dir: home().join("Podcasts"),
|
||||
socket: default_socket(),
|
||||
schedule: "every 60m".into(),
|
||||
interval_mins: None,
|
||||
organize: Organize::Feed,
|
||||
max_total_gb: 0.0,
|
||||
max_age_days: 0,
|
||||
@@ -176,8 +172,8 @@ impl Default for Torrent {
|
||||
}
|
||||
|
||||
impl General {
|
||||
/// Minutes between checks. Falls back to the legacy `interval_mins`, then to an hour.
|
||||
/// A malformed value warns rather than stopping the daemon.
|
||||
/// Minutes between checks, or an hour when `schedule` is empty or unreadable. A malformed
|
||||
/// value warns rather than stopping the daemon.
|
||||
pub fn interval(&self) -> u64 {
|
||||
if let Some(n) = parse_interval(&self.schedule) {
|
||||
return n;
|
||||
@@ -185,7 +181,7 @@ impl General {
|
||||
if !self.schedule.trim().is_empty() {
|
||||
tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default");
|
||||
}
|
||||
self.interval_mins.filter(|n| *n > 0).unwrap_or(60)
|
||||
60
|
||||
}
|
||||
}
|
||||
|
||||
@@ -278,9 +274,7 @@ pub fn config_path() -> PathBuf {
|
||||
if let Ok(p) = std::env::var("IPX_CONFIG") {
|
||||
return PathBuf::from(p);
|
||||
}
|
||||
dirs::config_dir()
|
||||
.unwrap_or_else(|| home().join(".config"))
|
||||
.join("ipx/config.toml")
|
||||
xdg("XDG_CONFIG_HOME", ".config").join("ipx/config.toml")
|
||||
}
|
||||
|
||||
/// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`.
|
||||
@@ -288,9 +282,7 @@ pub fn data_dir() -> PathBuf {
|
||||
if let Ok(p) = std::env::var("IPX_DATA_DIR") {
|
||||
return PathBuf::from(p);
|
||||
}
|
||||
dirs::data_dir()
|
||||
.unwrap_or_else(|| home().join(".local/share"))
|
||||
.join("ipx")
|
||||
xdg("XDG_DATA_HOME", ".local/share").join("ipx")
|
||||
}
|
||||
|
||||
fn default_socket() -> PathBuf {
|
||||
@@ -346,8 +338,16 @@ pub fn unique_slug(text: &str, taken: &BTreeMap<String, Feed>) -> String {
|
||||
(2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap()
|
||||
}
|
||||
|
||||
/// `$var`, or `~/fallback` when it is unset or empty, as the XDG base directory spec says.
|
||||
fn xdg(var: &str, fallback: &str) -> PathBuf {
|
||||
std::env::var_os(var)
|
||||
.filter(|v| !v.is_empty())
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| home().join(fallback))
|
||||
}
|
||||
|
||||
fn home() -> PathBuf {
|
||||
dirs::home_dir().unwrap_or_else(|| PathBuf::from("."))
|
||||
std::env::var_os("HOME").map(PathBuf::from).unwrap_or_else(|| PathBuf::from("."))
|
||||
}
|
||||
|
||||
fn expand_tilde(p: &Path) -> PathBuf {
|
||||
@@ -367,6 +367,8 @@ mod tests {
|
||||
r#"
|
||||
[general]
|
||||
download_dir = "/tmp/pods"
|
||||
# A key older versions read. An old config that still has it has to load.
|
||||
interval_mins = 45
|
||||
|
||||
[feeds.example]
|
||||
url = "https://example.com/feed.xml"
|
||||
@@ -403,22 +405,17 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn interval_falls_back_through_legacy_then_default() {
|
||||
fn interval_falls_back_to_an_hour() {
|
||||
let mut g = General::default();
|
||||
assert_eq!(g.interval(), 60, "the default schedule");
|
||||
|
||||
g.schedule = "every 15m".into();
|
||||
assert_eq!(g.interval(), 15);
|
||||
|
||||
// A config written before `schedule` existed still works.
|
||||
// Empty or garbage must not stop the daemon.
|
||||
g.schedule = String::new();
|
||||
g.interval_mins = Some(45);
|
||||
assert_eq!(g.interval(), 45);
|
||||
|
||||
// Garbage must not stop the daemon.
|
||||
assert_eq!(g.interval(), 60);
|
||||
g.schedule = "whenever".into();
|
||||
assert_eq!(g.interval(), 45);
|
||||
g.interval_mins = None;
|
||||
assert_eq!(g.interval(), 60);
|
||||
}
|
||||
|
||||
|
||||
358
src/db.rs
358
src/db.rs
@@ -40,14 +40,10 @@ CREATE TABLE IF NOT EXISTS entries (
|
||||
published INTEGER,
|
||||
description TEXT,
|
||||
first_seen INTEGER NOT NULL,
|
||||
read INTEGER NOT NULL DEFAULT 0,
|
||||
flagged INTEGER NOT NULL DEFAULT 0,
|
||||
image TEXT,
|
||||
duration INTEGER,
|
||||
episode INTEGER,
|
||||
season INTEGER,
|
||||
-- Seconds into the audio, so playback resumes where it was left.
|
||||
position INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (feed_id, guid)
|
||||
);
|
||||
|
||||
@@ -93,7 +89,7 @@ CREATE TABLE IF NOT EXISTS subscriptions (
|
||||
PRIMARY KEY (user_id, feed_id)
|
||||
);
|
||||
|
||||
-- Read, starred and how far in. One row per person per item, created on first touch;
|
||||
-- Read, kept and how far in. One row per person per item, created on first touch;
|
||||
-- an item nobody has touched has no row at all, which is what unread means.
|
||||
CREATE TABLE IF NOT EXISTS entry_state (
|
||||
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
@@ -143,8 +139,8 @@ pub struct Managed {
|
||||
pub orphaned: bool,
|
||||
}
|
||||
|
||||
/// Adds columns that later versions introduced. CREATE TABLE IF NOT EXISTS does nothing to
|
||||
/// a table that already exists, so an installed database needs them added explicitly.
|
||||
/// Adds the columns later versions introduced and drops the ones they retired. CREATE TABLE IF
|
||||
/// NOT EXISTS does nothing to a table that already exists, so an installed database needs both.
|
||||
fn migrate(conn: &Connection) -> Result<()> {
|
||||
let wanted: &[(&str, &str, &str)] = &[
|
||||
("feeds", "image", "TEXT"),
|
||||
@@ -155,18 +151,29 @@ fn migrate(conn: &Connection) -> Result<()> {
|
||||
("entries", "duration", "INTEGER"),
|
||||
("entries", "episode", "INTEGER"),
|
||||
("entries", "season", "INTEGER"),
|
||||
("entries", "position", "INTEGER NOT NULL DEFAULT 0"),
|
||||
];
|
||||
for (table, column, ty) in wanted {
|
||||
// Read state from before accounts, long since moved to entry_state. Two bugs came from
|
||||
// queries still reading these after they stopped meaning anything, so they go.
|
||||
let retired: &[(&str, &str)] = &[("entries", "read"), ("entries", "flagged"), ("entries", "position")];
|
||||
let has = |table: &str, column: &str| -> Result<bool> {
|
||||
let mut stmt = conn.prepare(&format!("PRAGMA table_info({table})"))?;
|
||||
let existing: Vec<String> = stmt
|
||||
let names = stmt
|
||||
.query_map([], |r| r.get::<_, String>(1))?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
if !existing.iter().any(|c| c == column) {
|
||||
Ok(names.iter().any(|c| c == column))
|
||||
};
|
||||
for (table, column, ty) in wanted {
|
||||
if !has(table, column)? {
|
||||
tracing::info!(table, column, "adding column");
|
||||
conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?;
|
||||
}
|
||||
}
|
||||
for (table, column) in retired {
|
||||
if has(table, column)? {
|
||||
tracing::info!(table, column, "dropping column");
|
||||
conn.execute_batch(&format!("ALTER TABLE {table} DROP COLUMN {column}"))?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -345,22 +352,19 @@ impl Db {
|
||||
|
||||
/// Returns true when this entry had not been seen before.
|
||||
///
|
||||
/// A changed description or title flips `read` back to 0, which is what the original's
|
||||
/// textDiff dance was ultimately for -- minus the diff markup, which the UI can do.
|
||||
pub fn record_entry(&self, feed_id: &str, e: &crate::feed::Entry) -> Result<bool> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
let inserted = conn.execute(
|
||||
"INSERT OR IGNORE INTO entries
|
||||
(feed_id, guid, title, link, published, description, first_seen, read, flagged,
|
||||
(feed_id, guid, title, link, published, description, first_seen,
|
||||
image, duration, episode, season)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, 0, 0, ?8, ?9, ?10, ?11)",
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11)",
|
||||
rusqlite::params![
|
||||
feed_id, e.guid, e.title, e.link, e.published, e.description, now(),
|
||||
e.image, e.duration, e.episode, e.season
|
||||
],
|
||||
)?;
|
||||
if inserted == 0 {
|
||||
// The SET expressions see the pre-update row, so this compares old vs new.
|
||||
conn.execute(
|
||||
"UPDATE entries SET
|
||||
title = coalesce(?3, title),
|
||||
@@ -368,8 +372,7 @@ impl Db {
|
||||
image = coalesce(?5, image),
|
||||
duration = coalesce(?6, duration),
|
||||
episode = coalesce(?7, episode),
|
||||
season = coalesce(?8, season),
|
||||
read = CASE WHEN description IS NOT ?4 OR title IS NOT ?3 THEN 0 ELSE read END
|
||||
season = coalesce(?8, season)
|
||||
WHERE feed_id = ?1 AND guid = ?2",
|
||||
rusqlite::params![
|
||||
feed_id, e.guid, e.title, e.description,
|
||||
@@ -609,6 +612,36 @@ pub struct EntryRow {
|
||||
const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2
|
||||
OR lower(coalesce(e.description, '')) LIKE ?2)";
|
||||
|
||||
/// Which feeds a query covers: one, or every feed the person subscribes to. Both forms
|
||||
/// mention `?1`, since binding a parameter the statement does not use is an error.
|
||||
fn scope_sql(feed_id: Option<&str>, user_param: u8) -> String {
|
||||
match feed_id {
|
||||
Some(_) => "e.feed_id = ?1".into(),
|
||||
None => format!(
|
||||
"?1 IS NULL AND e.feed_id IN (SELECT feed_id FROM subscriptions WHERE user_id = ?{user_param})"
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// The item table's ORDER BY. The column name picks one of these fixed expressions, so nothing
|
||||
/// the caller sends reaches the query, and anything unrecognised is newest first. Ties fall back
|
||||
/// to newest first too, so a page boundary is stable across "Load more".
|
||||
///
|
||||
/// ponytail: file type and size look at the item's first and largest file. The row shows the file
|
||||
/// it summarises, which is almost always that one; sort by that one if they ever disagree.
|
||||
pub fn order_sql(col: &str, dir: &str) -> String {
|
||||
let expr = match col {
|
||||
"kept" => "coalesce(s.flagged, 0)",
|
||||
"title" => "lower(coalesce(e.title, ''))",
|
||||
"feed" => "(SELECT lower(coalesce(f.title, f.id)) FROM feeds f WHERE f.id = e.feed_id)",
|
||||
"type" => "(SELECT min(x.mime) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
|
||||
"size" => "(SELECT max(x.length) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
|
||||
_ => "coalesce(e.published, e.first_seen)",
|
||||
};
|
||||
let dir = if dir == "asc" { "ASC" } else { "DESC" };
|
||||
format!("{expr} {dir}, coalesce(e.published, e.first_seen) DESC, e.rowid DESC")
|
||||
}
|
||||
|
||||
/// Which slice of a feed the UI is asking for.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum Filter {
|
||||
@@ -657,16 +690,18 @@ pub struct EncRow {
|
||||
}
|
||||
|
||||
impl Db {
|
||||
/// One page of a feed's entries, newest first, each with its enclosures attached.
|
||||
/// `search` matches title and description, case-insensitively.
|
||||
pub fn entries(
|
||||
/// One page of entries, each with its enclosures attached: one feed's, or every feed the
|
||||
/// person subscribes to when `feed_id` is None (All Subscriptions). `search` matches title
|
||||
/// and description, case-insensitively.
|
||||
pub fn entries_in(
|
||||
&self,
|
||||
user_id: i64,
|
||||
feed_id: &str,
|
||||
feed_id: Option<&str>,
|
||||
filter: Filter,
|
||||
search: Option<&str>,
|
||||
offset: i64,
|
||||
limit: i64,
|
||||
order: &str,
|
||||
) -> Result<Vec<EntryRow>> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
let like = search
|
||||
@@ -679,9 +714,10 @@ impl Db {
|
||||
FROM entries e
|
||||
LEFT JOIN entry_state s
|
||||
ON s.user_id = ?5 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
||||
WHERE e.feed_id = ?1 AND {} AND {SEARCH}
|
||||
ORDER BY coalesce(e.published, e.first_seen) DESC, e.rowid DESC
|
||||
WHERE {} AND {} AND {SEARCH}
|
||||
ORDER BY {order}
|
||||
LIMIT ?4 OFFSET ?3",
|
||||
scope_sql(feed_id, 5),
|
||||
filter.sql()
|
||||
);
|
||||
let mut stmt = conn.prepare(&sql)?;
|
||||
@@ -711,14 +747,14 @@ impl Db {
|
||||
return Ok(rows);
|
||||
}
|
||||
|
||||
// Only the guids on this page, so a feed with thousands of entries stays cheap.
|
||||
// Only the guids on this page, so a feed with thousands of entries stays cheap. A page
|
||||
// can span feeds, so each file is matched to its row by feed as well as guid, below.
|
||||
let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(",");
|
||||
let sql = format!(
|
||||
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error
|
||||
FROM enclosures WHERE feed_id = ? AND guid IN ({placeholders}) ORDER BY id"
|
||||
FROM enclosures WHERE guid IN ({placeholders}) ORDER BY id"
|
||||
);
|
||||
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len() + 1);
|
||||
params.push(&feed_id);
|
||||
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len());
|
||||
for row in &rows {
|
||||
params.push(&row.guid);
|
||||
}
|
||||
@@ -740,18 +776,18 @@ impl Db {
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
|
||||
for enc in encs {
|
||||
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid) {
|
||||
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid && r.feed_id == enc.feed_id) {
|
||||
row.enclosures.push(enc);
|
||||
}
|
||||
}
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
/// How many entries match, so the UI knows whether there is another page.
|
||||
pub fn count_entries(
|
||||
/// How many entries `entries_in` would page through, so the UI knows whether there is more.
|
||||
pub fn count_in(
|
||||
&self,
|
||||
user_id: i64,
|
||||
feed_id: &str,
|
||||
feed_id: Option<&str>,
|
||||
filter: Filter,
|
||||
search: Option<&str>,
|
||||
) -> Result<i64> {
|
||||
@@ -763,7 +799,8 @@ impl Db {
|
||||
"SELECT count(*) FROM entries e
|
||||
LEFT JOIN entry_state s
|
||||
ON s.user_id = ?3 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
||||
WHERE e.feed_id = ?1 AND {} AND {SEARCH}",
|
||||
WHERE {} AND {} AND {SEARCH}",
|
||||
scope_sql(feed_id, 3),
|
||||
filter.sql()
|
||||
);
|
||||
Ok(conn.query_row(&sql, rusqlite::params![feed_id, like, user_id], |r| r.get(0))?)
|
||||
@@ -780,24 +817,16 @@ impl Db {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Marks every entry in a feed read, for the "mark all read" button.
|
||||
/// Moves a single-user library onto an account: everything read, starred or part-played
|
||||
/// becomes that person's, and they subscribe to every feed already in the catalogue.
|
||||
/// Runs once -- the moment there is a first account and no subscriptions yet.
|
||||
pub fn adopt_existing_library(&self, user_id: i64, catalogue: &[String]) -> Result<usize> {
|
||||
/// The first admin starts subscribed to the whole catalogue: whoever wrote config.toml meant
|
||||
/// to read those feeds, and without this a fresh install signs in to an empty sidebar. Runs
|
||||
/// only while nobody subscribes to anything, so an unsubscribe is never undone.
|
||||
pub fn adopt_catalogue(&self, user_id: i64, catalogue: &[String]) -> Result<usize> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
let already: i64 =
|
||||
conn.query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0))?;
|
||||
if already > 0 {
|
||||
return Ok(0);
|
||||
}
|
||||
let moved = conn.execute(
|
||||
"INSERT INTO entry_state (user_id, feed_id, guid, read, flagged, position)
|
||||
SELECT ?1, feed_id, guid, read, flagged, position FROM entries
|
||||
WHERE read = 1 OR flagged = 1 OR position > 0
|
||||
ON CONFLICT(user_id, feed_id, guid) DO NOTHING",
|
||||
[user_id],
|
||||
)?;
|
||||
for id in catalogue {
|
||||
conn.execute(
|
||||
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
|
||||
@@ -810,7 +839,7 @@ impl Db {
|
||||
SELECT ?1, id, ?2 FROM feeds",
|
||||
params![user_id, now()],
|
||||
)?;
|
||||
Ok(moved)
|
||||
Ok(catalogue.len())
|
||||
}
|
||||
|
||||
// ---- subscriptions ----
|
||||
@@ -862,14 +891,20 @@ impl Db {
|
||||
|
||||
/// Everyone's settings for one feed. The scanner merges these into what it fetches
|
||||
/// and downloads, since one file serves the lot.
|
||||
pub fn subscribers(&self, feed_id: &str) -> Result<Vec<Sub>> {
|
||||
/// In a group, whatever someone has not set on the feed itself comes from their
|
||||
/// subscription to the group, as the group's settings dialog has always said it does.
|
||||
pub fn subscribers(&self, feed_id: &str, group: Option<&str>) -> Result<Vec<Sub>> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT keywords, auto_download, allow_explicit, max_new_per_check
|
||||
FROM subscriptions WHERE feed_id = ?1",
|
||||
"SELECT coalesce(c.keywords, p.keywords), coalesce(c.auto_download, p.auto_download),
|
||||
coalesce(c.allow_explicit, p.allow_explicit),
|
||||
coalesce(c.max_new_per_check, p.max_new_per_check)
|
||||
FROM subscriptions c
|
||||
LEFT JOIN subscriptions p ON p.user_id = c.user_id AND p.feed_id = ?2
|
||||
WHERE c.feed_id = ?1",
|
||||
)?;
|
||||
let out = stmt
|
||||
.query_map([feed_id], |r| {
|
||||
.query_map(params![feed_id, group], |r| {
|
||||
Ok(Sub {
|
||||
feed_id: feed_id.to_string(),
|
||||
keywords: r
|
||||
@@ -1267,6 +1302,71 @@ impl Db {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Empties a feed of its items, leaving its files alone.
|
||||
pub fn clear_entries(&self, feed_id: &str) -> Result<()> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
conn.execute("DELETE FROM entries WHERE feed_id = ?1", [feed_id])?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Every feed the database holds rows for, as (id, url), removed ones included.
|
||||
pub fn feed_urls(&self) -> Result<Vec<(String, String)>> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
let mut stmt = conn.prepare("SELECT id, coalesce(url, '') FROM feeds")?;
|
||||
let out = stmt
|
||||
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))?
|
||||
.collect::<rusqlite::Result<_>>()?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Hands a feed in a group the enclosures its parent holds, as (guid, url), with everyone's
|
||||
/// read state for them. A Patreon creator read as one feed before it was split into shows
|
||||
/// owns every show's files, and `enclosures.url` is unique, so without this each show would
|
||||
/// list its items with nothing to play.
|
||||
pub fn adopt(&self, parent: &str, child: &str, listed: &[(&str, &str)]) -> Result<()> {
|
||||
let mut conn = self.conn.lock().unwrap();
|
||||
let holds: bool = conn.query_row(
|
||||
"SELECT EXISTS (SELECT 1 FROM enclosures WHERE feed_id = ?1)",
|
||||
[parent],
|
||||
|r| r.get(0),
|
||||
)?;
|
||||
if !holds {
|
||||
return Ok(()); // An OPML, or a creator already shared out.
|
||||
}
|
||||
let tx = conn.transaction()?;
|
||||
for &(guid, url) in listed {
|
||||
let moved = tx.execute(
|
||||
"UPDATE enclosures SET feed_id = ?3, guid = ?4 WHERE url = ?1 AND feed_id = ?2",
|
||||
params![url, parent, child, guid],
|
||||
)?;
|
||||
if moved == 1 {
|
||||
// Patreon gives a post the same guid in every feed it appears in.
|
||||
tx.execute(
|
||||
"UPDATE OR IGNORE entry_state SET feed_id = ?2 WHERE feed_id = ?1 AND guid = ?3",
|
||||
params![parent, child, guid],
|
||||
)?;
|
||||
}
|
||||
}
|
||||
tx.commit()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A feed's enclosures skipped by one of its filters, by URL, with the reason: the verdicts a
|
||||
/// change of settings can overturn. A torrent held back while torrents are off is not a
|
||||
/// filter's call.
|
||||
pub fn skipped_by_filter(&self, feed_id: &str) -> Result<std::collections::HashMap<String, String>> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT url, last_error FROM enclosures
|
||||
WHERE feed_id = ?1 AND state = 'skipped' AND last_error IS NOT NULL
|
||||
AND last_error != 'torrents disabled'",
|
||||
)?;
|
||||
let out = stmt
|
||||
.query_map([feed_id], |r| Ok((r.get(0)?, r.get(1)?)))?
|
||||
.collect::<rusqlite::Result<_>>()?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
pub fn set_orphaned(&self, feed_id: &str, on: bool) -> Result<()> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
conn.execute(
|
||||
@@ -1319,6 +1419,40 @@ pub fn now() -> i64 {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn every_sort_column_runs_and_orders_both_ways() {
|
||||
let db = Db::memory().unwrap();
|
||||
db.exec_for_test(
|
||||
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
|
||||
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(1,'g',0);
|
||||
INSERT INTO feeds (id, url, title) VALUES ('f','u','Zebra'),('g','v','Aardvark');
|
||||
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
|
||||
('f','a','banana',100),('g','b','Apple',200),('f','c','cherry',300);
|
||||
INSERT INTO enclosures (id, feed_id, guid, url, mime, length, state) VALUES
|
||||
(1,'f','a','u1','audio/mpeg',300,'pending'),(2,'g','b','u2','image/png',10,'pending'),
|
||||
(3,'f','c','u3','video/mp4',2000,'pending');",
|
||||
)
|
||||
.unwrap();
|
||||
let order = |col: &str, dir: &str| -> Vec<String> {
|
||||
db.entries_in(1, None, Filter::All, None, 0, 50, &order_sql(col, dir))
|
||||
.unwrap()
|
||||
.into_iter()
|
||||
.map(|e| e.guid)
|
||||
.collect()
|
||||
};
|
||||
assert_eq!(order("title", "asc"), ["b", "a", "c"], "Apple, banana, cherry: case folded");
|
||||
assert_eq!(order("title", "desc"), ["c", "a", "b"]);
|
||||
assert_eq!(order("feed", "asc"), ["b", "c", "a"], "Aardvark, then Zebra's newest first");
|
||||
assert_eq!(order("type", "asc"), ["a", "b", "c"], "audio, image, video");
|
||||
assert_eq!(order("size", "desc"), ["c", "a", "b"]);
|
||||
assert_eq!(order("published", "desc"), ["c", "b", "a"]);
|
||||
db.set_entry_flag(1, "f", "a", EntryFlag::Flagged, true).unwrap();
|
||||
assert_eq!(order("kept", "desc")[0], "a");
|
||||
// An unknown column or direction is newest first; the name itself never reaches the SQL.
|
||||
assert_eq!(order("title; DROP TABLE entries", "sideways"), ["c", "b", "a"]);
|
||||
assert!(!order_sql("x'; --", "asc").contains("x'"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deleting_a_shared_file_asks_about_everyone_else() {
|
||||
let db = Db::memory().unwrap();
|
||||
@@ -1365,8 +1499,9 @@ mod tests {
|
||||
// Starring and position are just as private.
|
||||
db.set_entry_flag(1, "f", "b", EntryFlag::Flagged, true).unwrap();
|
||||
db.set_position(2, "f", "b", 42).unwrap();
|
||||
let ray = db.entries(1, "f", Filter::All, None, 0, 50).unwrap();
|
||||
let sam = db.entries(2, "f", Filter::All, None, 0, 50).unwrap();
|
||||
let order = order_sql("published", "desc");
|
||||
let page = |user| db.entries_in(user, Some("f"), Filter::All, None, 0, 50, &order).unwrap();
|
||||
let (ray, sam) = (page(1), page(2));
|
||||
let ray_b = ray.iter().find(|e| e.guid == "b").unwrap();
|
||||
let sam_b = sam.iter().find(|e| e.guid == "b").unwrap();
|
||||
assert!(ray_b.flagged && ray_b.position == 0);
|
||||
@@ -1390,6 +1525,49 @@ mod tests {
|
||||
assert_eq!(sum.downloaded, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_old_database_loses_the_retired_read_columns() {
|
||||
let conn = Connection::open_in_memory().unwrap();
|
||||
conn.execute_batch(
|
||||
"CREATE TABLE entries (feed_id TEXT NOT NULL, guid TEXT NOT NULL,
|
||||
first_seen INTEGER NOT NULL, read INTEGER NOT NULL DEFAULT 0,
|
||||
flagged INTEGER NOT NULL DEFAULT 0, position INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (feed_id, guid));",
|
||||
)
|
||||
.unwrap();
|
||||
// The same order as open(): the schema leaves the old table alone, migrate() fixes it.
|
||||
conn.execute_batch(SCHEMA).unwrap();
|
||||
migrate(&conn).unwrap();
|
||||
let cols: Vec<String> = conn
|
||||
.prepare("PRAGMA table_info(entries)")
|
||||
.unwrap()
|
||||
.query_map([], |r| r.get(1))
|
||||
.unwrap()
|
||||
.collect::<rusqlite::Result<_>>()
|
||||
.unwrap();
|
||||
assert!(!cols.iter().any(|c| ["read", "flagged", "position"].contains(&c.as_str())), "{cols:?}");
|
||||
assert!(cols.iter().any(|c| c == "image"), "and it still gains the newer ones");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_first_admin_starts_with_the_catalogue_and_only_once() {
|
||||
// Cutting this along with the dead read columns left the browser suite's admin with an
|
||||
// empty sidebar: it is how a fresh install's first account gets config.toml's feeds.
|
||||
let db = Db::memory().unwrap();
|
||||
db.exec_for_test("INSERT INTO users (id, name, is_admin, created) VALUES (1,'admin',1,0);")
|
||||
.unwrap();
|
||||
let subs = || -> i64 {
|
||||
db.conn.lock().unwrap().query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0)).unwrap()
|
||||
};
|
||||
let catalogue = ["a".to_string(), "b".to_string()];
|
||||
assert_eq!(db.adopt_catalogue(1, &catalogue).unwrap(), 2);
|
||||
assert_eq!(subs(), 2);
|
||||
// Once anyone subscribes to anything it never runs again, so an unsubscribe sticks.
|
||||
db.unsubscribe(1, "a").unwrap();
|
||||
assert_eq!(db.adopt_catalogue(1, &catalogue).unwrap(), 0);
|
||||
assert_eq!(subs(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_filter_works_with_and_without_a_search_term() {
|
||||
// Regression: the search clause used to be omitted when no term was given, while
|
||||
@@ -1413,21 +1591,22 @@ mod tests {
|
||||
|
||||
for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] {
|
||||
// Both paths must run without erroring, and agree with each other.
|
||||
let rows = db.entries(7, "f", f, None, 0, 50).unwrap();
|
||||
let n = db.count_entries(7, "f", f, None).unwrap();
|
||||
let order = order_sql("published", "desc");
|
||||
let rows = db.entries_in(7, Some("f"), f, None, 0, 50, &order).unwrap();
|
||||
let n = db.count_in(7, Some("f"), f, None).unwrap();
|
||||
assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page");
|
||||
|
||||
let rows = db.entries(7, "f", f, Some("dive"), 0, 50).unwrap();
|
||||
let n = db.count_entries(7, "f", f, Some("dive")).unwrap();
|
||||
let rows = db.entries_in(7, Some("f"), f, Some("dive"), 0, 50, &order).unwrap();
|
||||
let n = db.count_in(7, Some("f"), f, Some("dive")).unwrap();
|
||||
assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees");
|
||||
}
|
||||
|
||||
assert_eq!(db.count_entries(7, "f", Filter::All, None).unwrap(), 3);
|
||||
assert_eq!(db.count_entries(7, "f", Filter::Unread, None).unwrap(), 1);
|
||||
assert_eq!(db.count_entries(7, "f", Filter::Downloaded, None).unwrap(), 1);
|
||||
assert_eq!(db.count_entries(7, "f", Filter::Flagged, None).unwrap(), 1);
|
||||
assert_eq!(db.count_entries(7, "f", Filter::All, Some("dive")).unwrap(), 2);
|
||||
assert_eq!(db.count_entries(7, "f", Filter::All, Some("NOTES two")).unwrap(), 1,
|
||||
assert_eq!(db.count_in(7, Some("f"), Filter::All, None).unwrap(), 3);
|
||||
assert_eq!(db.count_in(7, Some("f"), Filter::Unread, None).unwrap(), 1);
|
||||
assert_eq!(db.count_in(7, Some("f"), Filter::Downloaded, None).unwrap(), 1);
|
||||
assert_eq!(db.count_in(7, Some("f"), Filter::Flagged, None).unwrap(), 1);
|
||||
assert_eq!(db.count_in(7, Some("f"), Filter::All, Some("dive")).unwrap(), 2);
|
||||
assert_eq!(db.count_in(7, Some("f"), Filter::All, Some("NOTES two")).unwrap(), 1,
|
||||
"search is case-insensitive and covers the description");
|
||||
}
|
||||
|
||||
@@ -1469,6 +1648,61 @@ mod tests {
|
||||
assert_eq!(state(4), "done");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_show_takes_over_what_its_creator_held() {
|
||||
let db = Db::memory().unwrap();
|
||||
db.exec_for_test(
|
||||
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
|
||||
INSERT INTO enclosures (id, feed_id, guid, url, state, path, last_error) VALUES
|
||||
(1,'creator','a','u1','done','/x/a.mp3',NULL),
|
||||
(2,'creator','b','u2','skipped',NULL,'explicit'),
|
||||
(3,'creator','c','u3','skipped',NULL,'explicit'),
|
||||
(4,'other','d','u4','skipped',NULL,'torrents disabled');
|
||||
INSERT INTO entry_state (user_id, feed_id, guid, read) VALUES (1,'creator','a',1);",
|
||||
)
|
||||
.unwrap();
|
||||
db.adopt("creator", "show", &[("a", "u1"), ("b", "u2"), ("d", "u4")]).unwrap();
|
||||
{
|
||||
let conn = db.conn.lock().unwrap();
|
||||
let owner = |id: i64| -> String {
|
||||
conn.query_row("SELECT feed_id FROM enclosures WHERE id = ?1", [id], |r| r.get(0)).unwrap()
|
||||
};
|
||||
assert_eq!(owner(1), "show", "a downloaded file moves with its item");
|
||||
assert_eq!(owner(2), "show");
|
||||
assert_eq!(owner(3), "creator", "this show does not list it");
|
||||
assert_eq!(owner(4), "other", "only the parent's are taken");
|
||||
let read: String = conn
|
||||
.query_row("SELECT feed_id FROM entry_state WHERE user_id = 1 AND guid = 'a'", [], |r| r.get(0))
|
||||
.unwrap();
|
||||
assert_eq!(read, "show", "what you had read stays read");
|
||||
}
|
||||
|
||||
// Only a filter's verdict can be overturned by a change of settings.
|
||||
let skipped = db.skipped_by_filter("show").unwrap();
|
||||
assert_eq!(skipped.get("u2").map(String::as_str), Some("explicit"));
|
||||
assert_eq!(skipped.len(), 1);
|
||||
assert!(db.skipped_by_filter("other").unwrap().is_empty(), "torrents disabled is not a filter");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_feed_in_a_group_follows_your_settings_on_the_group() {
|
||||
let db = Db::memory().unwrap();
|
||||
db.exec_for_test(
|
||||
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
|
||||
INSERT INTO subscriptions (user_id, feed_id, allow_explicit, created) VALUES
|
||||
(1,'group',1,0),(1,'show',NULL,0),(2,'group',1,0),(2,'show',0,0);",
|
||||
)
|
||||
.unwrap();
|
||||
let explicit = |group| -> Vec<Option<bool>> {
|
||||
let mut v: Vec<_> =
|
||||
db.subscribers("show", group).unwrap().into_iter().map(|s| s.allow_explicit).collect();
|
||||
v.sort();
|
||||
v
|
||||
};
|
||||
assert_eq!(explicit(Some("group")), [Some(false), Some(true)], "ray inherits; sam's own choice on the show wins");
|
||||
assert_eq!(explicit(None), [None, Some(false)], "outside a group nothing is inherited");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn enclosure_url_is_the_dedupe_key() {
|
||||
let db = Db::memory().unwrap();
|
||||
|
||||
@@ -223,7 +223,7 @@ enum Sniffed {
|
||||
/// 2008 and so always answered 'data'.
|
||||
async fn sniff(path: &Path) -> Result<Sniffed> {
|
||||
let head = read_head(path, 512).await?;
|
||||
if infer::is(&head, "torrent") || head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
|
||||
if head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
|
||||
return Ok(Sniffed::Torrent);
|
||||
}
|
||||
let text = String::from_utf8_lossy(&head);
|
||||
|
||||
136
src/feed.rs
136
src/feed.rs
@@ -117,6 +117,103 @@ pub fn opml_title(bytes: &[u8]) -> Option<String> {
|
||||
.filter(|t| !t.is_empty())
|
||||
}
|
||||
|
||||
/// The token and show of a Patreon feed link, or None for any other URL.
|
||||
///
|
||||
/// Patreon gives each patron one token per creator. With no show it stands for the creator,
|
||||
/// whose feed carries every show at once.
|
||||
fn patreon_parts(url: &str) -> Option<(String, Option<String>)> {
|
||||
let u = url::Url::parse(url).ok()?;
|
||||
if !matches!(u.host_str()?, "patreon.com" | "www.patreon.com") || !u.path().starts_with("/rss") {
|
||||
return None;
|
||||
}
|
||||
let param = |name: &str| u.query_pairs().find(|(k, _)| k == name).map(|(_, v)| v.into_owned());
|
||||
Some((param("auth")?, param("show")))
|
||||
}
|
||||
|
||||
/// A Patreon link naming a creator but no show.
|
||||
pub fn is_patreon_creator(url: &str) -> bool {
|
||||
matches!(patreon_parts(url), Some((_, None)))
|
||||
}
|
||||
|
||||
/// What was typed into Add feed, as a URL. A bare Patreon token is taken as its creator's
|
||||
/// feed, since the token alone says whose it is.
|
||||
pub fn expand_input(input: &str) -> String {
|
||||
let s = input.trim();
|
||||
let token = s.len() >= 20 && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_');
|
||||
if token { format!("https://www.patreon.com/rss?auth={s}") } else { s.to_owned() }
|
||||
}
|
||||
|
||||
/// Whether two URLs are the same feed. One Patreon show has several spellings -- by the
|
||||
/// creator's name, by number, or with no creator at all -- and the token and show are what
|
||||
/// identify it.
|
||||
pub fn same_feed(a: &str, b: &str) -> bool {
|
||||
a == b || patreon_parts(a).is_some_and(|p| Some(p) == patreon_parts(b))
|
||||
}
|
||||
|
||||
/// A Patreon creator's name and shows, each show as (title, feed URL).
|
||||
///
|
||||
/// ponytail: Patreon's own web API, undocumented, asked without signing in. If it changes,
|
||||
/// finding shows stops and the show feeds already found keep working. The documented API
|
||||
/// needs an OAuth client per install and does not list shows.
|
||||
pub async fn patreon_shows(
|
||||
client: &reqwest::Client,
|
||||
url: &str,
|
||||
) -> Result<(Option<String>, Vec<(String, String)>)> {
|
||||
// The creator feed names its campaign by number in its self link, a few hundred bytes in.
|
||||
// The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
|
||||
let mut resp = client.get(url).send().await.context("connecting")?;
|
||||
if !resp.status().is_success() {
|
||||
return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
|
||||
}
|
||||
let mut head = Vec::new();
|
||||
while patreon_campaign(&head).is_none() && head.len() < 64 * 1024 {
|
||||
let Some(chunk) = resp.chunk().await.context("reading the feed")? else { break };
|
||||
head.extend_from_slice(&chunk);
|
||||
}
|
||||
let campaign = patreon_campaign(&head)
|
||||
.ok_or_else(|| anyhow!("the Patreon feed does not say whose it is"))?;
|
||||
|
||||
let api = format!(
|
||||
"https://www.patreon.com/api/campaigns/{campaign}\
|
||||
?include=shows&fields%5Bcampaign%5D=name&fields%5Bcollection%5D=title"
|
||||
);
|
||||
let resp = client.get(api).send().await.context("asking Patreon for the shows")?;
|
||||
if !resp.status().is_success() {
|
||||
return Err(anyhow!("Patreon would not list the shows: HTTP {}", resp.status()));
|
||||
}
|
||||
let (name, shows) = parse_patreon_shows(&resp.bytes().await.context("reading the shows")?)?;
|
||||
Ok((name, shows.into_iter().map(|(id, title)| (title, format!("{url}&show={id}"))).collect()))
|
||||
}
|
||||
|
||||
/// The campaign number in the start of a Patreon feed.
|
||||
fn patreon_campaign(head: &[u8]) -> Option<String> {
|
||||
let text = String::from_utf8_lossy(head);
|
||||
text.match_indices("patreon.com/rss/").find_map(|(i, m)| {
|
||||
let id: String = text[i + m.len()..].chars().take_while(char::is_ascii_digit).collect();
|
||||
(!id.is_empty()).then_some(id)
|
||||
})
|
||||
}
|
||||
|
||||
/// A campaign's name and its shows as (id, title), from Patreon's JSON:API answer.
|
||||
fn parse_patreon_shows(json: &[u8]) -> Result<(Option<String>, Vec<(String, String)>)> {
|
||||
let v: serde_json::Value = serde_json::from_slice(json).context("Patreon's answer is not JSON")?;
|
||||
// Missing is not the same as none. Read as no shows, the creator feed would be scanned as
|
||||
// a plain feed, claim every show's files, and leave the shows empty once the list returned.
|
||||
let ids = v["data"]["relationships"]["shows"]["data"]
|
||||
.as_array()
|
||||
.ok_or_else(|| anyhow!("Patreon's answer does not list the shows"))?;
|
||||
let title = |id: &str| -> Option<String> {
|
||||
let show = v["included"].as_array()?.iter().find(|x| x["type"] == "collection" && x["id"] == id)?;
|
||||
show["attributes"]["title"].as_str().map(|t| t.trim().to_owned())
|
||||
};
|
||||
let shows = ids
|
||||
.iter()
|
||||
.filter_map(|s| s["id"].as_str())
|
||||
.map(|id| (id.to_owned(), title(id).unwrap_or_else(|| format!("Show {id}"))))
|
||||
.collect();
|
||||
Ok((v["data"]["attributes"]["name"].as_str().map(str::to_owned), shows))
|
||||
}
|
||||
|
||||
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
|
||||
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
|
||||
match rss::Channel::read_from(bytes) {
|
||||
@@ -550,6 +647,45 @@ mod tests {
|
||||
assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_patreon_creator_is_a_list_of_its_shows() {
|
||||
let tok = "AbCdEfGhIjKlMnOpQrStUvWxYz012_-9";
|
||||
assert_eq!(expand_input(&format!(" {tok} ")), format!("https://www.patreon.com/rss?auth={tok}"));
|
||||
assert_eq!(expand_input("https://example.com/rss"), "https://example.com/rss");
|
||||
|
||||
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss/glasscannon?auth={tok}")));
|
||||
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss?auth={tok}")));
|
||||
assert!(!is_patreon_creator(&format!("https://www.patreon.com/rss/x?auth={tok}&show=1")), "one show is a feed");
|
||||
assert!(!is_patreon_creator(&format!("https://example.com/rss?auth={tok}")));
|
||||
|
||||
// The show you already have by name is the one a bare token would add by number.
|
||||
assert!(same_feed(
|
||||
&format!("https://www.patreon.com/rss/glasscannon?auth={tok}&show=2073588"),
|
||||
&format!("https://www.patreon.com/rss?auth={tok}&show=2073588"),
|
||||
));
|
||||
assert!(!same_feed(
|
||||
&format!("https://www.patreon.com/rss?auth={tok}&show=1"),
|
||||
&format!("https://www.patreon.com/rss?auth={tok}&show=2"),
|
||||
));
|
||||
|
||||
// The self link carries the campaign by number, whichever spelling was asked for.
|
||||
let head = br#"<rss><channel><link>https://www.patreon.com/glasscannon</link>
|
||||
<atom:link href="https://www.patreon.com/rss/369921?auth=t" rel="self"/>"#;
|
||||
assert_eq!(patreon_campaign(head).as_deref(), Some("369921"));
|
||||
assert_eq!(patreon_campaign(b"<rss><channel><title>T"), None);
|
||||
|
||||
let json = br#"{"data":{"id":"369921","type":"campaign","attributes":{"name":"The Glass Cannon Network"},
|
||||
"relationships":{"shows":{"data":[{"id":"2073588","type":"collection"},{"id":"2073636","type":"collection"}]}}},
|
||||
"included":[{"id":"2073588","type":"collection","attributes":{"title":"Get in the Trunk "}},
|
||||
{"id":"2073636","type":"collection","attributes":{"title":"Shadowdark"}}]}"#;
|
||||
let (name, shows) = parse_patreon_shows(json).unwrap();
|
||||
assert_eq!(name.as_deref(), Some("The Glass Cannon Network"));
|
||||
assert_eq!(shows, [("2073588".into(), "Get in the Trunk".into()), ("2073636".into(), "Shadowdark".into())]);
|
||||
|
||||
// An answer that stops naming the shows is an error, never "this creator has none".
|
||||
assert!(parse_patreon_shows(br#"{"data":{"attributes":{"name":"X"}}}"#).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_item_may_carry_several_enclosures() {
|
||||
// The rss crate keeps only one per item -- the last -- so these come from the XML.
|
||||
|
||||
@@ -62,10 +62,10 @@ impl Event {
|
||||
Event::DownloadDone { path, .. } => format!(" saved {path}"),
|
||||
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
|
||||
Event::Reaped { path, bytes } => {
|
||||
format!("reap {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
|
||||
format!("deleted {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
|
||||
}
|
||||
Event::ReapDone { files, bytes } => format!(
|
||||
"reaped {files} file(s), {:.1} MB",
|
||||
"deleted {files} old file(s), {:.1} MB",
|
||||
*bytes as f64 / 1_048_576.0
|
||||
),
|
||||
Event::Status { feeds, pending, downloaded } => {
|
||||
|
||||
@@ -111,18 +111,11 @@ impl Visit for Collect {
|
||||
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
|
||||
self.add(field, format!("{value:?}"));
|
||||
}
|
||||
// Numbers and bools reach record_debug through the trait's defaults, which prints them the
|
||||
// same way. A string would print quoted there, hence its own method.
|
||||
fn record_str(&mut self, field: &Field, value: &str) {
|
||||
self.add(field, value.to_owned());
|
||||
}
|
||||
fn record_i64(&mut self, field: &Field, value: i64) {
|
||||
self.add(field, value.to_string());
|
||||
}
|
||||
fn record_u64(&mut self, field: &Field, value: u64) {
|
||||
self.add(field, value.to_string());
|
||||
}
|
||||
fn record_bool(&mut self, field: &Field, value: bool) {
|
||||
self.add(field, value.to_string());
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
151
src/main.rs
151
src/main.rs
@@ -334,8 +334,7 @@ async fn daemon(
|
||||
anyhow::bail!("a daemon is already listening on {}", socket.display());
|
||||
}
|
||||
|
||||
// A database with nobody in it cannot be signed into, and an install that predates
|
||||
// accounts still has to serve its owner. Both get the same starting point.
|
||||
// A database with nobody in it cannot be signed into.
|
||||
if ctx.db.users()?.is_empty() {
|
||||
ctx.db.create_user("admin", Some(&crate::auth::hash_password(DEFAULT_PASSWORD)?), true)?;
|
||||
tracing::warn!(
|
||||
@@ -344,22 +343,15 @@ async fn daemon(
|
||||
);
|
||||
}
|
||||
|
||||
// A library that predates accounts belongs to whoever was using it: the admin.
|
||||
if let Some(admin) = ctx.db.users()?.into_iter().find(|u| u.is_admin) {
|
||||
let catalogue: Vec<String> = ctx.cfg().feeds.keys().cloned().collect();
|
||||
match ctx.db.adopt_existing_library(admin.id, &catalogue) {
|
||||
match ctx.db.adopt_catalogue(admin.id, &catalogue) {
|
||||
Ok(0) => {}
|
||||
Ok(n) => tracing::info!(user = %admin.name, entries = n, "adopted the existing library"),
|
||||
Err(e) => tracing::error!(error = %e, "could not adopt the existing library"),
|
||||
Ok(n) => tracing::info!(user = %admin.name, feeds = n, "subscribed the first admin to the catalogue"),
|
||||
Err(e) => tracing::error!(error = %e, "could not subscribe the first admin to the catalogue"),
|
||||
}
|
||||
}
|
||||
|
||||
match migrate_opml_children(&ctx) {
|
||||
Ok(n) if n > 0 => tracing::info!(count = n, "moved OPML feeds out of config.toml into the database"),
|
||||
Ok(_) => {}
|
||||
Err(e) => tracing::warn!(error = ?e, "could not tidy OPML feeds out of the config"),
|
||||
}
|
||||
|
||||
match ctx.db.requeue_interrupted() {
|
||||
Ok(n) if n > 0 => tracing::info!(count = n, "requeued downloads interrupted by a restart"),
|
||||
Ok(_) => {}
|
||||
@@ -468,7 +460,7 @@ async fn start_web(
|
||||
let mut fresh = (*cfg).clone();
|
||||
fresh.web.enabled = true;
|
||||
fresh.web.bind = bind.clone();
|
||||
fresh.web.token = web::generate_token();
|
||||
fresh.web.token = crate::auth::new_session_token();
|
||||
fresh.save(config_path)?;
|
||||
ctx.reload_cfg(config_path)?;
|
||||
println!("web ui token generated. Open:\n http://{bind}/?token={}", fresh.web.token);
|
||||
@@ -517,8 +509,9 @@ async fn add(
|
||||
keywords: Vec<String>,
|
||||
) -> Result<()> {
|
||||
let mut cfg = (*ctx.cfg()).clone();
|
||||
let url = &feed::expand_input(url);
|
||||
// Includes feeds derived from an OPML, or the same show could be added twice.
|
||||
if let Some(existing) = subscriptions(ctx)?.iter().find(|s| s.cfg.url == url) {
|
||||
if let Some(existing) = subscriptions(ctx)?.iter().find(|s| feed::same_feed(&s.cfg.url, url)) {
|
||||
anyhow::bail!("already subscribed as {:?}", existing.id);
|
||||
}
|
||||
let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
|
||||
@@ -569,10 +562,17 @@ pub async fn add_one(
|
||||
|
||||
// Slugs must be unique across derived feeds too, or a new feed can collide with one
|
||||
// an OPML already introduced.
|
||||
let taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)?
|
||||
let mut taken: std::collections::BTreeMap<String, config::Feed> = subscriptions(ctx)?
|
||||
.into_iter()
|
||||
.map(|s| (s.id, s.cfg))
|
||||
.collect();
|
||||
// A removed feed keeps its rows, so its id is only free again for the same feed: re-adding
|
||||
// it gets its history back, and a different feed does not inherit someone else's.
|
||||
for (id, other) in ctx.db.feed_urls()? {
|
||||
if !feed::same_feed(&other, url) {
|
||||
taken.entry(id).or_insert_with(|| probe.clone());
|
||||
}
|
||||
}
|
||||
let id = config::unique_slug(&title, &taken);
|
||||
cfg.feeds.insert(id.clone(), probe);
|
||||
Ok(id)
|
||||
@@ -807,7 +807,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool) -> Result<()> {
|
||||
ctx.out.emit(Event::FeedSkip {
|
||||
feed: id.clone(),
|
||||
reason: format!(
|
||||
"OPML: {total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing",
|
||||
"{total} feed(s) listed, {} added, {removed} unsubscribed, {kept} kept without a listing",
|
||||
added.len()
|
||||
),
|
||||
});
|
||||
@@ -908,36 +908,6 @@ pub fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Moves OPML children that older versions wrote into config.toml over to the database.
|
||||
/// They were never yours to edit, and 80-odd of them made the file unreadable.
|
||||
fn migrate_opml_children(ctx: &Ctx) -> Result<usize> {
|
||||
let cfg = (*ctx.cfg()).clone();
|
||||
let children: Vec<(String, config::Feed)> = cfg
|
||||
.feeds
|
||||
.iter()
|
||||
.filter(|(_, f)| f.group.is_some())
|
||||
.map(|(id, f)| (id.clone(), f.clone()))
|
||||
.collect();
|
||||
if children.is_empty() {
|
||||
return Ok(0);
|
||||
}
|
||||
let mut fresh = cfg.clone();
|
||||
for (id, f) in &children {
|
||||
let group = f.group.clone().unwrap_or_default();
|
||||
let title = ctx
|
||||
.db
|
||||
.feed_summary(id)
|
||||
.ok()
|
||||
.and_then(|s| s.title)
|
||||
.unwrap_or_else(|| id.clone());
|
||||
ctx.db.upsert_managed(id, &f.url, &title, &group)?;
|
||||
fresh.feeds.remove(id);
|
||||
}
|
||||
fresh.save(&ctx.config_path)?;
|
||||
ctx.reload_cfg(&ctx.config_path)?;
|
||||
Ok(children.len())
|
||||
}
|
||||
|
||||
/// Seconds to wait before re-checking a feed.
|
||||
///
|
||||
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the
|
||||
@@ -963,7 +933,7 @@ struct Scan {
|
||||
enum Outcome {
|
||||
NotModified,
|
||||
Feed(Scan),
|
||||
/// The URL served an OPML document, so it is a subscription list rather than a feed.
|
||||
/// The URL is a list of feeds rather than a feed: an OPML, or a Patreon creator's shows.
|
||||
Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
|
||||
}
|
||||
|
||||
@@ -973,6 +943,31 @@ async fn scan_one(
|
||||
feed_cfg: &config::Feed,
|
||||
state: &db::HttpState,
|
||||
) -> Result<Outcome> {
|
||||
// A Patreon creator with more than one show is a list of feeds, like an OPML.
|
||||
if feed::is_patreon_creator(&feed_cfg.url) {
|
||||
match feed::patreon_shows(&ctx.client, &feed_cfg.url).await {
|
||||
Ok((name, shows)) if shows.len() > 1 => {
|
||||
ctx.db.touch_feed(id, &feed_cfg.url)?;
|
||||
if let Some(name) = name {
|
||||
ctx.db.set_title(id, &name)?;
|
||||
}
|
||||
// Read as one feed before it was split, it listed every show's items in one
|
||||
// heap. The items go; its files and read state move to each show as the show
|
||||
// lists them (`Db::adopt`), so no show comes up empty for want of a URL.
|
||||
ctx.db.clear_entries(id)?;
|
||||
return sync_group(ctx, id, feed_cfg, &shows).await;
|
||||
}
|
||||
Ok(_) => {} // One show: the creator's feed is that show.
|
||||
// Already split: keep the shows it has rather than read the creator as one heap.
|
||||
Err(e) if ctx.db.managed_feeds()?.iter().any(|m| m.group_id == id) => return Err(e),
|
||||
Err(e) => tracing::warn!(
|
||||
feed = id,
|
||||
error = %format!("{e:#}"),
|
||||
"could not list the Patreon shows; reading it as one feed"
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
let mut fetched = feed::fetch(
|
||||
&ctx.client,
|
||||
feed_cfg,
|
||||
@@ -1018,19 +1013,38 @@ async fn scan_one(
|
||||
)?;
|
||||
|
||||
let policy = policy_for(ctx, id, feed_cfg)?;
|
||||
if let Some(parent) = &feed_cfg.group {
|
||||
let listed: Vec<(&str, &str)> = parsed
|
||||
.entries
|
||||
.iter()
|
||||
.flat_map(|e| e.enclosures.iter().map(move |x| (e.guid.as_str(), x.url.as_str())))
|
||||
.collect();
|
||||
ctx.db.adopt(parent, id, &listed)?;
|
||||
}
|
||||
// Verdicts are recorded in `state`, so the download queue below is just "everything still
|
||||
// pending". A filter's verdict is looked at again on every scan, though: made once, at
|
||||
// discovery, it outlived the setting behind it, and allowing explicit items afterwards
|
||||
// changed nothing however often the feed was scanned.
|
||||
let skipped = ctx.db.skipped_by_filter(id)?;
|
||||
let mut scan = Scan::default();
|
||||
for entry in &parsed.entries {
|
||||
if ctx.db.record_entry(id, entry)? {
|
||||
scan.new_entries += 1;
|
||||
}
|
||||
for enc in &entry.enclosures {
|
||||
if !ctx.db.record_enclosure(id, &entry.guid, enc)? {
|
||||
continue; // Seen before: downloaded, skipped or deliberately reaped.
|
||||
let was = if ctx.db.record_enclosure(id, &entry.guid, enc)? {
|
||||
None
|
||||
} else if let Some(reason) = skipped.get(&enc.url) {
|
||||
Some(reason.as_str())
|
||||
} else {
|
||||
continue; // Settled: queued, downloaded, reaped, or another feed's file.
|
||||
};
|
||||
let now = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc);
|
||||
if now != was {
|
||||
match now {
|
||||
Some(reason) => ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?,
|
||||
None => ctx.db.mark_enclosure(&enc.url, "pending", None)?,
|
||||
}
|
||||
// Filters run once, at discovery, and are recorded in `state`. The download
|
||||
// queue below is then just "everything still pending".
|
||||
if let Some(reason) = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc) {
|
||||
ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1114,12 +1128,6 @@ async fn scan_one(
|
||||
Ok(Outcome::Feed(scan))
|
||||
}
|
||||
|
||||
/// Brings the feed list in step with a subscribed OPML.
|
||||
///
|
||||
/// New entries are added under the OPML's group and folder. An entry that has gone from
|
||||
/// the OPML is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
|
||||
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
|
||||
/// the UI to explain them.
|
||||
async fn sync_opml(
|
||||
ctx: &Arc<Ctx>,
|
||||
parent_id: &str,
|
||||
@@ -1130,25 +1138,44 @@ async fn sync_opml(
|
||||
if let Some(title) = feed::opml_title(bytes) {
|
||||
ctx.db.set_title(parent_id, &title)?;
|
||||
}
|
||||
sync_group(ctx, parent_id, parent, &listed).await
|
||||
}
|
||||
|
||||
/// Brings the feed list in step with a list of feeds: a subscribed OPML, or a Patreon
|
||||
/// creator's shows.
|
||||
///
|
||||
/// New entries are added under the list's group and folder. An entry that has gone from
|
||||
/// the list is unsubscribed *only if nothing was ever downloaded for it* -- otherwise it
|
||||
/// is kept and flagged, because dropping it would orphan files on disk with nothing in
|
||||
/// the UI to explain them.
|
||||
async fn sync_group(
|
||||
ctx: &Arc<Ctx>,
|
||||
parent_id: &str,
|
||||
parent: &config::Feed,
|
||||
listed: &[(String, String)],
|
||||
) -> Result<Outcome> {
|
||||
let cfg = ctx.cfg();
|
||||
let existing = ctx.db.managed_feeds()?;
|
||||
let mut added = vec![];
|
||||
|
||||
for (title, url) in &listed {
|
||||
for (title, url) in listed {
|
||||
// Already known, whether derived or promoted into the config.
|
||||
if let Some(m) = existing.iter().find(|m| &m.url == url) {
|
||||
ctx.db.upsert_managed(&m.id, url, title, parent_id)?;
|
||||
continue;
|
||||
}
|
||||
if cfg.feeds.values().any(|f| &f.url == url) {
|
||||
// A Patreon show you added by hand may be spelled differently from the one listed.
|
||||
if cfg.feeds.values().any(|f| feed::same_feed(&f.url, url)) {
|
||||
continue;
|
||||
}
|
||||
// A removed feed keeps its rows, so its id is only free again for the same feed.
|
||||
let known = ctx.db.feed_urls()?;
|
||||
let taken: std::collections::BTreeMap<String, config::Feed> = cfg
|
||||
.feeds
|
||||
.keys()
|
||||
.chain(existing.iter().map(|m| &m.id))
|
||||
.chain(added.iter())
|
||||
.chain(known.iter().filter(|(_, u)| !feed::same_feed(u, url)).map(|(id, _)| id))
|
||||
.map(|id| (id.clone(), parent.clone()))
|
||||
.collect();
|
||||
let id = config::unique_slug(title, &taken);
|
||||
@@ -1255,7 +1282,7 @@ pub struct Policy {
|
||||
|
||||
fn policy_for(ctx: &Ctx, id: &str, feed_cfg: &config::Feed) -> Result<Policy> {
|
||||
let global = ctx.cfg().general.max_new_per_check;
|
||||
Ok(merge_policy(&ctx.db.subscribers(id)?, feed_cfg, global))
|
||||
Ok(merge_policy(&ctx.db.subscribers(id, feed_cfg.group.as_deref())?, feed_cfg, global))
|
||||
}
|
||||
|
||||
fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Policy {
|
||||
|
||||
228
src/web.rs
228
src/web.rs
@@ -12,9 +12,7 @@ use axum::{
|
||||
},
|
||||
routing::{delete, get, patch, post},
|
||||
};
|
||||
use futures_util::StreamExt;
|
||||
use serde::Deserialize;
|
||||
use tokio_stream::wrappers::BroadcastStream;
|
||||
use tower::ServiceExt;
|
||||
use tower_http::services::ServeFile;
|
||||
use serde::Serialize;
|
||||
@@ -35,28 +33,6 @@ pub struct WebState {
|
||||
pub events: broadcast::Sender<Event>,
|
||||
}
|
||||
|
||||
/// A 32-hex-character shared secret, generated when config.toml has none.
|
||||
///
|
||||
/// ponytail: /dev/urandom rather than a CSPRNG crate -- 16 bytes, once, on a Unix-only
|
||||
/// binary. Falls back to the clock only if urandom is somehow unreadable, which would be a
|
||||
/// weak token, so that case is logged loudly.
|
||||
pub fn generate_token() -> String {
|
||||
use std::io::Read;
|
||||
let mut bytes = [0u8; 16];
|
||||
match std::fs::File::open("/dev/urandom").and_then(|mut f| f.read_exact(&mut bytes)) {
|
||||
Ok(()) => {}
|
||||
Err(e) => {
|
||||
tracing::error!(error = %e, "could not read /dev/urandom; token is NOT secure");
|
||||
let n = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_nanos() as u64)
|
||||
.unwrap_or(0);
|
||||
bytes[..8].copy_from_slice(&n.to_le_bytes());
|
||||
}
|
||||
}
|
||||
bytes.iter().map(|b| format!("{b:02x}")).collect()
|
||||
}
|
||||
|
||||
pub fn router(state: WebState) -> Router {
|
||||
Router::new()
|
||||
.route("/", get(index))
|
||||
@@ -65,9 +41,11 @@ pub fn router(state: WebState) -> Router {
|
||||
.route("/api/feeds", get(feeds).post(add_feed))
|
||||
.route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
|
||||
.route("/api/feeds/{id}/entries", get(entries))
|
||||
.route("/api/entries", get(all_entries))
|
||||
.route("/api/entries/{feed_id}/{guid}/flags", post(set_flags))
|
||||
.route("/api/entries/{feed_id}/{guid}/position", post(set_position))
|
||||
.route("/api/feeds/{id}/read-all", post(read_all))
|
||||
.route("/api/read-all", post(read_all_mine))
|
||||
.route("/api/feeds/{id}/download-latest", post(download_latest))
|
||||
.route("/api/enclosures/{id}/download", post(download_now))
|
||||
.route("/api/enclosures/{id}", delete(delete_file))
|
||||
@@ -75,6 +53,7 @@ pub fn router(state: WebState) -> Router {
|
||||
.route("/api/opml", get(export_opml).post(import_opml))
|
||||
.route("/api/settings", get(get_settings).patch(patch_settings))
|
||||
.route("/api/popular", get(get_popular))
|
||||
.route("/api/directory", get(get_directory))
|
||||
.route("/api/popular/{id}", post(subscribe_popular))
|
||||
.route("/api/users", get(list_users).post(add_user))
|
||||
.route("/api/users/{id}", patch(patch_user).delete(remove_user))
|
||||
@@ -85,6 +64,7 @@ pub fn router(state: WebState) -> Router {
|
||||
// Signing in cannot require being signed in, so these sit outside the auth layer.
|
||||
.route("/login", get(login_page))
|
||||
.route("/api/login", post(login))
|
||||
.route("/icon.png", get(icon))
|
||||
.layer(middleware::from_fn(access_log))
|
||||
.with_state(state)
|
||||
}
|
||||
@@ -430,6 +410,15 @@ async fn login_page() -> Html<&'static str> {
|
||||
Html(include_str!("../web/login.html"))
|
||||
}
|
||||
|
||||
/// The 2004 icon, served once for both pages rather than inlined as base64 into each. The
|
||||
/// sign-in page shows it, so it sits outside the auth layer with /login.
|
||||
async fn icon() -> impl IntoResponse {
|
||||
(
|
||||
[(header::CONTENT_TYPE, "image/png"), (header::CACHE_CONTROL, "max-age=86400")],
|
||||
include_bytes!("../web/ipodderx-icon.png").as_slice(),
|
||||
)
|
||||
}
|
||||
|
||||
fn constant_time_eq(a: &str, b: &str) -> bool {
|
||||
let (a, b) = (a.as_bytes(), b.as_bytes());
|
||||
if a.len() != b.len() {
|
||||
@@ -492,6 +481,9 @@ async fn feeds(
|
||||
let mut out = Vec::with_capacity(mine.len());
|
||||
for sub in &subs {
|
||||
let (id, feed) = (&sub.id, &sub.cfg);
|
||||
// In a group, what you have not set on the feed comes from your settings on the group,
|
||||
// the same fallback the scanner uses (`Db::subscribers`).
|
||||
let up = feed.group.as_deref().and_then(|g| mine.get(g));
|
||||
let Some(mine) = mine.get(id) else { continue };
|
||||
let s = state.ctx.db.feed_summary(id)?;
|
||||
let st = state.ctx.db.http_state(id)?;
|
||||
@@ -501,11 +493,22 @@ async fn feeds(
|
||||
title: s.title,
|
||||
image: s.image,
|
||||
folder: feed.folder.clone(),
|
||||
keywords: mine.keywords.clone().unwrap_or_else(|| feed.keywords.clone()),
|
||||
allow_explicit: mine.allow_explicit.unwrap_or(feed.allow_explicit),
|
||||
auto_download: mine.auto_download.unwrap_or(feed.auto_download),
|
||||
keywords: mine
|
||||
.keywords
|
||||
.clone()
|
||||
.or_else(|| up.and_then(|u| u.keywords.clone()))
|
||||
.unwrap_or_else(|| feed.keywords.clone()),
|
||||
allow_explicit: mine
|
||||
.allow_explicit
|
||||
.or(up.and_then(|u| u.allow_explicit))
|
||||
.unwrap_or(feed.allow_explicit),
|
||||
auto_download: mine
|
||||
.auto_download
|
||||
.or(up.and_then(|u| u.auto_download))
|
||||
.unwrap_or(feed.auto_download),
|
||||
max_new_per_check: mine
|
||||
.max_new_per_check
|
||||
.or(up.and_then(|u| u.max_new_per_check))
|
||||
.map(|n| n as usize)
|
||||
.or(feed.max_new_per_check),
|
||||
group: feed.group.clone(),
|
||||
@@ -537,14 +540,20 @@ async fn feeds(
|
||||
/// A feed that carries a credential is someone's paid or private subscription. Listing it
|
||||
/// would let anyone signed in subscribe to it and read what they pay for.
|
||||
///
|
||||
/// ponytail: a heuristic. A token hidden in the URL's path gets through; a per-feed
|
||||
/// `unlisted` flag is the upgrade if that ever happens.
|
||||
/// ponytail: a heuristic. A key hidden in the path of a host not in `PAID_HOSTS` gets
|
||||
/// through; a per-feed `unlisted` flag is the upgrade if that happens again.
|
||||
fn looks_private(feed: &crate::config::Feed) -> bool {
|
||||
if feed.username.is_some() || feed.password.is_some() || feed.password_env.is_some() {
|
||||
return true;
|
||||
}
|
||||
let Ok(u) = url::Url::parse(&feed.url) else { return true };
|
||||
!u.username().is_empty()
|
||||
// Paid-feed services put the subscriber's key in the path as often as in the query, and a
|
||||
// long path segment alone proves nothing (acast's public show ids look the same). So a
|
||||
// feed from one of them is private whatever its URL looks like. Supercast is why this
|
||||
// exists: `feeds.supercast.com/feeds/<key>` reached the directory before it did.
|
||||
let host = u.host_str().unwrap_or("");
|
||||
PAID_HOSTS.iter().any(|h| host == *h || host.ends_with(&format!(".{h}")))
|
||||
|| !u.username().is_empty()
|
||||
|| u.password().is_some()
|
||||
|| u.query_pairs().any(|(k, _)| {
|
||||
let k = k.to_ascii_lowercase();
|
||||
@@ -554,6 +563,10 @@ fn looks_private(feed: &crate::config::Feed) -> bool {
|
||||
})
|
||||
}
|
||||
|
||||
/// Services whose feeds are always one subscriber's own.
|
||||
const PAID_HOSTS: &[&str] =
|
||||
&["patreon.com", "supercast.com", "supportingcast.fm", "glow.fm", "memberful.com"];
|
||||
|
||||
/// Only an id, a title, artwork and a count: never a URL, which is where a key would be.
|
||||
#[derive(Serialize)]
|
||||
struct PopularRow {
|
||||
@@ -561,10 +574,13 @@ struct PopularRow {
|
||||
title: Option<String>,
|
||||
image: Option<String>,
|
||||
subscribers: i64,
|
||||
/// Yours already. Everyone counts, you included, so your own feeds are listed too.
|
||||
subscribed: bool,
|
||||
}
|
||||
|
||||
/// What other people here subscribe to that you don't, most subscribers first. What the
|
||||
/// Add feed screen offers, and all that `subscribe_popular` will subscribe you to.
|
||||
/// Every feed that may be listed, with everyone counted, you included, most subscribers
|
||||
/// first. Popular is the top of it, the directory is all of it, and it is all that
|
||||
/// `subscribe_popular` will subscribe you to.
|
||||
fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||
let db = &state.ctx.db;
|
||||
let mine: std::collections::HashSet<String> =
|
||||
@@ -576,14 +592,14 @@ fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||
// A feed from an OPML rides on the OPML: everyone subscribed to it counts every feed
|
||||
// inside, which would bury everything anyone chose on purpose.
|
||||
let from_opml = s.managed || s.cfg.group.is_some();
|
||||
if n == 0 || from_opml || mine.contains(&s.id) || looks_private(&s.cfg) {
|
||||
if n == 0 || from_opml || looks_private(&s.cfg) {
|
||||
continue;
|
||||
}
|
||||
let sum = db.feed_summary(&s.id)?;
|
||||
out.push(PopularRow { id: s.id, title: sum.title, image: sum.image, subscribers: n });
|
||||
let subscribed = mine.contains(&s.id);
|
||||
out.push(PopularRow { id: s.id, title: sum.title, image: sum.image, subscribers: n, subscribed });
|
||||
}
|
||||
let name = |p: &PopularRow| p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase();
|
||||
out.sort_by(|a, b| b.subscribers.cmp(&a.subscribers).then_with(|| name(a).cmp(&name(b))));
|
||||
out.sort_by(|a, b| b.subscribers.cmp(&a.subscribers).then_with(|| sort_name(a).cmp(&sort_name(b))));
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
@@ -592,10 +608,24 @@ async fn get_popular(
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||
let mut rows = popular(&state, user.id)?;
|
||||
rows.truncate(20);
|
||||
rows.truncate(10);
|
||||
Ok(Json(rows))
|
||||
}
|
||||
|
||||
/// Every feed that may be listed, A to Z.
|
||||
async fn get_directory(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||
let mut rows = popular(&state, user.id)?;
|
||||
rows.sort_by_key(sort_name);
|
||||
Ok(Json(rows))
|
||||
}
|
||||
|
||||
fn sort_name(p: &PopularRow) -> String {
|
||||
p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase()
|
||||
}
|
||||
|
||||
/// Subscribes by id, because the list never shows a URL. Checked against the same list, so
|
||||
/// a guessed id cannot reach a private feed.
|
||||
async fn subscribe_popular(
|
||||
@@ -695,6 +725,11 @@ mod tests {
|
||||
// Patreon's shape: the key is a query parameter.
|
||||
assert!(looks_private(&f("https://www.patreon.com/rss/x?auth=abc123&show=2073588")));
|
||||
assert!(looks_private(&f("https://example.com/rss?api_key=abc")));
|
||||
// Paid-feed services put the key in the path; the host gives them away.
|
||||
assert!(looks_private(&f("https://feeds.supercast.com/feeds/abcdefghijklmnopqrstuvwx")));
|
||||
assert!(looks_private(&f("https://someshow.supportingcast.fm/content/abc123.rss")));
|
||||
// A long path segment alone is not a key: acast's public show ids look the same.
|
||||
assert!(!looks_private(&f("https://feeds.acast.com/public/shows/0123456789abcdef01234567")));
|
||||
assert!(looks_private(&f("https://ray:hunter2@example.com/rss")));
|
||||
assert!(looks_private(&f("not a url")), "unparseable is not safe to list");
|
||||
let mut basic = f("https://example.com/rss");
|
||||
@@ -768,15 +803,6 @@ mod tests {
|
||||
"another feed already has that URL"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generated_tokens_are_32_hex_chars_and_not_repeated() {
|
||||
let a = generate_token();
|
||||
let b = generate_token();
|
||||
assert_eq!(a.len(), 32);
|
||||
assert!(a.chars().all(|c| c.is_ascii_hexdigit()));
|
||||
assert_ne!(a, b);
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
@@ -789,6 +815,12 @@ struct Page {
|
||||
filter: Option<String>,
|
||||
#[serde(default)]
|
||||
q: Option<String>,
|
||||
/// A column name and asc or desc. Anything unrecognised is newest first: the name picks a
|
||||
/// fixed expression in the query and never reaches it itself.
|
||||
#[serde(default)]
|
||||
sort: Option<String>,
|
||||
#[serde(default)]
|
||||
dir: Option<String>,
|
||||
}
|
||||
|
||||
fn fifty() -> i64 {
|
||||
@@ -806,20 +838,42 @@ async fn entries(
|
||||
Path(id): Path<String>,
|
||||
user: crate::db::User,
|
||||
Query(page): Query<Page>,
|
||||
) -> Result<Json<EntryPage>, ApiError> {
|
||||
entry_page(&state, user.id, Some(&id), &page)
|
||||
}
|
||||
|
||||
/// Every subscribed feed's items together, newest first: All Subscriptions.
|
||||
async fn all_entries(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
Query(page): Query<Page>,
|
||||
) -> Result<Json<EntryPage>, ApiError> {
|
||||
entry_page(&state, user.id, None, &page)
|
||||
}
|
||||
|
||||
/// One feed's page of items, or every subscribed feed's when `feed` is None.
|
||||
fn entry_page(
|
||||
state: &WebState,
|
||||
user_id: i64,
|
||||
feed: Option<&str>,
|
||||
page: &Page,
|
||||
) -> Result<Json<EntryPage>, ApiError> {
|
||||
let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all"));
|
||||
let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty());
|
||||
let mut rows = state
|
||||
.ctx
|
||||
.db
|
||||
.entries(user.id, &id, filter, search, page.offset, page.limit.clamp(1, 200))?;
|
||||
let db = &state.ctx.db;
|
||||
let order = crate::db::order_sql(
|
||||
page.sort.as_deref().unwrap_or("published"),
|
||||
page.dir.as_deref().unwrap_or("desc"),
|
||||
);
|
||||
let mut rows =
|
||||
db.entries_in(user_id, feed, filter, search, page.offset, page.limit.clamp(1, 200), &order)?;
|
||||
// Feed HTML is untrusted: it reaches the page only after ammonia has been through it.
|
||||
for row in &mut rows {
|
||||
if let Some(d) = &row.description {
|
||||
row.description = Some(ammonia::clean(d));
|
||||
}
|
||||
}
|
||||
let total = state.ctx.db.count_entries(user.id, &id, filter, search)?;
|
||||
let total = db.count_in(user_id, feed, filter, search)?;
|
||||
Ok(Json(EntryPage { total, entries: rows }))
|
||||
}
|
||||
|
||||
@@ -830,6 +884,19 @@ struct NewFeed {
|
||||
folder: Option<String>,
|
||||
#[serde(default)]
|
||||
keywords: Vec<String>,
|
||||
#[serde(default)]
|
||||
allow_explicit: bool,
|
||||
}
|
||||
|
||||
/// The Add feed dialog's explicit box. Like everything on a feed's own dialog it is yours, so it
|
||||
/// goes on your subscription, and before the first scan, which would otherwise skip every
|
||||
/// explicit item.
|
||||
fn explicit_on_add(state: &WebState, user_id: i64, feed_id: &str, allow: bool) -> Result<(), ApiError> {
|
||||
if allow {
|
||||
let sub = crate::db::Sub { feed_id: feed_id.to_owned(), allow_explicit: Some(true), ..Default::default() };
|
||||
state.ctx.db.set_subscription(user_id, &sub)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn add_feed(
|
||||
@@ -838,25 +905,42 @@ async fn add_feed(
|
||||
Json(body): Json<NewFeed>,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
let mut cfg = (*state.ctx.cfg()).clone();
|
||||
let url = crate::feed::expand_input(&body.url);
|
||||
// Someone else may already have it. Then adding costs nothing: no second fetch, no
|
||||
// second copy on disk, just another name against the same feed.
|
||||
if let Some(existing) = crate::subscriptions(&state.ctx)?
|
||||
.into_iter()
|
||||
.find(|s| s.cfg.url == body.url)
|
||||
.find(|s| crate::feed::same_feed(&s.cfg.url, &url))
|
||||
{
|
||||
let already = state.ctx.db.subscription(user.id, &existing.id)?.is_some();
|
||||
state.ctx.db.subscribe(user.id, &existing.id)?;
|
||||
if !already {
|
||||
explicit_on_add(&state, user.id, &existing.id, body.allow_explicit)?;
|
||||
}
|
||||
scan_soon(&state, Some(existing.id.clone())).await;
|
||||
return Ok(Json(
|
||||
serde_json::json!({ "id": existing.id, "existing": already }),
|
||||
));
|
||||
}
|
||||
let id = crate::add_one(&state.ctx, &mut cfg, &body.url, body.folder, body.keywords).await?;
|
||||
let id = crate::add_one(&state.ctx, &mut cfg, &url, body.folder, body.keywords).await?;
|
||||
cfg.save(&state.config_path)?;
|
||||
state.ctx.reload_cfg(&state.config_path)?;
|
||||
state.ctx.db.subscribe(user.id, &id)?;
|
||||
explicit_on_add(&state, user.id, &id, body.allow_explicit)?;
|
||||
scan_soon(&state, Some(id.clone())).await;
|
||||
Ok(Json(serde_json::json!({ "id": id, "existing": false })))
|
||||
}
|
||||
|
||||
/// Queues a scan, so a feed just added shows its items without anyone pressing Scan now.
|
||||
/// `None` scans whatever is due, which a feed never checked always is. The add has
|
||||
/// succeeded either way, so a daemon not taking commands is only logged.
|
||||
async fn scan_soon(state: &WebState, feed: Option<String>) {
|
||||
let force = feed.is_some();
|
||||
if state.cmds.send(Command::Fetch { feed, force }).await.is_err() {
|
||||
tracing::warn!("could not queue a scan: the daemon is not accepting commands");
|
||||
}
|
||||
}
|
||||
|
||||
/// Absent means "leave alone"; JSON `null` means "clear this".
|
||||
///
|
||||
/// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a
|
||||
@@ -1089,9 +1173,9 @@ async fn delete_file(
|
||||
let complaint = match (starred, unread) {
|
||||
(0, 0) => None,
|
||||
(0, u) => Some(format!("{} subscribed to this feed {} not played it yet", people(u), if u == 1 { "has" } else { "have" })),
|
||||
(st, 0) => Some(format!("another {} starred it to keep", people(st))),
|
||||
(st, 0) => Some(format!("another {} kept it", people(st))),
|
||||
(st, u) => Some(format!(
|
||||
"another {} starred it to keep, and {} not played it yet",
|
||||
"another {} kept it, and {} not played it yet",
|
||||
people(st),
|
||||
if u == 1 { "one person has".to_string() } else { format!("{u} have") }
|
||||
)),
|
||||
@@ -1140,9 +1224,20 @@ async fn fetch_now(
|
||||
|
||||
/// The same broadcast the socket clients read, as server-sent events.
|
||||
async fn events(State(state): State<WebState>) -> Sse<impl futures_util::Stream<Item = Result<SseEvent, std::convert::Infallible>>> {
|
||||
let stream = BroadcastStream::new(state.events.subscribe()).filter_map(|ev| async move {
|
||||
let ev = ev.ok()?;
|
||||
Some(Ok(SseEvent::default().data(serde_json::to_string(&ev).ok()?)))
|
||||
// A client that falls behind skips what it missed rather than being cut off.
|
||||
let stream = futures_util::stream::unfold(state.events.subscribe(), |mut rx| async move {
|
||||
loop {
|
||||
match rx.recv().await {
|
||||
Ok(ev) => {
|
||||
if let Ok(data) = serde_json::to_string(&ev) {
|
||||
let ev = Ok::<_, std::convert::Infallible>(SseEvent::default().data(data));
|
||||
return Some((ev, rx));
|
||||
}
|
||||
}
|
||||
Err(broadcast::error::RecvError::Lagged(_)) => {}
|
||||
Err(broadcast::error::RecvError::Closed) => return None,
|
||||
}
|
||||
}
|
||||
});
|
||||
Sse::new(stream).keep_alive(axum::response::sse::KeepAlive::default())
|
||||
}
|
||||
@@ -1197,6 +1292,18 @@ async fn read_all(
|
||||
Ok(Json(serde_json::json!({ "marked": n })))
|
||||
}
|
||||
|
||||
/// Everything read in every feed you subscribe to: exactly what All Subscriptions lists, since
|
||||
/// that view is scoped by the same subscriptions.
|
||||
async fn read_all_mine(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
let ids: Vec<String> =
|
||||
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
|
||||
let n = state.ctx.db.mark_all_read(user.id, &ids)?;
|
||||
Ok(Json(serde_json::json!({ "marked": n })))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct HowMany {
|
||||
#[serde(default = "five")]
|
||||
@@ -1285,6 +1392,9 @@ async fn import_opml(
|
||||
let doc = opml::OPML::from_str(&body.xml)
|
||||
.map_err(|e| ApiError::bad_request(format!("that is not an OPML file: {e}")))?;
|
||||
let (added, already) = crate::subscribe_opml(&state.ctx, &state.config_path, &doc, user.id)?;
|
||||
if added > 0 {
|
||||
scan_soon(&state, None).await;
|
||||
}
|
||||
Ok(Json(serde_json::json!({ "added": added, "already": already })))
|
||||
}
|
||||
|
||||
@@ -1338,8 +1448,6 @@ async fn patch_settings(
|
||||
)));
|
||||
}
|
||||
cfg.general.schedule = sched;
|
||||
// The legacy key would otherwise keep shadowing intent in the file.
|
||||
cfg.general.interval_mins = None;
|
||||
}
|
||||
if let Some(v) = body.max_new_per_check {
|
||||
cfg.general.max_new_per_check = v;
|
||||
|
||||
@@ -15,7 +15,7 @@ const script = html.split('<script>')[1].split('</script>')[0];
|
||||
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
|
||||
|
||||
const missing = [];
|
||||
const el = (name) => new Proxy({ style: {}, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
|
||||
const el = (name) => new Proxy({ style: { setProperty(){}, getPropertyValue(){ return ''; } }, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
|
||||
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
|
||||
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
|
||||
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
|
||||
@@ -46,7 +46,10 @@ const ctx = {
|
||||
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
|
||||
: String(url).includes('/api/users')
|
||||
? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }]
|
||||
: []),
|
||||
: /\/api\/(popular|directory)/.test(String(url))
|
||||
? [{ id: 'f', title: 'A Feed', image: null, subscribers: 2, subscribed: true },
|
||||
{ id: 'g', title: null, image: null, subscribers: 1, subscribed: false }]
|
||||
: /entries/.test(String(url)) ? { total: 0, entries: [] } : []),
|
||||
}),
|
||||
EventSource: function () { this.close = () => {}; },
|
||||
MediaMetadata: function () {},
|
||||
@@ -83,6 +86,9 @@ const drive = [
|
||||
['prefsModal', () => ctx.prefsModal()],
|
||||
['usersModal', () => ctx.usersModal()],
|
||||
['opmlModal', () => ctx.opmlModal()],
|
||||
['selectFeed (directory)', () => ctx.selectFeed(':directory')],
|
||||
['selectFeed (popular)', () => ctx.selectFeed(':popular')],
|
||||
['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
|
||||
['logsModal', () => ctx.logsModal()],
|
||||
// `const S` is not reachable from here: top-level const/let do not become properties
|
||||
// of a vm context the way var and function declarations do.
|
||||
|
||||
@@ -29,6 +29,18 @@ test('the theme toggle actually changes the theme', async ({ page }) => {
|
||||
.not.toBe(before);
|
||||
});
|
||||
|
||||
test('the theme button steps through dark, light and classic, and remembers', async ({ page }) => {
|
||||
const theme = () => page.evaluate(() => document.documentElement.dataset.theme);
|
||||
for (let i = 0; i < 3 && (await theme()) !== 'classic'; i++) await page.locator('#theme').click();
|
||||
expect(await theme()).toBe('classic');
|
||||
await expect(page.locator('#theme')).toHaveAttribute('title', /Classic.*Click for Dark/);
|
||||
|
||||
await page.reload();
|
||||
await expect.poll(theme).toBe('classic');
|
||||
// The 2004 Mac app set its type in Lucida Grande.
|
||||
expect(await page.evaluate(() => getComputedStyle(document.body).fontFamily)).toContain('Lucida Grande');
|
||||
});
|
||||
|
||||
test('settings opens and saves the global schedule', async ({ page }) => {
|
||||
await page.locator('#prefs').click();
|
||||
await expect(page.locator('#modal.on')).toBeVisible();
|
||||
@@ -69,20 +81,21 @@ test('the three panes are there and the item text lands in the bottom one', asyn
|
||||
|
||||
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
||||
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||
// The enclosure travels with the item, into the same pane.
|
||||
await expect(page.locator('#detail .encbox')).toHaveCount(1);
|
||||
// The enclosure goes to the Files pane beside the list, as the original's did.
|
||||
await expect(page.locator('#files')).toBeVisible();
|
||||
await expect(page.locator('#files .encbox')).toHaveCount(1);
|
||||
|
||||
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
|
||||
// its chip rather than assuming which episode the daemon happened to fetch.
|
||||
const downloaded = page.locator('.ep', { hasText: 'downloaded' }).first();
|
||||
const downloaded = page.locator('.ep', { has: page.locator('.kind.here') }).first();
|
||||
await downloaded.click();
|
||||
await expect(page.locator('#detail audio')).toBeVisible();
|
||||
await expect(page.locator('#detail .encbox .btn', { hasText: 'Save' })).toBeVisible();
|
||||
await expect(page.locator('#files [data-a="play"]')).toBeVisible();
|
||||
await expect(page.locator('#files .encbox [title="Save to this computer"]')).toBeVisible();
|
||||
|
||||
// Selecting another item replaces the pane rather than stacking.
|
||||
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
||||
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||
await expect(page.locator('#detail audio')).toHaveCount(0);
|
||||
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
||||
@@ -96,12 +109,12 @@ test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
||||
await row.click();
|
||||
|
||||
await expect(page.locator('#detail .dt')).toHaveText('An Article');
|
||||
await expect(page.locator('#detail audio')).toHaveCount(0);
|
||||
await expect(page.locator('#detail .encbox')).toContainText('image');
|
||||
await expect(page.locator('#detail .encbox')).toContainText('downloaded');
|
||||
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
|
||||
// What it is and that it is here: one icon, green, with the words in its tooltip.
|
||||
await expect(page.locator('#files .encbox .kind.here')).toHaveAttribute('title', 'image, downloaded');
|
||||
// Still offered as a file, just not as an episode: viewable and keepable.
|
||||
await expect(page.locator('#detail .btn', { hasText: 'Save' })).toBeVisible();
|
||||
const view = page.locator('#detail a', { hasText: 'View' });
|
||||
await expect(page.locator('#files [title="Save to this computer"]')).toBeVisible();
|
||||
const view = page.locator('#files a[title="View in a new tab"]');
|
||||
await expect(view).toHaveAttribute('target', '_blank');
|
||||
await expect(view).toHaveAttribute('rel', /noopener/);
|
||||
await expect(view).toHaveAttribute('href', /\/media\/\d+/);
|
||||
@@ -115,9 +128,9 @@ test('an item with several enclosures lists them all', async ({ page }) => {
|
||||
await expect(row).toContainText('+1 more file');
|
||||
|
||||
await row.click();
|
||||
// The pane below lists every one: the audio and the image.
|
||||
await expect(page.locator('#detail .encbox')).toHaveCount(2);
|
||||
await expect(page.locator('#detail .encbox').nth(1)).toContainText('image');
|
||||
// The Files pane lists every one: the audio and the image.
|
||||
await expect(page.locator('#files .encbox')).toHaveCount(2);
|
||||
await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
|
||||
});
|
||||
|
||||
test('the filter tabs change what is listed', async ({ page }) => {
|
||||
@@ -129,13 +142,13 @@ test('the filter tabs change what is listed', async ({ page }) => {
|
||||
await page.locator('.tabs button', { hasText: 'Unread' }).first().click();
|
||||
expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all);
|
||||
|
||||
await page.locator('.tabs button', { hasText: 'Flagged' }).first().click();
|
||||
await page.locator('.tabs button', { hasText: 'Kept' }).first().click();
|
||||
await expect(page.locator('#count')).toContainText('0 items');
|
||||
});
|
||||
|
||||
test('a feed URL is editable and has a copy button', async ({ page }) => {
|
||||
await page.getByText('Test Show').click();
|
||||
await page.locator('.btn', { hasText: 'Settings' }).first().click();
|
||||
await page.locator('#content .acts [data-a="settings"]').click();
|
||||
await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
|
||||
await expect(page.locator('#scopy')).toBeVisible();
|
||||
|
||||
@@ -174,9 +187,11 @@ test('an OPML subscription is a collapsible folder', async ({ page }) => {
|
||||
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
|
||||
}));
|
||||
|
||||
// Every row reserves the chevron slot for alignment; only a folder's is clickable.
|
||||
// Only a folder has a triangle, and it is a button that says whether the folder is open.
|
||||
const chev = page.locator('.feed.group .chev');
|
||||
await expect(chev).toBeVisible({ timeout: 20_000 });
|
||||
await expect(page.locator('.feed:not(.group) .chev')).toHaveCount(0);
|
||||
await expect(chev).toHaveAttribute('aria-expanded', 'false');
|
||||
|
||||
// Closed by default: the children are not listed until the folder is opened.
|
||||
const before = await page.locator('.feed').count();
|
||||
@@ -212,7 +227,7 @@ test('inside an OPML, feeds with unread items are listed first', async ({ page }
|
||||
await side.getByText('Grouped Show').click();
|
||||
const gs = page.locator('.ep', { hasText: 'Grouped Ep' });
|
||||
await gs.click();
|
||||
await page.locator('#detail button', { hasText: 'Mark unread' }).click();
|
||||
await page.locator('#detail button[title="Mark unread"]').click();
|
||||
await expect(gs).not.toHaveClass(/read/);
|
||||
|
||||
// Aardvark comes first alphabetically and in the OPML, so only the unread sort puts
|
||||
@@ -238,7 +253,7 @@ test('marking an OPML subscription read covers the feeds inside it', async ({ pa
|
||||
await expect(folder.locator('.badge')).not.toHaveText('0');
|
||||
await folder.click();
|
||||
|
||||
await page.locator('#content .acts button', { hasText: 'Mark all read' }).click();
|
||||
await page.locator('#content .acts [data-a="read"]').click();
|
||||
await expect(folder.locator('.badge')).toHaveText('0');
|
||||
});
|
||||
|
||||
@@ -280,9 +295,9 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
|
||||
// Another test may have opened this item already, so start from a known state: the
|
||||
// toggle in the text below flips it back -- which used to recurse until the stack blew.
|
||||
await row().click();
|
||||
await page.locator('#detail button', { hasText: 'Mark unread' }).click();
|
||||
await page.locator('#detail button[title="Mark unread"]').click();
|
||||
await expect(row()).not.toHaveClass(/read/);
|
||||
await expect(page.locator('#detail button', { hasText: 'Mark read' })).toBeVisible();
|
||||
await expect(page.locator('#detail button[title="Mark read"]')).toBeVisible();
|
||||
|
||||
// Opening it is reading it.
|
||||
await row().click();
|
||||
@@ -291,6 +306,30 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
|
||||
expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
test('the toolbar acts on the selected item', async ({ page }) => {
|
||||
await page.getByText('Test Show').click();
|
||||
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
|
||||
await expect(row()).toBeVisible({ timeout: 20_000 });
|
||||
// Nothing selected, nothing to act on.
|
||||
await expect(page.locator('#tbRead')).toBeDisabled();
|
||||
|
||||
await row().click(); // opening it reads it
|
||||
await expect(row()).toHaveClass(/read/);
|
||||
await page.locator('#tbRead').click();
|
||||
await expect(row()).not.toHaveClass(/read/);
|
||||
|
||||
await page.locator('#tbFlag').click();
|
||||
await expect(row().locator('.fl')).toHaveClass(/on/);
|
||||
await page.locator('#tbFlag').click(); // and back, so later tests see it unkept
|
||||
await expect(row().locator('.fl')).not.toHaveClass(/on/);
|
||||
|
||||
// Second Episode is the one the daemon downloaded, so it plays from the toolbar.
|
||||
await expect(page.locator('#tbPlay')).toBeEnabled();
|
||||
await page.locator('#tbPlay').click();
|
||||
await expect(page.locator('#player')).toBeVisible();
|
||||
await page.locator('#pclose').click();
|
||||
});
|
||||
|
||||
test('a second person has their own feeds and their own read state', async ({ browser }) => {
|
||||
const { execFileSync } = require('child_process');
|
||||
const setup = require('./global-setup');
|
||||
@@ -309,6 +348,10 @@ test('a second person has their own feeds and their own read state', async ({ br
|
||||
const ctx = await browser.newContext();
|
||||
const page = await ctx.newPage();
|
||||
await page.goto('/login');
|
||||
// The sign-in page shows the icon, so it has to load before anyone has signed in.
|
||||
const icon = await page.request.get('/icon.png');
|
||||
expect(icon.status()).toBe(200);
|
||||
expect(icon.headers()['content-type']).toBe('image/png');
|
||||
await page.locator('#name').fill('sam');
|
||||
await page.locator('#pw').fill('sampassword');
|
||||
await page.locator('button[type=submit]').click();
|
||||
@@ -350,8 +393,9 @@ test('deleting a shared file warns that it is everyone\'s copy', async ({ page }
|
||||
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||
await row.click();
|
||||
|
||||
const del = page.locator('#detail button', { hasText: 'Delete' });
|
||||
await expect(del).toHaveText('Delete for everyone');
|
||||
// An icon now; what it does, and to whom, is in its tooltip.
|
||||
const del = page.locator('#files button[data-a="del"]');
|
||||
await expect(del).toHaveAttribute('title', /^Delete for everyone \(shared with 1 other person/);
|
||||
|
||||
// Two prompts: the page's own, then the server's, because someone else has not played
|
||||
// it. Accept the first, decline the second, and the file must survive.
|
||||
@@ -417,7 +461,7 @@ test('Settings exports your OPML and imports a pasted one', async ({ page }) =>
|
||||
await page.locator('#prefs').click();
|
||||
const [dl] = await Promise.all([
|
||||
page.waitForEvent('download'),
|
||||
page.locator('#modalCard a', { hasText: 'Export OPML' }).click(),
|
||||
page.locator('#modalCard a[title="Export OPML"]').click(),
|
||||
]);
|
||||
expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml');
|
||||
const out = require('fs').readFileSync(await dl.path(), 'utf8');
|
||||
@@ -545,7 +589,7 @@ test('ipx import subscribes the admin, and ipx export writes the feeds out', asy
|
||||
expect(xml).toContain('http://127.0.0.1:8792/two.xml');
|
||||
});
|
||||
|
||||
test('Add feed offers what other people here read, but never a private feed', async ({ browser }) => {
|
||||
test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
|
||||
const { execFileSync } = require('child_process');
|
||||
const setup = require('./global-setup');
|
||||
const env = {
|
||||
@@ -566,7 +610,7 @@ test('Add feed offers what other people here read, but never a private feed', as
|
||||
await piper.locator('button[type=submit]').click();
|
||||
await expect(piper.locator('#feedlist')).toContainText('No feeds.');
|
||||
|
||||
await piper.locator('#addFeed').click();
|
||||
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||
const offered = piper.locator('#popular .childrow');
|
||||
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||
// An OPML's own feeds ride on the OPML, and a key in a URL marks someone's paid feed.
|
||||
@@ -579,9 +623,164 @@ test('Add feed offers what other people here read, but never a private feed', as
|
||||
expect(listed).not.toContain('.xml');
|
||||
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
|
||||
|
||||
await offered.filter({ hasText: 'Test Show' }).locator('button', { hasText: 'Subscribe' }).click();
|
||||
// Popular is the top ten of the directory, and the directory is every listed feed, A to Z.
|
||||
const dir = await (await piper.request.get('/api/directory')).json();
|
||||
const top = await (await piper.request.get('/api/popular')).json();
|
||||
const names = dir.map(p => (p.title || p.id).toLowerCase());
|
||||
expect(names).toEqual([...names].sort());
|
||||
expect(top.length).toBe(Math.min(10, dir.length));
|
||||
expect(top.every(t => dir.some(d => d.id === t.id))).toBe(true);
|
||||
expect(dir.map(p => p.id)).not.toContain('paid-show');
|
||||
|
||||
// Subscribe from the directory this time; the popular list shares the same rows.
|
||||
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||
await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
|
||||
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible();
|
||||
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||
|
||||
const row = async () =>
|
||||
(await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
|
||||
const before = await row();
|
||||
expect(before.subscribed).toBe(false);
|
||||
await offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
|
||||
await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||
// Once it is yours, it is no longer offered.
|
||||
expect(await (await piper.request.get('/api/popular')).text()).not.toContain('"test-show"');
|
||||
|
||||
// Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
|
||||
expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
|
||||
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||
await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
|
||||
await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
|
||||
|
||||
// All Subscriptions is every item from piper's feeds and only those: the admin's Picture
|
||||
// Blog is not among them.
|
||||
await piper.locator('#feedlist .place', { hasText: 'All Subscriptions' }).click();
|
||||
const first = piper.locator('.ep', { hasText: 'First Episode' });
|
||||
await expect(first).toBeVisible({ timeout: 20_000 });
|
||||
await expect(first.locator('.fd')).toHaveText('Test Show');
|
||||
await expect(piper.locator('.ep', { hasText: 'An Article' })).toHaveCount(0);
|
||||
await expect(piper.locator('#count')).toContainText('All Subscriptions:');
|
||||
await ctx.close();
|
||||
});
|
||||
|
||||
test('adding a feed scans it straight away', async ({ page }) => {
|
||||
await page.locator('#addFeed').click();
|
||||
await page.locator('#nurl').fill('http://127.0.0.1:8792/fresh.xml');
|
||||
await page.locator('#nsave').click();
|
||||
// Nobody pressed Scan. The scheduler's tick is a minute, so this is the add scanning it.
|
||||
await expect(page.locator('.ep', { hasText: 'Fresh Ep' })).toBeVisible({ timeout: 10_000 });
|
||||
});
|
||||
|
||||
test('a deleted file looks as if it was never downloaded', async ({ page }) => {
|
||||
// Other people subscribe to Picture Blog by now, so both prompts come; take them.
|
||||
page.on('dialog', d => d.accept());
|
||||
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
|
||||
const row = page.locator('.ep', { hasText: 'An Article' });
|
||||
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||
await row.click();
|
||||
await page.locator('#files button[data-a="del"]').click();
|
||||
|
||||
// No "reaped", no chip at all: just the way to get it again.
|
||||
await expect(row.locator('.kind.here')).toHaveCount(0);
|
||||
await expect(row).not.toContainText(/reaped/i);
|
||||
await row.click();
|
||||
await expect(page.locator('#files')).not.toContainText(/reaped/i);
|
||||
await expect(page.locator('#files button[title="Download to the server"]')).toBeVisible();
|
||||
});
|
||||
|
||||
test('one action, one icon: the toolbar, the page and every dialog agree', async ({ page }) => {
|
||||
const icon = loc => loc.locator('svg path').first().getAttribute('d');
|
||||
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
|
||||
|
||||
// Unsubscribe is a minus in the toolbar and the feed header, never the x that closes things.
|
||||
expect(await icon(page.locator('#content .acts [data-a="rm"]'))).toBe(await icon(page.locator('#tbRemove')));
|
||||
|
||||
// The toolbar's read and keep show the selected item's state, as its own buttons do, and follow
|
||||
// a change made from the toolbar.
|
||||
await page.locator('.ep').first().click();
|
||||
const pair = async a => [await icon(page.locator(a === 'read' ? '#tbRead' : '#tbFlag')),
|
||||
await icon(page.locator(`#detail [data-a="${a}"]`))];
|
||||
for (const a of ['read', 'flag']) { const [tb, own] = await pair(a); expect(tb).toBe(own); }
|
||||
const [kept] = await pair('flag');
|
||||
await page.locator('#tbFlag').click();
|
||||
await expect.poll(async () => { const [tb, own] = await pair('flag'); return tb === own && tb !== kept; }).toBe(true);
|
||||
await page.locator('#tbFlag').click(); // leave it as it was
|
||||
await expect.poll(async () => (await pair('flag'))[0]).toBe(kept);
|
||||
|
||||
// Every button in every dialog is an icon with its words in the tooltip.
|
||||
const dialogs = [
|
||||
() => page.locator('#addFeed').click(),
|
||||
() => page.locator('#prefs').click(),
|
||||
async () => { await page.locator('#prefs').click(); await page.locator('#gusers').click(); },
|
||||
async () => { await page.locator('#prefs').click(); await page.locator('#gopml').click(); },
|
||||
() => page.locator('#logs').click(),
|
||||
() => page.locator('#content .acts [data-a="settings"]').click(),
|
||||
() => page.locator('#content .acts [data-a="dl"]').click(),
|
||||
() => page.locator('#content .acts [data-a="rm"]').click(),
|
||||
];
|
||||
for (const open of dialogs) {
|
||||
await open();
|
||||
const btns = page.locator('#modalCard .btn');
|
||||
await expect(btns.first()).toBeVisible();
|
||||
for (const b of await btns.all()) {
|
||||
await expect(b.locator('svg')).toHaveCount(1);
|
||||
await expect(b).toHaveAttribute('title', /\S/);
|
||||
}
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(page.locator('#modal.on')).toBeHidden();
|
||||
}
|
||||
});
|
||||
|
||||
test('All Subscriptions marks everything read, across every feed', async ({ page }) => {
|
||||
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
|
||||
await all.click();
|
||||
// Earlier tests read things; make sure something is unread. Opening an item reads it, and
|
||||
// its own button makes it unread again.
|
||||
await page.locator('.ep').first().click();
|
||||
await page.locator('#detail [data-a="read"][title="Mark unread"]').click();
|
||||
await expect(all.locator('.badge')).not.toHaveText('0');
|
||||
|
||||
page.once('dialog', d => d.accept());
|
||||
await page.locator('#content .acts [data-a="readall"]').click();
|
||||
await expect(all.locator('.badge')).toHaveText('0');
|
||||
await page.locator('.tabs button', { hasText: 'Unread' }).click();
|
||||
await expect(page.locator('.ep')).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('the item table sorts by any column, both ways, and remembers', async ({ page }) => {
|
||||
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
|
||||
await all.click();
|
||||
const head = k => page.locator(`#list .ephead [data-sort="${k}"]`);
|
||||
const titles = () => page.locator('#eps .ep .t').allTextContents();
|
||||
// Byte order on lower case, which is what SQLite gives for lower(...).
|
||||
const cmp = (a, b) => (a.toLowerCase() < b.toLowerCase() ? -1 : a.toLowerCase() > b.toLowerCase() ? 1 : 0);
|
||||
const sorted = (t, dir) => JSON.stringify(t) === JSON.stringify([...t].sort((a, b) => cmp(a, b) * dir));
|
||||
await expect(page.locator('#eps .ep').nth(2)).toBeVisible({ timeout: 20_000 });
|
||||
expect(new Set(await titles()).size).toBeGreaterThan(2); // or both orders would prove nothing
|
||||
|
||||
await expect(head('title')).toHaveText('Title');
|
||||
await head('title').click();
|
||||
await expect.poll(async () => sorted(await titles(), 1)).toBe(true);
|
||||
await head('title').click();
|
||||
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
|
||||
|
||||
// Kept across a reload.
|
||||
await page.reload();
|
||||
await all.click();
|
||||
await expect(head('title').locator('.arr.desc')).toBeVisible();
|
||||
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
|
||||
|
||||
// Size has its own column; the file column is just what the file is.
|
||||
await expect(page.locator('#eps .ep .size', { hasText: /\d/ }).first()).toBeVisible();
|
||||
await expect(page.locator('#eps .ep .file', { hasText: /\d/ })).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('play in the Files pane plays once, in the player bar', async ({ page }) => {
|
||||
// Regression: the pane had an <audio> of its own, and playing it started the player bar too,
|
||||
// so the same file played twice at once.
|
||||
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
|
||||
await page.locator('.ep', { has: page.locator('.kind.here') }).first().click();
|
||||
await page.locator('#files [data-a="play"]').click();
|
||||
await expect(page.locator('#player')).toBeVisible();
|
||||
await expect(page.locator('audio')).toHaveCount(1); // the player bar's, and nothing else
|
||||
await page.locator('#pclose').click();
|
||||
});
|
||||
|
||||
5
tests/ui/fixtures/fresh.xml
Normal file
5
tests/ui/fixtures/fresh.xml
Normal file
@@ -0,0 +1,5 @@
|
||||
<?xml version="1.0"?>
|
||||
<rss version="2.0"><channel><title>Fresh Show</title><link>http://127.0.0.1:8792/</link>
|
||||
<description>Added in the browser suite, and scanned by adding it.</description>
|
||||
<item><title>Fresh Ep</title><guid>fresh-1</guid><description>x</description></item>
|
||||
</channel></rss>
|
||||
1034
web/index.html
1034
web/index.html
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user