Compare commits
26 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ae8123250b | |||
| 2c34a144ba | |||
| 669e8b5124 | |||
| 5362436766 | |||
| 0efc49519c | |||
| 57dcba2d1a | |||
| e95cccc66f | |||
| 7295be8b25 | |||
| f1b0d97b81 | |||
| ae47e31a97 | |||
| 8a309eb652 | |||
| 1a2b0d87c6 | |||
| d7fac2d0e7 | |||
| df9b7645d6 | |||
| f3825cfc57 | |||
| c0f4b0bcb2 | |||
| 8b8b48302b | |||
| 5d3fdde4da | |||
| 8784d0a3fd | |||
| 5e95557cbb | |||
| 6114add4a6 | |||
| c47c224372 | |||
| 686851b448 | |||
| f0d03c79c8 | |||
| 7df4ee7dde | |||
| d46ec73261 |
206
CHANGELOG.md
Normal file
206
CHANGELOG.md
Normal file
@@ -0,0 +1,206 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||||
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
The long form, with what was wrong before and how it was found, is in
|
||||||
|
[docs/history.md](docs/history.md).
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
## [0.4.0] - 2026-09-11
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- A Classic theme after the 2004 Mac app, beside Dark and Light: brushed-metal toolbar, Aqua
|
||||||
|
blue selection, red unread badges, a striped table and Lucida Grande. The theme button steps
|
||||||
|
through all three and remembers the choice.
|
||||||
|
- A toolbar across the top, after the original iPodderX: add and unsubscribe, play, mark read
|
||||||
|
and keep for the selected item, scan, a search box for what is showing, and Settings and Log.
|
||||||
|
- Directory, Popular and All Subscriptions at the top of the feed list, opening in the main pane.
|
||||||
|
Directory lists every feed anyone here subscribes to, A to Z (`GET /api/directory`). All
|
||||||
|
Subscriptions lists every item from every feed you subscribe to (`GET /api/entries`).
|
||||||
|
- Items show as a table (unread, kept, title, feed, file, size, published) with a Files pane beside
|
||||||
|
it, and a status bar with the totals.
|
||||||
|
- Mark everything read from All Subscriptions, across every feed you subscribe to
|
||||||
|
(`POST /api/read-all`). It asks first. All Subscriptions can also check every feed from its header.
|
||||||
|
- Click a column heading in the item table to sort by it (kept, title, feed, file type, size,
|
||||||
|
published); click again to reverse. The server sorts, so it covers the whole list, not just the
|
||||||
|
fifty shown, and the choice is remembered.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Popular shows the top 10, not 20, and counts everyone, you included. Your own feeds stay on it,
|
||||||
|
marked Subscribed, and clicking one opens it.
|
||||||
|
- Adding a feed scans it straight away, and an OPML import that added feeds scans them, so their
|
||||||
|
items show without pressing Scan.
|
||||||
|
- A file deleted to save space, or by hand, looks as if it was never downloaded: no "reaped"
|
||||||
|
label, just the Download button. The retention summary says "deleted", not "reaped".
|
||||||
|
- Buttons are icons, with the words in their tooltips: the Files pane (save, delete, view,
|
||||||
|
download), an item's own buttons (mark read, keep, open the original), the feed header (scan,
|
||||||
|
download latest, mark all read, settings, unsubscribe), and the Settings, feed settings and
|
||||||
|
Download latest dialogs (save, download, cancel). The icons are Font Awesome Free, embedded as
|
||||||
|
SVG: only the ones used, no font to download, and nothing fetched from anyone else. They
|
||||||
|
replace font characters such as ⟳ ⤓ ↗, which came out thin and tiny and differed from font to
|
||||||
|
font. Keep is a flag everywhere, as it was in the original, and mark unread is an envelope.
|
||||||
|
- One meaning per icon. Minus unsubscribes, x closes or cancels, plus adds or subscribes, and a
|
||||||
|
dialog's confirm button carries the icon of what it does. The feed header's unsubscribe was an x
|
||||||
|
and read as closing the page. The remaining word buttons are icons too:
|
||||||
|
- Log, Add feed, Users, Unsubscribe and OPML.
|
||||||
|
- Popular's Subscribe, Copy and Sign out.
|
||||||
|
- The Subscribed label in Popular, the Directory and Add feed, which is now a green check.
|
||||||
|
- The player's back, play, forward and close, which were font characters, and the folder arrow.
|
||||||
|
- The toolbar's read and keep buttons show the selected item's state, with the same icons as the
|
||||||
|
item's own buttons. Play, read and keep sit together, and Scan sits with add and unsubscribe.
|
||||||
|
- An OPML subscription's page has the same header as a feed's, with its buttons in the same places.
|
||||||
|
- The item table's size has its own column, apart from the file's type, and shows KB for small
|
||||||
|
files instead of "0 MB". The Item heading is now Title.
|
||||||
|
- A file's type is an icon (audio, video, image, PDF, torrent, other), green once it is
|
||||||
|
downloaded and red when the download failed, with the details in its tooltip. One icon per
|
||||||
|
row keeps the column lined up. The DOWNLOADED and PENDING labels are gone.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Playing a file from the Files pane played it twice at once, in the pane and in the player bar.
|
||||||
|
The pane has a play button now, and the player bar is the only player.
|
||||||
|
- The password box in Manage users was white in the dark theme.
|
||||||
|
- An item with no date showed a stray dot in its details.
|
||||||
|
- Escape did not close a dialog while the cursor was in one of its boxes, so Add feed, which opens
|
||||||
|
in its URL box, could not be closed with Escape.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- Feeds from paid-feed services (Patreon, Supercast, Supporting Cast, Glow, Memberful) are never
|
||||||
|
listed in Popular or the Directory. A Supercast feed, which keeps its key in the URL's path
|
||||||
|
rather than the query, was being listed.
|
||||||
|
|
||||||
|
## [0.3.0] - 2026-09-11
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Add feed lists what other people on this server subscribe to, most subscribers first, and
|
||||||
|
subscribes you by id (`GET /api/popular`, `POST /api/popular/{id}`). Feeds from an OPML, and
|
||||||
|
feeds with a login or a key in their URL, are never listed.
|
||||||
|
- Upload an OPML file to import, beside the paste box. The page checks it looks like OPML before
|
||||||
|
sending it and clears the picker afterwards.
|
||||||
|
- Settings → Manage users: add and remove accounts, and choose who is an admin
|
||||||
|
(`GET`/`POST /api/users`, `PATCH`/`DELETE /api/users/{id}`).
|
||||||
|
- Per-user subscriptions, and per-user read, starred and playback state. The existing library is
|
||||||
|
adopted by the admin on first start.
|
||||||
|
- Subscribing to a feed someone else already has costs no second fetch and no second copy. Scanning
|
||||||
|
merges every subscriber's wants.
|
||||||
|
- Delete on a shared feed reads **Delete for everyone**, and the server answers `409` while anyone
|
||||||
|
else has starred the item or not played it (`?force=true` overrides).
|
||||||
|
- A shared feed's header says how many other people read it.
|
||||||
|
- `docs/` for configuration, the CLI, users, SSO and architecture, and `CLAUDE.md` for anyone
|
||||||
|
working on the code.
|
||||||
|
- Browser tests for OPML import and export by every route, user admin, unread ordering, and
|
||||||
|
`ipx import`/`ipx export`.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Feeds inside an OPML subscription list the ones with unread items first.
|
||||||
|
- OPML import subscribes you to every feed in the file. `ipx import` subscribes the first admin.
|
||||||
|
- OPML export lists only your own subscriptions.
|
||||||
|
- Production runs as a Docker image pushed to `192.168.1.130:5000` and recreated with
|
||||||
|
`docker compose`.
|
||||||
|
- This changelog follows Keep a Changelog. The long-form entries moved to `docs/history.md`.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Importing another account's OPML export subscribed nobody and reported "Imported 0 feed(s)".
|
||||||
|
Feeds it added had no subscriber, so they were never scanned.
|
||||||
|
- Importing something that is not OPML answered `500`. It is now `400` "that is not an OPML file",
|
||||||
|
refused before anything changes.
|
||||||
|
- Starring stopped protecting a file from the quota and age sweeps once read state became per-user.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- The log is admin-only (`GET /api/logs` answers `403`, and the Log button is hidden). It names
|
||||||
|
every account, every feed and every failed sign-in.
|
||||||
|
- OPML export no longer hands anyone signed in the whole catalogue, including other people's
|
||||||
|
private feed URLs.
|
||||||
|
|
||||||
|
## [0.2.0] - 2026-09-10
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Web UI served by the daemon: plain HTML and JS compiled into the binary, with feeds, items,
|
||||||
|
filters, search, sanitised show notes and live progress over SSE.
|
||||||
|
- Player bar with resume, speed, keyboard shortcuts and lock-screen controls.
|
||||||
|
- Three-pane layout: feeds beside, items above, and the selected item's text and files below.
|
||||||
|
- Phone layout.
|
||||||
|
- Accounts and sign-in: Argon2id passwords, session cookies, `ipx user add|list|passwd|rm`, and a
|
||||||
|
trusted proxy header for Cloudflare Zero Trust or Authentik (`docs/sso.md`).
|
||||||
|
- Subscribing to an OPML: it is re-read every scan and its feeds show as a folder. A feed dropped
|
||||||
|
from it is removed unless it has downloads.
|
||||||
|
- Scheduling: a global interval with per-feed overrides (`every 30m`, `4h`, `1d`, `2w`).
|
||||||
|
- `[general] media_types`, default audio and video, and `max_new_per_check`, default 3.
|
||||||
|
- Every enclosure of an item, a View link for files that are not audio or video, and per-item
|
||||||
|
artwork.
|
||||||
|
- In-app log view with Daemon I/O, Scans and HTTP tabs.
|
||||||
|
- Mark all read on an OPML subscription.
|
||||||
|
- Editable feed URL with a copy button.
|
||||||
|
- Docker image whose healthcheck goes through the control socket.
|
||||||
|
- The iPodderX name, icon, and a colour scheme taken from the icon.
|
||||||
|
- `tests/page-smoke.js` and a Playwright browser suite.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Global settings and scan schedules are admin-only. The per-feed schedule picker is gone.
|
||||||
|
- Feeds from an OPML live in the database, not `config.toml`.
|
||||||
|
- Opening an item marks it read. Playing it marks it read only at the end or past 90%.
|
||||||
|
- "Episodes" became "items", since half the library is text.
|
||||||
|
- A burst of scan events causes one refresh, not one per feed.
|
||||||
|
- All is the default filter. OPML import and export, Settings and Log moved out of the header.
|
||||||
|
- Torrents run detached, two at a time.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Download fetched the next queued episodes instead of the one clicked.
|
||||||
|
- Pressing play made an item vanish from the Unread list.
|
||||||
|
- Clearing a folder, schedule or cap from the UI did nothing.
|
||||||
|
- The daemon ignored SIGTERM until the current download finished.
|
||||||
|
- A missing function stopped the page script and left the whole UI dead.
|
||||||
|
- One download painted progress on every pending row.
|
||||||
|
- A torrent could freeze scanning for up to an hour.
|
||||||
|
- Downloading from an OPML feed failed with "belongs to unsubscribed feed".
|
||||||
|
- Image enclosures were downloaded, counted as episodes and given a play button.
|
||||||
|
- A feed whose entries had been deleted stayed empty, because the server kept answering `304`.
|
||||||
|
- An item with several enclosures kept only the last.
|
||||||
|
- OPML folders rendered open by default.
|
||||||
|
- Mark read in the text pane recursed until the stack overflowed.
|
||||||
|
- The Unread, Downloaded and Flagged filters answered `500` without a search term.
|
||||||
|
- An OPML subscription always showed 0 unread.
|
||||||
|
- Folder names kept doubled spaces where separators were stripped.
|
||||||
|
- Sidebar rows had four different left edges.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- The web UI needs a token or a sign-in. The token is compared in constant time, and an empty
|
||||||
|
token refuses to serve.
|
||||||
|
- Show notes are sanitised with `ammonia`.
|
||||||
|
- A proxy's user header is honoured only from an address in `trusted_proxies`.
|
||||||
|
- A feed URL must be http(s), so `file:///etc/passwd` is refused.
|
||||||
|
- Download folders are sanitised per path segment, so `../../etc/Show` cannot climb out.
|
||||||
|
|
||||||
|
## [0.1.0] - 2026-09-09
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- `ipx`, a Rust rewrite of the iPodderX engine: TOML config, SQLite state, and `ipx list`, `add`,
|
||||||
|
`rm` and `fetch`.
|
||||||
|
- RSS and Atom parsing with conditional GET, `<ttl>` and basic auth.
|
||||||
|
- Streaming downloads with explicit, keyword and per-scan filters, deduplicated by enclosure URL.
|
||||||
|
- Quota and age retention that never touches a starred file, and `ipx reap [--dry-run]`.
|
||||||
|
- `ipx daemon` with a JSON-lines Unix socket. CLI commands proxy to a running daemon.
|
||||||
|
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
|
||||||
|
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
|
||||||
|
|
||||||
|
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.4.0...main
|
||||||
|
[0.4.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...v0.4.0
|
||||||
|
[0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0
|
||||||
|
[0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0
|
||||||
|
[0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0
|
||||||
144
CLAUDE.md
Normal file
144
CLAUDE.md
Normal file
@@ -0,0 +1,144 @@
|
|||||||
|
# Working on ipodderx-rs
|
||||||
|
|
||||||
|
Notes for whoever picks this up next. Read [docs/architecture.md](docs/architecture.md) for how the
|
||||||
|
thing is built; this file is about working on it without repeating mistakes that have already been
|
||||||
|
made here.
|
||||||
|
|
||||||
|
## Where things are
|
||||||
|
|
||||||
|
Production is the `iPodderX` container on Tower (192.168.1.130), the `ipodderx` service of the
|
||||||
|
Arcane project `content`: `/mnt/fast/arcane/projects/content/compose.yaml`. That file is what runs;
|
||||||
|
`docker-compose.yml` in this repo is a copy, and editing it changes nothing in production.
|
||||||
|
|
||||||
|
| | Host | In the container |
|
||||||
|
|---|---|---|
|
||||||
|
| Image | `192.168.1.130:5000/ipodderx:latest` | |
|
||||||
|
| Config | `/mnt/fast/appdata/ipodderx/config.toml` | `/config/config.toml` |
|
||||||
|
| Database | `/mnt/user/ipodderx/state.db` | `/data/state.db` |
|
||||||
|
| Downloads | `/mnt/user/ipodderx/downloads` | `/downloads` |
|
||||||
|
| Web UI | `192.168.1.130:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel | `0.0.0.0:8099` |
|
||||||
|
|
||||||
|
Deploying a change is: build and push the image, then pull it and recreate the container.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
|
||||||
|
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml pull ipodderx
|
||||||
|
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml up -d ipodderx
|
||||||
|
docker logs --tail 20 iPodderX
|
||||||
|
```
|
||||||
|
|
||||||
|
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
|
||||||
|
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
|
||||||
|
|
||||||
|
**A build that fails with `429 Too Many Requests` on a base image** is Docker Hub rate-limiting
|
||||||
|
this host. There is no Docker Hub login here, and the build asks about `debian:bookworm-slim` and
|
||||||
|
`rust:1-slim-bookworm` every time unless they are already stored locally. Pull them from Google's
|
||||||
|
mirror and tag them; the build then uses the local copies without asking Docker Hub:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker pull mirror.gcr.io/library/debian:bookworm-slim
|
||||||
|
docker tag mirror.gcr.io/library/debian:bookworm-slim debian:bookworm-slim
|
||||||
|
docker pull mirror.gcr.io/library/rust:1-slim-bookworm
|
||||||
|
docker tag mirror.gcr.io/library/rust:1-slim-bookworm rust:1-slim-bookworm
|
||||||
|
```
|
||||||
|
|
||||||
|
Run those again now and then, or the local copies go stale.
|
||||||
|
|
||||||
|
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
|
||||||
|
the worker is alive, not just the web port. The container restarts on its own after a reboot.
|
||||||
|
|
||||||
|
Before the container, ipx ran by hand in code-server, with its files in `/config/.config/ipx/` and
|
||||||
|
`/config/.local/share/ipx/`. Those are still there and the container does not read them. If you run
|
||||||
|
a daemon by hand for testing, stop it with **`pkill -x ipx`, never `pkill -f ipx`**. `-f` matches
|
||||||
|
the shell running the command and kills the session (exit 144). This has happened more than once.
|
||||||
|
|
||||||
|
## Before you touch the page
|
||||||
|
|
||||||
|
`web/index.html` is `include_str!`d into the binary, so **every page change needs a rebuild** before
|
||||||
|
it is visible. It is one file: markup, CSS and script.
|
||||||
|
|
||||||
|
After any edit to it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
node tests/page-smoke.js
|
||||||
|
```
|
||||||
|
|
||||||
|
That loads the script against a stub DOM and checks every selector it wires at load actually
|
||||||
|
exists. It exists because a patch once anchored on a deleted function, `String.replace` silently
|
||||||
|
matched nothing, and the whole UI died with a `ReferenceError` while every server-side test passed.
|
||||||
|
|
||||||
|
Patching that file by guessing an anchor string has failed repeatedly. Read the exact block first
|
||||||
|
(`sed -n 'START,ENDp'`), match it verbatim, and assert the replacement happened rather than hoping.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test # ~51 tests: parsing, filters, retention, schedules, SQL, per-user state
|
||||||
|
node tests/page-smoke.js
|
||||||
|
npx playwright test # 16 browser tests against a real daemon on fixture feeds
|
||||||
|
```
|
||||||
|
|
||||||
|
Things about the browser suite that have cost time:
|
||||||
|
|
||||||
|
* It starts **its own daemon and database** under `/tmp/ipx-ui-test`, wiped once per run. Playwright
|
||||||
|
re-imports the config in every worker, so `prepare()` guards on `TEST_WORKER_INDEX` — without
|
||||||
|
that guard a worker deleted the database out from under the running daemon, which then kept
|
||||||
|
serving from the unlinked inode while everything else saw an empty file.
|
||||||
|
* Tests **share that daemon and run in order**. A test that opens an item marks it read and changes
|
||||||
|
what later tests see. Write assertions that do not depend on what ran before, or normalise the
|
||||||
|
state first.
|
||||||
|
* Fixture feeds must not share an enclosure URL, because `enclosures.url` is globally unique and
|
||||||
|
whichever feed is scanned first claims it.
|
||||||
|
* `webServer` starts **before** `globalSetup`, which is why the fixture config is written at
|
||||||
|
config-load time instead.
|
||||||
|
|
||||||
|
Non-trivial logic leaves one runnable check behind. Pure functions (`merge_policy`, `pick`,
|
||||||
|
`matches_keywords`, `parse_interval`) are the easiest place to put it.
|
||||||
|
|
||||||
|
## Things that are true and easy to get wrong
|
||||||
|
|
||||||
|
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
|
||||||
|
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
|
||||||
|
* **`entries.read`, `entries.flagged` and `entries.position` are dead columns.** Read state lives in
|
||||||
|
`entry_state` per user. Two bugs have already come from queries still reading the old ones
|
||||||
|
(retention, and the entry pruner) — grep before adding a third.
|
||||||
|
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
|
||||||
|
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
|
||||||
|
derived and never written to config.
|
||||||
|
* **One fetch serves everyone**, so scan policy is a union of subscribers' wants (`merge_policy`).
|
||||||
|
Anyone wanting an item is enough to fetch it.
|
||||||
|
* **The UI hiding a control is not enforcement.** Admin-only actions check `user.is_admin` in the
|
||||||
|
handler and return `403`.
|
||||||
|
* **A `tokio::select!` only races its branches at the point of selection.** A long download has to
|
||||||
|
watch the shutdown channel itself; the daemon ignored SIGTERM for exactly this reason.
|
||||||
|
* Only one daemon per socket. Removing the socket file defeats the guard and you get two daemons
|
||||||
|
fighting over the database, with the stale one still holding the port.
|
||||||
|
* `/api/settings` answering `200` does **not** mean the worker is alive — it is a different task.
|
||||||
|
Probe the control socket (`ipx status`) to check that.
|
||||||
|
|
||||||
|
## House style
|
||||||
|
|
||||||
|
Comments explain **why**, not what. If a line looks odd, the comment says what went wrong without
|
||||||
|
it. No emoji, no exclamation marks, no "obviously". Prose in the UI and docs is plain English and
|
||||||
|
addressed to the person using it.
|
||||||
|
|
||||||
|
Every change gets one line under `## [Unreleased]` in [CHANGELOG.md](CHANGELOG.md), in its
|
||||||
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) group: Added, Changed, Deprecated,
|
||||||
|
Removed, Fixed or Security. Say it the way someone using ipx would notice it. When there is more to
|
||||||
|
say, such as what was wrong before or what it cost to find out, write it up at the top of
|
||||||
|
[docs/history.md](docs/history.md), dated. That record has been more useful than the git log more
|
||||||
|
than once.
|
||||||
|
|
||||||
|
Cutting a release: rename `[Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD` and open a new empty
|
||||||
|
`[Unreleased]` above it, bump `version` in `Cargo.toml`, tag the commit `vX.Y.Z`, and update the
|
||||||
|
compare links at the bottom of the changelog.
|
||||||
|
|
||||||
|
Deliberate simplifications get a `ponytail:` comment naming the ceiling and the upgrade path, e.g.
|
||||||
|
`// ponytail: global connection mutex, move to a pool if feed count makes it contend`.
|
||||||
|
|
||||||
|
## Known gaps
|
||||||
|
|
||||||
|
* Cloudflare's `Cf-Access-Jwt-Assertion` is not verified — ipx trusts the hop plus `trusted_proxies`
|
||||||
|
(documented in [docs/sso.md](docs/sso.md)).
|
||||||
|
* A feed's `<description>` subtitle is dropped whenever `content:encoded` exists, which loses
|
||||||
|
Substack-style subtitles.
|
||||||
2
Cargo.lock
generated
2
Cargo.lock
generated
@@ -1634,7 +1634,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "ipx"
|
name = "ipx"
|
||||||
version = "0.1.0"
|
version = "0.4.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"ammonia",
|
"ammonia",
|
||||||
"anyhow",
|
"anyhow",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "ipx"
|
name = "ipx"
|
||||||
version = "0.1.0"
|
version = "0.4.0"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
|
|||||||
265
README.md
265
README.md
@@ -1,14 +1,14 @@
|
|||||||
# ipodderx-rs
|
# ipodderx-rs
|
||||||
|
|
||||||
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent),
|
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent), files them
|
||||||
files them into per-feed folders, and reaps old episodes to stay under a disk quota.
|
into per-feed folders, and reaps old files to stay under a disk quota. Runs as a one-shot CLI or as
|
||||||
Runs as a one-shot CLI or as a daemon with a Unix-socket JSON event stream for a UI to attach to.
|
a daemon with a web UI, serving any number of people from one copy of the data.
|
||||||
|
|
||||||
## Lineage
|
## Lineage
|
||||||
|
|
||||||
This is a modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the
|
A modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python 2
|
||||||
Python 2 engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), which was
|
engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), open-sourced under the MIT
|
||||||
open-sourced under the MIT License in 2010.
|
License in 2010.
|
||||||
|
|
||||||
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
|
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
|
||||||
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
|
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
|
||||||
@@ -16,223 +16,108 @@ oldest-first disk quota reaper.
|
|||||||
|
|
||||||
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
|
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
|
||||||
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
|
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
|
||||||
3DES-encrypted preferences, and the `printMSG` stdout protocol (replaced by a JSON-lines socket).
|
3DES-encrypted preferences, and the `printMSG` stdout protocol -- replaced by a JSON-lines socket.
|
||||||
|
|
||||||
## Quickstart
|
## Quick start
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cargo install --path .
|
cargo build --release
|
||||||
|
install -m755 target/release/ipx ~/.cargo/bin/
|
||||||
|
|
||||||
ipx add https://atp.fm/rss # names the feed from its own title
|
ipx add https://atp.fm/rss # subscribe
|
||||||
ipx list
|
ipx fetch # scan and download
|
||||||
ipx fetch # scan now
|
ipx daemon # scheduler, control socket and web UI
|
||||||
ipx daemon # or run continuously, honouring each feed's <ttl>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Config lives at `~/.config/ipx/config.toml` (mode 0600, since it may hold feed passwords);
|
On first start with `[web] enabled = true` the daemon mints a token, writes it to config.toml and
|
||||||
state at `~/.local/share/ipx/state.db`. Override with `IPX_CONFIG` and `IPX_DATA_DIR`.
|
prints the URL to open. A database with no accounts starts with **admin / ipodderx** at `/login` --
|
||||||
Set `IPX_LOG=ipx=debug` for verbose logging on stderr.
|
change it with `echo -n '<password>' | ipx user passwd admin`.
|
||||||
|
|
||||||
## Commands
|
## Documentation
|
||||||
|
|
||||||
| command | what it does |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `ipx add <url> [--folder X] [--keywords a,b]` | subscribe; the id comes from the feed title |
|
| [docs/configuration.md](docs/configuration.md) | Every config key, paths, environment variables |
|
||||||
| `ipx rm <feed>` | unsubscribe; downloads and history are kept |
|
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
|
||||||
| `ipx list` / `ipx status` | subscriptions and their state |
|
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
|
||||||
| `ipx fetch [FEED] [--force]` | scan; `--force` ignores the TTL |
|
| [docs/sso.md](docs/sso.md) | Cloudflare Zero Trust or Authentik in front |
|
||||||
| `ipx reap [--dry-run]` | run retention now |
|
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, socket, HTTP API |
|
||||||
| `ipx import/export <file.opml>` | move subscriptions in or out |
|
| [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
|
||||||
| `ipx daemon` | scheduler plus the control socket |
|
| [docs/history.md](docs/history.md) | How it was built: the long form, with what was wrong and why |
|
||||||
|
| [CLAUDE.md](CLAUDE.md) | Notes for anyone (or anything) working on the code |
|
||||||
|
|
||||||
Any command with a wire form probes the socket first: if a daemon is running it does the work,
|
## The web UI
|
||||||
and the CLI just renders the events it streams back. `--local` forces in-process execution.
|
|
||||||
|
|
||||||
## Configuration
|
`ipx daemon` serves it in the same process, so it reads SQLite and the event bus directly.
|
||||||
|
|
||||||
```toml
|
Feeds down the side; the selected feed's items across the top; the selected item's text and its
|
||||||
[general]
|
enclosures below, which is where you play, download or delete them. The divider drags and its
|
||||||
download_dir = "~/Podcasts"
|
position is remembered. Playback serves Range requests, so seeking works. An OPML subscription is a
|
||||||
socket = "/run/user/1000/ipx.sock" # default: $XDG_RUNTIME_DIR/ipx.sock
|
collapsible folder whose page lists the feeds inside it.
|
||||||
interval_mins = 60 # default poll; a feed's own <ttl> wins when longer
|
|
||||||
organize = "feed" # "feed" | "date"
|
|
||||||
max_total_gb = 50 # 0 = unlimited
|
|
||||||
max_age_days = 30 # 0 = keep forever
|
|
||||||
max_new_per_check = 3 # per feed, per scan. 0 = unlimited (pulls whole back catalogues)
|
|
||||||
media_types = ["audio", "video"] # what downloads automatically. Anything else is still
|
|
||||||
# listed and can be fetched by hand -- blog feeds put article
|
|
||||||
# images in enclosures. Empty takes everything.
|
|
||||||
|
|
||||||
[torrent]
|
An item may carry several enclosures; all of them appear below, and anything that is not audio or
|
||||||
enabled = true
|
video gets a View link rather than a player -- the publisher's copy until it is downloaded, the
|
||||||
seed_ratio = 1.0 # stop seeding at this ratio ...
|
local one after. Opening an item marks it read. Show notes are untrusted feed HTML, sanitized with
|
||||||
seed_time_mins = 60 # ... or after this long, whichever comes first
|
`ammonia` server-side before they reach the page.
|
||||||
port_range = "6881-6889"
|
|
||||||
stall_mins = 30 # give up on a torrent making no progress
|
|
||||||
|
|
||||||
[feeds.atp]
|
The **Log** button shows the running daemon live in four tabs: *Daemon I/O* is the control protocol
|
||||||
url = "https://atp.fm/rss"
|
itself, every command in and event out; *Scans* is feed and download activity; *HTTP* is web
|
||||||
folder = "Accidental Tech Podcast" # default: the feed title
|
requests; *All* is everything, with level and text filters and a copy button. It reads a ring buffer
|
||||||
keywords = ["deep dive"] # OR across keywords, AND within one
|
held in the process, not a file, so it works the same under Docker.
|
||||||
allow_explicit = false
|
|
||||||
auto_download = true
|
|
||||||
max_new_per_check = 3 # the rest wait for the next scan
|
|
||||||
username = "ray" # optional HTTP basic auth
|
|
||||||
password_env = "IPX_ATP_PASS" # or a literal `password`
|
|
||||||
```
|
|
||||||
|
|
||||||
Retention keeps files that are `flagged` in the database, and deletes read episodes before unread
|
It is plain HTTP. On a LAN bind the token and everything else cross the network in the clear, and a
|
||||||
ones, oldest first.
|
feed URL can itself carry a credential. Put TLS in front of it if that matters.
|
||||||
|
|
||||||
## Socket protocol
|
|
||||||
|
|
||||||
Newline-delimited JSON over a Unix socket, both directions.
|
|
||||||
|
|
||||||
```sh
|
|
||||||
$ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
|
||||||
{"ev":"feed_start","feed":"atp"}
|
|
||||||
{"ev":"progress","feed":"atp","url":"...","file":"ep1.mp3","done":8192,"total":3000000}
|
|
||||||
{"ev":"download_done","feed":"atp","url":"...","path":"...","bytes":3000000}
|
|
||||||
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
|
|
||||||
{"ev":"scan_done","feeds":1}
|
|
||||||
```
|
|
||||||
|
|
||||||
Commands: `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `status`.
|
|
||||||
Events: `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
|
||||||
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
|
||||||
`scan_done`, `reap_done` and `status` are terminal -- a client that asked for work stops there.
|
|
||||||
|
|
||||||
Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy
|
|
||||||
daemon also sees that daemon's other work.
|
|
||||||
|
|
||||||
## OPML
|
## OPML
|
||||||
|
|
||||||
Two different things, both supported:
|
**Importing and exporting** a file copies subscriptions in or out once: `ipx import subs.opml`,
|
||||||
|
`ipx export subs.opml`, or Settings → Subscriptions in the UI.
|
||||||
**Importing and exporting** a file copies subscriptions in or out once — `ipx import subs.opml`,
|
|
||||||
`ipx export subs.opml`, or the OPML button in the UI.
|
|
||||||
|
|
||||||
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
|
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
|
||||||
other feed; every scan re-reads it and keeps your feed list in step. An OPML is recognised by its
|
other feed; every scan re-reads it and keeps your list in step. The feeds inside are not written to
|
||||||
content, so a URL without a `.opml` extension still works.
|
config.toml -- the OPML is the source of truth, so they are re-derived each scan and held in the
|
||||||
|
database. They show as a folder, download into one nested folder, and inherit the subscription's
|
||||||
|
settings until you change one, which gives it its own entry.
|
||||||
|
|
||||||
The feeds inside it are **not written to `config.toml`** — the OPML is the source of truth, so they
|
When a feed drops out of the OPML upstream, it is unsubscribed and removed -- unless it has
|
||||||
are re-derived each scan and held in the database. Your config keeps only what you chose. They show
|
downloads, in which case it is kept and flagged in the UI as no longer listed. A downloaded file is
|
||||||
as a collapsible folder in the sidebar, download into one nested folder, and newly listed ones are
|
never left behind with nothing explaining where it came from.
|
||||||
scanned in the same run rather than waiting for the next interval. They inherit the subscription's
|
|
||||||
settings; change anything on one and it gets its own config entry from then on.
|
|
||||||
|
|
||||||
When a feed drops out of the OPML upstream:
|
|
||||||
|
|
||||||
| it has downloads | what happens |
|
|
||||||
|---|---|
|
|
||||||
| no | unsubscribed and removed from the config |
|
|
||||||
| yes | kept, flagged in the UI as no longer listed |
|
|
||||||
|
|
||||||
A downloaded file is never left behind with nothing explaining where it came from.
|
|
||||||
|
|
||||||
## Web UI
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[web]
|
|
||||||
enabled = true
|
|
||||||
bind = "0.0.0.0:8080" # 127.0.0.1:8080 by default
|
|
||||||
token = "" # generated and written back on first run
|
|
||||||
```
|
|
||||||
|
|
||||||
`ipx daemon` then serves it in the same process (`ipx daemon --web ADDR` overrides the bind for one
|
|
||||||
run). On first start it mints a token, saves it to config.toml, and prints the URL to open:
|
|
||||||
|
|
||||||
```
|
|
||||||
web ui token generated. Open:
|
|
||||||
http://0.0.0.0:8080/?token=1f4c…
|
|
||||||
```
|
|
||||||
|
|
||||||
`?token=` sets a year-long cookie, so you only paste it once per browser. Everything is behind that
|
|
||||||
token, including `/media/...` — a cookie rather than a header precisely because an `<audio>` element
|
|
||||||
cannot send headers.
|
|
||||||
|
|
||||||
Browse feeds, read show notes, play episodes in the browser (Range requests are served, so seeking
|
|
||||||
works), download or delete individual files, mark episodes read or flag them to keep, and edit a
|
|
||||||
feed's folder/keywords/explicit/auto-download/limit settings. Config edits are written to
|
|
||||||
config.toml and hot-reloaded — no daemon restart.
|
|
||||||
|
|
||||||
Show notes are feed-supplied HTML from an untrusted source; they are sanitized with `ammonia`
|
|
||||||
server-side before they reach the page.
|
|
||||||
|
|
||||||
**It is plain HTTP.** On a LAN bind, the token and everything else crosses the network in the
|
|
||||||
clear — and a feed URL can itself contain a credential (Patreon's, for one, carries an auth token).
|
|
||||||
Put it behind a reverse proxy with TLS if that matters to you.
|
|
||||||
|
|
||||||
## Layout
|
|
||||||
|
|
||||||
Feeds down the side; the selected feed's episodes across the top; the selected episode's text and
|
|
||||||
its enclosures in the pane below, which is where you play, download or delete them. The divider
|
|
||||||
between the two panes drags and the position is remembered. An OPML subscription's page instead
|
|
||||||
lists the feeds inside it.
|
|
||||||
|
|
||||||
An item may carry several enclosures. All of them appear in the pane below; the row summarises the
|
|
||||||
one you would act on and notes how many others there are. Anything that is not audio or video gets a
|
|
||||||
View link — opening the publisher's copy, or the local one once downloaded — rather than a player.
|
|
||||||
|
|
||||||
## Log view
|
|
||||||
|
|
||||||
The **Log** button in the sidebar shows the running daemon's output live, in four tabs:
|
|
||||||
**Daemon I/O** is the control protocol itself — every command in and every event out, as JSON;
|
|
||||||
**Scans** is feed and download activity; **HTTP** is web requests; **All** is everything: feed scans, downloads,
|
|
||||||
torrent activity and every HTTP request, with level and text filters and a copy button. It reads a
|
|
||||||
2000-line ring buffer held inside the process (`/api/logs`), not a file — so it works the same under
|
|
||||||
Docker, where logs go to stdout and there is no file to tail. The buffer keeps `debug` even when the terminal does not, so protocol traffic and routine
|
|
||||||
skips are there without making stderr unreadable — `IPX_UI_LOG` changes what it captures and
|
|
||||||
`IPX_LOG` what reaches stderr.
|
|
||||||
|
|
||||||
## Tests
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cargo test # the server: parsing, filters, retention, schedules, SQL
|
|
||||||
node tests/page-smoke.js # the page script loads without throwing
|
|
||||||
npx playwright test # a real browser against a real daemon
|
|
||||||
```
|
|
||||||
|
|
||||||
The Rust tests cover the server and the smoke test catches a script that fails to load, but neither
|
|
||||||
can see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which
|
|
||||||
is what has actually slipped through. The Playwright suite drives a headless browser against a
|
|
||||||
daemon started on fixture feeds, and each test maps to a bug that reached a user: the page rendering
|
|
||||||
empty, a dead theme toggle, settings not persisting, episode metadata, filter tabs, the feed URL
|
|
||||||
field and its copy button, the log tabs, and OPML folders.
|
|
||||||
|
|
||||||
`npm install` gets the test runner; the browser itself comes from
|
|
||||||
`npx playwright install --with-deps chromium` (in `install.sh`).
|
|
||||||
|
|
||||||
## Docker
|
## Docker
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker compose up -d # builds the image and starts it
|
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
|
||||||
docker compose logs -f ipx # the token is printed on first start
|
docker compose pull ipodderx && docker compose up -d ipodderx
|
||||||
|
docker compose logs -f ipodderx # the first start prints the default admin password
|
||||||
```
|
```
|
||||||
|
|
||||||
`docker-compose.yml` mounts `./config`, `./data` and a downloads directory, publishes 8099 for the
|
`docker-compose.yml` runs the image from the registry above rather than building it, so build and
|
||||||
UI and 6881 (TCP **and** UDP — DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
|
push first; change the tag in both places to use another registry. It mounts `/config` (config.toml),
|
||||||
land owned the way Unraid shares expect. On first start the entrypoint writes a config bound to
|
`/data` (state.db) and `/downloads` from this install's host paths, which you will want to change for
|
||||||
`0.0.0.0`, since a container's loopback is not reachable from outside it, and prints the URL with
|
yours. It publishes 8099 for the
|
||||||
its generated token.
|
UI and 6881 (TCP **and** UDP -- DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
|
||||||
|
land owned the way Unraid shares expect. The healthcheck runs `ipx status` through the control
|
||||||
The healthcheck runs `ipx status`, which goes through the control socket to the command worker — so
|
socket, so it catches a daemon that is alive but wedged rather than merely one that has died.
|
||||||
it catches a daemon that is alive but wedged, not merely one that has died.
|
|
||||||
|
|
||||||
## Running it as a service
|
## Running it as a service
|
||||||
|
|
||||||
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
|
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
|
||||||
rather run periodic scans with no daemon (in which case there is no socket for a UI to attach to).
|
rather run periodic scans with no daemon -- in which case there is no socket for a UI to attach to.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
|
||||||
|
node tests/page-smoke.js # the page script loads without throwing
|
||||||
|
npx playwright test # a real browser against a real daemon
|
||||||
|
```
|
||||||
|
|
||||||
|
`npm install` gets the test runner; the browser comes from
|
||||||
|
`npx playwright install --with-deps chromium` (in `install.sh`).
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
MIT. See [LICENSE](LICENSE).
|
MIT. See [LICENSE](LICENSE).
|
||||||
|
|
||||||
## Signing in
|
The icons are [Font Awesome Free](https://fontawesome.com) 7.3.1 by @fontawesome, under
|
||||||
|
[CC BY 4.0](https://fontawesome.com/license/free), embedded as SVG in `web/index.html`.
|
||||||
A new database starts with **admin / ipodderx** at `/login`; change it with
|
|
||||||
`echo -n <password> | ipx user passwd admin`. Accounts are managed with `ipx user add|list|passwd|rm`.
|
|
||||||
|
|
||||||
To let Cloudflare Zero Trust or Authentik do the authenticating instead, see
|
|
||||||
[docs/sso.md](docs/sso.md).
|
|
||||||
|
|||||||
139
docs/architecture.md
Normal file
139
docs/architecture.md
Normal file
@@ -0,0 +1,139 @@
|
|||||||
|
# How it works
|
||||||
|
|
||||||
|
One binary, `ipx`. `ipx daemon` runs three things in one process: a scheduler, a Unix-socket
|
||||||
|
control server, and the web UI. Everything else is a CLI that either does the work itself or hands
|
||||||
|
it to a running daemon.
|
||||||
|
|
||||||
|
## Modules
|
||||||
|
|
||||||
|
| File | Responsibility | What it replaced in the Python |
|
||||||
|
|---|---|---|
|
||||||
|
| `src/main.rs` | CLI, dispatch, scan loop, download policy | `iPXAgent.py` |
|
||||||
|
| `src/config.rs` | TOML load/save, `General`/`Feed`/`Web`, intervals, slugs | `iPXSettings.py`, `feeds.plist` |
|
||||||
|
| `src/db.rs` | SQLite schema, migrations, every query | `.ipxd` plists, `history.dat`, `qmcache.dat` |
|
||||||
|
| `src/feed.rs` | Conditional GET, RSS/Atom/OPML parsing | `FeedData.__getFeed/__getEntries` |
|
||||||
|
| `src/download.rs` | Streaming download, naming, type sniffing, placement | `iPXDownloader.getFile` |
|
||||||
|
| `src/torrent.rs` | librqbit session, seeding limits, stall abort | vendored BitTorrent 4.2.1 |
|
||||||
|
| `src/retention.rs` | Quota and age sweeps | `iPXQuotaManager.py` |
|
||||||
|
| `src/ipc.rs` | Event and command types, the socket server | `printMSG` on stdout |
|
||||||
|
| `src/auth.rs` | Argon2id hashing, session tokens, header names | — |
|
||||||
|
| `src/web.rs` | axum: HTTP API, auth, SSE, media streaming | — |
|
||||||
|
| `src/logbuf.rs` | Ring buffer behind the UI's Log view | — |
|
||||||
|
| `web/index.html` | The whole front end, `include_str!`d into the binary | — |
|
||||||
|
|
||||||
|
The page is compiled in, so **editing `web/index.html` needs a rebuild**.
|
||||||
|
|
||||||
|
## A scan
|
||||||
|
|
||||||
|
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
|
||||||
|
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
|
||||||
|
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
|
||||||
|
re-derived into the database each scan, never written to config.toml.
|
||||||
|
4. Record entries. A changed title or description flips the item back to unread.
|
||||||
|
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
|
||||||
|
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
|
||||||
|
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
|
||||||
|
a reason.
|
||||||
|
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
|
||||||
|
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
|
||||||
|
or an error page — and is deleted.
|
||||||
|
|
||||||
|
## Data model
|
||||||
|
|
||||||
|
```
|
||||||
|
feeds id, url, title, image, etag, last_modified, last_checked, ttl_mins,
|
||||||
|
last_error, orphaned, group_id, managed
|
||||||
|
entries feed_id, guid, title, link, published, description, first_seen,
|
||||||
|
image, duration, episode, season PK (feed_id, guid)
|
||||||
|
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
|
||||||
|
bytes_done, downloaded_at, last_error
|
||||||
|
users id, name, pass_hash, is_admin, created
|
||||||
|
sessions token, user_id, created, seen
|
||||||
|
subscriptions user_id, feed_id, keywords, auto_download, allow_explicit,
|
||||||
|
max_new_per_check, created PK (user_id, feed_id)
|
||||||
|
entry_state user_id, feed_id, guid, read, flagged, position
|
||||||
|
PK (user_id, feed_id, guid)
|
||||||
|
```
|
||||||
|
|
||||||
|
`entries` still has `read`, `flagged` and `position` columns from before accounts existed. They are
|
||||||
|
**dead** — the migration copied them into `entry_state` and nothing reads them now. Anything found
|
||||||
|
querying them is a bug; two were.
|
||||||
|
|
||||||
|
Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in
|
||||||
|
`migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the
|
||||||
|
same path as `Db::open`, so a migration-only column cannot pass tests while missing in production.
|
||||||
|
|
||||||
|
## Control socket
|
||||||
|
|
||||||
|
Newline-delimited JSON, both directions, over `$XDG_RUNTIME_DIR/ipx.sock`.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
||||||
|
{"ev":"feed_start","feed":"atp"}
|
||||||
|
{"ev":"progress","feed":"atp","enclosure":42,"file":"ep1.mp3","done":8192,"total":3000000}
|
||||||
|
{"ev":"download_done","feed":"atp","enclosure":42,"path":"…","bytes":3000000}
|
||||||
|
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
|
||||||
|
{"ev":"scan_done","feeds":1}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Commands** — `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `download`
|
||||||
|
(`enclosure`), `status`.
|
||||||
|
|
||||||
|
**Events** — `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
||||||
|
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
||||||
|
`scan_done`, `reap_done` and `status` are terminal: a client that asked for work stops reading
|
||||||
|
there.
|
||||||
|
|
||||||
|
Progress carries the enclosure id, without which a UI cannot tell one download from another and
|
||||||
|
ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast,
|
||||||
|
so a client attached to a busy daemon also sees that daemon's other work.
|
||||||
|
|
||||||
|
Inside the process the same events go over a `tokio::broadcast`; commands arrive on an `mpsc` and
|
||||||
|
are handled by a single worker, so nothing races over the same download. Shutdown is a `watch`
|
||||||
|
channel raced *inside* each job — `tokio::select!` only races branches at the point of selection,
|
||||||
|
so a long download had to be able to notice the signal itself.
|
||||||
|
|
||||||
|
## HTTP API
|
||||||
|
|
||||||
|
Everything below `/api` needs a signed-in user; the browser gets a redirect to `/login`, anything
|
||||||
|
else a `401`.
|
||||||
|
|
||||||
|
| Route | |
|
||||||
|
|---|---|
|
||||||
|
| `GET /` | the app |
|
||||||
|
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
|
||||||
|
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
|
||||||
|
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
|
||||||
|
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable, sortable (`sort` = kept, title, feed, type, size or published; `dir` = asc or desc) |
|
||||||
|
| `GET /api/entries` | the same, across every feed you subscribe to (All Subscriptions) |
|
||||||
|
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
|
||||||
|
| `POST /api/read-all` | everything read in every feed you subscribe to (All Subscriptions) |
|
||||||
|
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, starred, position |
|
||||||
|
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
|
||||||
|
| `POST /api/fetch` | |
|
||||||
|
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
|
||||||
|
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
|
||||||
|
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
|
||||||
|
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
|
||||||
|
| `GET /api/events` | SSE, the same broadcast the socket carries |
|
||||||
|
| `GET /api/logs` | admin-only; the ring buffer, with a sequence cursor |
|
||||||
|
| `GET /media/{id}` | the file, with Range support so seeking works |
|
||||||
|
|
||||||
|
Show notes are feed-supplied HTML from an untrusted source, sanitized with `ammonia` server-side
|
||||||
|
before they reach the page.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test # parsing, filters, retention, schedules, SQL, per-user isolation
|
||||||
|
node tests/page-smoke.js # the page script loads and every selector it wires at load exists
|
||||||
|
npx playwright test # a real browser against a real daemon on fixture feeds
|
||||||
|
```
|
||||||
|
|
||||||
|
The Rust tests cannot see a wrong selector, a handler that runs and does nothing, or a page that
|
||||||
|
renders empty — which is what has actually reached users. Each Playwright case maps to a bug that
|
||||||
|
did.
|
||||||
|
|
||||||
|
The suite starts its own daemon and database under `/tmp/ipx-ui-test`, wiped once per run. Tests
|
||||||
|
share that daemon and run in order, so a test that marks something read changes what later tests
|
||||||
|
see — make assertions that do not depend on earlier ones.
|
||||||
91
docs/cli.md
Normal file
91
docs/cli.md
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
# Command line
|
||||||
|
|
||||||
|
```
|
||||||
|
ipx [--config PATH] [--local] <command>
|
||||||
|
```
|
||||||
|
|
||||||
|
Every command that has a wire form probes the control socket first: if a daemon is running, the
|
||||||
|
daemon does the work and the CLI just renders the events it streams back. That is deliberate — two
|
||||||
|
processes must never download the same thing. `--local` forces the work to happen in-process.
|
||||||
|
|
||||||
|
| Command | What it does |
|
||||||
|
|---|---|
|
||||||
|
| `ipx list` | Subscriptions and their state |
|
||||||
|
| `ipx status` | Counts: feeds, pending, downloaded |
|
||||||
|
| `ipx fetch [FEED] [--force]` | Scan everything, or one feed. `--force` ignores the TTL |
|
||||||
|
| `ipx add <url> [--folder X] [--keywords a,b]` | Subscribe; the id comes from the feed title |
|
||||||
|
| `ipx rm <feed>` | Unsubscribe; downloads and history are kept |
|
||||||
|
| `ipx import <file.opml>` / `ipx export <file.opml>` | Move subscriptions in or out. Import subscribes the first admin, as the shared web token does; in the web UI it subscribes whoever is signed in |
|
||||||
|
| `ipx reap [--dry-run]` | Run retention now |
|
||||||
|
| `ipx user <add\|list\|passwd\|rm>` | Accounts for the web UI |
|
||||||
|
| `ipx daemon [--web ADDR]` | Scheduler, control socket and web UI |
|
||||||
|
|
||||||
|
## Accounts
|
||||||
|
|
||||||
|
Passwords are read from **stdin**, so they miss the shell history and any `ps` listing.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
echo -n 'a good password' | ipx user add ray # local account
|
||||||
|
ipx user add ray@example.com --no-password # signs in through the proxy only
|
||||||
|
echo -n 'a good password' | ipx user passwd admin # change a password
|
||||||
|
ipx user list # who exists, and how each signs in
|
||||||
|
ipx user rm sam # account, subscriptions and read state
|
||||||
|
```
|
||||||
|
|
||||||
|
The first account created is an admin; later ones are ordinary users. A database with no accounts
|
||||||
|
at all gets **admin / ipodderx** on the next daemon start, announced in the log — change it.
|
||||||
|
|
||||||
|
To avoid even the command line, read it interactively:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
read -s PW && echo -n "$PW" | ipx user passwd admin
|
||||||
|
```
|
||||||
|
|
||||||
|
## Scanning
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx fetch # everything due
|
||||||
|
ipx fetch atp --force # one feed, ignoring its TTL and schedule
|
||||||
|
```
|
||||||
|
|
||||||
|
A scan: conditional GET (`If-None-Match` / `If-Modified-Since`), parse, record new entries, apply
|
||||||
|
the filters, then download up to the per-scan cap, newest first. A feed nothing has changed in
|
||||||
|
answers `304` and costs one request.
|
||||||
|
|
||||||
|
## Retention
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx reap --dry-run # what would go, oldest first
|
||||||
|
ipx reap # actually delete
|
||||||
|
```
|
||||||
|
|
||||||
|
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
|
||||||
|
with no file are pruned from the database. **Starred by anyone keeps a file**, and one only counts
|
||||||
|
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
|
||||||
|
stops the next scan fetching it again.
|
||||||
|
|
||||||
|
## The daemon
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx daemon # scheduler + socket + web UI
|
||||||
|
ipx daemon --web 0.0.0.0:8099 # override the configured bind for one run
|
||||||
|
```
|
||||||
|
|
||||||
|
One daemon per socket; a second refuses to start rather than fight over the database. It shuts down
|
||||||
|
cleanly on SIGTERM, including mid-download.
|
||||||
|
|
||||||
|
To kill it, match the binary exactly:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pkill -x ipx
|
||||||
|
```
|
||||||
|
|
||||||
|
`pkill -f ipx` matches the shell running the command too, and kills your own session.
|
||||||
|
|
||||||
|
## Talking to it directly
|
||||||
|
|
||||||
|
```sh
|
||||||
|
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
||||||
|
```
|
||||||
|
|
||||||
|
See [architecture.md](architecture.md#control-socket) for the protocol.
|
||||||
118
docs/configuration.md
Normal file
118
docs/configuration.md
Normal file
@@ -0,0 +1,118 @@
|
|||||||
|
# Configuration
|
||||||
|
|
||||||
|
One TOML file, read at startup and re-read whenever the web UI writes to it — most changes take
|
||||||
|
effect without a restart. Default location `$XDG_CONFIG_HOME/ipx/config.toml`
|
||||||
|
(`~/.config/ipx/config.toml`), overridden with `--config` or `$IPX_CONFIG`.
|
||||||
|
|
||||||
|
| What | Where | Override |
|
||||||
|
|---|---|---|
|
||||||
|
| Config | `~/.config/ipx/config.toml` | `--config`, `$IPX_CONFIG` |
|
||||||
|
| Database | `~/.local/share/ipx/state.db` | `$IPX_DATA_DIR` |
|
||||||
|
| Control socket | `$XDG_RUNTIME_DIR/ipx.sock` | `[general] socket` |
|
||||||
|
| Downloads | `[general] download_dir` | — |
|
||||||
|
|
||||||
|
`~` is expanded in paths. The database is SQLite in WAL mode; back it up by copying `state.db`
|
||||||
|
while the daemon is stopped, or with `sqlite3 state.db .backup`.
|
||||||
|
|
||||||
|
## `[general]`
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[general]
|
||||||
|
download_dir = "~/Podcasts"
|
||||||
|
socket = "/run/user/1000/ipx.sock"
|
||||||
|
schedule = "every 1h" # "every 30m", "every 4h", "2d", "90" (minutes)
|
||||||
|
organize = "feed" # "feed" | "date"
|
||||||
|
max_total_gb = 50 # 0 = unlimited
|
||||||
|
max_age_days = 30 # 0 = keep forever
|
||||||
|
max_new_per_check = 3 # per feed, per scan. 0 = unlimited
|
||||||
|
media_types = ["audio", "video"]
|
||||||
|
```
|
||||||
|
|
||||||
|
* **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
|
||||||
|
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
|
||||||
|
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
|
||||||
|
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
|
||||||
|
pad. Starred items are never deleted, and a file only counts as read once every subscriber has
|
||||||
|
read it. `0` disables it entirely.
|
||||||
|
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
|
||||||
|
Starred ones stay. `0` disables it.
|
||||||
|
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
|
||||||
|
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
|
||||||
|
fetched 216 files and 22 GB in one scan.
|
||||||
|
* **`media_types`** — top-level MIME types taken automatically. Anything else is still listed and
|
||||||
|
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
|
||||||
|
without this the disk fills with artwork. Empty takes everything.
|
||||||
|
|
||||||
|
`interval_mins` from older configs is still read, and `schedule` supersedes it.
|
||||||
|
|
||||||
|
## `[torrent]`
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[torrent]
|
||||||
|
enabled = true
|
||||||
|
seed_ratio = 1.0 # stop seeding at this ratio ...
|
||||||
|
seed_time_mins = 60 # ... or after this long, whichever comes first
|
||||||
|
port_range = "6881-6889"
|
||||||
|
stall_mins = 30 # give up on a torrent making no progress
|
||||||
|
```
|
||||||
|
|
||||||
|
A `.torrent` body is handed to the torrent path whatever MIME type it was advertised as. Torrents
|
||||||
|
run on their own tasks (two at a time) so a slow swarm never blocks a scan.
|
||||||
|
|
||||||
|
## `[web]`
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[web]
|
||||||
|
enabled = true
|
||||||
|
bind = "0.0.0.0:8099" # 127.0.0.1:8080 by default
|
||||||
|
token = "" # generated and saved on first run
|
||||||
|
trusted_header = "" # e.g. "Cf-Access-Authenticated-User-Email"
|
||||||
|
trusted_proxies = ["127.0.0.1", "::1"]
|
||||||
|
auto_create_users = true
|
||||||
|
session_days = 30
|
||||||
|
```
|
||||||
|
|
||||||
|
* **`token`** — the shared secret, which signs in as the **admin**. `?token=…` sets a cookie, so
|
||||||
|
you paste it once per browser. It is what the Docker healthcheck and any scripts use.
|
||||||
|
* **`trusted_header`** — a header naming the signed-in user, set by whatever fronts ipx. Empty
|
||||||
|
disables that path. See [sso.md](sso.md).
|
||||||
|
* **`trusted_proxies`** — addresses allowed to assert that header, and the entire security boundary
|
||||||
|
for it. Name the proxy, never a subnet.
|
||||||
|
* **`auto_create_users`** — create an account the first time the proxy vouches for a new name.
|
||||||
|
* **`session_days`** — sign a session out after this long without a request.
|
||||||
|
|
||||||
|
It is plain HTTP. On a LAN bind everything crosses the network in the clear — and a feed URL can
|
||||||
|
itself carry a credential. Put TLS in front of it if that matters.
|
||||||
|
|
||||||
|
## `[feeds.<id>]`
|
||||||
|
|
||||||
|
The table key is the feed id: stable, human-readable, and used in paths and the API. `ipx add`
|
||||||
|
derives it from the feed title.
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[feeds.atp]
|
||||||
|
url = "https://atp.fm/rss"
|
||||||
|
folder = "Accidental Tech Podcast" # default: the feed title
|
||||||
|
schedule = "every 6h" # overrides [general] for this feed
|
||||||
|
media_types = ["audio"] # overrides [general] for this feed
|
||||||
|
username = "ray" # HTTP basic auth
|
||||||
|
password_env = "IPX_ATP_PASS" # preferred over a literal `password`
|
||||||
|
```
|
||||||
|
|
||||||
|
With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
|
||||||
|
`max_new_per_check` live on each person's subscription in the database**, not here — the values in
|
||||||
|
config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
|
||||||
|
and are the same for everyone. See [users.md](users.md).
|
||||||
|
|
||||||
|
Feeds derived from a subscribed OPML are **not** written here: the OPML is the source of truth and
|
||||||
|
they are re-derived on every scan. Editing one in the UI promotes it to a real config entry.
|
||||||
|
|
||||||
|
## Environment
|
||||||
|
|
||||||
|
| Variable | Effect |
|
||||||
|
|---|---|
|
||||||
|
| `IPX_CONFIG` | Config file path |
|
||||||
|
| `IPX_DATA_DIR` | Directory holding `state.db` |
|
||||||
|
| `IPX_LOG` | What reaches stderr (`ipx=debug`, `ipx::scan=debug`, …) |
|
||||||
|
| `IPX_UI_LOG` | What the in-process log buffer captures for the UI's Log view |
|
||||||
|
| `http_proxy` / `https_proxy` | Honoured for feed and enclosure fetches |
|
||||||
@@ -1,58 +1,284 @@
|
|||||||
# Progress
|
# How ipx was built
|
||||||
|
|
||||||
Running record of what has actually landed. Newest entry first.
|
The long-form record, newest first: what landed each day, what was wrong before, and what it cost
|
||||||
The full design and step list live in the plan file at
|
to find out. [CHANGELOG.md](../CHANGELOG.md) has the short version, by release; this is where the
|
||||||
`/config/.claude/plans/i-want-to-create-playful-quiche.md`.
|
reasoning lives. New write-ups go at the top.
|
||||||
|
|
||||||
## Build order
|
See [README.md](../README.md) for what the thing is.
|
||||||
|
|
||||||
- [x] **1. Repo skeleton** — git init (`main`), `cargo init --name ipx`, deps pinned, LICENSE,
|
## 2026-09-11 — One meaning per icon, sortable columns, and one player
|
||||||
README, this file.
|
|
||||||
- [x] **2. `config.rs` + `db.rs`** — TOML config structs + SQLite schema.
|
|
||||||
- [x] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries.
|
|
||||||
- [x] **4. `download.rs`** — downloads, filters, dedupe.
|
|
||||||
- [x] **5. `retention.rs`** — oldest-first quota + age reaper.
|
|
||||||
- [x] **6. `ipc.rs` + daemon** — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon.
|
|
||||||
- [x] **7. `torrent.rs`** — librqbit, seed to ratio/time, stall abort. (swarm download unverified —
|
|
||||||
see the step 7 entry)
|
|
||||||
- [x] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd units, README.
|
|
||||||
|
|
||||||
### Phase 2 — web front end
|
Ray asked for a pass over the whole UI: consistent icons, and buttons placed next to what they act
|
||||||
|
on. The pass started from screenshots of every view and dialog, taken against the browser suite's
|
||||||
|
own daemon and fixtures. Several things did not agree:
|
||||||
|
|
||||||
Decided with Ray: axum serving plain HTML/JS (no WASM toolchain), running **inside the daemon**
|
- Unsubscribe was a minus in the toolbar but an x in the feed header. There it read as "close this
|
||||||
process so it reads SQLite and the event bus directly, LAN-bindable with a shared token.
|
page", and x also meant Cancel in every dialog.
|
||||||
|
- Half the dialogs still had word buttons: Log, Add feed, Users, Unsubscribe, OPML, Popular's
|
||||||
|
Subscribe, the Copy buttons and Sign out.
|
||||||
|
- The player bar and the folder arrow were font characters, and so was the row's "playing" marker.
|
||||||
|
Those came out thin and varied from font to font.
|
||||||
|
- The OPML subscription page laid out its header differently from a feed's, and All Subscriptions
|
||||||
|
had no buttons at all.
|
||||||
|
|
||||||
- [x] **9. Config hot-reload + web skeleton.** `Ctx.cfg` becomes `RwLock<Arc<Config>>` so the UI can
|
The rule now is one meaning per icon. Minus unsubscribes, x closes or cancels, plus adds, subscribes
|
||||||
edit feeds without a daemon restart. `[web]` config section (enabled/bind/token, token
|
or imports, and a dialog's confirm button carries the icon of what it does. The words go in the
|
||||||
auto-generated and saved on first run). axum server started by `ipx daemon`, token checked by
|
tooltip. The toolbar is grouped by what it acts on: add, unsubscribe and scan act on feeds; play,
|
||||||
middleware, `?token=` sets a cookie so `<audio>` requests authenticate too.
|
read and keep act on the selected item. Its read and keep buttons now show that item's state, as
|
||||||
*Done when:* `ipx daemon` serves a page on the configured bind, and a wrong token gets 401.
|
the item's own buttons already did. A browser test opens all eight dialogs and fails on any button
|
||||||
- [x] **10. Browsing.** `/api/feeds`, `/api/feeds/:id/entries`, entry detail. Descriptions are
|
without an icon and a tooltip. It also checks that the toolbar and the page use the same icon for
|
||||||
untrusted feed HTML — sanitized with `ammonia` before they reach the page.
|
the same action.
|
||||||
*Done when:* the Glass Cannon feed's 131 entries browse and read correctly.
|
|
||||||
- [x] **11. Media actions.** Range-request audio streaming (`tower-http` ServeFile) so seeking
|
|
||||||
works, download-on-demand for a pending enclosure, delete a file, mark read/flagged.
|
|
||||||
*Done when:* an episode plays and seeks in a browser, and delete reaps the row.
|
|
||||||
- [x] **12. Feed configuration.** Add/remove feeds and edit folder, keywords, allow_explicit,
|
|
||||||
auto_download, max_new_per_check from the UI, written back to config.toml and hot-reloaded.
|
|
||||||
*Done when:* flipping allow_explicit in the UI takes effect on the next scan with no restart.
|
|
||||||
- [x] **13. Live progress + polish.** SSE from the existing broadcast bus so downloads show live.
|
|
||||||
README section, screenshot-free usage notes.
|
|
||||||
*Done when:* starting a fetch from the UI shows progress advancing without a reload.
|
|
||||||
|
|
||||||
Note: `read`/`flagged` finally get a writer here. Retention orders by them (see the step 5 entry),
|
The pass turned up three bugs:
|
||||||
and until now nothing set them.
|
|
||||||
|
|
||||||
## Smoke tests
|
- **Escape ignored inside a dialog's text box.** The keydown handler returned early for any key
|
||||||
|
typed in a text box, and Escape sat below that check. So Add feed, which opens with the cursor in
|
||||||
|
its URL box, could not be closed with Escape. It showed up because the screenshot run stalled:
|
||||||
|
the dialog stayed open, and every later click landed on its overlay until the run timed out.
|
||||||
|
- **A white password box in the dark theme.** The input rule listed text, search and number
|
||||||
|
fields, but not password ones.
|
||||||
|
- **A stray "• •" under an item with no date.** Each detail carried its own trailing dot, whether
|
||||||
|
or not the next one was there.
|
||||||
|
|
||||||
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
|
All Subscriptions gained Mark everything read, as `POST /api/read-all`. It marks the feeds from the
|
||||||
2. `ipx fetch` again → no re-download, feed skipped for TTL.
|
person's subscriptions, which is the same set the view's query is scoped by. It asks first, since
|
||||||
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
|
one click covers every feed.
|
||||||
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
|
|
||||||
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
|
**Sorting** happens on the server. A list arrives fifty at a time, so sorting in the page would
|
||||||
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
|
only have reordered what was loaded. The column name picks one of a fixed set of SQL expressions
|
||||||
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
|
(`order_sql`), so nothing the browser sends reaches the query, and an unknown name means newest
|
||||||
`reaped`.
|
first. Ties also fall back to newest first, so "Load more" does not skip or repeat items at a page
|
||||||
|
boundary. A database test runs every column both ways. Size came out of the File column into its
|
||||||
|
own, and small files show KB, since whole megabytes made every article image "0 MB".
|
||||||
|
|
||||||
|
**The double play.** Ray found that pressing play in the Files pane played the file twice at once.
|
||||||
|
The pane drew its own `<audio controls>`, and its `onplay` handler also started the player bar on
|
||||||
|
the same file. That meant two audio elements from one click. The pane now has a play button that
|
||||||
|
hands that exact file to the player bar, the only player. `play()` now takes the file. Its check
|
||||||
|
for resuming compares the file as well as the item, so another of an item's files starts from its
|
||||||
|
beginning instead of carrying on with the first. A test plays from the pane and checks the page
|
||||||
|
has one `<audio>` element.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — The original's layout
|
||||||
|
|
||||||
|
Ray pointed at a screenshot of the Mac app (techpp.com, 2012) and asked for its panes, its
|
||||||
|
toolbar, and a Directory that lives in the feed list rather than behind a button. What it had, and
|
||||||
|
what ipx now does:
|
||||||
|
|
||||||
|
- A toolbar across the window: subscribe and unsubscribe, play, flag, refresh, and a search box
|
||||||
|
scoped to the feed on show. ipx's has the same groups, acting on the selected feed and item, with
|
||||||
|
Settings and Log at the right end, for admins only.
|
||||||
|
- A source list opening with Directory, Playlist Builder and All Subscriptions above the feeds.
|
||||||
|
ipx has Directory, Popular and All Subscriptions there, opening in the main pane. Playlist
|
||||||
|
Builder is left out, since nothing here builds playlists.
|
||||||
|
- The entries as a table, with a Files pane beside it and the entry below. The columns are unread,
|
||||||
|
kept, the item, its feed, its file and when. Sorting by column is not done yet.
|
||||||
|
- A status bar with the totals for what is on show.
|
||||||
|
|
||||||
|
All Subscriptions needed one new endpoint, `GET /api/entries`. It is the per-feed query with
|
||||||
|
`feed_id = ?` swapped for the person's subscriptions. The enclosure lookup that follows it used to
|
||||||
|
filter by feed as well; a page can now span feeds, so each file is matched to its row by feed and
|
||||||
|
guid instead.
|
||||||
|
|
||||||
|
A phone has no room for a pane beside the table, so there the files follow the item's text in the
|
||||||
|
full-screen reader, and the table drops to title and date.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Popular on this server
|
||||||
|
|
||||||
|
The old iPodderX had a directory of podcasts and a top-feeds list. The open-sourced engine shows how
|
||||||
|
the list was fed. After every successful download, `iPXClass.py:1249` called `doPing`, which sent
|
||||||
|
the enclosure URL and the feed URL, unencoded and over plain HTTP, to
|
||||||
|
`directory.iPodderX.com/feedData/survey/files`. It was on by default, and only the Mac app's
|
||||||
|
preferences could switch it off (`anonFeedback`). The directory itself lived on that server and in
|
||||||
|
the Cocoa app, and both are gone.
|
||||||
|
|
||||||
|
ipx keeps the idea and drops the ping. Add feed lists what other accounts on this server subscribe
|
||||||
|
to, ranked by subscriber count. The old ping would have sent a Patreon feed's `auth=` key along
|
||||||
|
with everything else, so this is built around not doing that:
|
||||||
|
|
||||||
|
- A row carries an id, a title, artwork and a count. No URL reaches the page, so no key can.
|
||||||
|
- Subscribing goes by id through `POST /api/popular/{id}`, which checks the id against the same
|
||||||
|
list. Guessing a private feed's id gets a 400.
|
||||||
|
- Anything that looks private is never listed: a login configured for the feed, credentials in
|
||||||
|
the URL, or a query key containing `auth`, `token`, `key`, `secret`, `pass`, `sig`, `session`,
|
||||||
|
`user` or `uid`. It is a heuristic, and a token hidden in the URL's path gets through. A per-feed
|
||||||
|
`unlisted` flag is the upgrade if that happens.
|
||||||
|
|
||||||
|
It happened the same day. The first screenshot of the new Directory listed "Glass Cannon Live!
|
||||||
|
Ascension (for Ray Slakinski)", a Supercast feed at `feeds.supercast.com/feeds/<key>`. Treating
|
||||||
|
any long path segment as a key would have hidden public feeds too: acast's show ids look the
|
||||||
|
same. So paid-feed services are named instead (Patreon, Supercast, Supporting Cast, Glow,
|
||||||
|
Memberful), and any feed from one of them is private whatever its URL looks like. The per-feed
|
||||||
|
flag is still the upgrade for a service not on that list.
|
||||||
|
- Feeds from an OPML are left out. Everyone subscribed to an OPML counts every feed inside it, so
|
||||||
|
they would bury everything anyone chose on purpose.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Import an OPML file by uploading it
|
||||||
|
|
||||||
|
The import screen now has a file picker as well as the paste box. The page reads the file and
|
||||||
|
checks it looks like OPML before sending anything. If it doesn't, the page says so and clears the
|
||||||
|
picker, and it also clears the picker after an import. The file travels as text and is never
|
||||||
|
written to disk on the server, so there is nothing to clean up there.
|
||||||
|
|
||||||
|
The server now parses the OPML before touching anything, and returns `400` "that is not an OPML
|
||||||
|
file" instead of a `500`. `subscribe_opml` takes a parsed document, so `ipx import` also refuses a
|
||||||
|
non-OPML file by name before changing anything.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — OPML import and export are per person
|
||||||
|
|
||||||
|
Importing an OPML now subscribes you to every feed in it. Feeds already in the catalogue cost
|
||||||
|
nothing, and unknown ones are added under the OPML's title. Before this, import only added URLs
|
||||||
|
missing from `config.toml` and subscribed nobody. So importing an export from another account did
|
||||||
|
nothing at all, and a genuinely new feed had no subscriber and was never scanned. The page said
|
||||||
|
"Imported 0 feed(s)". `ipx import` had the same gap; it now subscribes the first admin. Both go
|
||||||
|
through `subscribe_opml`.
|
||||||
|
|
||||||
|
Export now lists only your own subscriptions. It used to write out the whole catalogue to anyone
|
||||||
|
signed in, including other people's feeds and any private URLs in them.
|
||||||
|
|
||||||
|
Tests now cover every way in and out:
|
||||||
|
- Settings' Export OPML download and paste-to-import screen.
|
||||||
|
- A round trip that imports the admin's actual export into a second account and checks both
|
||||||
|
exports match. That round trip is exactly what failed.
|
||||||
|
- `ipx import` and `ipx export`, run in their own scratch config and database.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — The log is admin-only
|
||||||
|
|
||||||
|
`GET /api/logs` now returns `403` to anyone who is not an admin, and the page hides the Log button
|
||||||
|
from them. Before this, every signed-in person could read the whole log. That includes every
|
||||||
|
account's name, every feed anyone subscribes to, and every failed sign-in. `/api/events` stays open
|
||||||
|
to everyone, because it carries the scan progress each person's page shows.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Managing users from the web
|
||||||
|
|
||||||
|
Settings has a **Manage users…** screen for an admin. From it you can add someone, with a password
|
||||||
|
or with none for someone the proxy signs in, tick or untick Admin, and remove an account. It is
|
||||||
|
backed by `GET/POST /api/users` and `PATCH/DELETE /api/users/{id}`, which return `403` for anyone
|
||||||
|
who is not an admin. The only admin cannot be demoted or removed, because nobody would then be
|
||||||
|
able to manage accounts except from the CLI on the box. Before this, accounts could only be managed
|
||||||
|
with `ipx user`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Unread feeds first inside an OPML
|
||||||
|
|
||||||
|
An OPML subscription's feeds, both in the sidebar folder and on its own page, now list the ones with
|
||||||
|
unread items first. They were listed alphabetically, so with dozens of feeds the few with anything
|
||||||
|
new were scattered through the list. Within each half the order is still alphabetical. The browser
|
||||||
|
suite's fixture OPML gained a second feed, Aardvark Radio, which sorts first by name and by position,
|
||||||
|
so the new test only passes if unread wins.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Deploying is a Docker image
|
||||||
|
|
||||||
|
Production moved from a hand-started daemon in code-server to the `iPodderX` container in the Arcane
|
||||||
|
project `content`. `CLAUDE.md` now deploys by pushing to the registry at `192.168.1.130:5000` and
|
||||||
|
recreating that one service with `docker compose`. The old instructions copied a binary over a
|
||||||
|
process nobody supervised, so it did not come back after a reboot. Two known gaps are gone: the
|
||||||
|
image does support accounts, and the entrypoint drops to `99:100`, so downloads are no longer
|
||||||
|
owned by root. The README's Docker section said `docker compose up -d` builds the image and named the
|
||||||
|
service `ipx`; the compose file pulls from the registry and the service is `ipodderx`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Documentation
|
||||||
|
|
||||||
|
`PROGRESS.md` became this changelog; the finished step lists moved to an appendix. The README is now
|
||||||
|
an overview that points at `docs/`: [configuration](configuration.md) (every key, path and
|
||||||
|
environment variable), [cli](cli.md), [users](users.md) (what several people share),
|
||||||
|
[sso](sso.md) (refreshed for accounts and the admin-only settings), and
|
||||||
|
[architecture](architecture.md) (modules, the scan, the schema, the socket protocol, the HTTP
|
||||||
|
API). `CLAUDE.md` collects what someone working on the code needs to know and has already been
|
||||||
|
learned the hard way here -- `pkill -x` not `-f`, the page being compiled in, the dead columns on
|
||||||
|
`entries`, the Playwright worker that deleted its own database.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Pruning respects a star from anyone
|
||||||
|
|
||||||
|
Same defect as the reaper, one function along: `prune_entries` guarded on `entries.flagged`, which
|
||||||
|
nothing has written since read state moved to `entry_state`. Starring a text item with no file would
|
||||||
|
not have saved it from the age sweep. It follows the reaper's rule now, and takes orphaned read
|
||||||
|
state with whatever it deletes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Step D: one file, and everyone who wants it
|
||||||
|
|
||||||
|
The last of it, which is all about telling the truth before acting:
|
||||||
|
|
||||||
|
* **Delete says whose file it is.** A feed with other subscribers labels the button *Delete for
|
||||||
|
everyone*, and the confirmation names how many people share it. The server then has the last
|
||||||
|
word: if anyone else has starred the item or not played it yet, `DELETE /api/enclosures/{id}`
|
||||||
|
returns **409** with the reason in plain words, and only `?force=true` goes through. So the
|
||||||
|
rule is enforced where it matters rather than in the page that happens to be asking.
|
||||||
|
* **A feed's header says it is shared** -- "shared with 1 other person" -- which is the answer to
|
||||||
|
"why is there a file here I never asked for": someone else's subscription fetched it, and one
|
||||||
|
copy serves you both.
|
||||||
|
* Retention already respects the same rule from the entry before this one: starred by anyone keeps
|
||||||
|
it, read by everyone releases it.
|
||||||
|
|
||||||
|
`others_wanting` is tested with three subscribers disagreeing, and a browser test walks the whole
|
||||||
|
delete flow: the label, both prompts, declining the second, and the file still being there.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Retention caught up with per-user state
|
||||||
|
|
||||||
|
Moving read and starred into `entry_state` left `reap_candidates` reading `entries.read` and
|
||||||
|
`entries.flagged`, which nothing writes any more: **starring stopped protecting a file**, and the
|
||||||
|
"delete the ones already read first" ordering was dead. Harmless while the quota and age limits are
|
||||||
|
0 -- the reaper deletes nothing at all then -- but it would have bitten the moment one was set.
|
||||||
|
|
||||||
|
One file serves every subscriber, so both tests are now about all of them: **anyone** starring it
|
||||||
|
keeps it, and it only counts as read once **everyone** subscribed has read it. A file whose feed
|
||||||
|
nobody subscribes to has nobody left to keep it, so it sorts with the read ones. Tested with two
|
||||||
|
subscribers disagreeing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-09-11 — Steps B and C: what is yours, what is everyone's
|
||||||
|
|
||||||
|
Read, starred and playback position moved out of `entries` into `entry_state (user_id, feed_id,
|
||||||
|
guid, ...)`; subscriptions became rows in `subscriptions (user_id, feed_id, ...)` carrying **your**
|
||||||
|
keywords, auto-download, explicit and per-scan limit. The feed list, unread counts, filters and
|
||||||
|
mark-all-read are all per person now. On first start the existing library is adopted by the admin:
|
||||||
|
2438 read/starred items and all 86 feeds, so nothing was lost.
|
||||||
|
|
||||||
|
The split follows from the file being shared:
|
||||||
|
|
||||||
|
* **Yours**: read state, starred, position, keywords, auto-download, explicit, per-scan limit,
|
||||||
|
and which feeds you see at all.
|
||||||
|
* **Everyone's**: the feed URL, its download folder, and when it is scanned -- there is one copy of
|
||||||
|
a file however many people subscribe, so those describe the file, not a preference. Admin-only,
|
||||||
|
refused with a 403 for anyone else rather than merely hidden.
|
||||||
|
|
||||||
|
Scanning merges the subscribers' wants, because one fetch and one file serve them all: an item is
|
||||||
|
downloaded if **anyone** wants it (any one person's keyword set matching is enough, and one person
|
||||||
|
taking everything removes the filter), auto-download is on if anyone has it on, and the per-scan cap
|
||||||
|
is the largest anyone asked for. `merge_policy` is a pure function with a test covering each of
|
||||||
|
those. Subscribing to a feed someone already has costs no second fetch and no second copy on disk;
|
||||||
|
unsubscribing takes it off your list alone, and only when the last subscriber leaves does the feed
|
||||||
|
stop being scanned.
|
||||||
|
|
||||||
|
**A test-harness bug worth naming**: Playwright imports the config in every worker, so the fixture's
|
||||||
|
`prepare()` ran again mid-run and deleted the data directory out from under the daemon. The daemon
|
||||||
|
kept serving from the unlinked inode while the CLI and any query opened a fresh empty database at
|
||||||
|
the same path -- which looked exactly like sign-in being broken. Only the launching process wipes
|
||||||
|
now (a worker has `TEST_WORKER_INDEX`).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -107,34 +333,6 @@ who is signed in above the sidebar footer, with a sign-out, and a 401 sends the
|
|||||||
Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B.
|
Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Multi-user — the plan
|
|
||||||
|
|
||||||
Decided with Ray: **stay on SQLite** (Postgres was considered and dropped -- it is a deployment
|
|
||||||
choice, not a capability one, and nothing here contends for writes). Sign-in is either a local
|
|
||||||
username and password or the Cloudflare Zero Trust that already fronts `ipodderx.sdf1.net`, which
|
|
||||||
puts the authenticated identity in `Cf-Access-Authenticated-User-Email`. Feeds, items and files are **shared**; read state and subscriptions are **per user**.
|
|
||||||
|
|
||||||
The point of sharing: two people subscribed to the same show cost one fetch, one parse, and one file
|
|
||||||
on disk. `enclosures.url` is already globally UNIQUE, so the file half is nearly free.
|
|
||||||
|
|
||||||
- [x] **A. Users, sessions, sign-in.** `users` + `sessions` tables, Argon2 hashing, session cookie,
|
|
||||||
`ipx user add|list|passwd|rm`. A proxy header (`trusted_header` in `[web]`) signs in and
|
|
||||||
optionally creates a user -- honoured only from a `trusted_proxies` address, so a LAN client
|
|
||||||
cannot simply assert it. The existing shared token keeps working and resolves to the admin, so
|
|
||||||
the healthcheck and any scripts survive. Login page for direct access.
|
|
||||||
- [ ] **B. Per-user read state.** `entry_state(user_id, feed_id, guid, read, flagged, position)`;
|
|
||||||
the current columns on `entries` migrate into the first user's rows. Unread counts, filters and
|
|
||||||
playback position all become per user.
|
|
||||||
- [ ] **C. Per-user subscriptions.** `subscriptions(user_id, feed_id)`. config.toml stays the feed
|
|
||||||
catalogue; the UI lists only what you subscribe to. Adding a feed someone else already has costs
|
|
||||||
nothing. A feed nobody subscribes to stops being scanned but keeps its files.
|
|
||||||
- [ ] **D. One file, many users.** Auto-download when *any* subscriber wants it; retention never
|
|
||||||
deletes a file another user has starred or not yet played; deleting a download says so when
|
|
||||||
someone else still has it.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-10 — Items, not episodes
|
## 2026-09-10 — Items, not episodes
|
||||||
|
|
||||||
Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search
|
Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search
|
||||||
@@ -1185,3 +1383,37 @@ Gotcha worth keeping: three feature names in the plan were wrong against current
|
|||||||
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
|
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
|
||||||
is the fix. Whole tree is rustls-only now, no C TLS dependency.
|
is the fix. Whole tree is rustls-only now, no C TLS dependency.
|
||||||
Next: step 2 — `config.rs` + `db.rs`. (done)
|
Next: step 2 — `config.rs` + `db.rs`. (done)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How this was built
|
||||||
|
|
||||||
|
Three planned phases, all complete. The dated entries above are the record of each.
|
||||||
|
|
||||||
|
**The engine (steps 1-8)** — repo skeleton with pinned dependencies; `config.rs` + `db.rs` (TOML
|
||||||
|
config, SQLite schema); `feed.rs` (conditional GET, RSS-then-Atom parsing); `download.rs`
|
||||||
|
(streaming downloads, filters, URL dedupe); `retention.rs` (oldest-first quota and age reaper);
|
||||||
|
`ipc.rs` plus the daemon (Unix-socket JSON lines, TTL scheduler, CLI proxying to a running daemon);
|
||||||
|
`torrent.rs` (librqbit, seeding to a ratio or a time, stall abort); OPML import/export and polish.
|
||||||
|
|
||||||
|
**The web front end (steps 9-13)** — axum serving plain HTML and JS, no WASM toolchain, running
|
||||||
|
*inside* the daemon so it reads SQLite and the event bus directly: config hot-reload and the `[web]`
|
||||||
|
section; browsing feeds and entries with `ammonia`-sanitized show notes; media actions (Range
|
||||||
|
streaming, download on demand, delete, read/starred); feed configuration written back to
|
||||||
|
config.toml; live progress over SSE from the existing broadcast bus.
|
||||||
|
|
||||||
|
**Multi-user (steps A-D)** — accounts, sessions and sign-in (Argon2id, session cookie, a trusted
|
||||||
|
proxy header for Cloudflare Zero Trust); per-user read state; per-user subscriptions; and the rules
|
||||||
|
that follow from one file serving everyone. SQLite stayed: Postgres was considered and dropped, as
|
||||||
|
nothing here contends for writes.
|
||||||
|
|
||||||
|
### The live checks each step had to pass
|
||||||
|
|
||||||
|
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
|
||||||
|
2. `ipx fetch` again → no re-download, feed skipped for TTL.
|
||||||
|
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
|
||||||
|
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
|
||||||
|
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
|
||||||
|
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
|
||||||
|
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
|
||||||
|
`reaped`.
|
||||||
26
docs/sso.md
26
docs/sso.md
@@ -35,7 +35,15 @@ Restart the daemon after editing. Accounts made this way have **no password**: t
|
|||||||
arrive through the proxy. `ipx user list` marks them `proxy only`.
|
arrive through the proxy. `ipx user list` marks them `proxy only`.
|
||||||
|
|
||||||
The first account created is an admin. Every later one is an ordinary user, and an ordinary user
|
The first account created is an admin. Every later one is an ordinary user, and an ordinary user
|
||||||
cannot change global settings or how often feeds are scanned. Promote someone with:
|
cannot change global settings, a feed's URL or folder, or how often feeds are scanned: the API
|
||||||
|
refuses those with a `403`, not just the UI. Everything else about a feed (which items they want,
|
||||||
|
whether to fetch them, how many at a time) is theirs alone; see [users.md](users.md).
|
||||||
|
|
||||||
|
Somebody arriving through the proxy for the first time starts with **no feeds**, because
|
||||||
|
subscriptions are per person. Adding a feed someone else already reads costs no second fetch and no
|
||||||
|
second copy on disk.
|
||||||
|
|
||||||
|
Promote someone with:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
ipx user list
|
ipx user list
|
||||||
@@ -43,7 +51,9 @@ echo -n 'a good password' | ipx user passwd <name> # optional: also lets them
|
|||||||
```
|
```
|
||||||
|
|
||||||
Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN
|
Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN
|
||||||
when the tunnel is down. A brand new database starts with **admin / ipodderx** — change it.
|
when the tunnel is down. So does the shared `[web] token`, which signs in as the admin: that is
|
||||||
|
what the Docker healthcheck uses, and the way back in if you lock yourself out. A brand new database
|
||||||
|
starts with **admin / ipodderx** — change it.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -206,7 +216,13 @@ ipx user rm sam # remove the account
|
|||||||
|
|
||||||
Set `auto_create_users = false` once everyone who should have an account has one. After that the
|
Set `auto_create_users = false` once everyone who should have an account has one. After that the
|
||||||
proxy vouching for an unknown name is logged and refused, rather than quietly making an account.
|
proxy vouching for an unknown name is logged and refused, rather than quietly making an account.
|
||||||
|
Pre-create people instead with `ipx user add <name> --no-password`, using exactly the name the
|
||||||
|
header will carry (Cloudflare sends the email address, lower-cased).
|
||||||
|
|
||||||
Scanning intervals, the disk quota, retention and the download folder are **admin-only** — the
|
Scanning intervals, the disk quota, retention, the download folder and a feed's URL are
|
||||||
Settings button is hidden for everyone else, and the API refuses the change even if the request is
|
**admin-only**: the Settings button is hidden for everyone else, and the API refuses the change even
|
||||||
made by hand. Ordinary users still control their own folders, keywords and downloads per feed.
|
if the request is made by hand. Everyone controls their own keywords, auto-download, explicit
|
||||||
|
setting and per-scan cap, along with their own read state and which feeds they see.
|
||||||
|
|
||||||
|
See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
|
||||||
|
for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.
|
||||||
|
|||||||
97
docs/users.md
Normal file
97
docs/users.md
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
# Accounts, and what several people share
|
||||||
|
|
||||||
|
ipx serves any number of people from one copy of the data. The rule that decides everything else:
|
||||||
|
**there is one file on disk per enclosure URL.** Two people subscribed to the same show cost one
|
||||||
|
fetch, one parse and one file.
|
||||||
|
|
||||||
|
## What is yours, what is everyone's
|
||||||
|
|
||||||
|
| Yours alone | The same for everyone |
|
||||||
|
|---|---|
|
||||||
|
| Read, starred, playback position | The feed's URL |
|
||||||
|
| Which feeds you see at all | Its download folder |
|
||||||
|
| Keywords, auto-download, explicit, per-scan cap | When it is scanned |
|
||||||
|
| | The file on disk |
|
||||||
|
|
||||||
|
The right-hand column describes the feed and the file rather than a preference — two people wanting
|
||||||
|
different folders would mean two copies. Those three are **admin-only**, and the API returns `403`
|
||||||
|
for anyone else rather than merely hiding the controls.
|
||||||
|
|
||||||
|
## How the scanner merges everyone's wants
|
||||||
|
|
||||||
|
One fetch serves every subscriber, so the policy is a union:
|
||||||
|
|
||||||
|
* an item is downloaded if **anyone** wants it — one person's keyword set matching is enough, and
|
||||||
|
one person with no keywords removes the filter for that feed entirely
|
||||||
|
* auto-download is on if **anyone** has it on
|
||||||
|
* the per-scan cap is the **largest** anyone asked for
|
||||||
|
|
||||||
|
So "auto-download off" means *I don't cause downloads*, not *I never see them*. If someone else's
|
||||||
|
subscription pulls an item, you see it listed as downloaded and can play it, because the enclosure
|
||||||
|
is shared.
|
||||||
|
|
||||||
|
## Deleting
|
||||||
|
|
||||||
|
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
|
||||||
|
for everyone** and names them in the confirmation, and the server has the last word: if anyone else
|
||||||
|
has starred the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
|
||||||
|
reason, and only `?force=true` goes through.
|
||||||
|
|
||||||
|
Retention follows the same rule: starred by anyone keeps a file, and it counts as read only once
|
||||||
|
every subscriber has read it.
|
||||||
|
|
||||||
|
## Signing in
|
||||||
|
|
||||||
|
Three ways, tried in order of how specific the claim is:
|
||||||
|
|
||||||
|
1. **A proxy header** naming the user — Cloudflare Zero Trust or Authentik. Honoured only from an
|
||||||
|
address in `trusted_proxies`. See [sso.md](sso.md).
|
||||||
|
2. **A session cookie** from signing in at `/login`. Argon2id hashes, sessions in the database,
|
||||||
|
idle timeout `session_days`.
|
||||||
|
3. **The shared `[web] token`**, which signs in as the admin — this is what the Docker healthcheck
|
||||||
|
and any scripts use.
|
||||||
|
|
||||||
|
A database with no accounts creates **admin / ipodderx** on the next daemon start and says so in
|
||||||
|
the log. Change it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
echo -n 'a good password' | ipx user passwd admin
|
||||||
|
```
|
||||||
|
|
||||||
|
## Adding someone
|
||||||
|
|
||||||
|
```sh
|
||||||
|
echo -n 'their password' | ipx user add sam
|
||||||
|
```
|
||||||
|
|
||||||
|
They sign in at `/login` and start with **no feeds**: subscriptions are per person. Adding a feed
|
||||||
|
someone else already has costs nothing — no second fetch, no second copy — it just appears on their
|
||||||
|
list with their own read state. Unsubscribing removes it from their list alone; only when the last
|
||||||
|
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
|
||||||
|
re-subscribing does not pull the back catalogue again.
|
||||||
|
|
||||||
|
**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
|
||||||
|
shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
|
||||||
|
included. Directory lists every one of them A to Z. Your own feeds are marked Subscribed.
|
||||||
|
It shows a title, artwork and a count, never a URL or who reads it. Feeds from an
|
||||||
|
OPML subscription are left out, since they come with the OPML. So is anything that looks private: a
|
||||||
|
login configured for the feed, credentials in its URL, or a key such as `auth=` or `token=` in the
|
||||||
|
query, or a feed from a paid-feed service such as Patreon or Supercast, which put the key in the
|
||||||
|
path. Those are someone's paid subscriptions, and listing them would let anyone here read what they
|
||||||
|
pay for.
|
||||||
|
|
||||||
|
An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none
|
||||||
|
for someone the proxy signs in), tick or untick Admin, or remove an account. Removing one takes its
|
||||||
|
subscriptions and read state with it; downloaded files stay. The only admin cannot be demoted or
|
||||||
|
removed there, so there is always someone who can manage the rest.
|
||||||
|
|
||||||
|
## Admin
|
||||||
|
|
||||||
|
The first account is an admin. An admin can change global settings (scanning interval, quota,
|
||||||
|
retention, media types, download folder), a feed's URL, folder and schedule, and who has an account
|
||||||
|
and who else is an admin, and read the log, which names everyone's feeds and sign-ins. Everyone else
|
||||||
|
gets the Settings and Log buttons hidden and a `403` if they ask anyway.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx user list # the admin column says who
|
||||||
|
```
|
||||||
580
src/db.rs
580
src/db.rs
@@ -80,6 +80,31 @@ CREATE TABLE IF NOT EXISTS users (
|
|||||||
created INTEGER NOT NULL
|
created INTEGER NOT NULL
|
||||||
);
|
);
|
||||||
|
|
||||||
|
-- What one person wants from a feed. The feed, its items and its files are shared; this
|
||||||
|
-- is the part that is not. NULL in a column means: follow the feed's own setting.
|
||||||
|
CREATE TABLE IF NOT EXISTS subscriptions (
|
||||||
|
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
feed_id TEXT NOT NULL,
|
||||||
|
keywords TEXT,
|
||||||
|
auto_download INTEGER,
|
||||||
|
allow_explicit INTEGER,
|
||||||
|
max_new_per_check INTEGER,
|
||||||
|
created INTEGER NOT NULL,
|
||||||
|
PRIMARY KEY (user_id, feed_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Read, starred and how far in. One row per person per item, created on first touch;
|
||||||
|
-- an item nobody has touched has no row at all, which is what unread means.
|
||||||
|
CREATE TABLE IF NOT EXISTS entry_state (
|
||||||
|
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
feed_id TEXT NOT NULL,
|
||||||
|
guid TEXT NOT NULL,
|
||||||
|
read INTEGER NOT NULL DEFAULT 0,
|
||||||
|
flagged INTEGER NOT NULL DEFAULT 0,
|
||||||
|
position INTEGER NOT NULL DEFAULT 0,
|
||||||
|
PRIMARY KEY (user_id, feed_id, guid)
|
||||||
|
);
|
||||||
|
|
||||||
CREATE TABLE IF NOT EXISTS sessions (
|
CREATE TABLE IF NOT EXISTS sessions (
|
||||||
token TEXT PRIMARY KEY,
|
token TEXT PRIMARY KEY,
|
||||||
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
@@ -88,6 +113,16 @@ CREATE TABLE IF NOT EXISTS sessions (
|
|||||||
);
|
);
|
||||||
";
|
";
|
||||||
|
|
||||||
|
/// One person's wants for one feed. `None` in a field means the feed's own setting stands.
|
||||||
|
#[derive(Debug, Clone, Default)]
|
||||||
|
pub struct Sub {
|
||||||
|
pub feed_id: String,
|
||||||
|
pub keywords: Option<Vec<String>>,
|
||||||
|
pub auto_download: Option<bool>,
|
||||||
|
pub allow_explicit: Option<bool>,
|
||||||
|
pub max_new_per_check: Option<i64>,
|
||||||
|
}
|
||||||
|
|
||||||
/// Someone who can sign in. `pass_hash` is None for an account that only ever arrives
|
/// Someone who can sign in. `pass_hash` is None for an account that only ever arrives
|
||||||
/// through the proxy.
|
/// through the proxy.
|
||||||
#[derive(Debug, Clone)]
|
#[derive(Debug, Clone)]
|
||||||
@@ -431,23 +466,31 @@ pub struct Candidate {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl Db {
|
impl Db {
|
||||||
/// Files on disk, flagged ones excluded, read before unread and oldest first within
|
/// Files on disk that may be deleted to get back under quota: starred by nobody,
|
||||||
/// each group.
|
/// with the ones everybody has finished going first, oldest first within each group.
|
||||||
///
|
///
|
||||||
/// The Python intended `read = 1 AND flagged = 0` but never achieved it (a missing
|
/// One file serves every subscriber, so both tests are about all of them: **anyone**
|
||||||
/// plistlib import and an `EntreiesData` typo meant the filter always threw). Requiring
|
/// starring it keeps it, and it only counts as read when **everyone** subscribed has
|
||||||
/// `read = 1` outright would be just as dead here, since nothing marks episodes read
|
/// read it. A file whose feed nobody subscribes to has no one left to keep it, so it
|
||||||
/// until a UI exists -- so `flagged` is the keep-forever marker, and `read` only decides
|
/// sorts with the read ones.
|
||||||
/// what goes first.
|
///
|
||||||
|
/// (The Python intended `read = 1 AND flagged = 0` but never achieved it -- a missing
|
||||||
|
/// plistlib import and an `EntreiesData` typo meant the filter always threw.)
|
||||||
pub fn reap_candidates(&self) -> Result<Vec<Candidate>> {
|
pub fn reap_candidates(&self) -> Result<Vec<Candidate>> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let mut stmt = conn.prepare(
|
let mut stmt = conn.prepare(
|
||||||
"SELECT e.id, e.url, e.path, e.bytes_done,
|
"SELECT e.id, e.url, e.path, e.bytes_done, coalesce(e.downloaded_at, 0),
|
||||||
coalesce(e.downloaded_at, 0), coalesce(n.read, 0), coalesce(n.flagged, 0)
|
CASE WHEN coalesce(readers.n, 0) >= coalesce(subs.n, 0) THEN 1 ELSE 0 END
|
||||||
FROM enclosures e
|
FROM enclosures e
|
||||||
LEFT JOIN entries n ON n.feed_id = e.feed_id AND n.guid = e.guid
|
LEFT JOIN (SELECT feed_id, count(*) n FROM subscriptions GROUP BY feed_id) subs
|
||||||
WHERE e.path IS NOT NULL AND coalesce(n.flagged, 0) = 0
|
ON subs.feed_id = e.feed_id
|
||||||
ORDER BY coalesce(n.read, 0) DESC, coalesce(e.downloaded_at, 0) ASC, e.id ASC",
|
LEFT JOIN (SELECT feed_id, guid, count(*) n FROM entry_state
|
||||||
|
WHERE read = 1 GROUP BY feed_id, guid) readers
|
||||||
|
ON readers.feed_id = e.feed_id AND readers.guid = e.guid
|
||||||
|
WHERE e.path IS NOT NULL
|
||||||
|
AND NOT EXISTS (SELECT 1 FROM entry_state s
|
||||||
|
WHERE s.feed_id = e.feed_id AND s.guid = e.guid AND s.flagged = 1)
|
||||||
|
ORDER BY 6 DESC, coalesce(e.downloaded_at, 0) ASC, e.id ASC",
|
||||||
)?;
|
)?;
|
||||||
let rows = stmt
|
let rows = stmt
|
||||||
.query_map([], |r| {
|
.query_map([], |r| {
|
||||||
@@ -493,24 +536,40 @@ impl Db {
|
|||||||
pub fn prune_entries(&self, older_than: i64) -> Result<usize> {
|
pub fn prune_entries(&self, older_than: i64) -> Result<usize> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let n = conn.execute(
|
let n = conn.execute(
|
||||||
"DELETE FROM entries WHERE flagged = 0
|
"DELETE FROM entries
|
||||||
AND coalesce(published, first_seen) < ?1
|
WHERE coalesce(published, first_seen) < ?1
|
||||||
AND NOT EXISTS (
|
AND NOT EXISTS (
|
||||||
SELECT 1 FROM enclosures e
|
SELECT 1 FROM enclosures e
|
||||||
WHERE e.feed_id = entries.feed_id AND e.guid = entries.guid
|
WHERE e.feed_id = entries.feed_id AND e.guid = entries.guid
|
||||||
AND e.path IS NOT NULL)",
|
AND e.path IS NOT NULL)
|
||||||
|
-- Starred by anyone keeps it, the same rule the reaper follows.
|
||||||
|
AND NOT EXISTS (
|
||||||
|
SELECT 1 FROM entry_state s
|
||||||
|
WHERE s.feed_id = entries.feed_id AND s.guid = entries.guid
|
||||||
|
AND s.flagged = 1)",
|
||||||
[older_than],
|
[older_than],
|
||||||
)?;
|
)?;
|
||||||
|
// Whatever went takes everyone's read state with it, rather than leaving rows
|
||||||
|
// pointing at an item that no longer exists.
|
||||||
|
conn.execute(
|
||||||
|
"DELETE FROM entry_state WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM entries e
|
||||||
|
WHERE e.feed_id = entry_state.feed_id AND e.guid = entry_state.guid)",
|
||||||
|
[],
|
||||||
|
)?;
|
||||||
Ok(n)
|
Ok(n)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Db {
|
impl Db {
|
||||||
pub fn unread_count(&self, feed_id: &str) -> Result<i64> {
|
pub fn unread_count(&self, user_id: i64, feed_id: &str) -> Result<i64> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
Ok(conn.query_row(
|
Ok(conn.query_row(
|
||||||
"SELECT count(*) FROM entries WHERE feed_id = ?1 AND read = 0",
|
"SELECT count(*) FROM entries e
|
||||||
[feed_id],
|
LEFT JOIN entry_state s
|
||||||
|
ON s.user_id = ?2 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
||||||
|
WHERE e.feed_id = ?1 AND coalesce(s.read, 0) = 0",
|
||||||
|
rusqlite::params![feed_id, user_id],
|
||||||
|r| r.get(0),
|
|r| r.get(0),
|
||||||
)?)
|
)?)
|
||||||
}
|
}
|
||||||
@@ -550,6 +609,36 @@ pub struct EntryRow {
|
|||||||
const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2
|
const SEARCH: &str = "(?2 = '' OR lower(coalesce(e.title, '')) LIKE ?2
|
||||||
OR lower(coalesce(e.description, '')) LIKE ?2)";
|
OR lower(coalesce(e.description, '')) LIKE ?2)";
|
||||||
|
|
||||||
|
/// Which feeds a query covers: one, or every feed the person subscribes to. Both forms
|
||||||
|
/// mention `?1`, since binding a parameter the statement does not use is an error.
|
||||||
|
fn scope_sql(feed_id: Option<&str>, user_param: u8) -> String {
|
||||||
|
match feed_id {
|
||||||
|
Some(_) => "e.feed_id = ?1".into(),
|
||||||
|
None => format!(
|
||||||
|
"?1 IS NULL AND e.feed_id IN (SELECT feed_id FROM subscriptions WHERE user_id = ?{user_param})"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The item table's ORDER BY. The column name picks one of these fixed expressions, so nothing
|
||||||
|
/// the caller sends reaches the query, and anything unrecognised is newest first. Ties fall back
|
||||||
|
/// to newest first too, so a page boundary is stable across "Load more".
|
||||||
|
///
|
||||||
|
/// ponytail: file type and size look at the item's first and largest file. The row shows the file
|
||||||
|
/// it summarises, which is almost always that one; sort by that one if they ever disagree.
|
||||||
|
pub fn order_sql(col: &str, dir: &str) -> String {
|
||||||
|
let expr = match col {
|
||||||
|
"kept" => "coalesce(s.flagged, 0)",
|
||||||
|
"title" => "lower(coalesce(e.title, ''))",
|
||||||
|
"feed" => "(SELECT lower(coalesce(f.title, f.id)) FROM feeds f WHERE f.id = e.feed_id)",
|
||||||
|
"type" => "(SELECT min(x.mime) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
|
||||||
|
"size" => "(SELECT max(x.length) FROM enclosures x WHERE x.feed_id = e.feed_id AND x.guid = e.guid)",
|
||||||
|
_ => "coalesce(e.published, e.first_seen)",
|
||||||
|
};
|
||||||
|
let dir = if dir == "asc" { "ASC" } else { "DESC" };
|
||||||
|
format!("{expr} {dir}, coalesce(e.published, e.first_seen) DESC, e.rowid DESC")
|
||||||
|
}
|
||||||
|
|
||||||
/// Which slice of a feed the UI is asking for.
|
/// Which slice of a feed the UI is asking for.
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub enum Filter {
|
pub enum Filter {
|
||||||
@@ -574,8 +663,8 @@ impl Filter {
|
|||||||
fn sql(self) -> &'static str {
|
fn sql(self) -> &'static str {
|
||||||
match self {
|
match self {
|
||||||
Self::All => "1=1",
|
Self::All => "1=1",
|
||||||
Self::Unread => "e.read = 0",
|
Self::Unread => "coalesce(s.read, 0) = 0",
|
||||||
Self::Flagged => "e.flagged = 1",
|
Self::Flagged => "coalesce(s.flagged, 0) = 1",
|
||||||
Self::Downloaded => {
|
Self::Downloaded => {
|
||||||
"EXISTS (SELECT 1 FROM enclosures x
|
"EXISTS (SELECT 1 FROM enclosures x
|
||||||
WHERE x.feed_id = e.feed_id AND x.guid = e.guid AND x.path IS NOT NULL)"
|
WHERE x.feed_id = e.feed_id AND x.guid = e.guid AND x.path IS NOT NULL)"
|
||||||
@@ -602,23 +691,43 @@ impl Db {
|
|||||||
/// `search` matches title and description, case-insensitively.
|
/// `search` matches title and description, case-insensitively.
|
||||||
pub fn entries(
|
pub fn entries(
|
||||||
&self,
|
&self,
|
||||||
|
user_id: i64,
|
||||||
feed_id: &str,
|
feed_id: &str,
|
||||||
filter: Filter,
|
filter: Filter,
|
||||||
search: Option<&str>,
|
search: Option<&str>,
|
||||||
offset: i64,
|
offset: i64,
|
||||||
limit: i64,
|
limit: i64,
|
||||||
|
) -> Result<Vec<EntryRow>> {
|
||||||
|
self.entries_in(user_id, Some(feed_id), filter, search, offset, limit, &order_sql("published", "desc"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `entries` for one feed, or across every feed the person subscribes to when `feed_id`
|
||||||
|
/// is None: the All Subscriptions view.
|
||||||
|
pub fn entries_in(
|
||||||
|
&self,
|
||||||
|
user_id: i64,
|
||||||
|
feed_id: Option<&str>,
|
||||||
|
filter: Filter,
|
||||||
|
search: Option<&str>,
|
||||||
|
offset: i64,
|
||||||
|
limit: i64,
|
||||||
|
order: &str,
|
||||||
) -> Result<Vec<EntryRow>> {
|
) -> Result<Vec<EntryRow>> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let like = search
|
let like = search
|
||||||
.map(|q| format!("%{}%", q.trim().to_lowercase()))
|
.map(|q| format!("%{}%", q.trim().to_lowercase()))
|
||||||
.unwrap_or_default();
|
.unwrap_or_default();
|
||||||
let sql = format!(
|
let sql = format!(
|
||||||
"SELECT e.guid, e.feed_id, e.title, e.link, e.published, e.description, e.read,
|
"SELECT e.guid, e.feed_id, e.title, e.link, e.published, e.description,
|
||||||
e.flagged, e.image, e.duration, e.episode, e.season, e.position
|
coalesce(s.read, 0), coalesce(s.flagged, 0), e.image, e.duration,
|
||||||
|
e.episode, e.season, coalesce(s.position, 0)
|
||||||
FROM entries e
|
FROM entries e
|
||||||
WHERE e.feed_id = ?1 AND {} AND {SEARCH}
|
LEFT JOIN entry_state s
|
||||||
ORDER BY coalesce(e.published, e.first_seen) DESC, e.rowid DESC
|
ON s.user_id = ?5 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
||||||
|
WHERE {} AND {} AND {SEARCH}
|
||||||
|
ORDER BY {order}
|
||||||
LIMIT ?4 OFFSET ?3",
|
LIMIT ?4 OFFSET ?3",
|
||||||
|
scope_sql(feed_id, 5),
|
||||||
filter.sql()
|
filter.sql()
|
||||||
);
|
);
|
||||||
let mut stmt = conn.prepare(&sql)?;
|
let mut stmt = conn.prepare(&sql)?;
|
||||||
@@ -641,21 +750,21 @@ impl Db {
|
|||||||
})
|
})
|
||||||
};
|
};
|
||||||
let mut rows: Vec<EntryRow> = stmt
|
let mut rows: Vec<EntryRow> = stmt
|
||||||
.query_map(rusqlite::params![feed_id, like, offset, limit], map)?
|
.query_map(rusqlite::params![feed_id, like, offset, limit, user_id], map)?
|
||||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||||
|
|
||||||
if rows.is_empty() {
|
if rows.is_empty() {
|
||||||
return Ok(rows);
|
return Ok(rows);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Only the guids on this page, so a feed with thousands of entries stays cheap.
|
// Only the guids on this page, so a feed with thousands of entries stays cheap. A page
|
||||||
|
// can span feeds, so each file is matched to its row by feed as well as guid, below.
|
||||||
let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(",");
|
let placeholders = std::iter::repeat_n("?", rows.len()).collect::<Vec<_>>().join(",");
|
||||||
let sql = format!(
|
let sql = format!(
|
||||||
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error
|
"SELECT id, feed_id, guid, url, mime, length, path, state, last_error
|
||||||
FROM enclosures WHERE feed_id = ? AND guid IN ({placeholders}) ORDER BY id"
|
FROM enclosures WHERE guid IN ({placeholders}) ORDER BY id"
|
||||||
);
|
);
|
||||||
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len() + 1);
|
let mut params: Vec<&dyn rusqlite::ToSql> = Vec::with_capacity(rows.len());
|
||||||
params.push(&feed_id);
|
|
||||||
for row in &rows {
|
for row in &rows {
|
||||||
params.push(&row.guid);
|
params.push(&row.guid);
|
||||||
}
|
}
|
||||||
@@ -677,7 +786,7 @@ impl Db {
|
|||||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||||
|
|
||||||
for enc in encs {
|
for enc in encs {
|
||||||
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid) {
|
if let Some(row) = rows.iter_mut().find(|r| r.guid == enc.guid && r.feed_id == enc.feed_id) {
|
||||||
row.enclosures.push(enc);
|
row.enclosures.push(enc);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -685,29 +794,254 @@ impl Db {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// How many entries match, so the UI knows whether there is another page.
|
/// How many entries match, so the UI knows whether there is another page.
|
||||||
pub fn count_entries(&self, feed_id: &str, filter: Filter, search: Option<&str>) -> Result<i64> {
|
pub fn count_entries(
|
||||||
|
&self,
|
||||||
|
user_id: i64,
|
||||||
|
feed_id: &str,
|
||||||
|
filter: Filter,
|
||||||
|
search: Option<&str>,
|
||||||
|
) -> Result<i64> {
|
||||||
|
self.count_in(user_id, Some(feed_id), filter, search)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `count_entries` for one feed, or across every feed the person subscribes to.
|
||||||
|
pub fn count_in(
|
||||||
|
&self,
|
||||||
|
user_id: i64,
|
||||||
|
feed_id: Option<&str>,
|
||||||
|
filter: Filter,
|
||||||
|
search: Option<&str>,
|
||||||
|
) -> Result<i64> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let like = search
|
let like = search
|
||||||
.map(|q| format!("%{}%", q.trim().to_lowercase()))
|
.map(|q| format!("%{}%", q.trim().to_lowercase()))
|
||||||
.unwrap_or_default();
|
.unwrap_or_default();
|
||||||
let sql = format!(
|
let sql = format!(
|
||||||
"SELECT count(*) FROM entries e WHERE e.feed_id = ?1 AND {} AND {SEARCH}",
|
"SELECT count(*) FROM entries e
|
||||||
|
LEFT JOIN entry_state s
|
||||||
|
ON s.user_id = ?3 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
||||||
|
WHERE {} AND {} AND {SEARCH}",
|
||||||
|
scope_sql(feed_id, 3),
|
||||||
filter.sql()
|
filter.sql()
|
||||||
);
|
);
|
||||||
Ok(conn.query_row(&sql, rusqlite::params![feed_id, like], |r| r.get(0))?)
|
Ok(conn.query_row(&sql, rusqlite::params![feed_id, like, user_id], |r| r.get(0))?)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Where playback got to, so it resumes there next time.
|
/// Where playback got to, so it resumes there next time -- for this listener only.
|
||||||
pub fn set_position(&self, feed_id: &str, guid: &str, secs: i64) -> Result<()> {
|
pub fn set_position(&self, user_id: i64, feed_id: &str, guid: &str, secs: i64) -> Result<()> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE entries SET position = ?3 WHERE feed_id = ?1 AND guid = ?2",
|
"INSERT INTO entry_state (user_id, feed_id, guid, position) VALUES (?1, ?2, ?3, ?4)
|
||||||
rusqlite::params![feed_id, guid, secs.max(0)],
|
ON CONFLICT(user_id, feed_id, guid) DO UPDATE SET position = excluded.position",
|
||||||
|
rusqlite::params![user_id, feed_id, guid, secs.max(0)],
|
||||||
)?;
|
)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Marks every entry in a feed read, for the "mark all read" button.
|
/// Marks every entry in a feed read, for the "mark all read" button.
|
||||||
|
/// Moves a single-user library onto an account: everything read, starred or part-played
|
||||||
|
/// becomes that person's, and they subscribe to every feed already in the catalogue.
|
||||||
|
/// Runs once -- the moment there is a first account and no subscriptions yet.
|
||||||
|
pub fn adopt_existing_library(&self, user_id: i64, catalogue: &[String]) -> Result<usize> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let already: i64 =
|
||||||
|
conn.query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0))?;
|
||||||
|
if already > 0 {
|
||||||
|
return Ok(0);
|
||||||
|
}
|
||||||
|
let moved = conn.execute(
|
||||||
|
"INSERT INTO entry_state (user_id, feed_id, guid, read, flagged, position)
|
||||||
|
SELECT ?1, feed_id, guid, read, flagged, position FROM entries
|
||||||
|
WHERE read = 1 OR flagged = 1 OR position > 0
|
||||||
|
ON CONFLICT(user_id, feed_id, guid) DO NOTHING",
|
||||||
|
[user_id],
|
||||||
|
)?;
|
||||||
|
for id in catalogue {
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
|
||||||
|
params![user_id, id, now()],
|
||||||
|
)?;
|
||||||
|
}
|
||||||
|
// Feeds that exist only in the database (OPML children) count too.
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created)
|
||||||
|
SELECT ?1, id, ?2 FROM feeds",
|
||||||
|
params![user_id, now()],
|
||||||
|
)?;
|
||||||
|
Ok(moved)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- subscriptions ----
|
||||||
|
|
||||||
|
/// What this person wants from a feed. Absent means they do not subscribe at all.
|
||||||
|
pub fn subscription(&self, user_id: i64, feed_id: &str) -> Result<Option<Sub>> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let mut stmt = conn.prepare(
|
||||||
|
"SELECT keywords, auto_download, allow_explicit, max_new_per_check
|
||||||
|
FROM subscriptions WHERE user_id = ?1 AND feed_id = ?2",
|
||||||
|
)?;
|
||||||
|
let mut rows = stmt.query(params![user_id, feed_id])?;
|
||||||
|
Ok(match rows.next()? {
|
||||||
|
Some(r) => Some(Sub {
|
||||||
|
feed_id: feed_id.to_string(),
|
||||||
|
keywords: r
|
||||||
|
.get::<_, Option<String>>(0)?
|
||||||
|
.and_then(|j| serde_json::from_str(&j).ok()),
|
||||||
|
auto_download: r.get::<_, Option<i64>>(1)?.map(|v| v != 0),
|
||||||
|
allow_explicit: r.get::<_, Option<i64>>(2)?.map(|v| v != 0),
|
||||||
|
max_new_per_check: r.get(3)?,
|
||||||
|
}),
|
||||||
|
None => None,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every feed this person subscribes to, with their settings.
|
||||||
|
pub fn subscriptions_for(&self, user_id: i64) -> Result<Vec<Sub>> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let mut stmt = conn.prepare(
|
||||||
|
"SELECT feed_id, keywords, auto_download, allow_explicit, max_new_per_check
|
||||||
|
FROM subscriptions WHERE user_id = ?1",
|
||||||
|
)?;
|
||||||
|
let out = stmt
|
||||||
|
.query_map([user_id], |r| {
|
||||||
|
Ok(Sub {
|
||||||
|
feed_id: r.get(0)?,
|
||||||
|
keywords: r
|
||||||
|
.get::<_, Option<String>>(1)?
|
||||||
|
.and_then(|j| serde_json::from_str(&j).ok()),
|
||||||
|
auto_download: r.get::<_, Option<i64>>(2)?.map(|v| v != 0),
|
||||||
|
allow_explicit: r.get::<_, Option<i64>>(3)?.map(|v| v != 0),
|
||||||
|
max_new_per_check: r.get(4)?,
|
||||||
|
})
|
||||||
|
})?
|
||||||
|
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Everyone's settings for one feed. The scanner merges these into what it fetches
|
||||||
|
/// and downloads, since one file serves the lot.
|
||||||
|
pub fn subscribers(&self, feed_id: &str) -> Result<Vec<Sub>> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let mut stmt = conn.prepare(
|
||||||
|
"SELECT keywords, auto_download, allow_explicit, max_new_per_check
|
||||||
|
FROM subscriptions WHERE feed_id = ?1",
|
||||||
|
)?;
|
||||||
|
let out = stmt
|
||||||
|
.query_map([feed_id], |r| {
|
||||||
|
Ok(Sub {
|
||||||
|
feed_id: feed_id.to_string(),
|
||||||
|
keywords: r
|
||||||
|
.get::<_, Option<String>>(0)?
|
||||||
|
.and_then(|j| serde_json::from_str(&j).ok()),
|
||||||
|
auto_download: r.get::<_, Option<i64>>(1)?.map(|v| v != 0),
|
||||||
|
allow_explicit: r.get::<_, Option<i64>>(2)?.map(|v| v != 0),
|
||||||
|
max_new_per_check: r.get(3)?,
|
||||||
|
})
|
||||||
|
})?
|
||||||
|
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Subscribers per feed, for the whole catalogue in one query -- the feed list would
|
||||||
|
/// otherwise ask once per feed.
|
||||||
|
pub fn subscriber_counts(&self) -> Result<std::collections::HashMap<String, i64>> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let mut stmt =
|
||||||
|
conn.prepare("SELECT feed_id, count(*) FROM subscriptions GROUP BY feed_id")?;
|
||||||
|
let out = stmt
|
||||||
|
.query_map([], |r| Ok((r.get::<_, String>(0)?, r.get::<_, i64>(1)?)))?
|
||||||
|
.collect::<rusqlite::Result<std::collections::HashMap<_, _>>>()?;
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Who else would miss this file: subscribers other than `user_id` who have starred
|
||||||
|
/// the item or have not read it yet. Deleting is deleting their copy too.
|
||||||
|
pub fn others_wanting(&self, enclosure_id: i64, user_id: i64) -> Result<(i64, i64)> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let mut stmt = conn.prepare(
|
||||||
|
"SELECT
|
||||||
|
sum(CASE WHEN coalesce(st.flagged, 0) = 1 THEN 1 ELSE 0 END),
|
||||||
|
sum(CASE WHEN coalesce(st.read, 0) = 0 THEN 1 ELSE 0 END)
|
||||||
|
FROM enclosures e
|
||||||
|
JOIN subscriptions s ON s.feed_id = e.feed_id AND s.user_id != ?2
|
||||||
|
LEFT JOIN entry_state st
|
||||||
|
ON st.user_id = s.user_id AND st.feed_id = e.feed_id AND st.guid = e.guid
|
||||||
|
WHERE e.id = ?1",
|
||||||
|
)?;
|
||||||
|
let (starred, unread) = stmt.query_row(params![enclosure_id, user_id], |r| {
|
||||||
|
Ok((r.get::<_, Option<i64>>(0)?.unwrap_or(0), r.get::<_, Option<i64>>(1)?.unwrap_or(0)))
|
||||||
|
})?;
|
||||||
|
Ok((starred, unread))
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn subscribe(&self, user_id: i64, feed_id: &str) -> Result<()> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
|
||||||
|
params![user_id, feed_id, now()],
|
||||||
|
)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn unsubscribe(&self, user_id: i64, feed_id: &str) -> Result<()> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
conn.execute(
|
||||||
|
"DELETE FROM subscriptions WHERE user_id = ?1 AND feed_id = ?2",
|
||||||
|
params![user_id, feed_id],
|
||||||
|
)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How many people want this feed. Nobody means it stops being scanned.
|
||||||
|
pub fn subscriber_count(&self, feed_id: &str) -> Result<i64> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
Ok(conn.query_row(
|
||||||
|
"SELECT count(*) FROM subscriptions WHERE feed_id = ?1",
|
||||||
|
[feed_id],
|
||||||
|
|r| r.get(0),
|
||||||
|
)?)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Overwrites one person's settings for a feed. A None field means: follow the feed.
|
||||||
|
pub fn set_subscription(&self, user_id: i64, sub: &Sub) -> Result<()> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let kw = sub
|
||||||
|
.keywords
|
||||||
|
.as_ref()
|
||||||
|
.map(|k| serde_json::to_string(k))
|
||||||
|
.transpose()?;
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO subscriptions
|
||||||
|
(user_id, feed_id, keywords, auto_download, allow_explicit, max_new_per_check, created)
|
||||||
|
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
|
||||||
|
ON CONFLICT(user_id, feed_id) DO UPDATE SET
|
||||||
|
keywords = excluded.keywords,
|
||||||
|
auto_download = excluded.auto_download,
|
||||||
|
allow_explicit = excluded.allow_explicit,
|
||||||
|
max_new_per_check = excluded.max_new_per_check",
|
||||||
|
params![
|
||||||
|
user_id,
|
||||||
|
sub.feed_id,
|
||||||
|
kw,
|
||||||
|
sub.auto_download.map(|v| v as i64),
|
||||||
|
sub.allow_explicit.map(|v| v as i64),
|
||||||
|
sub.max_new_per_check,
|
||||||
|
now()
|
||||||
|
],
|
||||||
|
)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Feeds with at least one subscriber. What the scanner walks.
|
||||||
|
pub fn subscribed_feed_ids(&self) -> Result<Vec<String>> {
|
||||||
|
let conn = self.conn.lock().unwrap();
|
||||||
|
let mut stmt = conn.prepare("SELECT DISTINCT feed_id FROM subscriptions")?;
|
||||||
|
let out = stmt
|
||||||
|
.query_map([], |r| r.get::<_, String>(0))?
|
||||||
|
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
// ---- users and sessions ----
|
// ---- users and sessions ----
|
||||||
|
|
||||||
pub fn create_user(&self, name: &str, pass_hash: Option<&str>, admin: bool) -> Result<i64> {
|
pub fn create_user(&self, name: &str, pass_hash: Option<&str>, admin: bool) -> Result<i64> {
|
||||||
@@ -827,11 +1161,19 @@ impl Db {
|
|||||||
|
|
||||||
/// Marks every entry of the given feeds read. Takes a list because an OPML subscription
|
/// Marks every entry of the given feeds read. Takes a list because an OPML subscription
|
||||||
/// holds no entries itself -- marking it read means the feeds inside it.
|
/// holds no entries itself -- marking it read means the feeds inside it.
|
||||||
pub fn mark_all_read(&self, feed_ids: &[String]) -> Result<usize> {
|
pub fn mark_all_read(&self, user_id: i64, feed_ids: &[String]) -> Result<usize> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let mut n = 0;
|
let mut n = 0;
|
||||||
for id in feed_ids {
|
for id in feed_ids {
|
||||||
n += conn.execute("UPDATE entries SET read = 1 WHERE feed_id = ?1 AND read = 0", [id])?;
|
n += conn.execute(
|
||||||
|
"INSERT INTO entry_state (user_id, feed_id, guid, read)
|
||||||
|
SELECT ?1, e.feed_id, e.guid, 1 FROM entries e
|
||||||
|
LEFT JOIN entry_state s
|
||||||
|
ON s.user_id = ?1 AND s.feed_id = e.feed_id AND s.guid = e.guid
|
||||||
|
WHERE e.feed_id = ?2 AND coalesce(s.read, 0) = 0
|
||||||
|
ON CONFLICT(user_id, feed_id, guid) DO UPDATE SET read = 1",
|
||||||
|
rusqlite::params![user_id, id],
|
||||||
|
)?;
|
||||||
}
|
}
|
||||||
Ok(n)
|
Ok(n)
|
||||||
}
|
}
|
||||||
@@ -876,14 +1218,28 @@ impl Db {
|
|||||||
.optional()?)
|
.optional()?)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `read` and `flagged` finally get a writer: retention orders by them.
|
/// Read and starred, per person. The row is created on first touch.
|
||||||
pub fn set_entry_flag(&self, feed_id: &str, guid: &str, field: EntryFlag, on: bool) -> Result<()> {
|
pub fn set_entry_flag(
|
||||||
|
&self,
|
||||||
|
user_id: i64,
|
||||||
|
feed_id: &str,
|
||||||
|
guid: &str,
|
||||||
|
field: EntryFlag,
|
||||||
|
on: bool,
|
||||||
|
) -> Result<()> {
|
||||||
let conn = self.conn.lock().unwrap();
|
let conn = self.conn.lock().unwrap();
|
||||||
let sql = match field {
|
let col = match field {
|
||||||
EntryFlag::Read => "UPDATE entries SET read = ?3 WHERE feed_id = ?1 AND guid = ?2",
|
EntryFlag::Read => "read",
|
||||||
EntryFlag::Flagged => "UPDATE entries SET flagged = ?3 WHERE feed_id = ?1 AND guid = ?2",
|
EntryFlag::Flagged => "flagged",
|
||||||
};
|
};
|
||||||
conn.execute(sql, rusqlite::params![feed_id, guid, on as i64])?;
|
conn.execute(
|
||||||
|
&format!(
|
||||||
|
"INSERT INTO entry_state (user_id, feed_id, guid, {col})
|
||||||
|
VALUES (?1, ?2, ?3, ?4)
|
||||||
|
ON CONFLICT(user_id, feed_id, guid) DO UPDATE SET {col} = excluded.{col}"
|
||||||
|
),
|
||||||
|
rusqlite::params![user_id, feed_id, guid, on as i64],
|
||||||
|
)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1021,6 +1377,99 @@ pub fn now() -> i64 {
|
|||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn every_sort_column_runs_and_orders_both_ways() {
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
db.exec_for_test(
|
||||||
|
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
|
||||||
|
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(1,'g',0);
|
||||||
|
INSERT INTO feeds (id, url, title) VALUES ('f','u','Zebra'),('g','v','Aardvark');
|
||||||
|
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
|
||||||
|
('f','a','banana',100),('g','b','Apple',200),('f','c','cherry',300);
|
||||||
|
INSERT INTO enclosures (id, feed_id, guid, url, mime, length, state) VALUES
|
||||||
|
(1,'f','a','u1','audio/mpeg',300,'pending'),(2,'g','b','u2','image/png',10,'pending'),
|
||||||
|
(3,'f','c','u3','video/mp4',2000,'pending');",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let order = |col: &str, dir: &str| -> Vec<String> {
|
||||||
|
db.entries_in(1, None, Filter::All, None, 0, 50, &order_sql(col, dir))
|
||||||
|
.unwrap()
|
||||||
|
.into_iter()
|
||||||
|
.map(|e| e.guid)
|
||||||
|
.collect()
|
||||||
|
};
|
||||||
|
assert_eq!(order("title", "asc"), ["b", "a", "c"], "Apple, banana, cherry: case folded");
|
||||||
|
assert_eq!(order("title", "desc"), ["c", "a", "b"]);
|
||||||
|
assert_eq!(order("feed", "asc"), ["b", "c", "a"], "Aardvark, then Zebra's newest first");
|
||||||
|
assert_eq!(order("type", "asc"), ["a", "b", "c"], "audio, image, video");
|
||||||
|
assert_eq!(order("size", "desc"), ["c", "a", "b"]);
|
||||||
|
assert_eq!(order("published", "desc"), ["c", "b", "a"]);
|
||||||
|
db.set_entry_flag(1, "f", "a", EntryFlag::Flagged, true).unwrap();
|
||||||
|
assert_eq!(order("kept", "desc")[0], "a");
|
||||||
|
// An unknown column or direction is newest first; the name itself never reaches the SQL.
|
||||||
|
assert_eq!(order("title; DROP TABLE entries", "sideways"), ["c", "b", "a"]);
|
||||||
|
assert!(!order_sql("x'; --", "asc").contains("x'"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn deleting_a_shared_file_asks_about_everyone_else() {
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
db.exec_for_test(
|
||||||
|
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0),(3,'kit',0,0);
|
||||||
|
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0),(3,'f',0);
|
||||||
|
INSERT INTO entries (feed_id, guid, first_seen) VALUES ('f','a',0);
|
||||||
|
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
||||||
|
(1,'f','a','u1','/tmp/a','done');",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
// Nobody has touched it: both others still have it unplayed.
|
||||||
|
assert_eq!(db.others_wanting(1, 1).unwrap(), (0, 2));
|
||||||
|
|
||||||
|
// Sam reads it, Kit stars it.
|
||||||
|
db.set_entry_flag(2, "f", "a", EntryFlag::Read, true).unwrap();
|
||||||
|
db.set_entry_flag(3, "f", "a", EntryFlag::Flagged, true).unwrap();
|
||||||
|
assert_eq!(db.others_wanting(1, 1).unwrap(), (1, 1), "one starred it, one has not played it");
|
||||||
|
|
||||||
|
// Asking as Kit, only Ray and Sam count -- and Kit's own star is not a reason to
|
||||||
|
// warn Kit.
|
||||||
|
db.set_entry_flag(1, "f", "a", EntryFlag::Read, true).unwrap();
|
||||||
|
assert_eq!(db.others_wanting(1, 3).unwrap(), (0, 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn read_state_belongs_to_one_person() {
|
||||||
|
let db = Db::memory().unwrap();
|
||||||
|
db.exec_for_test(
|
||||||
|
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
|
||||||
|
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
|
||||||
|
('f','a','One',100),('f','b','Two',200);",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(db.unread_count(1, "f").unwrap(), 2);
|
||||||
|
assert_eq!(db.unread_count(2, "f").unwrap(), 2);
|
||||||
|
|
||||||
|
db.set_entry_flag(1, "f", "a", EntryFlag::Read, true).unwrap();
|
||||||
|
assert_eq!(db.unread_count(1, "f").unwrap(), 1, "ray read one of them");
|
||||||
|
assert_eq!(db.unread_count(2, "f").unwrap(), 2, "sam has read nothing");
|
||||||
|
|
||||||
|
// Starring and position are just as private.
|
||||||
|
db.set_entry_flag(1, "f", "b", EntryFlag::Flagged, true).unwrap();
|
||||||
|
db.set_position(2, "f", "b", 42).unwrap();
|
||||||
|
let ray = db.entries(1, "f", Filter::All, None, 0, 50).unwrap();
|
||||||
|
let sam = db.entries(2, "f", Filter::All, None, 0, 50).unwrap();
|
||||||
|
let ray_b = ray.iter().find(|e| e.guid == "b").unwrap();
|
||||||
|
let sam_b = sam.iter().find(|e| e.guid == "b").unwrap();
|
||||||
|
assert!(ray_b.flagged && ray_b.position == 0);
|
||||||
|
assert!(!sam_b.flagged && sam_b.position == 42);
|
||||||
|
|
||||||
|
// Marking a whole feed read is likewise one person's business.
|
||||||
|
assert_eq!(db.mark_all_read(2, &["f".to_string()]).unwrap(), 2);
|
||||||
|
assert_eq!(db.unread_count(2, "f").unwrap(), 0);
|
||||||
|
assert_eq!(db.unread_count(1, "f").unwrap(), 1);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn schema_is_idempotent_and_summary_handles_unknown_feeds() {
|
fn schema_is_idempotent_and_summary_handles_unknown_feeds() {
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
@@ -1040,32 +1489,37 @@ mod tests {
|
|||||||
// so plain filtering failed with "Wrong number of parameters passed to query".
|
// so plain filtering failed with "Wrong number of parameters passed to query".
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO entries (feed_id, guid, title, description, first_seen, read, flagged) VALUES
|
"INSERT INTO entries (feed_id, guid, title, description, first_seen) VALUES
|
||||||
('f','a','Alpha dive','notes one',100,0,0),
|
('f','a','Alpha dive','notes one',100),
|
||||||
('f','b','Beta', 'notes two',200,1,0),
|
('f','b','Beta', 'notes two',200),
|
||||||
('f','c','Gamma dive','notes three',300,1,1);
|
('f','c','Gamma dive','notes three',300);
|
||||||
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
||||||
(1,'f','b','u1','/tmp/b','done');",
|
(1,'f','b','u1','/tmp/b','done');
|
||||||
|
-- Read and starred belong to a person now, so say which one.
|
||||||
|
INSERT INTO users (id, name, is_admin, created) VALUES (7,'reader',1,0);
|
||||||
|
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
|
||||||
|
(7,'f','b',1,0),
|
||||||
|
(7,'f','c',1,1);",
|
||||||
)
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
|
||||||
for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] {
|
for f in [Filter::All, Filter::Unread, Filter::Downloaded, Filter::Flagged] {
|
||||||
// Both paths must run without erroring, and agree with each other.
|
// Both paths must run without erroring, and agree with each other.
|
||||||
let rows = db.entries("f", f, None, 0, 50).unwrap();
|
let rows = db.entries(7, "f", f, None, 0, 50).unwrap();
|
||||||
let n = db.count_entries("f", f, None).unwrap();
|
let n = db.count_entries(7, "f", f, None).unwrap();
|
||||||
assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page");
|
assert_eq!(rows.len() as i64, n, "{f:?} count disagrees with the page");
|
||||||
|
|
||||||
let rows = db.entries("f", f, Some("dive"), 0, 50).unwrap();
|
let rows = db.entries(7, "f", f, Some("dive"), 0, 50).unwrap();
|
||||||
let n = db.count_entries("f", f, Some("dive")).unwrap();
|
let n = db.count_entries(7, "f", f, Some("dive")).unwrap();
|
||||||
assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees");
|
assert_eq!(rows.len() as i64, n, "{f:?} with search disagrees");
|
||||||
}
|
}
|
||||||
|
|
||||||
assert_eq!(db.count_entries("f", Filter::All, None).unwrap(), 3);
|
assert_eq!(db.count_entries(7, "f", Filter::All, None).unwrap(), 3);
|
||||||
assert_eq!(db.count_entries("f", Filter::Unread, None).unwrap(), 1);
|
assert_eq!(db.count_entries(7, "f", Filter::Unread, None).unwrap(), 1);
|
||||||
assert_eq!(db.count_entries("f", Filter::Downloaded, None).unwrap(), 1);
|
assert_eq!(db.count_entries(7, "f", Filter::Downloaded, None).unwrap(), 1);
|
||||||
assert_eq!(db.count_entries("f", Filter::Flagged, None).unwrap(), 1);
|
assert_eq!(db.count_entries(7, "f", Filter::Flagged, None).unwrap(), 1);
|
||||||
assert_eq!(db.count_entries("f", Filter::All, Some("dive")).unwrap(), 2);
|
assert_eq!(db.count_entries(7, "f", Filter::All, Some("dive")).unwrap(), 2);
|
||||||
assert_eq!(db.count_entries("f", Filter::All, Some("NOTES two")).unwrap(), 1,
|
assert_eq!(db.count_entries(7, "f", Filter::All, Some("NOTES two")).unwrap(), 1,
|
||||||
"search is case-insensitive and covers the description");
|
"search is case-insensitive and covers the description");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -62,10 +62,10 @@ impl Event {
|
|||||||
Event::DownloadDone { path, .. } => format!(" saved {path}"),
|
Event::DownloadDone { path, .. } => format!(" saved {path}"),
|
||||||
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
|
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
|
||||||
Event::Reaped { path, bytes } => {
|
Event::Reaped { path, bytes } => {
|
||||||
format!("reap {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
|
format!("deleted {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
|
||||||
}
|
}
|
||||||
Event::ReapDone { files, bytes } => format!(
|
Event::ReapDone { files, bytes } => format!(
|
||||||
"reaped {files} file(s), {:.1} MB",
|
"deleted {files} old file(s), {:.1} MB",
|
||||||
*bytes as f64 / 1_048_576.0
|
*bytes as f64 / 1_048_576.0
|
||||||
),
|
),
|
||||||
Event::Status { feeds, pending, downloaded } => {
|
Event::Status { feeds, pending, downloaded } => {
|
||||||
|
|||||||
298
src/main.rs
298
src/main.rs
@@ -344,6 +344,16 @@ async fn daemon(
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A library that predates accounts belongs to whoever was using it: the admin.
|
||||||
|
if let Some(admin) = ctx.db.users()?.into_iter().find(|u| u.is_admin) {
|
||||||
|
let catalogue: Vec<String> = ctx.cfg().feeds.keys().cloned().collect();
|
||||||
|
match ctx.db.adopt_existing_library(admin.id, &catalogue) {
|
||||||
|
Ok(0) => {}
|
||||||
|
Ok(n) => tracing::info!(user = %admin.name, entries = n, "adopted the existing library"),
|
||||||
|
Err(e) => tracing::error!(error = %e, "could not adopt the existing library"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
match migrate_opml_children(&ctx) {
|
match migrate_opml_children(&ctx) {
|
||||||
Ok(n) if n > 0 => tracing::info!(count = n, "moved OPML feeds out of config.toml into the database"),
|
Ok(n) if n > 0 => tracing::info!(count = n, "moved OPML feeds out of config.toml into the database"),
|
||||||
Ok(_) => {}
|
Ok(_) => {}
|
||||||
@@ -592,44 +602,94 @@ fn rm(ctx: &Ctx, config_path: &std::path::Path, feed: &str) -> Result<()> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async fn import(ctx: &Ctx, config_path: &std::path::Path, file: &std::path::Path) -> Result<()> {
|
async fn import(ctx: &Ctx, config_path: &std::path::Path, file: &std::path::Path) -> Result<()> {
|
||||||
let mut cfg = (*ctx.cfg()).clone();
|
|
||||||
let text = std::fs::read_to_string(file)
|
let text = std::fs::read_to_string(file)
|
||||||
.with_context(|| format!("reading {}", file.display()))?;
|
.with_context(|| format!("reading {}", file.display()))?;
|
||||||
let doc = opml::OPML::from_str(&text).map_err(|e| anyhow::anyhow!("parsing OPML: {e}"))?;
|
// The CLI speaks for the operator, as the shared web token does.
|
||||||
|
let admin = ctx
|
||||||
|
.db
|
||||||
|
.users()?
|
||||||
|
.into_iter()
|
||||||
|
.find(|u| u.is_admin)
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("no admin account to subscribe: ipx user add <name> --admin"))?;
|
||||||
|
let doc = opml::OPML::from_str(&text)
|
||||||
|
.map_err(|e| anyhow::anyhow!("{} is not OPML: {e}", file.display()))?;
|
||||||
|
let (added, had) = subscribe_opml(ctx, config_path, &doc, admin.id)?;
|
||||||
|
println!("subscribed {} to {added} feed(s); {had} already there", admin.name);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Subscribes one person to every feed in an OPML document, for the CLI and the web alike.
|
||||||
|
/// A feed already in the catalogue costs nothing; an unknown one is added under the OPML's
|
||||||
|
/// title rather than refetching each. Returns (newly subscribed, already subscribed).
|
||||||
|
///
|
||||||
|
/// Before accounts, importing only added unknown URLs to config.toml. Once subscriptions
|
||||||
|
/// decided what each person sees, that imported nothing at all for a feed someone else
|
||||||
|
/// already had, and a new one had no subscriber, so it was never scanned.
|
||||||
|
///
|
||||||
|
/// The caller parses the document, so each refuses a file that is not OPML in its own terms,
|
||||||
|
/// before anything is touched: a 400 from the web, a message from the CLI.
|
||||||
|
pub fn subscribe_opml(
|
||||||
|
ctx: &Ctx,
|
||||||
|
config_path: &std::path::Path,
|
||||||
|
doc: &opml::OPML,
|
||||||
|
user_id: i64,
|
||||||
|
) -> Result<(usize, usize)> {
|
||||||
let mut found = vec![];
|
let mut found = vec![];
|
||||||
collect_outlines(&doc.body.outlines, &mut found);
|
collect_outlines(&doc.body.outlines, &mut found);
|
||||||
|
|
||||||
let mut added = 0;
|
let known = subscriptions(ctx)?;
|
||||||
|
let mut cfg = (*ctx.cfg()).clone();
|
||||||
|
let mut ids = vec![];
|
||||||
|
let mut grew = false;
|
||||||
for (title, url) in found {
|
for (title, url) in found {
|
||||||
if cfg.feeds.values().any(|f| f.url == url) {
|
let existing = known
|
||||||
continue;
|
.iter()
|
||||||
}
|
.find(|s| s.cfg.url == url)
|
||||||
// Name it from the OPML title rather than refetching every feed.
|
.map(|s| s.id.clone())
|
||||||
let id = config::unique_slug(&title, &cfg.feeds);
|
// The same URL listed twice in one file.
|
||||||
cfg.feeds.insert(
|
.or_else(|| cfg.feeds.iter().find(|(_, f)| f.url == url).map(|(id, _)| id.clone()));
|
||||||
id.clone(),
|
let id = match existing {
|
||||||
config::Feed {
|
Some(id) => id,
|
||||||
url,
|
None => {
|
||||||
folder: None,
|
let id = config::unique_slug(&title, &cfg.feeds);
|
||||||
group: None,
|
cfg.feeds.insert(
|
||||||
media_types: None,
|
id.clone(),
|
||||||
schedule: None,
|
config::Feed {
|
||||||
keywords: vec![],
|
url,
|
||||||
allow_explicit: false,
|
folder: None,
|
||||||
auto_download: true,
|
group: None,
|
||||||
max_new_per_check: None,
|
media_types: None,
|
||||||
username: None,
|
schedule: None,
|
||||||
password: None,
|
keywords: vec![],
|
||||||
password_env: None,
|
allow_explicit: false,
|
||||||
},
|
auto_download: true,
|
||||||
);
|
max_new_per_check: None,
|
||||||
println!("added {id}");
|
username: None,
|
||||||
added += 1;
|
password: None,
|
||||||
|
password_env: None,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
grew = true;
|
||||||
|
id
|
||||||
|
}
|
||||||
|
};
|
||||||
|
ids.push(id);
|
||||||
}
|
}
|
||||||
cfg.save(config_path)?;
|
if grew {
|
||||||
println!("{added} feed(s) imported");
|
cfg.save(config_path)?;
|
||||||
Ok(())
|
ctx.reload_cfg(config_path)?;
|
||||||
|
}
|
||||||
|
|
||||||
|
let (mut added, mut had) = (0, 0);
|
||||||
|
for id in ids {
|
||||||
|
if ctx.db.subscription(user_id, &id)?.is_some() {
|
||||||
|
had += 1;
|
||||||
|
} else {
|
||||||
|
ctx.db.subscribe(user_id, &id)?;
|
||||||
|
added += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok((added, had))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// OPML nests feeds inside folder outlines, so this walks the whole tree.
|
/// OPML nests feeds inside folder outlines, so this walks the whole tree.
|
||||||
@@ -957,6 +1017,7 @@ async fn scan_one(
|
|||||||
parsed.image.as_deref(),
|
parsed.image.as_deref(),
|
||||||
)?;
|
)?;
|
||||||
|
|
||||||
|
let policy = policy_for(ctx, id, feed_cfg)?;
|
||||||
let mut scan = Scan::default();
|
let mut scan = Scan::default();
|
||||||
for entry in &parsed.entries {
|
for entry in &parsed.entries {
|
||||||
if ctx.db.record_entry(id, entry)? {
|
if ctx.db.record_entry(id, entry)? {
|
||||||
@@ -968,18 +1029,14 @@ async fn scan_one(
|
|||||||
}
|
}
|
||||||
// Filters run once, at discovery, and are recorded in `state`. The download
|
// Filters run once, at discovery, and are recorded in `state`. The download
|
||||||
// queue below is then just "everything still pending".
|
// queue below is then just "everything still pending".
|
||||||
if let Some(reason) = reject(&ctx.cfg(), feed_cfg, entry, enc) {
|
if let Some(reason) = reject(&ctx.cfg(), feed_cfg, &policy, entry, enc) {
|
||||||
ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?;
|
ctx.db.mark_enclosure(&enc.url, "skipped", Some(reason))?;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// An unset per-feed cap follows the global one; 0 there means unlimited.
|
let budget = policy.budget;
|
||||||
let budget = feed_cfg.max_new_per_check.unwrap_or_else(|| {
|
if policy.auto_download && budget > 0 {
|
||||||
let g = ctx.cfg().general.max_new_per_check;
|
|
||||||
if g == 0 { usize::MAX } else { g }
|
|
||||||
});
|
|
||||||
if feed_cfg.auto_download && budget > 0 {
|
|
||||||
let cfg = ctx.cfg();
|
let cfg = ctx.cfg();
|
||||||
let folder = download::folder_for(&cfg, id, feed_cfg, parsed.title.as_deref());
|
let folder = download::folder_for(&cfg, id, feed_cfg, parsed.title.as_deref());
|
||||||
let dest_dir = cfg.general.download_dir.join(&folder);
|
let dest_dir = cfg.general.download_dir.join(&folder);
|
||||||
@@ -1099,6 +1156,23 @@ async fn sync_opml(
|
|||||||
added.push(id);
|
added.push(id);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Whoever subscribes to the OPML subscribes to what it lists: that is what taking a
|
||||||
|
// subscription means. Their own feeds are untouched.
|
||||||
|
for id in ctx
|
||||||
|
.db
|
||||||
|
.managed_feeds()?
|
||||||
|
.iter()
|
||||||
|
.filter(|m| m.group_id == parent_id)
|
||||||
|
.map(|m| m.id.clone())
|
||||||
|
.chain(std::iter::once(parent_id.to_string()))
|
||||||
|
{
|
||||||
|
for user in ctx.db.users()? {
|
||||||
|
if ctx.db.subscription(user.id, parent_id)?.is_some() {
|
||||||
|
ctx.db.subscribe(user.id, &id)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Anything in this group the OPML no longer lists.
|
// Anything in this group the OPML no longer lists.
|
||||||
let mut removed = 0;
|
let mut removed = 0;
|
||||||
let mut kept = 0;
|
let mut kept = 0;
|
||||||
@@ -1125,11 +1199,12 @@ async fn sync_opml(
|
|||||||
fn reject(
|
fn reject(
|
||||||
cfg: &config::Config,
|
cfg: &config::Config,
|
||||||
feed_cfg: &config::Feed,
|
feed_cfg: &config::Feed,
|
||||||
|
policy: &Policy,
|
||||||
entry: &feed::Entry,
|
entry: &feed::Entry,
|
||||||
enc: &feed::Enclosure,
|
enc: &feed::Enclosure,
|
||||||
) -> Option<&'static str> {
|
) -> Option<&'static str> {
|
||||||
let url = enc.url.as_str();
|
let url = enc.url.as_str();
|
||||||
if !feed_cfg.auto_download {
|
if !policy.auto_download {
|
||||||
return Some("auto_download is off");
|
return Some("auto_download is off");
|
||||||
}
|
}
|
||||||
// Blog feeds put the article's header image in an <enclosure>; without this a text
|
// Blog feeds put the article's header image in an <enclosure>; without this a text
|
||||||
@@ -1141,7 +1216,7 @@ fn reject(
|
|||||||
if !config::wanted_media(enc.mime.as_deref(), wanted) {
|
if !config::wanted_media(enc.mime.as_deref(), wanted) {
|
||||||
return Some("not audio or video");
|
return Some("not audio or video");
|
||||||
}
|
}
|
||||||
if entry.explicit && !feed_cfg.allow_explicit {
|
if entry.explicit && !policy.allow_explicit {
|
||||||
return Some("explicit");
|
return Some("explicit");
|
||||||
}
|
}
|
||||||
let categories = entry.categories.join(" ");
|
let categories = entry.categories.join(" ");
|
||||||
@@ -1151,12 +1226,79 @@ fn reject(
|
|||||||
entry.description.as_deref().unwrap_or(""),
|
entry.description.as_deref().unwrap_or(""),
|
||||||
categories.as_str(),
|
categories.as_str(),
|
||||||
];
|
];
|
||||||
if !download::matches_keywords(&feed_cfg.keywords, &haystacks) {
|
// One file serves everyone subscribed, so an item is wanted if it is wanted by
|
||||||
|
// anyone: any one person's keyword set matching is enough.
|
||||||
|
let wanted_by_someone = policy.keyword_sets.is_empty()
|
||||||
|
|| policy
|
||||||
|
.keyword_sets
|
||||||
|
.iter()
|
||||||
|
.any(|set| download::matches_keywords(set, &haystacks));
|
||||||
|
if !wanted_by_someone {
|
||||||
return Some("no keyword match");
|
return Some("no keyword match");
|
||||||
}
|
}
|
||||||
None
|
None
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// What the scanner should do for a feed, merged across everyone subscribed to it. The
|
||||||
|
/// feed is fetched once and its files are downloaded once, so the merge is a union: if
|
||||||
|
/// one person wants a thing, it is fetched, and everyone else simply sees it listed.
|
||||||
|
///
|
||||||
|
/// With no subscribers at all -- a hand-written config entry nobody has claimed yet --
|
||||||
|
/// the feed's own settings stand, which is how a single-user install behaves.
|
||||||
|
pub struct Policy {
|
||||||
|
pub auto_download: bool,
|
||||||
|
pub allow_explicit: bool,
|
||||||
|
/// Empty means take everything. Otherwise one set per subscriber who filters.
|
||||||
|
pub keyword_sets: Vec<Vec<String>>,
|
||||||
|
pub budget: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn policy_for(ctx: &Ctx, id: &str, feed_cfg: &config::Feed) -> Result<Policy> {
|
||||||
|
let global = ctx.cfg().general.max_new_per_check;
|
||||||
|
Ok(merge_policy(&ctx.db.subscribers(id)?, feed_cfg, global))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Policy {
|
||||||
|
let cap = |n: Option<usize>| n.unwrap_or(if global == 0 { usize::MAX } else { global });
|
||||||
|
|
||||||
|
if subs.is_empty() {
|
||||||
|
return Policy {
|
||||||
|
auto_download: feed_cfg.auto_download,
|
||||||
|
allow_explicit: feed_cfg.allow_explicit,
|
||||||
|
keyword_sets: if feed_cfg.keywords.is_empty() {
|
||||||
|
vec![]
|
||||||
|
} else {
|
||||||
|
vec![feed_cfg.keywords.clone()]
|
||||||
|
},
|
||||||
|
budget: cap(feed_cfg.max_new_per_check),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut policy = Policy {
|
||||||
|
auto_download: false,
|
||||||
|
allow_explicit: false,
|
||||||
|
keyword_sets: vec![],
|
||||||
|
budget: 0,
|
||||||
|
};
|
||||||
|
for sub in subs {
|
||||||
|
if !sub.auto_download.unwrap_or(feed_cfg.auto_download) {
|
||||||
|
continue; // Not fetching for this person, so their wants add nothing.
|
||||||
|
}
|
||||||
|
policy.auto_download = true;
|
||||||
|
policy.allow_explicit |= sub.allow_explicit.unwrap_or(feed_cfg.allow_explicit);
|
||||||
|
policy.budget = policy
|
||||||
|
.budget
|
||||||
|
.max(cap(sub.max_new_per_check.map(|n| n as usize).or(feed_cfg.max_new_per_check)));
|
||||||
|
let kw = sub.keywords.clone().unwrap_or_else(|| feed_cfg.keywords.clone());
|
||||||
|
if kw.is_empty() {
|
||||||
|
// Somebody takes everything, so no filter can apply to the shared copy.
|
||||||
|
return Policy { keyword_sets: vec![], ..policy };
|
||||||
|
}
|
||||||
|
policy.keyword_sets.push(kw);
|
||||||
|
}
|
||||||
|
policy
|
||||||
|
}
|
||||||
|
|
||||||
async fn fetch_one(
|
async fn fetch_one(
|
||||||
ctx: &Arc<Ctx>,
|
ctx: &Arc<Ctx>,
|
||||||
feed_id: &str,
|
feed_id: &str,
|
||||||
@@ -1352,3 +1494,79 @@ fn duration(secs: u64) -> String {
|
|||||||
s => format!("{}d", s / 86_400),
|
s => format!("{}d", s / 86_400),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn feed() -> config::Feed {
|
||||||
|
// Whatever `ipx add` would write, which is the shape every code path sees.
|
||||||
|
let mut cfg = config::Config::default();
|
||||||
|
let f = add_one_cfg(&mut cfg, "http://x/f.xml", None, vec![]);
|
||||||
|
f
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The feed entry `add` builds, without the network round trip it does for a title.
|
||||||
|
fn add_one_cfg(
|
||||||
|
_cfg: &mut config::Config,
|
||||||
|
url: &str,
|
||||||
|
folder: Option<String>,
|
||||||
|
keywords: Vec<String>,
|
||||||
|
) -> config::Feed {
|
||||||
|
config::Feed {
|
||||||
|
url: url.into(),
|
||||||
|
folder,
|
||||||
|
keywords,
|
||||||
|
allow_explicit: false,
|
||||||
|
auto_download: true,
|
||||||
|
group: None,
|
||||||
|
media_types: None,
|
||||||
|
schedule: None,
|
||||||
|
max_new_per_check: None,
|
||||||
|
username: None,
|
||||||
|
password: None,
|
||||||
|
password_env: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sub(kw: Option<&[&str]>, auto: Option<bool>, max: Option<i64>) -> db::Sub {
|
||||||
|
db::Sub {
|
||||||
|
feed_id: "f".into(),
|
||||||
|
keywords: kw.map(|k| k.iter().map(|s| s.to_string()).collect()),
|
||||||
|
auto_download: auto,
|
||||||
|
allow_explicit: None,
|
||||||
|
max_new_per_check: max,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_shared_feed_is_fetched_for_whoever_wants_the_most() {
|
||||||
|
// Nobody subscribed: the feed's own settings stand, as in a single-user install.
|
||||||
|
let p = merge_policy(&[], &feed(), 3);
|
||||||
|
assert!(p.auto_download);
|
||||||
|
assert_eq!(p.budget, 3);
|
||||||
|
assert!(p.keyword_sets.is_empty());
|
||||||
|
|
||||||
|
// Two filters: an item wanted by either of them is fetched, since one file serves
|
||||||
|
// both. The larger per-scan cap wins for the same reason.
|
||||||
|
let p = merge_policy(
|
||||||
|
&[sub(Some(&["rust"]), None, Some(2)), sub(Some(&["sqlite"]), None, Some(9))],
|
||||||
|
&feed(),
|
||||||
|
3,
|
||||||
|
);
|
||||||
|
assert_eq!(p.keyword_sets.len(), 2);
|
||||||
|
assert_eq!(p.budget, 9);
|
||||||
|
|
||||||
|
// One person taking everything removes the filter for the shared copy.
|
||||||
|
let p = merge_policy(&[sub(Some(&["rust"]), None, None), sub(Some(&[]), None, None)], &feed(), 3);
|
||||||
|
assert!(p.keyword_sets.is_empty());
|
||||||
|
|
||||||
|
// Everyone has auto-download off: nothing is fetched automatically.
|
||||||
|
let p = merge_policy(&[sub(None, Some(false), None), sub(None, Some(false), None)], &feed(), 3);
|
||||||
|
assert!(!p.auto_download);
|
||||||
|
|
||||||
|
// One of them wants it, so it is fetched.
|
||||||
|
let p = merge_policy(&[sub(None, Some(false), None), sub(None, Some(true), None)], &feed(), 3);
|
||||||
|
assert!(p.auto_download);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -150,33 +150,52 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn query_never_offers_flagged_files_and_prefers_read_ones() {
|
fn query_never_offers_a_file_anyone_starred_and_prefers_ones_everyone_read() {
|
||||||
|
// One file serves both subscribers, so it takes both of them to release it.
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO entries (feed_id, guid, first_seen, read, flagged) VALUES
|
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
|
||||||
('f', 'keep', 0, 1, 1),
|
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0);
|
||||||
('f', 'unread', 0, 0, 0),
|
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
||||||
('f', 'read', 0, 1, 0);
|
('f', 'keep', 0),
|
||||||
|
('f', 'half', 0),
|
||||||
|
('f', 'unread', 0),
|
||||||
|
('f', 'read', 0);
|
||||||
|
-- Starred by one of the two, so it stays whatever the other thinks.
|
||||||
|
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
|
||||||
|
(1, 'f', 'keep', 1, 1),
|
||||||
|
(2, 'f', 'keep', 1, 0),
|
||||||
|
(1, 'f', 'half', 1, 0),
|
||||||
|
(1, 'f', 'read', 1, 0),
|
||||||
|
(2, 'f', 'read', 1, 0);
|
||||||
INSERT INTO enclosures (id, feed_id, guid, url, path, bytes_done, state, downloaded_at) VALUES
|
INSERT INTO enclosures (id, feed_id, guid, url, path, bytes_done, state, downloaded_at) VALUES
|
||||||
(1, 'f', 'keep', 'u1', '/tmp/keep', 10, 'done', 10),
|
(1, 'f', 'keep', 'u1', '/tmp/keep', 10, 'done', 10),
|
||||||
(2, 'f', 'unread', 'u2', '/tmp/unread', 10, 'done', 20),
|
(2, 'f', 'half', 'u2', '/tmp/half', 10, 'done', 20),
|
||||||
(3, 'f', 'read', 'u3', '/tmp/read', 10, 'done', 30);",
|
(3, 'f', 'unread', 'u3', '/tmp/unread', 10, 'done', 30),
|
||||||
|
(4, 'f', 'read', 'u4', '/tmp/read', 10, 'done', 40);",
|
||||||
)
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
|
||||||
let got: Vec<i64> = db.reap_candidates().unwrap().iter().map(|c| c.id).collect();
|
let got: Vec<i64> = db.reap_candidates().unwrap().iter().map(|c| c.id).collect();
|
||||||
assert_eq!(got, vec![3, 2], "flagged excluded; read goes before unread");
|
assert_eq!(
|
||||||
|
got,
|
||||||
|
vec![4, 2, 3],
|
||||||
|
"starred by anyone is never offered; read by everyone goes first, and one \
|
||||||
|
person still having it unread keeps it back with the unread ones"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn prune_keeps_entries_that_still_have_a_file() {
|
fn prune_keeps_entries_that_still_have_a_file() {
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO entries (feed_id, guid, first_seen, read, flagged) VALUES
|
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
|
||||||
('f', 'has-file', 100, 1, 0),
|
INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',1);
|
||||||
('f', 'no-file', 100, 1, 0),
|
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
||||||
('f', 'flagged', 100, 1, 1),
|
('f', 'has-file', 100),
|
||||||
('f', 'recent', 900, 1, 0);
|
('f', 'no-file', 100),
|
||||||
|
('f', 'flagged', 100),
|
||||||
|
('f', 'recent', 900);
|
||||||
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
||||||
(1, 'f', 'has-file', 'u1', '/tmp/x', 'done');",
|
(1, 'f', 'has-file', 'u1', '/tmp/x', 'done');",
|
||||||
)
|
)
|
||||||
|
|||||||
568
src/web.rs
568
src/web.rs
@@ -65,15 +65,22 @@ pub fn router(state: WebState) -> Router {
|
|||||||
.route("/api/feeds", get(feeds).post(add_feed))
|
.route("/api/feeds", get(feeds).post(add_feed))
|
||||||
.route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
|
.route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
|
||||||
.route("/api/feeds/{id}/entries", get(entries))
|
.route("/api/feeds/{id}/entries", get(entries))
|
||||||
|
.route("/api/entries", get(all_entries))
|
||||||
.route("/api/entries/{feed_id}/{guid}/flags", post(set_flags))
|
.route("/api/entries/{feed_id}/{guid}/flags", post(set_flags))
|
||||||
.route("/api/entries/{feed_id}/{guid}/position", post(set_position))
|
.route("/api/entries/{feed_id}/{guid}/position", post(set_position))
|
||||||
.route("/api/feeds/{id}/read-all", post(read_all))
|
.route("/api/feeds/{id}/read-all", post(read_all))
|
||||||
|
.route("/api/read-all", post(read_all_mine))
|
||||||
.route("/api/feeds/{id}/download-latest", post(download_latest))
|
.route("/api/feeds/{id}/download-latest", post(download_latest))
|
||||||
.route("/api/enclosures/{id}/download", post(download_now))
|
.route("/api/enclosures/{id}/download", post(download_now))
|
||||||
.route("/api/enclosures/{id}", delete(delete_file))
|
.route("/api/enclosures/{id}", delete(delete_file))
|
||||||
.route("/api/fetch", post(fetch_now))
|
.route("/api/fetch", post(fetch_now))
|
||||||
.route("/api/opml", get(export_opml).post(import_opml))
|
.route("/api/opml", get(export_opml).post(import_opml))
|
||||||
.route("/api/settings", get(get_settings).patch(patch_settings))
|
.route("/api/settings", get(get_settings).patch(patch_settings))
|
||||||
|
.route("/api/popular", get(get_popular))
|
||||||
|
.route("/api/directory", get(get_directory))
|
||||||
|
.route("/api/popular/{id}", post(subscribe_popular))
|
||||||
|
.route("/api/users", get(list_users).post(add_user))
|
||||||
|
.route("/api/users/{id}", patch(patch_user).delete(remove_user))
|
||||||
.route("/api/logs", get(logs))
|
.route("/api/logs", get(logs))
|
||||||
.route("/api/events", get(events))
|
.route("/api/events", get(events))
|
||||||
.route("/media/{id}", get(media))
|
.route("/media/{id}", get(media))
|
||||||
@@ -303,6 +310,125 @@ async fn me(user: crate::db::User) -> Json<serde_json::Value> {
|
|||||||
Json(serde_json::json!({ "name": user.name, "admin": user.is_admin }))
|
Json(serde_json::json!({ "name": user.name, "admin": user.is_admin }))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---- accounts: admin only ----
|
||||||
|
|
||||||
|
fn require_admin(user: &crate::db::User) -> Result<(), ApiError> {
|
||||||
|
if user.is_admin {
|
||||||
|
Ok(())
|
||||||
|
} else {
|
||||||
|
Err(ApiError::forbidden("only an admin manages accounts"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Demoting or removing this account would leave nobody able to manage anyone, and the only
|
||||||
|
/// way back would be `ipx user` on the box.
|
||||||
|
fn last_admin(users: &[crate::db::User], id: i64) -> bool {
|
||||||
|
let admins: Vec<i64> = users.iter().filter(|u| u.is_admin).map(|u| u.id).collect();
|
||||||
|
admins == [id]
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list_users(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
|
require_admin(&user)?;
|
||||||
|
let users: Vec<_> = state
|
||||||
|
.ctx
|
||||||
|
.db
|
||||||
|
.users()?
|
||||||
|
.iter()
|
||||||
|
.map(|u| {
|
||||||
|
serde_json::json!({
|
||||||
|
"id": u.id, "name": u.name, "admin": u.is_admin, "password": u.pass_hash.is_some(),
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
Ok(Json(serde_json::json!(users)))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
struct NewUser {
|
||||||
|
name: String,
|
||||||
|
#[serde(default)]
|
||||||
|
password: String,
|
||||||
|
#[serde(default)]
|
||||||
|
admin: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn add_user(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
Json(body): Json<NewUser>,
|
||||||
|
) -> Result<StatusCode, ApiError> {
|
||||||
|
require_admin(&user)?;
|
||||||
|
// The same rules as a name a proxy vouches for, so either way of signing in finds it.
|
||||||
|
let name = crate::auth::name_from_header(&body.name).ok_or_else(|| {
|
||||||
|
ApiError::bad_request("a name is required, without commas, semicolons or line breaks")
|
||||||
|
})?;
|
||||||
|
if state.ctx.db.user_by_name(&name)?.is_some() {
|
||||||
|
return Err(ApiError::bad_request(format!("{name} already exists")));
|
||||||
|
}
|
||||||
|
// No password is someone the proxy signs in, as with `ipx user add --no-password`.
|
||||||
|
let hash = if body.password.is_empty() {
|
||||||
|
None
|
||||||
|
} else {
|
||||||
|
Some(crate::auth::hash_password(&body.password).map_err(|e| ApiError::bad_request(format!("{e:#}")))?)
|
||||||
|
};
|
||||||
|
state.ctx.db.create_user(&name, hash.as_deref(), body.admin)?;
|
||||||
|
tracing::info!(by = %user.name, user = %name, admin = body.admin, "account added");
|
||||||
|
Ok(StatusCode::CREATED)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
struct UserPatch {
|
||||||
|
admin: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn patch_user(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
Path(id): Path<i64>,
|
||||||
|
Json(body): Json<UserPatch>,
|
||||||
|
) -> Result<StatusCode, ApiError> {
|
||||||
|
require_admin(&user)?;
|
||||||
|
let users = state.ctx.db.users()?;
|
||||||
|
let target = users
|
||||||
|
.iter()
|
||||||
|
.find(|u| u.id == id)
|
||||||
|
.ok_or_else(|| ApiError::bad_request(format!("no account with id {id}")))?;
|
||||||
|
if !body.admin && last_admin(&users, id) {
|
||||||
|
return Err(ApiError::bad_request(format!(
|
||||||
|
"{} is the only admin; make someone else an admin first",
|
||||||
|
target.name
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
state.ctx.db.set_admin(id, body.admin)?;
|
||||||
|
tracing::info!(by = %user.name, user = %target.name, admin = body.admin, "admin changed");
|
||||||
|
Ok(StatusCode::NO_CONTENT)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn remove_user(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
Path(id): Path<i64>,
|
||||||
|
) -> Result<StatusCode, ApiError> {
|
||||||
|
require_admin(&user)?;
|
||||||
|
let users = state.ctx.db.users()?;
|
||||||
|
let target = users
|
||||||
|
.iter()
|
||||||
|
.find(|u| u.id == id)
|
||||||
|
.ok_or_else(|| ApiError::bad_request(format!("no account with id {id}")))?;
|
||||||
|
if last_admin(&users, id) {
|
||||||
|
return Err(ApiError::bad_request(format!(
|
||||||
|
"{} is the only admin; make someone else an admin first",
|
||||||
|
target.name
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
state.ctx.db.delete_user(id)?;
|
||||||
|
tracing::info!(by = %user.name, user = %target.name, "account removed");
|
||||||
|
Ok(StatusCode::NO_CONTENT)
|
||||||
|
}
|
||||||
|
|
||||||
async fn login_page() -> Html<&'static str> {
|
async fn login_page() -> Html<&'static str> {
|
||||||
Html(include_str!("../web/login.html"))
|
Html(include_str!("../web/login.html"))
|
||||||
}
|
}
|
||||||
@@ -346,15 +472,30 @@ struct FeedRow {
|
|||||||
entries: i64,
|
entries: i64,
|
||||||
downloaded: i64,
|
downloaded: i64,
|
||||||
unread: i64,
|
unread: i64,
|
||||||
|
/// Including you. More than one means every file here is shared.
|
||||||
|
subscribers: i64,
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn feeds(State(state): State<WebState>) -> Result<Json<Vec<FeedRow>>, ApiError> {
|
async fn feeds(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
) -> Result<Json<Vec<FeedRow>>, ApiError> {
|
||||||
let cfg = state.ctx.cfg();
|
let cfg = state.ctx.cfg();
|
||||||
// Config entries plus the feeds derived from OPML subscriptions.
|
// Config entries plus the feeds derived from OPML subscriptions -- the catalogue.
|
||||||
|
// What comes back is only the part of it this person subscribes to.
|
||||||
let subs = crate::subscriptions(&state.ctx)?;
|
let subs = crate::subscriptions(&state.ctx)?;
|
||||||
let mut out = Vec::with_capacity(subs.len());
|
let mine: std::collections::HashMap<String, crate::db::Sub> = state
|
||||||
|
.ctx
|
||||||
|
.db
|
||||||
|
.subscriptions_for(user.id)?
|
||||||
|
.into_iter()
|
||||||
|
.map(|s| (s.feed_id.clone(), s))
|
||||||
|
.collect();
|
||||||
|
let counts = state.ctx.db.subscriber_counts()?;
|
||||||
|
let mut out = Vec::with_capacity(mine.len());
|
||||||
for sub in &subs {
|
for sub in &subs {
|
||||||
let (id, feed) = (&sub.id, &sub.cfg);
|
let (id, feed) = (&sub.id, &sub.cfg);
|
||||||
|
let Some(mine) = mine.get(id) else { continue };
|
||||||
let s = state.ctx.db.feed_summary(id)?;
|
let s = state.ctx.db.feed_summary(id)?;
|
||||||
let st = state.ctx.db.http_state(id)?;
|
let st = state.ctx.db.http_state(id)?;
|
||||||
out.push(FeedRow {
|
out.push(FeedRow {
|
||||||
@@ -363,10 +504,13 @@ async fn feeds(State(state): State<WebState>) -> Result<Json<Vec<FeedRow>>, ApiE
|
|||||||
title: s.title,
|
title: s.title,
|
||||||
image: s.image,
|
image: s.image,
|
||||||
folder: feed.folder.clone(),
|
folder: feed.folder.clone(),
|
||||||
keywords: feed.keywords.clone(),
|
keywords: mine.keywords.clone().unwrap_or_else(|| feed.keywords.clone()),
|
||||||
allow_explicit: feed.allow_explicit,
|
allow_explicit: mine.allow_explicit.unwrap_or(feed.allow_explicit),
|
||||||
auto_download: feed.auto_download,
|
auto_download: mine.auto_download.unwrap_or(feed.auto_download),
|
||||||
max_new_per_check: feed.max_new_per_check,
|
max_new_per_check: mine
|
||||||
|
.max_new_per_check
|
||||||
|
.map(|n| n as usize)
|
||||||
|
.or(feed.max_new_per_check),
|
||||||
group: feed.group.clone(),
|
group: feed.group.clone(),
|
||||||
orphaned: s.orphaned,
|
orphaned: s.orphaned,
|
||||||
// Derived from an OPML and not written to config until you change something.
|
// Derived from an OPML and not written to config until you change something.
|
||||||
@@ -384,12 +528,118 @@ async fn feeds(State(state): State<WebState>) -> Result<Json<Vec<FeedRow>>, ApiE
|
|||||||
last_error: s.last_error,
|
last_error: s.last_error,
|
||||||
entries: s.entries,
|
entries: s.entries,
|
||||||
downloaded: s.downloaded,
|
downloaded: s.downloaded,
|
||||||
unread: state.ctx.db.unread_count(id)?,
|
unread: state.ctx.db.unread_count(user.id, id)?,
|
||||||
|
subscribers: counts.get(id).copied().unwrap_or(0),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
Ok(Json(out))
|
Ok(Json(out))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---- popular on this server ----
|
||||||
|
|
||||||
|
/// A feed that carries a credential is someone's paid or private subscription. Listing it
|
||||||
|
/// would let anyone signed in subscribe to it and read what they pay for.
|
||||||
|
///
|
||||||
|
/// ponytail: a heuristic. A key hidden in the path of a host not in `PAID_HOSTS` gets
|
||||||
|
/// through; a per-feed `unlisted` flag is the upgrade if that happens again.
|
||||||
|
fn looks_private(feed: &crate::config::Feed) -> bool {
|
||||||
|
if feed.username.is_some() || feed.password.is_some() || feed.password_env.is_some() {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
let Ok(u) = url::Url::parse(&feed.url) else { return true };
|
||||||
|
// Paid-feed services put the subscriber's key in the path as often as in the query, and a
|
||||||
|
// long path segment alone proves nothing (acast's public show ids look the same). So a
|
||||||
|
// feed from one of them is private whatever its URL looks like. Supercast is why this
|
||||||
|
// exists: `feeds.supercast.com/feeds/<key>` reached the directory before it did.
|
||||||
|
let host = u.host_str().unwrap_or("");
|
||||||
|
PAID_HOSTS.iter().any(|h| host == *h || host.ends_with(&format!(".{h}")))
|
||||||
|
|| !u.username().is_empty()
|
||||||
|
|| u.password().is_some()
|
||||||
|
|| u.query_pairs().any(|(k, _)| {
|
||||||
|
let k = k.to_ascii_lowercase();
|
||||||
|
["auth", "token", "key", "secret", "pass", "sig", "session", "user", "uid"]
|
||||||
|
.iter()
|
||||||
|
.any(|w| k.contains(w))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Services whose feeds are always one subscriber's own.
|
||||||
|
const PAID_HOSTS: &[&str] =
|
||||||
|
&["patreon.com", "supercast.com", "supportingcast.fm", "glow.fm", "memberful.com"];
|
||||||
|
|
||||||
|
/// Only an id, a title, artwork and a count: never a URL, which is where a key would be.
|
||||||
|
#[derive(Serialize)]
|
||||||
|
struct PopularRow {
|
||||||
|
id: String,
|
||||||
|
title: Option<String>,
|
||||||
|
image: Option<String>,
|
||||||
|
subscribers: i64,
|
||||||
|
/// Yours already. Everyone counts, you included, so your own feeds are listed too.
|
||||||
|
subscribed: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every feed that may be listed, with everyone counted, you included, most subscribers
|
||||||
|
/// first. Popular is the top of it, the directory is all of it, and it is all that
|
||||||
|
/// `subscribe_popular` will subscribe you to.
|
||||||
|
fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||||
|
let db = &state.ctx.db;
|
||||||
|
let mine: std::collections::HashSet<String> =
|
||||||
|
db.subscriptions_for(user_id)?.into_iter().map(|s| s.feed_id).collect();
|
||||||
|
let counts = db.subscriber_counts()?;
|
||||||
|
let mut out = vec![];
|
||||||
|
for s in crate::subscriptions(&state.ctx)? {
|
||||||
|
let n = counts.get(&s.id).copied().unwrap_or(0);
|
||||||
|
// A feed from an OPML rides on the OPML: everyone subscribed to it counts every feed
|
||||||
|
// inside, which would bury everything anyone chose on purpose.
|
||||||
|
let from_opml = s.managed || s.cfg.group.is_some();
|
||||||
|
if n == 0 || from_opml || looks_private(&s.cfg) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let sum = db.feed_summary(&s.id)?;
|
||||||
|
let subscribed = mine.contains(&s.id);
|
||||||
|
out.push(PopularRow { id: s.id, title: sum.title, image: sum.image, subscribers: n, subscribed });
|
||||||
|
}
|
||||||
|
out.sort_by(|a, b| b.subscribers.cmp(&a.subscribers).then_with(|| sort_name(a).cmp(&sort_name(b))));
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get_popular(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||||
|
let mut rows = popular(&state, user.id)?;
|
||||||
|
rows.truncate(10);
|
||||||
|
Ok(Json(rows))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every feed that may be listed, A to Z.
|
||||||
|
async fn get_directory(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||||
|
let mut rows = popular(&state, user.id)?;
|
||||||
|
rows.sort_by_key(sort_name);
|
||||||
|
Ok(Json(rows))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sort_name(p: &PopularRow) -> String {
|
||||||
|
p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Subscribes by id, because the list never shows a URL. Checked against the same list, so
|
||||||
|
/// a guessed id cannot reach a private feed.
|
||||||
|
async fn subscribe_popular(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
Path(id): Path<String>,
|
||||||
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
|
if !popular(&state, user.id)?.iter().any(|p| p.id == id) {
|
||||||
|
return Err(ApiError::bad_request(format!("{id:?} is not on the popular list")));
|
||||||
|
}
|
||||||
|
state.ctx.db.subscribe(user.id, &id)?;
|
||||||
|
Ok(Json(serde_json::json!({ "id": id })))
|
||||||
|
}
|
||||||
|
|
||||||
/// Validates a replacement feed URL: present, parseable, and not already subscribed under
|
/// Validates a replacement feed URL: present, parseable, and not already subscribed under
|
||||||
/// a different id. Returns the trimmed URL.
|
/// a different id. Returns the trimmed URL.
|
||||||
fn check_url(
|
fn check_url(
|
||||||
@@ -454,6 +704,47 @@ impl IntoResponse for ApiError {
|
|||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_feed_with_a_credential_is_never_popular() {
|
||||||
|
let f = |url: &str| crate::config::Feed {
|
||||||
|
url: url.into(),
|
||||||
|
folder: None,
|
||||||
|
group: None,
|
||||||
|
media_types: None,
|
||||||
|
schedule: None,
|
||||||
|
keywords: vec![],
|
||||||
|
allow_explicit: false,
|
||||||
|
auto_download: true,
|
||||||
|
max_new_per_check: None,
|
||||||
|
username: None,
|
||||||
|
password: None,
|
||||||
|
password_env: None,
|
||||||
|
};
|
||||||
|
assert!(!looks_private(&f("https://feeds.twit.tv/twit.xml")));
|
||||||
|
assert!(!looks_private(&f("https://example.com/rss?format=mp3")));
|
||||||
|
// Patreon's shape: the key is a query parameter.
|
||||||
|
assert!(looks_private(&f("https://www.patreon.com/rss/x?auth=abc123&show=2073588")));
|
||||||
|
assert!(looks_private(&f("https://example.com/rss?api_key=abc")));
|
||||||
|
// Paid-feed services put the key in the path; the host gives them away.
|
||||||
|
assert!(looks_private(&f("https://feeds.supercast.com/feeds/abcdefghijklmnopqrstuvwx")));
|
||||||
|
assert!(looks_private(&f("https://someshow.supportingcast.fm/content/abc123.rss")));
|
||||||
|
// A long path segment alone is not a key: acast's public show ids look the same.
|
||||||
|
assert!(!looks_private(&f("https://feeds.acast.com/public/shows/0123456789abcdef01234567")));
|
||||||
|
assert!(looks_private(&f("https://ray:hunter2@example.com/rss")));
|
||||||
|
assert!(looks_private(&f("not a url")), "unparseable is not safe to list");
|
||||||
|
let mut basic = f("https://example.com/rss");
|
||||||
|
basic.username = Some("ray".into());
|
||||||
|
assert!(looks_private(&basic), "a feed with a login configured");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn only_the_last_admin_is_protected() {
|
||||||
|
let u = |id, is_admin| crate::db::User { id, name: format!("u{id}"), pass_hash: None, is_admin };
|
||||||
|
assert!(last_admin(&[u(1, true), u(2, false)], 1));
|
||||||
|
assert!(!last_admin(&[u(1, true), u(2, true)], 1), "another admin remains");
|
||||||
|
assert!(!last_admin(&[u(1, true), u(2, false)], 2), "not an admin at all");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn token_comparison_rejects_mismatches_and_length_differences() {
|
fn token_comparison_rejects_mismatches_and_length_differences() {
|
||||||
assert!(constant_time_eq("abc123", "abc123"));
|
assert!(constant_time_eq("abc123", "abc123"));
|
||||||
@@ -533,6 +824,12 @@ struct Page {
|
|||||||
filter: Option<String>,
|
filter: Option<String>,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
q: Option<String>,
|
q: Option<String>,
|
||||||
|
/// A column name and asc or desc. Anything unrecognised is newest first: the name picks a
|
||||||
|
/// fixed expression in the query and never reaches it itself.
|
||||||
|
#[serde(default)]
|
||||||
|
sort: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
dir: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
fn fifty() -> i64 {
|
fn fifty() -> i64 {
|
||||||
@@ -548,21 +845,44 @@ struct EntryPage {
|
|||||||
async fn entries(
|
async fn entries(
|
||||||
State(state): State<WebState>,
|
State(state): State<WebState>,
|
||||||
Path(id): Path<String>,
|
Path(id): Path<String>,
|
||||||
|
user: crate::db::User,
|
||||||
Query(page): Query<Page>,
|
Query(page): Query<Page>,
|
||||||
|
) -> Result<Json<EntryPage>, ApiError> {
|
||||||
|
entry_page(&state, user.id, Some(&id), &page)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every subscribed feed's items together, newest first: All Subscriptions.
|
||||||
|
async fn all_entries(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
Query(page): Query<Page>,
|
||||||
|
) -> Result<Json<EntryPage>, ApiError> {
|
||||||
|
entry_page(&state, user.id, None, &page)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One feed's page of items, or every subscribed feed's when `feed` is None.
|
||||||
|
fn entry_page(
|
||||||
|
state: &WebState,
|
||||||
|
user_id: i64,
|
||||||
|
feed: Option<&str>,
|
||||||
|
page: &Page,
|
||||||
) -> Result<Json<EntryPage>, ApiError> {
|
) -> Result<Json<EntryPage>, ApiError> {
|
||||||
let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all"));
|
let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all"));
|
||||||
let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty());
|
let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty());
|
||||||
let mut rows = state
|
let db = &state.ctx.db;
|
||||||
.ctx
|
let order = crate::db::order_sql(
|
||||||
.db
|
page.sort.as_deref().unwrap_or("published"),
|
||||||
.entries(&id, filter, search, page.offset, page.limit.clamp(1, 200))?;
|
page.dir.as_deref().unwrap_or("desc"),
|
||||||
|
);
|
||||||
|
let mut rows =
|
||||||
|
db.entries_in(user_id, feed, filter, search, page.offset, page.limit.clamp(1, 200), &order)?;
|
||||||
// Feed HTML is untrusted: it reaches the page only after ammonia has been through it.
|
// Feed HTML is untrusted: it reaches the page only after ammonia has been through it.
|
||||||
for row in &mut rows {
|
for row in &mut rows {
|
||||||
if let Some(d) = &row.description {
|
if let Some(d) = &row.description {
|
||||||
row.description = Some(ammonia::clean(d));
|
row.description = Some(ammonia::clean(d));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
let total = state.ctx.db.count_entries(&id, filter, search)?;
|
let total = db.count_in(user_id, feed, filter, search)?;
|
||||||
Ok(Json(EntryPage { total, entries: rows }))
|
Ok(Json(EntryPage { total, entries: rows }))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -577,22 +897,41 @@ struct NewFeed {
|
|||||||
|
|
||||||
async fn add_feed(
|
async fn add_feed(
|
||||||
State(state): State<WebState>,
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
Json(body): Json<NewFeed>,
|
Json(body): Json<NewFeed>,
|
||||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
let mut cfg = (*state.ctx.cfg()).clone();
|
let mut cfg = (*state.ctx.cfg()).clone();
|
||||||
// Derived feeds count as subscribed: adding one an OPML already lists would duplicate it.
|
// Someone else may already have it. Then adding costs nothing: no second fetch, no
|
||||||
|
// second copy on disk, just another name against the same feed.
|
||||||
if let Some(existing) = crate::subscriptions(&state.ctx)?
|
if let Some(existing) = crate::subscriptions(&state.ctx)?
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.find(|s| s.cfg.url == body.url)
|
.find(|s| s.cfg.url == body.url)
|
||||||
{
|
{
|
||||||
return Ok(Json(serde_json::json!({ "id": existing.id, "existing": true })));
|
let already = state.ctx.db.subscription(user.id, &existing.id)?.is_some();
|
||||||
|
state.ctx.db.subscribe(user.id, &existing.id)?;
|
||||||
|
scan_soon(&state, Some(existing.id.clone())).await;
|
||||||
|
return Ok(Json(
|
||||||
|
serde_json::json!({ "id": existing.id, "existing": already }),
|
||||||
|
));
|
||||||
}
|
}
|
||||||
let id = crate::add_one(&state.ctx, &mut cfg, &body.url, body.folder, body.keywords).await?;
|
let id = crate::add_one(&state.ctx, &mut cfg, &body.url, body.folder, body.keywords).await?;
|
||||||
cfg.save(&state.config_path)?;
|
cfg.save(&state.config_path)?;
|
||||||
state.ctx.reload_cfg(&state.config_path)?;
|
state.ctx.reload_cfg(&state.config_path)?;
|
||||||
|
state.ctx.db.subscribe(user.id, &id)?;
|
||||||
|
scan_soon(&state, Some(id.clone())).await;
|
||||||
Ok(Json(serde_json::json!({ "id": id, "existing": false })))
|
Ok(Json(serde_json::json!({ "id": id, "existing": false })))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Queues a scan, so a feed just added shows its items without anyone pressing Scan now.
|
||||||
|
/// `None` scans whatever is due, which a feed never checked always is. The add has
|
||||||
|
/// succeeded either way, so a daemon not taking commands is only logged.
|
||||||
|
async fn scan_soon(state: &WebState, feed: Option<String>) {
|
||||||
|
let force = feed.is_some();
|
||||||
|
if state.cmds.send(Command::Fetch { feed, force }).await.is_err() {
|
||||||
|
tracing::warn!("could not queue a scan: the daemon is not accepting commands");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Absent means "leave alone"; JSON `null` means "clear this".
|
/// Absent means "leave alone"; JSON `null` means "clear this".
|
||||||
///
|
///
|
||||||
/// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a
|
/// That distinction needs `double_option`: serde maps `null` onto the *outer* `None` for a
|
||||||
@@ -626,10 +965,46 @@ async fn patch_feed(
|
|||||||
user: crate::db::User,
|
user: crate::db::User,
|
||||||
Json(body): Json<FeedPatch>,
|
Json(body): Json<FeedPatch>,
|
||||||
) -> Result<StatusCode, ApiError> {
|
) -> Result<StatusCode, ApiError> {
|
||||||
// How often a feed is polled is the operator's call: it costs bandwidth, it is what
|
// What one person wants -- which items, whether to fetch them, how many at a time --
|
||||||
// publishers notice, and one impatient setting affects everyone reading the feed.
|
// is theirs. It goes on their subscription and nobody else sees the change.
|
||||||
if body.schedule.is_some() && !user.is_admin {
|
if state.ctx.db.subscription(user.id, &id)?.is_some() {
|
||||||
return Err(ApiError::forbidden("only an admin sets when feeds are scanned"));
|
let mut mine = state
|
||||||
|
.ctx
|
||||||
|
.db
|
||||||
|
.subscription(user.id, &id)?
|
||||||
|
.unwrap_or_else(|| crate::db::Sub { feed_id: id.clone(), ..Default::default() });
|
||||||
|
let mut touched = false;
|
||||||
|
if let Some(v) = body.keywords.clone() {
|
||||||
|
mine.keywords = Some(v.into_iter().filter(|k| !k.trim().is_empty()).collect());
|
||||||
|
touched = true;
|
||||||
|
}
|
||||||
|
if let Some(v) = body.allow_explicit {
|
||||||
|
mine.allow_explicit = Some(v);
|
||||||
|
touched = true;
|
||||||
|
}
|
||||||
|
if let Some(v) = body.auto_download {
|
||||||
|
mine.auto_download = Some(v);
|
||||||
|
touched = true;
|
||||||
|
}
|
||||||
|
if let Some(v) = body.max_new_per_check {
|
||||||
|
mine.max_new_per_check = v.map(|n| n as i64);
|
||||||
|
touched = true;
|
||||||
|
}
|
||||||
|
if touched {
|
||||||
|
state.ctx.db.set_subscription(user.id, &mine)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The rest describes the feed itself -- where its files land, its address, when it is
|
||||||
|
// polled -- and there is one of those however many people read it.
|
||||||
|
let feed_level = body.url.is_some() || body.folder.is_some() || body.schedule.is_some();
|
||||||
|
if !feed_level {
|
||||||
|
return Ok(StatusCode::NO_CONTENT);
|
||||||
|
}
|
||||||
|
if !user.is_admin {
|
||||||
|
return Err(ApiError::forbidden(
|
||||||
|
"the feed's address, folder and schedule are the same for everyone, so only an admin changes them",
|
||||||
|
));
|
||||||
}
|
}
|
||||||
let mut cfg = (*state.ctx.cfg()).clone();
|
let mut cfg = (*state.ctx.cfg()).clone();
|
||||||
|
|
||||||
@@ -675,18 +1050,6 @@ async fn patch_feed(
|
|||||||
if let Some(v) = body.folder {
|
if let Some(v) = body.folder {
|
||||||
feed.folder = v.filter(|s| !s.trim().is_empty());
|
feed.folder = v.filter(|s| !s.trim().is_empty());
|
||||||
}
|
}
|
||||||
if let Some(v) = body.keywords {
|
|
||||||
feed.keywords = v.into_iter().filter(|k| !k.trim().is_empty()).collect();
|
|
||||||
}
|
|
||||||
if let Some(v) = body.allow_explicit {
|
|
||||||
feed.allow_explicit = v;
|
|
||||||
}
|
|
||||||
if let Some(v) = body.auto_download {
|
|
||||||
feed.auto_download = v;
|
|
||||||
}
|
|
||||||
if let Some(v) = body.max_new_per_check {
|
|
||||||
feed.max_new_per_check = v;
|
|
||||||
}
|
|
||||||
cfg.save(&state.config_path)?;
|
cfg.save(&state.config_path)?;
|
||||||
state.ctx.reload_cfg(&state.config_path)?;
|
state.ctx.reload_cfg(&state.config_path)?;
|
||||||
if url_changed {
|
if url_changed {
|
||||||
@@ -700,7 +1063,23 @@ async fn patch_feed(
|
|||||||
async fn remove_feed(
|
async fn remove_feed(
|
||||||
State(state): State<WebState>,
|
State(state): State<WebState>,
|
||||||
Path(id): Path<String>,
|
Path(id): Path<String>,
|
||||||
|
user: crate::db::User,
|
||||||
) -> Result<StatusCode, ApiError> {
|
) -> Result<StatusCode, ApiError> {
|
||||||
|
// Unsubscribing is personal: it takes the feed off your list and leaves everyone
|
||||||
|
// else's alone.
|
||||||
|
state.ctx.db.unsubscribe(user.id, &id)?;
|
||||||
|
for child in crate::subscriptions(&state.ctx)?
|
||||||
|
.iter()
|
||||||
|
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
|
||||||
|
{
|
||||||
|
state.ctx.db.unsubscribe(user.id, &child.id)?;
|
||||||
|
}
|
||||||
|
if state.ctx.db.subscriber_count(&id)? > 0 {
|
||||||
|
return Ok(StatusCode::NO_CONTENT);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Nobody is left: the feed stops being scanned. Its files and history stay, so if
|
||||||
|
// someone subscribes again they do not pull the back catalogue a second time.
|
||||||
let mut cfg = (*state.ctx.cfg()).clone();
|
let mut cfg = (*state.ctx.cfg()).clone();
|
||||||
if cfg.feeds.remove(&id).is_none() {
|
if cfg.feeds.remove(&id).is_none() {
|
||||||
// A derived feed: forget it here, though the OPML will list it again on the next
|
// A derived feed: forget it here, though the OPML will list it again on the next
|
||||||
@@ -708,7 +1087,6 @@ async fn remove_feed(
|
|||||||
state.ctx.db.drop_managed(&id)?;
|
state.ctx.db.drop_managed(&id)?;
|
||||||
return Ok(StatusCode::NO_CONTENT);
|
return Ok(StatusCode::NO_CONTENT);
|
||||||
}
|
}
|
||||||
// Downloads and history stay, so re-adding does not re-pull the back catalogue.
|
|
||||||
cfg.save(&state.config_path)?;
|
cfg.save(&state.config_path)?;
|
||||||
state.ctx.reload_cfg(&state.config_path)?;
|
state.ctx.reload_cfg(&state.config_path)?;
|
||||||
Ok(StatusCode::NO_CONTENT)
|
Ok(StatusCode::NO_CONTENT)
|
||||||
@@ -723,14 +1101,15 @@ struct Flags {
|
|||||||
async fn set_flags(
|
async fn set_flags(
|
||||||
State(state): State<WebState>,
|
State(state): State<WebState>,
|
||||||
Path((feed_id, guid)): Path<(String, String)>,
|
Path((feed_id, guid)): Path<(String, String)>,
|
||||||
|
user: crate::db::User,
|
||||||
Json(body): Json<Flags>,
|
Json(body): Json<Flags>,
|
||||||
) -> Result<StatusCode, ApiError> {
|
) -> Result<StatusCode, ApiError> {
|
||||||
use crate::db::EntryFlag;
|
use crate::db::EntryFlag;
|
||||||
if let Some(v) = body.read {
|
if let Some(v) = body.read {
|
||||||
state.ctx.db.set_entry_flag(&feed_id, &guid, EntryFlag::Read, v)?;
|
state.ctx.db.set_entry_flag(user.id, &feed_id, &guid, EntryFlag::Read, v)?;
|
||||||
}
|
}
|
||||||
if let Some(v) = body.flagged {
|
if let Some(v) = body.flagged {
|
||||||
state.ctx.db.set_entry_flag(&feed_id, &guid, EntryFlag::Flagged, v)?;
|
state.ctx.db.set_entry_flag(user.id, &feed_id, &guid, EntryFlag::Flagged, v)?;
|
||||||
}
|
}
|
||||||
Ok(StatusCode::NO_CONTENT)
|
Ok(StatusCode::NO_CONTENT)
|
||||||
}
|
}
|
||||||
@@ -759,15 +1138,46 @@ async fn download_now(
|
|||||||
Ok(StatusCode::ACCEPTED)
|
Ok(StatusCode::ACCEPTED)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
struct Force {
|
||||||
|
#[serde(default)]
|
||||||
|
force: bool,
|
||||||
|
}
|
||||||
|
|
||||||
async fn delete_file(
|
async fn delete_file(
|
||||||
State(state): State<WebState>,
|
State(state): State<WebState>,
|
||||||
Path(id): Path<i64>,
|
Path(id): Path<i64>,
|
||||||
|
user: crate::db::User,
|
||||||
|
Query(q): Query<Force>,
|
||||||
) -> Result<StatusCode, ApiError> {
|
) -> Result<StatusCode, ApiError> {
|
||||||
let enc = state
|
let enc = state
|
||||||
.ctx
|
.ctx
|
||||||
.db
|
.db
|
||||||
.enclosure(id)?
|
.enclosure(id)?
|
||||||
.ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
|
.ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
|
||||||
|
|
||||||
|
// There is one copy of the file: deleting it deletes everyone's. Say so before doing
|
||||||
|
// it, once, and let them decide.
|
||||||
|
if !q.force {
|
||||||
|
let (starred, unread) = state.ctx.db.others_wanting(id, user.id)?;
|
||||||
|
let people = |n: i64| if n == 1 { "person".to_string() } else { format!("{n} people") };
|
||||||
|
let complaint = match (starred, unread) {
|
||||||
|
(0, 0) => None,
|
||||||
|
(0, u) => Some(format!("{} subscribed to this feed {} not played it yet", people(u), if u == 1 { "has" } else { "have" })),
|
||||||
|
(st, 0) => Some(format!("another {} starred it to keep", people(st))),
|
||||||
|
(st, u) => Some(format!(
|
||||||
|
"another {} starred it to keep, and {} not played it yet",
|
||||||
|
people(st),
|
||||||
|
if u == 1 { "one person has".to_string() } else { format!("{u} have") }
|
||||||
|
)),
|
||||||
|
};
|
||||||
|
if let Some(why) = complaint {
|
||||||
|
return Err(ApiError {
|
||||||
|
error: anyhow::Error::msg(format!("There is one copy of this file and {why}.")),
|
||||||
|
status: StatusCode::CONFLICT,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
if let Some(path) = &enc.path
|
if let Some(path) = &enc.path
|
||||||
&& let Err(e) = std::fs::remove_file(path)
|
&& let Err(e) = std::fs::remove_file(path)
|
||||||
&& e.kind() != std::io::ErrorKind::NotFound
|
&& e.kind() != std::io::ErrorKind::NotFound
|
||||||
@@ -838,15 +1248,17 @@ struct Position {
|
|||||||
async fn set_position(
|
async fn set_position(
|
||||||
State(state): State<WebState>,
|
State(state): State<WebState>,
|
||||||
Path((feed_id, guid)): Path<(String, String)>,
|
Path((feed_id, guid)): Path<(String, String)>,
|
||||||
|
user: crate::db::User,
|
||||||
Json(body): Json<Position>,
|
Json(body): Json<Position>,
|
||||||
) -> Result<StatusCode, ApiError> {
|
) -> Result<StatusCode, ApiError> {
|
||||||
state.ctx.db.set_position(&feed_id, &guid, body.secs)?;
|
state.ctx.db.set_position(user.id, &feed_id, &guid, body.secs)?;
|
||||||
Ok(StatusCode::NO_CONTENT)
|
Ok(StatusCode::NO_CONTENT)
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn read_all(
|
async fn read_all(
|
||||||
State(state): State<WebState>,
|
State(state): State<WebState>,
|
||||||
Path(id): Path<String>,
|
Path(id): Path<String>,
|
||||||
|
user: crate::db::User,
|
||||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
// A subscription's own row has no entries, so marking it read means everything under it.
|
// A subscription's own row has no entries, so marking it read means everything under it.
|
||||||
let mut ids = vec![id.clone()];
|
let mut ids = vec![id.clone()];
|
||||||
@@ -856,7 +1268,19 @@ async fn read_all(
|
|||||||
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
|
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
|
||||||
.map(|s| s.id),
|
.map(|s| s.id),
|
||||||
);
|
);
|
||||||
let n = state.ctx.db.mark_all_read(&ids)?;
|
let n = state.ctx.db.mark_all_read(user.id, &ids)?;
|
||||||
|
Ok(Json(serde_json::json!({ "marked": n })))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Everything read in every feed you subscribe to: exactly what All Subscriptions lists, since
|
||||||
|
/// that view is scoped by the same subscriptions.
|
||||||
|
async fn read_all_mine(
|
||||||
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
|
let ids: Vec<String> =
|
||||||
|
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
|
||||||
|
let n = state.ctx.db.mark_all_read(user.id, &ids)?;
|
||||||
Ok(Json(serde_json::json!({ "marked": n })))
|
Ok(Json(serde_json::json!({ "marked": n })))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -890,8 +1314,14 @@ async fn download_latest(
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Subscriptions as OPML, so they can move to another podcast app.
|
/// Subscriptions as OPML, so they can move to another podcast app.
|
||||||
async fn export_opml(State(state): State<WebState>) -> Result<Response, ApiError> {
|
async fn export_opml(
|
||||||
let cfg = state.ctx.cfg();
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
|
) -> Result<Response, ApiError> {
|
||||||
|
// Yours, not the whole catalogue: other people's feeds, and any private URLs in them, are
|
||||||
|
// not yours to download. This used to export config.toml to whoever asked.
|
||||||
|
let mine: std::collections::HashSet<String> =
|
||||||
|
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
|
||||||
let mut doc = opml::OPML {
|
let mut doc = opml::OPML {
|
||||||
head: Some(opml::Head {
|
head: Some(opml::Head {
|
||||||
title: Some("ipx subscriptions".into()),
|
title: Some("ipx subscriptions".into()),
|
||||||
@@ -899,15 +1329,19 @@ async fn export_opml(State(state): State<WebState>) -> Result<Response, ApiError
|
|||||||
}),
|
}),
|
||||||
..Default::default()
|
..Default::default()
|
||||||
};
|
};
|
||||||
for (id, feed) in &cfg.feeds {
|
for s in crate::subscriptions(&state.ctx)? {
|
||||||
|
// A feed from an OPML subscription comes back with the OPML itself.
|
||||||
|
if s.managed || !mine.contains(&s.id) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
let title = state
|
let title = state
|
||||||
.ctx
|
.ctx
|
||||||
.db
|
.db
|
||||||
.feed_summary(id)
|
.feed_summary(&s.id)
|
||||||
.ok()
|
.ok()
|
||||||
.and_then(|s| s.title)
|
.and_then(|sum| sum.title)
|
||||||
.unwrap_or_else(|| id.clone());
|
.unwrap_or_else(|| s.id.clone());
|
||||||
doc.add_feed(&title, &feed.url);
|
doc.add_feed(&title, &s.cfg.url);
|
||||||
}
|
}
|
||||||
let xml = doc.to_string().map_err(|e| anyhow::anyhow!("writing OPML: {e}"))?;
|
let xml = doc.to_string().map_err(|e| anyhow::anyhow!("writing OPML: {e}"))?;
|
||||||
Ok((
|
Ok((
|
||||||
@@ -930,42 +1364,18 @@ struct OpmlBody {
|
|||||||
|
|
||||||
async fn import_opml(
|
async fn import_opml(
|
||||||
State(state): State<WebState>,
|
State(state): State<WebState>,
|
||||||
|
user: crate::db::User,
|
||||||
Json(body): Json<OpmlBody>,
|
Json(body): Json<OpmlBody>,
|
||||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||||
|
// Refused before anything is touched. Nothing reaches the disk either way: an uploaded
|
||||||
|
// file arrives as text, is read here, and is gone when the request ends.
|
||||||
let doc = opml::OPML::from_str(&body.xml)
|
let doc = opml::OPML::from_str(&body.xml)
|
||||||
.map_err(|e| anyhow::anyhow!("that does not parse as OPML: {e}"))?;
|
.map_err(|e| ApiError::bad_request(format!("that is not an OPML file: {e}")))?;
|
||||||
let mut found = vec![];
|
let (added, already) = crate::subscribe_opml(&state.ctx, &state.config_path, &doc, user.id)?;
|
||||||
crate::collect_outlines(&doc.body.outlines, &mut found);
|
if added > 0 {
|
||||||
|
scan_soon(&state, None).await;
|
||||||
let mut cfg = (*state.ctx.cfg()).clone();
|
|
||||||
let mut added = 0;
|
|
||||||
for (title, url) in found {
|
|
||||||
if cfg.feeds.values().any(|f| f.url == url) {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
let id = crate::config::unique_slug(&title, &cfg.feeds);
|
|
||||||
cfg.feeds.insert(
|
|
||||||
id,
|
|
||||||
crate::config::Feed {
|
|
||||||
url,
|
|
||||||
folder: None,
|
|
||||||
group: None,
|
|
||||||
media_types: None,
|
|
||||||
schedule: None,
|
|
||||||
keywords: vec![],
|
|
||||||
allow_explicit: false,
|
|
||||||
auto_download: true,
|
|
||||||
max_new_per_check: None,
|
|
||||||
username: None,
|
|
||||||
password: None,
|
|
||||||
password_env: None,
|
|
||||||
},
|
|
||||||
);
|
|
||||||
added += 1;
|
|
||||||
}
|
}
|
||||||
cfg.save(&state.config_path)?;
|
Ok(Json(serde_json::json!({ "added": added, "already": already })))
|
||||||
state.ctx.reload_cfg(&state.config_path)?;
|
|
||||||
Ok(Json(serde_json::json!({ "added": added })))
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Serialize)]
|
#[derive(Serialize)]
|
||||||
@@ -1061,9 +1471,13 @@ struct LogPage {
|
|||||||
latest: u64,
|
latest: u64,
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn logs(Query(q): Query<LogQuery>) -> Json<LogPage> {
|
async fn logs(user: crate::db::User, Query(q): Query<LogQuery>) -> Result<Json<LogPage>, ApiError> {
|
||||||
|
// The log names every account, every feed and every failed sign-in, not just yours.
|
||||||
|
if !user.is_admin {
|
||||||
|
return Err(ApiError::forbidden("only an admin reads the log"));
|
||||||
|
}
|
||||||
let (lines, latest) = crate::logbuf::since(q.after, q.limit.clamp(1, 2000));
|
let (lines, latest) = crate::logbuf::since(q.after, q.limit.clamp(1, 2000));
|
||||||
Json(LogPage { lines, latest })
|
Ok(Json(LogPage { lines, latest }))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One line per HTTP request, so the web side shows up in the same log as the daemon.
|
/// One line per HTTP request, so the web side shows up in the same log as the daemon.
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ const script = html.split('<script>')[1].split('</script>')[0];
|
|||||||
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
|
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
|
||||||
|
|
||||||
const missing = [];
|
const missing = [];
|
||||||
const el = (name) => new Proxy({ style: {}, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
|
const el = (name) => new Proxy({ style: { setProperty(){}, getPropertyValue(){ return ''; } }, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
|
||||||
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
|
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
|
||||||
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
|
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
|
||||||
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
|
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
|
||||||
@@ -44,7 +44,12 @@ const ctx = {
|
|||||||
json: () => Promise.resolve(
|
json: () => Promise.resolve(
|
||||||
String(url).includes('/api/settings')
|
String(url).includes('/api/settings')
|
||||||
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
|
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
|
||||||
: []),
|
: String(url).includes('/api/users')
|
||||||
|
? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }]
|
||||||
|
: /\/api\/(popular|directory)/.test(String(url))
|
||||||
|
? [{ id: 'f', title: 'A Feed', image: null, subscribers: 2, subscribed: true },
|
||||||
|
{ id: 'g', title: null, image: null, subscribers: 1, subscribed: false }]
|
||||||
|
: /entries/.test(String(url)) ? { total: 0, entries: [] } : []),
|
||||||
}),
|
}),
|
||||||
EventSource: function () { this.close = () => {}; },
|
EventSource: function () { this.close = () => {}; },
|
||||||
MediaMetadata: function () {},
|
MediaMetadata: function () {},
|
||||||
@@ -79,6 +84,11 @@ const drive = [
|
|||||||
['downloadLatestModal', () => ctx.downloadLatestModal(feed)],
|
['downloadLatestModal', () => ctx.downloadLatestModal(feed)],
|
||||||
['removeFeed', () => ctx.removeFeed(feed)],
|
['removeFeed', () => ctx.removeFeed(feed)],
|
||||||
['prefsModal', () => ctx.prefsModal()],
|
['prefsModal', () => ctx.prefsModal()],
|
||||||
|
['usersModal', () => ctx.usersModal()],
|
||||||
|
['opmlModal', () => ctx.opmlModal()],
|
||||||
|
['selectFeed (directory)', () => ctx.selectFeed(':directory')],
|
||||||
|
['selectFeed (popular)', () => ctx.selectFeed(':popular')],
|
||||||
|
['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
|
||||||
['logsModal', () => ctx.logsModal()],
|
['logsModal', () => ctx.logsModal()],
|
||||||
// `const S` is not reachable from here: top-level const/let do not become properties
|
// `const S` is not reachable from here: top-level const/let do not become properties
|
||||||
// of a vm context the way var and function declarations do.
|
// of a vm context the way var and function declarations do.
|
||||||
|
|||||||
@@ -10,8 +10,8 @@ test.beforeEach(async ({ page }) => {
|
|||||||
test('the page loads and lists the configured feeds', async ({ page }) => {
|
test('the page loads and lists the configured feeds', async ({ page }) => {
|
||||||
// Regression: a ReferenceError in the script left the shell rendered and the sidebar
|
// Regression: a ReferenceError in the script left the shell rendered and the sidebar
|
||||||
// empty, with every handler below the error dead. Server-side checks all passed.
|
// empty, with every handler below the error dead. Server-side checks all passed.
|
||||||
// Three top-level feeds in the fixture config; the OPML's child is inside a closed folder.
|
// Four top-level feeds in the fixture config; the OPML's children are inside a closed folder.
|
||||||
await expect(page.locator('.feed')).toHaveCount(4, { timeout: 15_000 });
|
await expect(page.locator('.feed')).toHaveCount(5, { timeout: 15_000 });
|
||||||
await expect(page.getByText('Test Show')).toBeVisible();
|
await expect(page.getByText('Test Show')).toBeVisible();
|
||||||
const errors = [];
|
const errors = [];
|
||||||
page.on('pageerror', e => errors.push(e.message));
|
page.on('pageerror', e => errors.push(e.message));
|
||||||
@@ -29,6 +29,18 @@ test('the theme toggle actually changes the theme', async ({ page }) => {
|
|||||||
.not.toBe(before);
|
.not.toBe(before);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('the theme button steps through dark, light and classic, and remembers', async ({ page }) => {
|
||||||
|
const theme = () => page.evaluate(() => document.documentElement.dataset.theme);
|
||||||
|
for (let i = 0; i < 3 && (await theme()) !== 'classic'; i++) await page.locator('#theme').click();
|
||||||
|
expect(await theme()).toBe('classic');
|
||||||
|
await expect(page.locator('#theme')).toHaveAttribute('title', /Classic.*Click for Dark/);
|
||||||
|
|
||||||
|
await page.reload();
|
||||||
|
await expect.poll(theme).toBe('classic');
|
||||||
|
// The 2004 Mac app set its type in Lucida Grande.
|
||||||
|
expect(await page.evaluate(() => getComputedStyle(document.body).fontFamily)).toContain('Lucida Grande');
|
||||||
|
});
|
||||||
|
|
||||||
test('settings opens and saves the global schedule', async ({ page }) => {
|
test('settings opens and saves the global schedule', async ({ page }) => {
|
||||||
await page.locator('#prefs').click();
|
await page.locator('#prefs').click();
|
||||||
await expect(page.locator('#modal.on')).toBeVisible();
|
await expect(page.locator('#modal.on')).toBeVisible();
|
||||||
@@ -69,20 +81,21 @@ test('the three panes are there and the item text lands in the bottom one', asyn
|
|||||||
|
|
||||||
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
||||||
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||||
// The enclosure travels with the item, into the same pane.
|
// The enclosure goes to the Files pane beside the list, as the original's did.
|
||||||
await expect(page.locator('#detail .encbox')).toHaveCount(1);
|
await expect(page.locator('#files')).toBeVisible();
|
||||||
|
await expect(page.locator('#files .encbox')).toHaveCount(1);
|
||||||
|
|
||||||
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
|
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
|
||||||
// its chip rather than assuming which episode the daemon happened to fetch.
|
// its chip rather than assuming which episode the daemon happened to fetch.
|
||||||
const downloaded = page.locator('.ep', { hasText: 'downloaded' }).first();
|
const downloaded = page.locator('.ep', { has: page.locator('.kind.here') }).first();
|
||||||
await downloaded.click();
|
await downloaded.click();
|
||||||
await expect(page.locator('#detail audio')).toBeVisible();
|
await expect(page.locator('#files [data-a="play"]')).toBeVisible();
|
||||||
await expect(page.locator('#detail .encbox .btn', { hasText: 'Save' })).toBeVisible();
|
await expect(page.locator('#files .encbox [title="Save to this computer"]')).toBeVisible();
|
||||||
|
|
||||||
// Selecting another item replaces the pane rather than stacking.
|
// Selecting another item replaces the pane rather than stacking.
|
||||||
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
||||||
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||||
await expect(page.locator('#detail audio')).toHaveCount(0);
|
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
||||||
@@ -96,12 +109,12 @@ test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
|||||||
await row.click();
|
await row.click();
|
||||||
|
|
||||||
await expect(page.locator('#detail .dt')).toHaveText('An Article');
|
await expect(page.locator('#detail .dt')).toHaveText('An Article');
|
||||||
await expect(page.locator('#detail audio')).toHaveCount(0);
|
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
|
||||||
await expect(page.locator('#detail .encbox')).toContainText('image');
|
// What it is and that it is here: one icon, green, with the words in its tooltip.
|
||||||
await expect(page.locator('#detail .encbox')).toContainText('downloaded');
|
await expect(page.locator('#files .encbox .kind.here')).toHaveAttribute('title', 'image, downloaded');
|
||||||
// Still offered as a file, just not as an episode: viewable and keepable.
|
// Still offered as a file, just not as an episode: viewable and keepable.
|
||||||
await expect(page.locator('#detail .btn', { hasText: 'Save' })).toBeVisible();
|
await expect(page.locator('#files [title="Save to this computer"]')).toBeVisible();
|
||||||
const view = page.locator('#detail a', { hasText: 'View' });
|
const view = page.locator('#files a[title="View in a new tab"]');
|
||||||
await expect(view).toHaveAttribute('target', '_blank');
|
await expect(view).toHaveAttribute('target', '_blank');
|
||||||
await expect(view).toHaveAttribute('rel', /noopener/);
|
await expect(view).toHaveAttribute('rel', /noopener/);
|
||||||
await expect(view).toHaveAttribute('href', /\/media\/\d+/);
|
await expect(view).toHaveAttribute('href', /\/media\/\d+/);
|
||||||
@@ -115,9 +128,9 @@ test('an item with several enclosures lists them all', async ({ page }) => {
|
|||||||
await expect(row).toContainText('+1 more file');
|
await expect(row).toContainText('+1 more file');
|
||||||
|
|
||||||
await row.click();
|
await row.click();
|
||||||
// The pane below lists every one: the audio and the image.
|
// The Files pane lists every one: the audio and the image.
|
||||||
await expect(page.locator('#detail .encbox')).toHaveCount(2);
|
await expect(page.locator('#files .encbox')).toHaveCount(2);
|
||||||
await expect(page.locator('#detail .encbox').nth(1)).toContainText('image');
|
await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
|
||||||
});
|
});
|
||||||
|
|
||||||
test('the filter tabs change what is listed', async ({ page }) => {
|
test('the filter tabs change what is listed', async ({ page }) => {
|
||||||
@@ -135,7 +148,7 @@ test('the filter tabs change what is listed', async ({ page }) => {
|
|||||||
|
|
||||||
test('a feed URL is editable and has a copy button', async ({ page }) => {
|
test('a feed URL is editable and has a copy button', async ({ page }) => {
|
||||||
await page.getByText('Test Show').click();
|
await page.getByText('Test Show').click();
|
||||||
await page.locator('.btn', { hasText: 'Settings' }).first().click();
|
await page.locator('#content .acts [data-a="settings"]').click();
|
||||||
await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
|
await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
|
||||||
await expect(page.locator('#scopy')).toBeVisible();
|
await expect(page.locator('#scopy')).toBeVisible();
|
||||||
|
|
||||||
@@ -178,16 +191,50 @@ test('an OPML subscription is a collapsible folder', async ({ page }) => {
|
|||||||
const chev = page.locator('.feed.group .chev');
|
const chev = page.locator('.feed.group .chev');
|
||||||
await expect(chev).toBeVisible({ timeout: 20_000 });
|
await expect(chev).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
// Closed by default: the child is not listed until the folder is opened.
|
// Closed by default: the children are not listed until the folder is opened.
|
||||||
const before = await page.locator('.feed').count();
|
const before = await page.locator('.feed').count();
|
||||||
await chev.click();
|
await chev.click();
|
||||||
await expect(page.locator('.feed')).toHaveCount(before + 1);
|
await expect(page.locator('.feed')).toHaveCount(before + 2);
|
||||||
// Scoped to the sidebar: the name also appears as the page heading once selected.
|
// Scoped to the sidebar: the name also appears as the page heading once selected.
|
||||||
await expect(page.locator('#feedlist').getByText('Grouped Show')).toBeVisible();
|
await expect(page.locator('#feedlist').getByText('Grouped Show')).toBeVisible();
|
||||||
|
|
||||||
// The subscription's own page lists what is inside it.
|
// The subscription's own page lists what is inside it.
|
||||||
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
|
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
|
||||||
await expect(page.locator('.childrow')).toHaveCount(1);
|
await expect(page.locator('.childrow')).toHaveCount(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('inside an OPML, feeds with unread items are listed first', async ({ page }) => {
|
||||||
|
await page.evaluate(() =>
|
||||||
|
fetch('/api/fetch', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
|
||||||
|
}));
|
||||||
|
const chev = page.locator('.feed.group .chev');
|
||||||
|
await expect(chev).toBeVisible({ timeout: 20_000 });
|
||||||
|
await chev.click();
|
||||||
|
const side = page.locator('#feedlist');
|
||||||
|
await expect(side.getByText('Aardvark Radio')).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
|
// Other tests change read state, so set it here: Aardvark Radio read, Grouped Show not.
|
||||||
|
// Opening an item reads it; the toggle in the pane below flips it back.
|
||||||
|
await side.getByText('Aardvark Radio').click();
|
||||||
|
const aa = page.locator('.ep', { hasText: 'Aardvark Ep' });
|
||||||
|
await aa.click();
|
||||||
|
await expect(aa).toHaveClass(/read/);
|
||||||
|
await side.getByText('Grouped Show').click();
|
||||||
|
const gs = page.locator('.ep', { hasText: 'Grouped Ep' });
|
||||||
|
await gs.click();
|
||||||
|
await page.locator('#detail button[title="Mark unread"]').click();
|
||||||
|
await expect(gs).not.toHaveClass(/read/);
|
||||||
|
|
||||||
|
// Aardvark comes first alphabetically and in the OPML, so only the unread sort puts
|
||||||
|
// Grouped Show above it. The folder stays open across the reload (localStorage).
|
||||||
|
await page.reload();
|
||||||
|
const want = ['Grouped Show', 'Aardvark Radio'];
|
||||||
|
await expect(page.locator('#feedlist .feed.child b')).toHaveText(want, { timeout: 20_000 });
|
||||||
|
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
|
||||||
|
await expect(page.locator('.childrow b')).toHaveText(want);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('marking an OPML subscription read covers the feeds inside it', async ({ page }) => {
|
test('marking an OPML subscription read covers the feeds inside it', async ({ page }) => {
|
||||||
@@ -204,7 +251,7 @@ test('marking an OPML subscription read covers the feeds inside it', async ({ pa
|
|||||||
await expect(folder.locator('.badge')).not.toHaveText('0');
|
await expect(folder.locator('.badge')).not.toHaveText('0');
|
||||||
await folder.click();
|
await folder.click();
|
||||||
|
|
||||||
await page.locator('#content .acts button', { hasText: 'Mark all read' }).click();
|
await page.locator('#content .acts [data-a="read"]').click();
|
||||||
await expect(folder.locator('.badge')).toHaveText('0');
|
await expect(folder.locator('.badge')).toHaveText('0');
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -246,9 +293,9 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
|
|||||||
// Another test may have opened this item already, so start from a known state: the
|
// Another test may have opened this item already, so start from a known state: the
|
||||||
// toggle in the text below flips it back -- which used to recurse until the stack blew.
|
// toggle in the text below flips it back -- which used to recurse until the stack blew.
|
||||||
await row().click();
|
await row().click();
|
||||||
await page.locator('#detail button', { hasText: 'Mark unread' }).click();
|
await page.locator('#detail button[title="Mark unread"]').click();
|
||||||
await expect(row()).not.toHaveClass(/read/);
|
await expect(row()).not.toHaveClass(/read/);
|
||||||
await expect(page.locator('#detail button', { hasText: 'Mark read' })).toBeVisible();
|
await expect(page.locator('#detail button[title="Mark read"]')).toBeVisible();
|
||||||
|
|
||||||
// Opening it is reading it.
|
// Opening it is reading it.
|
||||||
await row().click();
|
await row().click();
|
||||||
@@ -256,3 +303,478 @@ test('opening an item marks it read, and the toggle flips it back', async ({ pag
|
|||||||
|
|
||||||
expect(errors).toEqual([]);
|
expect(errors).toEqual([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('the toolbar acts on the selected item', async ({ page }) => {
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
|
||||||
|
await expect(row()).toBeVisible({ timeout: 20_000 });
|
||||||
|
// Nothing selected, nothing to act on.
|
||||||
|
await expect(page.locator('#tbRead')).toBeDisabled();
|
||||||
|
|
||||||
|
await row().click(); // opening it reads it
|
||||||
|
await expect(row()).toHaveClass(/read/);
|
||||||
|
await page.locator('#tbRead').click();
|
||||||
|
await expect(row()).not.toHaveClass(/read/);
|
||||||
|
|
||||||
|
await page.locator('#tbFlag').click();
|
||||||
|
await expect(row().locator('.fl')).toHaveClass(/on/);
|
||||||
|
await page.locator('#tbFlag').click(); // and back, so later tests see it unkept
|
||||||
|
await expect(row().locator('.fl')).not.toHaveClass(/on/);
|
||||||
|
|
||||||
|
// Second Episode is the one the daemon downloaded, so it plays from the toolbar.
|
||||||
|
await expect(page.locator('#tbPlay')).toBeEnabled();
|
||||||
|
await page.locator('#tbPlay').click();
|
||||||
|
await expect(page.locator('#player')).toBeVisible();
|
||||||
|
await page.locator('#pclose').click();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a second person has their own feeds and their own read state', async ({ browser }) => {
|
||||||
|
const { execFileSync } = require('child_process');
|
||||||
|
const setup = require('./global-setup');
|
||||||
|
const env = {
|
||||||
|
...process.env,
|
||||||
|
IPX_CONFIG: `${setup.root}/config/config.toml`,
|
||||||
|
IPX_DATA_DIR: `${setup.root}/data`,
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
execFileSync('./target/debug/ipx', ['user', 'add', 'sam'], { input: 'sampassword', env });
|
||||||
|
} catch (e) {
|
||||||
|
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
|
||||||
|
}
|
||||||
|
|
||||||
|
// A fresh context, so none of the admin's cookies come along.
|
||||||
|
const ctx = await browser.newContext();
|
||||||
|
const page = await ctx.newPage();
|
||||||
|
await page.goto('/login');
|
||||||
|
await page.locator('#name').fill('sam');
|
||||||
|
await page.locator('#pw').fill('sampassword');
|
||||||
|
await page.locator('button[type=submit]').click();
|
||||||
|
await expect(page.locator('#feedlist')).toBeVisible();
|
||||||
|
|
||||||
|
// Sam subscribes to nothing yet, so sees nothing -- the admin's feeds are not theirs.
|
||||||
|
await expect(page.locator('#feedlist')).toContainText('No feeds.');
|
||||||
|
await expect(page.locator('#prefs')).toBeHidden(); // not an admin
|
||||||
|
// Hiding the button is not the guard; the server is.
|
||||||
|
expect((await page.request.get('/api/users')).status()).toBe(403);
|
||||||
|
await expect(page.locator('#logs')).toBeHidden();
|
||||||
|
expect((await page.request.get('/api/logs')).status()).toBe(403);
|
||||||
|
|
||||||
|
// Subscribing to a feed the admin already has costs no second fetch: same feed, same
|
||||||
|
// files, but Sam's own read state.
|
||||||
|
await page.locator('#addFeed').click();
|
||||||
|
await page.locator('#nurl').fill('http://127.0.0.1:8792/show.xml');
|
||||||
|
await page.locator('#nsave').click();
|
||||||
|
await expect(page.locator('.feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
|
await page.locator('.feed', { hasText: 'Test Show' }).click();
|
||||||
|
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
|
||||||
|
// The admin read these earlier in this file; for Sam they are all still unread.
|
||||||
|
const rows = await page.locator('.ep').count();
|
||||||
|
await page.locator('.tabs button', { hasText: 'Unread' }).click();
|
||||||
|
await expect(page.locator('.ep')).toHaveCount(rows);
|
||||||
|
|
||||||
|
// Two people now share this feed, so the page says so and Delete is honest about it.
|
||||||
|
await expect(page.locator('#content .sub').first()).toContainText('shared with 1 other person');
|
||||||
|
|
||||||
|
await ctx.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('deleting a shared file warns that it is everyone\'s copy', async ({ page }) => {
|
||||||
|
// Admin and Sam both subscribe to Test Show by now, and the daemon downloaded a file.
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
await page.locator('.tabs button', { hasText: 'Downloaded' }).click();
|
||||||
|
const row = page.locator('.ep').first();
|
||||||
|
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||||
|
await row.click();
|
||||||
|
|
||||||
|
// An icon now; what it does, and to whom, is in its tooltip.
|
||||||
|
const del = page.locator('#files button[data-a="del"]');
|
||||||
|
await expect(del).toHaveAttribute('title', /^Delete for everyone \(shared with 1 other person/);
|
||||||
|
|
||||||
|
// Two prompts: the page's own, then the server's, because someone else has not played
|
||||||
|
// it. Accept the first, decline the second, and the file must survive.
|
||||||
|
const seen = [];
|
||||||
|
page.on('dialog', d => {
|
||||||
|
seen.push(d.message());
|
||||||
|
if (seen.length === 1) d.accept();
|
||||||
|
else d.dismiss();
|
||||||
|
});
|
||||||
|
await del.click();
|
||||||
|
await expect.poll(() => seen.length, { timeout: 10_000 }).toBe(2);
|
||||||
|
expect(seen[0]).toContain('shared with 1 other person');
|
||||||
|
expect(seen[1]).toContain('one copy of this file');
|
||||||
|
|
||||||
|
await page.reload();
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
await page.locator('.tabs button', { hasText: 'Downloaded' }).click();
|
||||||
|
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
// Every row says "Admin" on its checkbox, so match the name exactly.
|
||||||
|
const userRow = (page, name) =>
|
||||||
|
page.locator('#modalCard [data-id]').filter({ has: page.locator('b', { hasText: new RegExp(`^${name}$`) }) });
|
||||||
|
|
||||||
|
async function openUsers(page) {
|
||||||
|
await page.locator('#prefs').click();
|
||||||
|
await page.locator('#gusers').click();
|
||||||
|
await expect(userRow(page, 'admin')).toBeVisible();
|
||||||
|
}
|
||||||
|
|
||||||
|
test('an admin adds someone, makes them an admin, and removes them', async ({ page }) => {
|
||||||
|
await openUsers(page);
|
||||||
|
await page.locator('#uname').fill('pat');
|
||||||
|
await page.locator('#upass').fill('patpassword');
|
||||||
|
await page.locator('#uadd').click();
|
||||||
|
const row = userRow(page, 'pat');
|
||||||
|
await expect(row).toBeVisible();
|
||||||
|
await expect(row.locator('[data-a="admin"]')).not.toBeChecked();
|
||||||
|
|
||||||
|
await row.locator('[data-a="admin"]').check();
|
||||||
|
// Not just the box: it has to have reached the database.
|
||||||
|
await expect.poll(async () =>
|
||||||
|
(await (await page.request.get('/api/users')).json()).find(u => u.name === 'pat')?.admin
|
||||||
|
).toBe(true);
|
||||||
|
|
||||||
|
page.once('dialog', d => d.accept());
|
||||||
|
await row.locator('[data-a="rm"]').click();
|
||||||
|
await expect(row).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the only admin cannot be demoted or removed', async ({ page }) => {
|
||||||
|
await openUsers(page);
|
||||||
|
await userRow(page, 'admin').locator('[data-a="admin"]').click();
|
||||||
|
await expect(page.locator('.toast.bad')).toContainText('only admin');
|
||||||
|
// Redrawn from the server, so the box is back.
|
||||||
|
await expect(userRow(page, 'admin').locator('[data-a="admin"]')).toBeChecked();
|
||||||
|
|
||||||
|
const me = (await (await page.request.get('/api/users')).json()).find(u => u.name === 'admin');
|
||||||
|
expect((await page.request.delete(`/api/users/${me.id}`)).status()).toBe(400);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Settings exports your OPML and imports a pasted one', async ({ page }) => {
|
||||||
|
await page.locator('#prefs').click();
|
||||||
|
const [dl] = await Promise.all([
|
||||||
|
page.waitForEvent('download'),
|
||||||
|
page.locator('#modalCard a[title="Export OPML"]').click(),
|
||||||
|
]);
|
||||||
|
expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml');
|
||||||
|
const out = require('fs').readFileSync(await dl.path(), 'utf8');
|
||||||
|
for (const f of ['show.xml', 'pics.xml', 'multi.xml', 'subs.opml']) expect(out).toContain(f);
|
||||||
|
// A feed from an OPML subscription comes back with the OPML itself, not on its own.
|
||||||
|
expect(out).not.toContain('other.xml');
|
||||||
|
|
||||||
|
// One feed new to everyone, one the admin already has.
|
||||||
|
await page.locator('#gopml').click();
|
||||||
|
await page.locator('#opmlText').fill('<opml version="2.0"><head><title>t</title></head><body>' +
|
||||||
|
'<outline text="Imported Show" xmlUrl="http://127.0.0.1:8792/imported.xml"/>' +
|
||||||
|
'<outline text="Test Show" xmlUrl="http://127.0.0.1:8792/show.xml"/></body></opml>');
|
||||||
|
await page.locator('#oimp').click();
|
||||||
|
await expect(page.locator('.toast', { hasText: 'Subscribed to' }))
|
||||||
|
.toHaveText('Subscribed to 1 feed(s), 1 you already had');
|
||||||
|
// Named from the OPML's id until the first scan reads the feed's own title.
|
||||||
|
await expect(page.locator('#feedlist .feed', { hasText: /Imported Show|imported-show/ }))
|
||||||
|
.toBeVisible({ timeout: 20_000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an uploaded OPML file imports, and a file that is not OPML is refused', async ({ page }) => {
|
||||||
|
await page.locator('#prefs').click();
|
||||||
|
await page.locator('#gopml').click();
|
||||||
|
const pick = page.locator('#opmlFile');
|
||||||
|
|
||||||
|
// An RSS feed is XML but not OPML: refused in the page, and the picker lets go of it.
|
||||||
|
await pick.setInputFiles({
|
||||||
|
name: 'feed.xml', mimeType: 'application/xml',
|
||||||
|
buffer: require('fs').readFileSync(require('path').join(__dirname, 'fixtures', 'show.xml')),
|
||||||
|
});
|
||||||
|
await page.locator('#oimp').click();
|
||||||
|
await expect(page.locator('.toast.bad', { hasText: 'feed.xml is not an OPML file' })).toBeVisible();
|
||||||
|
expect(await pick.evaluate(i => i.files.length)).toBe(0);
|
||||||
|
|
||||||
|
// Something that gets past the page's quick look is still refused by the server, untouched.
|
||||||
|
const sneaky = await page.request.post('/api/opml', {
|
||||||
|
data: { xml: '<rss version="2.0"><channel><title><opml></title></channel></rss>' },
|
||||||
|
});
|
||||||
|
expect(sneaky.status()).toBe(400);
|
||||||
|
expect(await sneaky.text()).toContain('not an OPML file');
|
||||||
|
|
||||||
|
// A real one. Multi Show is already the admin's, so it counts as already had.
|
||||||
|
await pick.setInputFiles({
|
||||||
|
name: 'subs.opml', mimeType: 'text/x-opml',
|
||||||
|
buffer: Buffer.from('<opml version="2.0"><head><title>t</title></head><body>' +
|
||||||
|
'<outline text="Multi Show" xmlUrl="http://127.0.0.1:8792/multi.xml"/></body></opml>'),
|
||||||
|
});
|
||||||
|
await page.locator('#oimp').click();
|
||||||
|
await expect(page.locator('.toast', { hasText: 'Subscribed to' }))
|
||||||
|
.toHaveText('Subscribed to 0 feed(s), 1 you already had');
|
||||||
|
await expect(page.locator('#modal.on')).toBeHidden();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an export from one account imports into another', async ({ page, browser }) => {
|
||||||
|
// Regression: import only added URLs the catalogue lacked and subscribed nobody, so importing
|
||||||
|
// the admin's export into a second account did nothing at all.
|
||||||
|
const { execFileSync } = require('child_process');
|
||||||
|
const setup = require('./global-setup');
|
||||||
|
const env = {
|
||||||
|
...process.env,
|
||||||
|
IPX_CONFIG: `${setup.root}/config/config.toml`,
|
||||||
|
IPX_DATA_DIR: `${setup.root}/data`,
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
execFileSync('./target/debug/ipx', ['user', 'add', 'opal'], { input: 'opalpassword', env });
|
||||||
|
} catch (e) {
|
||||||
|
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
|
||||||
|
}
|
||||||
|
const ctx = await browser.newContext();
|
||||||
|
const opal = await ctx.newPage();
|
||||||
|
await opal.goto('/login');
|
||||||
|
await opal.locator('#name').fill('opal');
|
||||||
|
await opal.locator('#pw').fill('opalpassword');
|
||||||
|
await opal.locator('button[type=submit]').click();
|
||||||
|
await expect(opal.locator('#feedlist')).toContainText('No feeds.');
|
||||||
|
|
||||||
|
// Export used to hand anyone the whole catalogue. Opal has nothing yet, so gets nothing.
|
||||||
|
const empty = await opal.request.get('/api/opml');
|
||||||
|
expect(empty.status()).toBe(200);
|
||||||
|
expect(await empty.text()).not.toContain('xmlUrl');
|
||||||
|
|
||||||
|
const urlsIn = xml => [...xml.matchAll(/xmlUrl="([^"]+)"/g)].map(m => m[1]).sort();
|
||||||
|
const exported = await (await page.request.get('/api/opml')).text(); // the admin's
|
||||||
|
const urls = urlsIn(exported);
|
||||||
|
expect(urls.length).toBeGreaterThan(2);
|
||||||
|
expect(await (await opal.request.post('/api/opml', { data: { xml: exported } })).json())
|
||||||
|
.toEqual({ added: urls.length, already: 0 });
|
||||||
|
expect(await (await opal.request.post('/api/opml', { data: { xml: exported } })).json())
|
||||||
|
.toEqual({ added: 0, already: urls.length });
|
||||||
|
|
||||||
|
await opal.reload();
|
||||||
|
await expect(opal.locator('.feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
|
// The round trip closes: opal's own export now lists what the admin's did.
|
||||||
|
expect(urlsIn(await (await opal.request.get('/api/opml')).text())).toEqual(urls);
|
||||||
|
await ctx.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('ipx import subscribes the admin, and ipx export writes the feeds out', async () => {
|
||||||
|
// Its own config and database. The CLI works in-process, and the suite's running daemon
|
||||||
|
// reads config.toml once at start, so it would not see what the CLI added anyway.
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
const { execFileSync } = require('child_process');
|
||||||
|
const setup = require('./global-setup');
|
||||||
|
const dir = path.join(setup.root, 'cli');
|
||||||
|
fs.rmSync(dir, { recursive: true, force: true });
|
||||||
|
fs.mkdirSync(path.join(dir, 'data'), { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(dir, 'config.toml'),
|
||||||
|
`[general]\ndownload_dir = "${dir}/downloads"\nsocket = "${dir}/ipx.sock"\n`);
|
||||||
|
const env = { ...process.env, IPX_CONFIG: path.join(dir, 'config.toml'), IPX_DATA_DIR: path.join(dir, 'data') };
|
||||||
|
const ipx = (args, input) => execFileSync('./target/debug/ipx', args, { env, input, encoding: 'utf8' });
|
||||||
|
|
||||||
|
ipx(['user', 'add', 'boss'], 'bosspassword'); // the first account is the admin
|
||||||
|
const opml = path.join(dir, 'in.opml');
|
||||||
|
fs.writeFileSync(opml, '<opml version="2.0"><head><title>t</title></head><body>' +
|
||||||
|
'<outline text="One" xmlUrl="http://127.0.0.1:8792/one.xml"/>' +
|
||||||
|
'<outline text="Two" xmlUrl="http://127.0.0.1:8792/two.xml"/></body></opml>');
|
||||||
|
expect(ipx(['import', opml])).toContain('subscribed boss to 2 feed(s); 0 already there');
|
||||||
|
expect(ipx(['import', opml])).toContain('subscribed boss to 0 feed(s); 2 already there');
|
||||||
|
|
||||||
|
const out = path.join(dir, 'out.opml');
|
||||||
|
ipx(['export', out]);
|
||||||
|
const xml = fs.readFileSync(out, 'utf8');
|
||||||
|
expect(xml).toContain('http://127.0.0.1:8792/one.xml');
|
||||||
|
expect(xml).toContain('http://127.0.0.1:8792/two.xml');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
|
||||||
|
const { execFileSync } = require('child_process');
|
||||||
|
const setup = require('./global-setup');
|
||||||
|
const env = {
|
||||||
|
...process.env,
|
||||||
|
IPX_CONFIG: `${setup.root}/config/config.toml`,
|
||||||
|
IPX_DATA_DIR: `${setup.root}/data`,
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
execFileSync('./target/debug/ipx', ['user', 'add', 'piper'], { input: 'piperpassword', env });
|
||||||
|
} catch (e) {
|
||||||
|
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
|
||||||
|
}
|
||||||
|
const ctx = await browser.newContext();
|
||||||
|
const piper = await ctx.newPage();
|
||||||
|
await piper.goto('/login');
|
||||||
|
await piper.locator('#name').fill('piper');
|
||||||
|
await piper.locator('#pw').fill('piperpassword');
|
||||||
|
await piper.locator('button[type=submit]').click();
|
||||||
|
await expect(piper.locator('#feedlist')).toContainText('No feeds.');
|
||||||
|
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||||
|
const offered = piper.locator('#popular .childrow');
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
|
// An OPML's own feeds ride on the OPML, and a key in a URL marks someone's paid feed.
|
||||||
|
await expect(offered.filter({ hasText: /Grouped Show|grouped-show/ })).toHaveCount(0);
|
||||||
|
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||||
|
|
||||||
|
// No URL reaches the page at all, so neither can a key, and the server holds the same line.
|
||||||
|
const listed = await (await piper.request.get('/api/popular')).text();
|
||||||
|
expect(listed).not.toContain('secret123');
|
||||||
|
expect(listed).not.toContain('.xml');
|
||||||
|
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
|
||||||
|
|
||||||
|
// Popular is the top ten of the directory, and the directory is every listed feed, A to Z.
|
||||||
|
const dir = await (await piper.request.get('/api/directory')).json();
|
||||||
|
const top = await (await piper.request.get('/api/popular')).json();
|
||||||
|
const names = dir.map(p => (p.title || p.id).toLowerCase());
|
||||||
|
expect(names).toEqual([...names].sort());
|
||||||
|
expect(top.length).toBe(Math.min(10, dir.length));
|
||||||
|
expect(top.every(t => dir.some(d => d.id === t.id))).toBe(true);
|
||||||
|
expect(dir.map(p => p.id)).not.toContain('paid-show');
|
||||||
|
|
||||||
|
// Subscribe from the directory this time; the popular list shares the same rows.
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||||
|
await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible();
|
||||||
|
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||||
|
|
||||||
|
const row = async () =>
|
||||||
|
(await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
|
||||||
|
const before = await row();
|
||||||
|
expect(before.subscribed).toBe(false);
|
||||||
|
await offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
|
||||||
|
await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
|
// Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
|
||||||
|
expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
|
||||||
|
|
||||||
|
// All Subscriptions is every item from piper's feeds and only those: the admin's Picture
|
||||||
|
// Blog is not among them.
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'All Subscriptions' }).click();
|
||||||
|
const first = piper.locator('.ep', { hasText: 'First Episode' });
|
||||||
|
await expect(first).toBeVisible({ timeout: 20_000 });
|
||||||
|
await expect(first.locator('.fd')).toHaveText('Test Show');
|
||||||
|
await expect(piper.locator('.ep', { hasText: 'An Article' })).toHaveCount(0);
|
||||||
|
await expect(piper.locator('#count')).toContainText('All Subscriptions:');
|
||||||
|
await ctx.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('adding a feed scans it straight away', async ({ page }) => {
|
||||||
|
await page.locator('#addFeed').click();
|
||||||
|
await page.locator('#nurl').fill('http://127.0.0.1:8792/fresh.xml');
|
||||||
|
await page.locator('#nsave').click();
|
||||||
|
// Nobody pressed Scan. The scheduler's tick is a minute, so this is the add scanning it.
|
||||||
|
await expect(page.locator('.ep', { hasText: 'Fresh Ep' })).toBeVisible({ timeout: 10_000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a deleted file looks as if it was never downloaded', async ({ page }) => {
|
||||||
|
// Other people subscribe to Picture Blog by now, so both prompts come; take them.
|
||||||
|
page.on('dialog', d => d.accept());
|
||||||
|
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
|
||||||
|
const row = page.locator('.ep', { hasText: 'An Article' });
|
||||||
|
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||||
|
await row.click();
|
||||||
|
await page.locator('#files button[data-a="del"]').click();
|
||||||
|
|
||||||
|
// No "reaped", no chip at all: just the way to get it again.
|
||||||
|
await expect(row.locator('.kind.here')).toHaveCount(0);
|
||||||
|
await expect(row).not.toContainText(/reaped/i);
|
||||||
|
await row.click();
|
||||||
|
await expect(page.locator('#files')).not.toContainText(/reaped/i);
|
||||||
|
await expect(page.locator('#files button[title="Download to the server"]')).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('one action, one icon: the toolbar, the page and every dialog agree', async ({ page }) => {
|
||||||
|
const icon = loc => loc.locator('svg path').first().getAttribute('d');
|
||||||
|
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
|
||||||
|
|
||||||
|
// Unsubscribe is a minus in the toolbar and the feed header, never the x that closes things.
|
||||||
|
expect(await icon(page.locator('#content .acts [data-a="rm"]'))).toBe(await icon(page.locator('#tbRemove')));
|
||||||
|
|
||||||
|
// The toolbar's read and keep show the selected item's state, as its own buttons do, and follow
|
||||||
|
// a change made from the toolbar.
|
||||||
|
await page.locator('.ep').first().click();
|
||||||
|
const pair = async a => [await icon(page.locator(a === 'read' ? '#tbRead' : '#tbFlag')),
|
||||||
|
await icon(page.locator(`#detail [data-a="${a}"]`))];
|
||||||
|
for (const a of ['read', 'flag']) { const [tb, own] = await pair(a); expect(tb).toBe(own); }
|
||||||
|
const [kept] = await pair('flag');
|
||||||
|
await page.locator('#tbFlag').click();
|
||||||
|
await expect.poll(async () => { const [tb, own] = await pair('flag'); return tb === own && tb !== kept; }).toBe(true);
|
||||||
|
await page.locator('#tbFlag').click(); // leave it as it was
|
||||||
|
await expect.poll(async () => (await pair('flag'))[0]).toBe(kept);
|
||||||
|
|
||||||
|
// Every button in every dialog is an icon with its words in the tooltip.
|
||||||
|
const dialogs = [
|
||||||
|
() => page.locator('#addFeed').click(),
|
||||||
|
() => page.locator('#prefs').click(),
|
||||||
|
async () => { await page.locator('#prefs').click(); await page.locator('#gusers').click(); },
|
||||||
|
async () => { await page.locator('#prefs').click(); await page.locator('#gopml').click(); },
|
||||||
|
() => page.locator('#logs').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="settings"]').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="dl"]').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="rm"]').click(),
|
||||||
|
];
|
||||||
|
for (const open of dialogs) {
|
||||||
|
await open();
|
||||||
|
const btns = page.locator('#modalCard .btn');
|
||||||
|
await expect(btns.first()).toBeVisible();
|
||||||
|
for (const b of await btns.all()) {
|
||||||
|
await expect(b.locator('svg')).toHaveCount(1);
|
||||||
|
await expect(b).toHaveAttribute('title', /\S/);
|
||||||
|
}
|
||||||
|
await page.keyboard.press('Escape');
|
||||||
|
await expect(page.locator('#modal.on')).toBeHidden();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('All Subscriptions marks everything read, across every feed', async ({ page }) => {
|
||||||
|
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
|
||||||
|
await all.click();
|
||||||
|
// Earlier tests read things; make sure something is unread. Opening an item reads it, and
|
||||||
|
// its own button makes it unread again.
|
||||||
|
await page.locator('.ep').first().click();
|
||||||
|
await page.locator('#detail [data-a="read"][title="Mark unread"]').click();
|
||||||
|
await expect(all.locator('.badge')).not.toHaveText('0');
|
||||||
|
|
||||||
|
page.once('dialog', d => d.accept());
|
||||||
|
await page.locator('#content .acts [data-a="readall"]').click();
|
||||||
|
await expect(all.locator('.badge')).toHaveText('0');
|
||||||
|
await page.locator('.tabs button', { hasText: 'Unread' }).click();
|
||||||
|
await expect(page.locator('.ep')).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the item table sorts by any column, both ways, and remembers', async ({ page }) => {
|
||||||
|
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
|
||||||
|
await all.click();
|
||||||
|
const head = k => page.locator(`#list .ephead [data-sort="${k}"]`);
|
||||||
|
const titles = () => page.locator('#eps .ep .t').allTextContents();
|
||||||
|
// Byte order on lower case, which is what SQLite gives for lower(...).
|
||||||
|
const cmp = (a, b) => (a.toLowerCase() < b.toLowerCase() ? -1 : a.toLowerCase() > b.toLowerCase() ? 1 : 0);
|
||||||
|
const sorted = (t, dir) => JSON.stringify(t) === JSON.stringify([...t].sort((a, b) => cmp(a, b) * dir));
|
||||||
|
await expect(page.locator('#eps .ep').nth(2)).toBeVisible({ timeout: 20_000 });
|
||||||
|
expect(new Set(await titles()).size).toBeGreaterThan(2); // or both orders would prove nothing
|
||||||
|
|
||||||
|
await expect(head('title')).toHaveText('Title');
|
||||||
|
await head('title').click();
|
||||||
|
await expect.poll(async () => sorted(await titles(), 1)).toBe(true);
|
||||||
|
await head('title').click();
|
||||||
|
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
|
||||||
|
|
||||||
|
// Kept across a reload.
|
||||||
|
await page.reload();
|
||||||
|
await all.click();
|
||||||
|
await expect(head('title').locator('.arr.desc')).toBeVisible();
|
||||||
|
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
|
||||||
|
|
||||||
|
// Size has its own column; the file column is just what the file is.
|
||||||
|
await expect(page.locator('#eps .ep .size', { hasText: /\d/ }).first()).toBeVisible();
|
||||||
|
await expect(page.locator('#eps .ep .file', { hasText: /\d/ })).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('play in the Files pane plays once, in the player bar', async ({ page }) => {
|
||||||
|
// Regression: the pane had an <audio> of its own, and playing it started the player bar too,
|
||||||
|
// so the same file played twice at once.
|
||||||
|
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
|
||||||
|
await page.locator('.ep', { has: page.locator('.kind.here') }).first().click();
|
||||||
|
await page.locator('#files [data-a="play"]').click();
|
||||||
|
await expect(page.locator('#player')).toBeVisible();
|
||||||
|
await expect(page.locator('audio')).toHaveCount(1); // the player bar's, and nothing else
|
||||||
|
await page.locator('#pclose').click();
|
||||||
|
});
|
||||||
|
|||||||
5
tests/ui/fixtures/aardvark.xml
Normal file
5
tests/ui/fixtures/aardvark.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Aardvark Radio</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Inside the OPML, and first in it and alphabetically.</description>
|
||||||
|
<item><title>Aardvark Ep</title><guid>aa-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
5
tests/ui/fixtures/fresh.xml
Normal file
5
tests/ui/fixtures/fresh.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Fresh Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Added in the browser suite, and scanned by adding it.</description>
|
||||||
|
<item><title>Fresh Ep</title><guid>fresh-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
5
tests/ui/fixtures/imported.xml
Normal file
5
tests/ui/fixtures/imported.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Imported Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Only ever arrives through an OPML import.</description>
|
||||||
|
<item><title>Imported Ep</title><guid>imp-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
5
tests/ui/fixtures/paid.xml
Normal file
5
tests/ui/fixtures/paid.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Paid Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Subscribed with a key in its URL, so it must never be offered to anyone else.</description>
|
||||||
|
<item><title>Paid Ep</title><guid>paid-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
@@ -1,4 +1,5 @@
|
|||||||
<opml version="2.0"><head><title>Test Subscriptions</title></head>
|
<opml version="2.0"><head><title>Test Subscriptions</title></head>
|
||||||
<body><outline text="Folder">
|
<body><outline text="Folder">
|
||||||
|
<outline type="rss" text="Aardvark Radio" xmlUrl="http://127.0.0.1:8792/aardvark.xml"/>
|
||||||
<outline type="rss" text="Grouped Show" xmlUrl="http://127.0.0.1:8792/other.xml"/>
|
<outline type="rss" text="Grouped Show" xmlUrl="http://127.0.0.1:8792/other.xml"/>
|
||||||
</outline></body></opml>
|
</outline></body></opml>
|
||||||
@@ -10,7 +10,15 @@ const TOKEN = 'testtokentesttokentesttoken12345'; // fixed, so tests need not
|
|||||||
// Called from playwright.config.js at load time, NOT as globalSetup: Playwright starts
|
// Called from playwright.config.js at load time, NOT as globalSetup: Playwright starts
|
||||||
// webServer *before* globalSetup, so a config written there does not exist yet when the
|
// webServer *before* globalSetup, so a config written there does not exist yet when the
|
||||||
// daemon launches -- it would fall back to the real config and fight the live daemon.
|
// daemon launches -- it would fall back to the real config and fight the live daemon.
|
||||||
|
// Playwright imports this config again in every worker process, so prepare() runs more
|
||||||
|
// than once per suite. Wiping on the second call deleted the data directory out from under
|
||||||
|
// the running daemon: it kept serving from the unlinked inode, while anything else opening
|
||||||
|
// that path -- the CLI, a query -- got a brand new empty database and disagreed with it.
|
||||||
function prepare() {
|
function prepare() {
|
||||||
|
// Only the process that launches the run may wipe. A worker gets TEST_WORKER_INDEX.
|
||||||
|
if (process.env.TEST_WORKER_INDEX !== undefined || process.env.PW_WORKER_INDEX !== undefined) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
fs.rmSync(root, { recursive: true, force: true });
|
fs.rmSync(root, { recursive: true, force: true });
|
||||||
for (const d of ['config', 'data', 'downloads']) {
|
for (const d of ['config', 'data', 'downloads']) {
|
||||||
fs.mkdirSync(path.join(root, d), { recursive: true });
|
fs.mkdirSync(path.join(root, d), { recursive: true });
|
||||||
@@ -47,6 +55,11 @@ auto_download = true
|
|||||||
[feeds.test-subscriptions]
|
[feeds.test-subscriptions]
|
||||||
url = "http://127.0.0.1:8792/subs.opml"
|
url = "http://127.0.0.1:8792/subs.opml"
|
||||||
auto_download = false
|
auto_download = false
|
||||||
|
|
||||||
|
# A key in its URL, like a Patreon feed: someone's paid subscription, never offered to others.
|
||||||
|
[feeds.paid-show]
|
||||||
|
url = "http://127.0.0.1:8792/paid.xml?auth=secret123"
|
||||||
|
auto_download = false
|
||||||
`);
|
`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
965
web/index.html
965
web/index.html
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user