Compare commits
72 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 9eb7aadced | |||
| c6bceaef37 | |||
| dc63d6acaf | |||
| 8937f35f00 | |||
| 0990f2a90d | |||
| 0cc002cdfa | |||
| 2c9e899762 | |||
| 2e416f96cf | |||
| 9269aa99f7 | |||
| ae8123250b | |||
| 2c34a144ba | |||
| 669e8b5124 | |||
| 5362436766 | |||
| 0efc49519c | |||
| 57dcba2d1a | |||
| e95cccc66f | |||
| 7295be8b25 | |||
| f1b0d97b81 | |||
| ae47e31a97 | |||
| 8a309eb652 | |||
| 1a2b0d87c6 | |||
| d7fac2d0e7 | |||
| df9b7645d6 | |||
| f3825cfc57 | |||
| c0f4b0bcb2 | |||
| 8b8b48302b | |||
| 5d3fdde4da | |||
| 8784d0a3fd | |||
| 5e95557cbb | |||
| 6114add4a6 | |||
| c47c224372 | |||
| 686851b448 | |||
| f0d03c79c8 | |||
| 7df4ee7dde | |||
| d46ec73261 | |||
| 4810bb5cfb | |||
| 06f182b555 | |||
| ed26fa2061 | |||
| 772dda5154 | |||
| 3817dbebdd | |||
| 3ee65f3155 | |||
| d157c88ba9 | |||
| 7473d5b7fb | |||
| 8f5e2749ff | |||
| dfcdf47143 | |||
| 470f3e1ff1 | |||
| c77d152015 | |||
| 5190a29cdb | |||
| dd9c0881ee | |||
| 6d6b4dd1e4 | |||
| ddb9dbb7e1 | |||
| a83f62ccd3 | |||
| 8b21d19365 | |||
| 4429f1119c | |||
| 665d5b8ecb | |||
| c86d698363 | |||
| 5f6e2a8dc1 | |||
| 19309d609f | |||
| e97f2b9c2f | |||
| 9960befed5 | |||
| 9dc4c1ddfa | |||
| 9c5e7716c5 | |||
| cbd5e68d13 | |||
| e05ea62051 | |||
| aecad18d22 | |||
| 44623098ae | |||
| 72280b9ccd | |||
| 5666166769 | |||
| 93b4815d84 | |||
| 74ec6e9281 | |||
| ed47e456d4 | |||
| c12e8ca19c |
4
.dockerignore
Normal file
4
.dockerignore
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
target/
|
||||||
|
.git/
|
||||||
|
*.md
|
||||||
|
tests/
|
||||||
3
.gitignore
vendored
3
.gitignore
vendored
@@ -1 +1,4 @@
|
|||||||
/target
|
/target
|
||||||
|
/node_modules
|
||||||
|
/test-results
|
||||||
|
/playwright-report
|
||||||
|
|||||||
266
CHANGELOG.md
Normal file
266
CHANGELOG.md
Normal file
@@ -0,0 +1,266 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||||
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
The long form, with what was wrong before and how it was found, is in
|
||||||
|
[docs/history.md](docs/history.md).
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
## [0.5.0] - 2026-09-12
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- A Patreon token pasted into Add feed, or a creator's RSS link without `&show=`, becomes a folder
|
||||||
|
of that creator's shows, kept in step on every scan like a subscribed OPML. A creator with only
|
||||||
|
one show stays a plain feed. One already added as a single long feed is split into its shows on
|
||||||
|
its next scan, keeping its files and what you had read.
|
||||||
|
- Add feed has an "Allow items marked explicit" box, so a new feed's first scan no longer skips
|
||||||
|
every explicit item.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Unread counts, unread dots and download progress are amber, the colour of the icon's EQ bars.
|
||||||
|
Blue is kept for the primary action and links, so a count no longer looks like a button.
|
||||||
|
- What is playing is marked by small EQ bars, in its row and in the player. They move only while it
|
||||||
|
plays.
|
||||||
|
- A folder in the sidebar shows its first four shows' art as a mosaic, and its shows sit under its
|
||||||
|
title. Only folders have a triangle, so every feed lines up with Directory and Popular above.
|
||||||
|
- Feeds without art get initials in a colour of their own, instead of all the same grey.
|
||||||
|
- The Flagged tab is called Kept, as the Keep button and Settings already said.
|
||||||
|
- A feed's header is one short line; when it checks next is in its tooltip.
|
||||||
|
- The item list takes more of the window, and the Files pane shows only when the item has files.
|
||||||
|
- Column headings and tags are in sentence case, and fewer things are bold.
|
||||||
|
- Unsubscribe is a round button beside the feed's other actions.
|
||||||
|
- Export and Import in Settings say what they do.
|
||||||
|
- The sign-in page shows the original icon large.
|
||||||
|
- Nothing animates when your system asks for reduced motion.
|
||||||
|
- The pages are about 90 KB smaller: the icon is served once instead of written into each.
|
||||||
|
- A web token generated for a new install is 64 characters instead of 32.
|
||||||
|
- The README is a short overview of what ipx does and how to run it, and points into `docs/` for
|
||||||
|
the rest. It still described the layout from before 0.4.0.
|
||||||
|
|
||||||
|
### Removed
|
||||||
|
|
||||||
|
- The systemd units in `contrib/`. Run ipx with Docker, or point a unit of your own at
|
||||||
|
`ipx daemon`.
|
||||||
|
- Upgrading from before 0.3.0 directly: what was read, kept or part-played before accounts is no
|
||||||
|
longer carried over to the admin, and OPML feeds that old versions wrote into `config.toml` are
|
||||||
|
no longer moved out of it. Upgrade through 0.4.0 first.
|
||||||
|
- `interval_mins` in `config.toml` is ignored; use `schedule`.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- The feed list works from the keyboard: Tab reaches every feed, Enter opens it, and Right and Left
|
||||||
|
open and close a folder. Every button shows where the focus is, including in the toolbar, which
|
||||||
|
used to clip the ring.
|
||||||
|
- "1 items" reads "1 item".
|
||||||
|
- A selected feed without art no longer loses its initials tile in Dark and Light.
|
||||||
|
- The player shows the feed's initials when there is no art, not the episode's.
|
||||||
|
- Turning on Allow explicit, or changing keywords or auto-download, brings back what those settings
|
||||||
|
had skipped on the feed's next scan. Before, an item was judged once, when first seen, and a
|
||||||
|
skipped one stayed skipped whatever you changed.
|
||||||
|
- Feeds inside an OPML or a Patreon creator follow your settings on the folder unless you set their
|
||||||
|
own, as the folder's settings dialog said they did. Before, the folder's settings reached nothing
|
||||||
|
inside it.
|
||||||
|
- A new feed no longer takes the name of one you removed earlier and shows that feed's old items.
|
||||||
|
Re-adding the same feed still gets its old name, and its history, back.
|
||||||
|
|
||||||
|
## [0.4.0] - 2026-09-11
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- A Classic theme after the 2004 Mac app, beside Dark and Light: brushed-metal toolbar, Aqua
|
||||||
|
blue selection, red unread badges, a striped table and Lucida Grande. The theme button steps
|
||||||
|
through all three and remembers the choice.
|
||||||
|
- A toolbar across the top, after the original iPodderX: add and unsubscribe, play, mark read
|
||||||
|
and keep for the selected item, scan, a search box for what is showing, and Settings and Log.
|
||||||
|
- Directory, Popular and All Subscriptions at the top of the feed list, opening in the main pane.
|
||||||
|
Directory lists every feed anyone here subscribes to, A to Z (`GET /api/directory`). All
|
||||||
|
Subscriptions lists every item from every feed you subscribe to (`GET /api/entries`).
|
||||||
|
- Items show as a table (unread, kept, title, feed, file, size, published) with a Files pane beside
|
||||||
|
it, and a status bar with the totals.
|
||||||
|
- Mark everything read from All Subscriptions, across every feed you subscribe to
|
||||||
|
(`POST /api/read-all`). It asks first. All Subscriptions can also check every feed from its header.
|
||||||
|
- Click a column heading in the item table to sort by it (kept, title, feed, file type, size,
|
||||||
|
published); click again to reverse. The server sorts, so it covers the whole list, not just the
|
||||||
|
fifty shown, and the choice is remembered.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Popular shows the top 10, not 20, and counts everyone, you included. Your own feeds stay on it,
|
||||||
|
marked Subscribed, and clicking one opens it.
|
||||||
|
- Adding a feed scans it straight away, and an OPML import that added feeds scans them, so their
|
||||||
|
items show without pressing Scan.
|
||||||
|
- A file deleted to save space, or by hand, looks as if it was never downloaded: no "reaped"
|
||||||
|
label, just the Download button. The retention summary says "deleted", not "reaped".
|
||||||
|
- Buttons are icons, with the words in their tooltips: the Files pane (save, delete, view,
|
||||||
|
download), an item's own buttons (mark read, keep, open the original), the feed header (scan,
|
||||||
|
download latest, mark all read, settings, unsubscribe), and the Settings, feed settings and
|
||||||
|
Download latest dialogs (save, download, cancel). The icons are Font Awesome Free, embedded as
|
||||||
|
SVG: only the ones used, no font to download, and nothing fetched from anyone else. They
|
||||||
|
replace font characters such as ⟳ ⤓ ↗, which came out thin and tiny and differed from font to
|
||||||
|
font. Keep is a flag everywhere, as it was in the original, and mark unread is an envelope.
|
||||||
|
- One meaning per icon. Minus unsubscribes, x closes or cancels, plus adds or subscribes, and a
|
||||||
|
dialog's confirm button carries the icon of what it does. The feed header's unsubscribe was an x
|
||||||
|
and read as closing the page. The remaining word buttons are icons too:
|
||||||
|
- Log, Add feed, Users, Unsubscribe and OPML.
|
||||||
|
- Popular's Subscribe, Copy and Sign out.
|
||||||
|
- The Subscribed label in Popular, the Directory and Add feed, which is now a green check.
|
||||||
|
- The player's back, play, forward and close, which were font characters, and the folder arrow.
|
||||||
|
- The toolbar's read and keep buttons show the selected item's state, with the same icons as the
|
||||||
|
item's own buttons. Play, read and keep sit together, and Scan sits with add and unsubscribe.
|
||||||
|
- An OPML subscription's page has the same header as a feed's, with its buttons in the same places.
|
||||||
|
- The item table's size has its own column, apart from the file's type, and shows KB for small
|
||||||
|
files instead of "0 MB". The Item heading is now Title.
|
||||||
|
- A file's type is an icon (audio, video, image, PDF, torrent, other), green once it is
|
||||||
|
downloaded and red when the download failed, with the details in its tooltip. One icon per
|
||||||
|
row keeps the column lined up. The DOWNLOADED and PENDING labels are gone.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Playing a file from the Files pane played it twice at once, in the pane and in the player bar.
|
||||||
|
The pane has a play button now, and the player bar is the only player.
|
||||||
|
- The password box in Manage users was white in the dark theme.
|
||||||
|
- An item with no date showed a stray dot in its details.
|
||||||
|
- Escape did not close a dialog while the cursor was in one of its boxes, so Add feed, which opens
|
||||||
|
in its URL box, could not be closed with Escape.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- Feeds from paid-feed services (Patreon, Supercast, Supporting Cast, Glow, Memberful) are never
|
||||||
|
listed in Popular or the Directory. A Supercast feed, which keeps its key in the URL's path
|
||||||
|
rather than the query, was being listed.
|
||||||
|
|
||||||
|
## [0.3.0] - 2026-09-11
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Add feed lists what other people on this server subscribe to, most subscribers first, and
|
||||||
|
subscribes you by id (`GET /api/popular`, `POST /api/popular/{id}`). Feeds from an OPML, and
|
||||||
|
feeds with a login or a key in their URL, are never listed.
|
||||||
|
- Upload an OPML file to import, beside the paste box. The page checks it looks like OPML before
|
||||||
|
sending it and clears the picker afterwards.
|
||||||
|
- Settings → Manage users: add and remove accounts, and choose who is an admin
|
||||||
|
(`GET`/`POST /api/users`, `PATCH`/`DELETE /api/users/{id}`).
|
||||||
|
- Per-user subscriptions, and per-user read, starred and playback state. The existing library is
|
||||||
|
adopted by the admin on first start.
|
||||||
|
- Subscribing to a feed someone else already has costs no second fetch and no second copy. Scanning
|
||||||
|
merges every subscriber's wants.
|
||||||
|
- Delete on a shared feed reads **Delete for everyone**, and the server answers `409` while anyone
|
||||||
|
else has starred the item or not played it (`?force=true` overrides).
|
||||||
|
- A shared feed's header says how many other people read it.
|
||||||
|
- `docs/` for configuration, the CLI, users, SSO and architecture, and `CLAUDE.md` for anyone
|
||||||
|
working on the code.
|
||||||
|
- Browser tests for OPML import and export by every route, user admin, unread ordering, and
|
||||||
|
`ipx import`/`ipx export`.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Feeds inside an OPML subscription list the ones with unread items first.
|
||||||
|
- OPML import subscribes you to every feed in the file. `ipx import` subscribes the first admin.
|
||||||
|
- OPML export lists only your own subscriptions.
|
||||||
|
- Production runs as a Docker image pushed to `192.168.1.130:5000` and recreated with
|
||||||
|
`docker compose`.
|
||||||
|
- This changelog follows Keep a Changelog. The long-form entries moved to `docs/history.md`.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Importing another account's OPML export subscribed nobody and reported "Imported 0 feed(s)".
|
||||||
|
Feeds it added had no subscriber, so they were never scanned.
|
||||||
|
- Importing something that is not OPML answered `500`. It is now `400` "that is not an OPML file",
|
||||||
|
refused before anything changes.
|
||||||
|
- Starring stopped protecting a file from the quota and age sweeps once read state became per-user.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- The log is admin-only (`GET /api/logs` answers `403`, and the Log button is hidden). It names
|
||||||
|
every account, every feed and every failed sign-in.
|
||||||
|
- OPML export no longer hands anyone signed in the whole catalogue, including other people's
|
||||||
|
private feed URLs.
|
||||||
|
|
||||||
|
## [0.2.0] - 2026-09-10
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Web UI served by the daemon: plain HTML and JS compiled into the binary, with feeds, items,
|
||||||
|
filters, search, sanitised show notes and live progress over SSE.
|
||||||
|
- Player bar with resume, speed, keyboard shortcuts and lock-screen controls.
|
||||||
|
- Three-pane layout: feeds beside, items above, and the selected item's text and files below.
|
||||||
|
- Phone layout.
|
||||||
|
- Accounts and sign-in: Argon2id passwords, session cookies, `ipx user add|list|passwd|rm`, and a
|
||||||
|
trusted proxy header for Cloudflare Zero Trust or Authentik (`docs/sso.md`).
|
||||||
|
- Subscribing to an OPML: it is re-read every scan and its feeds show as a folder. A feed dropped
|
||||||
|
from it is removed unless it has downloads.
|
||||||
|
- Scheduling: a global interval with per-feed overrides (`every 30m`, `4h`, `1d`, `2w`).
|
||||||
|
- `[general] media_types`, default audio and video, and `max_new_per_check`, default 3.
|
||||||
|
- Every enclosure of an item, a View link for files that are not audio or video, and per-item
|
||||||
|
artwork.
|
||||||
|
- In-app log view with Daemon I/O, Scans and HTTP tabs.
|
||||||
|
- Mark all read on an OPML subscription.
|
||||||
|
- Editable feed URL with a copy button.
|
||||||
|
- Docker image whose healthcheck goes through the control socket.
|
||||||
|
- The iPodderX name, icon, and a colour scheme taken from the icon.
|
||||||
|
- `tests/page-smoke.js` and a Playwright browser suite.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Global settings and scan schedules are admin-only. The per-feed schedule picker is gone.
|
||||||
|
- Feeds from an OPML live in the database, not `config.toml`.
|
||||||
|
- Opening an item marks it read. Playing it marks it read only at the end or past 90%.
|
||||||
|
- "Episodes" became "items", since half the library is text.
|
||||||
|
- A burst of scan events causes one refresh, not one per feed.
|
||||||
|
- All is the default filter. OPML import and export, Settings and Log moved out of the header.
|
||||||
|
- Torrents run detached, two at a time.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Download fetched the next queued episodes instead of the one clicked.
|
||||||
|
- Pressing play made an item vanish from the Unread list.
|
||||||
|
- Clearing a folder, schedule or cap from the UI did nothing.
|
||||||
|
- The daemon ignored SIGTERM until the current download finished.
|
||||||
|
- A missing function stopped the page script and left the whole UI dead.
|
||||||
|
- One download painted progress on every pending row.
|
||||||
|
- A torrent could freeze scanning for up to an hour.
|
||||||
|
- Downloading from an OPML feed failed with "belongs to unsubscribed feed".
|
||||||
|
- Image enclosures were downloaded, counted as episodes and given a play button.
|
||||||
|
- A feed whose entries had been deleted stayed empty, because the server kept answering `304`.
|
||||||
|
- An item with several enclosures kept only the last.
|
||||||
|
- OPML folders rendered open by default.
|
||||||
|
- Mark read in the text pane recursed until the stack overflowed.
|
||||||
|
- The Unread, Downloaded and Flagged filters answered `500` without a search term.
|
||||||
|
- An OPML subscription always showed 0 unread.
|
||||||
|
- Folder names kept doubled spaces where separators were stripped.
|
||||||
|
- Sidebar rows had four different left edges.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- The web UI needs a token or a sign-in. The token is compared in constant time, and an empty
|
||||||
|
token refuses to serve.
|
||||||
|
- Show notes are sanitised with `ammonia`.
|
||||||
|
- A proxy's user header is honoured only from an address in `trusted_proxies`.
|
||||||
|
- A feed URL must be http(s), so `file:///etc/passwd` is refused.
|
||||||
|
- Download folders are sanitised per path segment, so `../../etc/Show` cannot climb out.
|
||||||
|
|
||||||
|
## [0.1.0] - 2026-09-09
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- `ipx`, a Rust rewrite of the iPodderX engine: TOML config, SQLite state, and `ipx list`, `add`,
|
||||||
|
`rm` and `fetch`.
|
||||||
|
- RSS and Atom parsing with conditional GET, `<ttl>` and basic auth.
|
||||||
|
- Streaming downloads with explicit, keyword and per-scan filters, deduplicated by enclosure URL.
|
||||||
|
- Quota and age retention that never touches a starred file, and `ipx reap [--dry-run]`.
|
||||||
|
- `ipx daemon` with a JSON-lines Unix socket. CLI commands proxy to a running daemon.
|
||||||
|
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
|
||||||
|
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
|
||||||
|
|
||||||
|
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.0...main
|
||||||
|
[0.5.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.4.0...v0.5.0
|
||||||
|
[0.4.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.3.0...v0.4.0
|
||||||
|
[0.3.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.2.0...v0.3.0
|
||||||
|
[0.2.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.1.0...v0.2.0
|
||||||
|
[0.1.0]: https://git.sdf1.net/rays/ipodderx-rs/releases/tag/v0.1.0
|
||||||
148
CLAUDE.md
Normal file
148
CLAUDE.md
Normal file
@@ -0,0 +1,148 @@
|
|||||||
|
# Working on ipodderx-rs
|
||||||
|
|
||||||
|
Notes for whoever picks this up next. Read [docs/architecture.md](docs/architecture.md) for how the
|
||||||
|
thing is built; this file is about working on it without repeating mistakes that have already been
|
||||||
|
made here.
|
||||||
|
|
||||||
|
## Where things are
|
||||||
|
|
||||||
|
Production is the `iPodderX` container on Tower (192.168.1.130), the `ipodderx` service of the
|
||||||
|
Arcane project `content`: `/mnt/fast/arcane/projects/content/compose.yaml`. That file is what runs;
|
||||||
|
`docker-compose.yml` in this repo is a copy, and editing it changes nothing in production.
|
||||||
|
|
||||||
|
| | Host | In the container |
|
||||||
|
|---|---|---|
|
||||||
|
| Image | `192.168.1.130:5000/ipodderx:latest` | |
|
||||||
|
| Config | `/mnt/fast/appdata/ipodderx/config.toml` | `/config/config.toml` |
|
||||||
|
| Database | `/mnt/user/ipodderx/state.db` | `/data/state.db` |
|
||||||
|
| Downloads | `/mnt/user/ipodderx/downloads` | `/downloads` |
|
||||||
|
| Web UI | `192.168.1.130:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel | `0.0.0.0:8099` |
|
||||||
|
|
||||||
|
Deploying a change is: build and push the image, then pull it and recreate the container.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
|
||||||
|
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml pull ipodderx
|
||||||
|
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml up -d ipodderx
|
||||||
|
docker logs --tail 20 iPodderX
|
||||||
|
```
|
||||||
|
|
||||||
|
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
|
||||||
|
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
|
||||||
|
|
||||||
|
**A build that fails with `429 Too Many Requests` on a base image** is Docker Hub rate-limiting
|
||||||
|
this host. There is no Docker Hub login here, and the build asks about `debian:bookworm-slim` and
|
||||||
|
`rust:1-slim-bookworm` every time unless they are already stored locally. Pull them from Google's
|
||||||
|
mirror and tag them; the build then uses the local copies without asking Docker Hub:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker pull mirror.gcr.io/library/debian:bookworm-slim
|
||||||
|
docker tag mirror.gcr.io/library/debian:bookworm-slim debian:bookworm-slim
|
||||||
|
docker pull mirror.gcr.io/library/rust:1-slim-bookworm
|
||||||
|
docker tag mirror.gcr.io/library/rust:1-slim-bookworm rust:1-slim-bookworm
|
||||||
|
```
|
||||||
|
|
||||||
|
Run those again now and then, or the local copies go stale.
|
||||||
|
|
||||||
|
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
|
||||||
|
the worker is alive, not just the web port. The container restarts on its own after a reboot.
|
||||||
|
|
||||||
|
Before the container, ipx ran by hand in code-server, with its files in `/config/.config/ipx/` and
|
||||||
|
`/config/.local/share/ipx/`. Those are still there and the container does not read them. If you run
|
||||||
|
a daemon by hand for testing, stop it with **`pkill -x ipx`, never `pkill -f ipx`**. `-f` matches
|
||||||
|
the shell running the command and kills the session (exit 144). This has happened more than once.
|
||||||
|
|
||||||
|
## Before you touch the page
|
||||||
|
|
||||||
|
`web/index.html` is `include_str!`d into the binary, so **every page change needs a rebuild** before
|
||||||
|
it is visible. It is one file: markup, CSS and script.
|
||||||
|
|
||||||
|
After any edit to it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
node tests/page-smoke.js
|
||||||
|
```
|
||||||
|
|
||||||
|
That loads the script against a stub DOM and checks every selector it wires at load actually
|
||||||
|
exists. It exists because a patch once anchored on a deleted function, `String.replace` silently
|
||||||
|
matched nothing, and the whole UI died with a `ReferenceError` while every server-side test passed.
|
||||||
|
|
||||||
|
Patching that file by guessing an anchor string has failed repeatedly. Read the exact block first
|
||||||
|
(`sed -n 'START,ENDp'`), match it verbatim, and assert the replacement happened rather than hoping.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test # ~51 tests: parsing, filters, retention, schedules, SQL, per-user state
|
||||||
|
node tests/page-smoke.js
|
||||||
|
npx playwright test # 16 browser tests against a real daemon on fixture feeds
|
||||||
|
```
|
||||||
|
|
||||||
|
Things about the browser suite that have cost time:
|
||||||
|
|
||||||
|
* It starts **its own daemon and database** under `/tmp/ipx-ui-test`, wiped once per run. Playwright
|
||||||
|
re-imports the config in every worker, so `prepare()` guards on `TEST_WORKER_INDEX` — without
|
||||||
|
that guard a worker deleted the database out from under the running daemon, which then kept
|
||||||
|
serving from the unlinked inode while everything else saw an empty file.
|
||||||
|
* Tests **share that daemon and run in order**. A test that opens an item marks it read and changes
|
||||||
|
what later tests see. Write assertions that do not depend on what ran before, or normalise the
|
||||||
|
state first.
|
||||||
|
* Fixture feeds must not share an enclosure URL, because `enclosures.url` is globally unique and
|
||||||
|
whichever feed is scanned first claims it.
|
||||||
|
* `webServer` starts **before** `globalSetup`, which is why the fixture config is written at
|
||||||
|
config-load time instead.
|
||||||
|
|
||||||
|
Non-trivial logic leaves one runnable check behind. Pure functions (`merge_policy`, `pick`,
|
||||||
|
`matches_keywords`, `parse_interval`) are the easiest place to put it.
|
||||||
|
|
||||||
|
## Things that are true and easy to get wrong
|
||||||
|
|
||||||
|
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
|
||||||
|
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
|
||||||
|
* **Read state lives in `entry_state`, per user, and nowhere else.** `entries` had `read`, `flagged`
|
||||||
|
and `position` columns from before accounts; two bugs came from queries still reading them
|
||||||
|
(retention, and the entry pruner), and `migrate()` now drops them.
|
||||||
|
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
|
||||||
|
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
|
||||||
|
derived and never written to config.
|
||||||
|
* **One fetch serves everyone**, so scan policy is a union of subscribers' wants (`merge_policy`).
|
||||||
|
Anyone wanting an item is enough to fetch it.
|
||||||
|
* **The UI hiding a control is not enforcement.** Admin-only actions check `user.is_admin` in the
|
||||||
|
handler and return `403`.
|
||||||
|
* **A `tokio::select!` only races its branches at the point of selection.** A long download has to
|
||||||
|
watch the shutdown channel itself; the daemon ignored SIGTERM for exactly this reason.
|
||||||
|
* Only one daemon per socket. Removing the socket file defeats the guard and you get two daemons
|
||||||
|
fighting over the database, with the stale one still holding the port.
|
||||||
|
* `/api/settings` answering `200` does **not** mean the worker is alive — it is a different task.
|
||||||
|
Probe the control socket (`ipx status`) to check that.
|
||||||
|
* **Every `ipx` command runs `migrate()` when it opens the database**, the healthcheck's
|
||||||
|
`ipx status` included. A migration that rewrites a big table (`DROP COLUMN`) takes seconds on
|
||||||
|
production, and a command run meanwhile fails with `migrating schema`. It changes nothing; wait
|
||||||
|
for `daemon started` in the log. Copy `state.db` aside before deploying one.
|
||||||
|
|
||||||
|
## House style
|
||||||
|
|
||||||
|
Comments explain **why**, not what. If a line looks odd, the comment says what went wrong without
|
||||||
|
it. No emoji, no exclamation marks, no "obviously". Prose in the UI and docs is plain English and
|
||||||
|
addressed to the person using it.
|
||||||
|
|
||||||
|
Every change gets one line under `## [Unreleased]` in [CHANGELOG.md](CHANGELOG.md), in its
|
||||||
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) group: Added, Changed, Deprecated,
|
||||||
|
Removed, Fixed or Security. Say it the way someone using ipx would notice it. When there is more to
|
||||||
|
say, such as what was wrong before or what it cost to find out, write it up at the top of
|
||||||
|
[docs/history.md](docs/history.md), dated. That record has been more useful than the git log more
|
||||||
|
than once.
|
||||||
|
|
||||||
|
Cutting a release: rename `[Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD` and open a new empty
|
||||||
|
`[Unreleased]` above it, bump `version` in `Cargo.toml`, tag the commit `vX.Y.Z`, and update the
|
||||||
|
compare links at the bottom of the changelog.
|
||||||
|
|
||||||
|
Deliberate simplifications get a `ponytail:` comment naming the ceiling and the upgrade path, e.g.
|
||||||
|
`// ponytail: global connection mutex, move to a pool if feed count makes it contend`.
|
||||||
|
|
||||||
|
## Known gaps
|
||||||
|
|
||||||
|
* Cloudflare's `Cf-Access-Jwt-Assertion` is not verified — ipx trusts the hop plus `trusted_proxies`
|
||||||
|
(documented in [docs/sso.md](docs/sso.md)).
|
||||||
|
* A feed's `<description>` subtitle is dropped whenever `content:encoded` exists, which loses
|
||||||
|
Substack-style subtitles.
|
||||||
361
Cargo.lock
generated
361
Cargo.lock
generated
@@ -23,6 +23,18 @@ version = "0.2.21"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
|
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "ammonia"
|
||||||
|
version = "4.1.4"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "dc6d763210e2eb7670d1a5183a08bebefa3f97db2a738a684f2ce00bd49f681d"
|
||||||
|
dependencies = [
|
||||||
|
"cssparser",
|
||||||
|
"html5ever",
|
||||||
|
"maplit",
|
||||||
|
"url",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "android_system_properties"
|
name = "android_system_properties"
|
||||||
version = "0.1.6"
|
version = "0.1.6"
|
||||||
@@ -97,6 +109,18 @@ dependencies = [
|
|||||||
"rustversion",
|
"rustversion",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "argon2"
|
||||||
|
version = "0.6.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "134c52ddac6d63c576bef8168db10c83c49c26444ecbc68060fef078925a901c"
|
||||||
|
dependencies = [
|
||||||
|
"base64ct",
|
||||||
|
"blake2",
|
||||||
|
"cpufeatures",
|
||||||
|
"password-hash",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "arrayvec"
|
name = "arrayvec"
|
||||||
version = "0.7.8"
|
version = "0.7.8"
|
||||||
@@ -175,7 +199,7 @@ dependencies = [
|
|||||||
"chrono",
|
"chrono",
|
||||||
"derive_builder",
|
"derive_builder",
|
||||||
"diligent-date-parser",
|
"diligent-date-parser",
|
||||||
"quick-xml",
|
"quick-xml 0.41.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@@ -222,6 +246,7 @@ checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90"
|
|||||||
dependencies = [
|
dependencies = [
|
||||||
"axum-core",
|
"axum-core",
|
||||||
"bytes",
|
"bytes",
|
||||||
|
"form_urlencoded",
|
||||||
"futures-util",
|
"futures-util",
|
||||||
"http",
|
"http",
|
||||||
"http-body",
|
"http-body",
|
||||||
@@ -235,6 +260,9 @@ dependencies = [
|
|||||||
"percent-encoding",
|
"percent-encoding",
|
||||||
"pin-project-lite",
|
"pin-project-lite",
|
||||||
"serde_core",
|
"serde_core",
|
||||||
|
"serde_json",
|
||||||
|
"serde_path_to_error",
|
||||||
|
"serde_urlencoded",
|
||||||
"sync_wrapper",
|
"sync_wrapper",
|
||||||
"tokio",
|
"tokio",
|
||||||
"tower",
|
"tower",
|
||||||
@@ -310,6 +338,12 @@ version = "0.23.1"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
|
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "base64ct"
|
||||||
|
version = "1.8.3"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bitflags"
|
name = "bitflags"
|
||||||
version = "1.3.2"
|
version = "1.3.2"
|
||||||
@@ -334,6 +368,24 @@ dependencies = [
|
|||||||
"wyz",
|
"wyz",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "blake2"
|
||||||
|
version = "0.11.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "5b5d4d889834ee8ecfc0f8426ad30faf7cdcb10f741a8e6d7224d95325479f6f"
|
||||||
|
dependencies = [
|
||||||
|
"digest",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "block-buffer"
|
||||||
|
version = "0.12.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
|
||||||
|
dependencies = [
|
||||||
|
"hybrid-array",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "bs58"
|
name = "bs58"
|
||||||
version = "0.5.1"
|
version = "0.5.1"
|
||||||
@@ -384,17 +436,6 @@ dependencies = [
|
|||||||
"shlex",
|
"shlex",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "cfb"
|
|
||||||
version = "0.14.0"
|
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
||||||
checksum = "a347dcabdae9c31b0825fd6a8bed285ec9c2acb89c47827126d52fa4f59cece3"
|
|
||||||
dependencies = [
|
|
||||||
"fnv",
|
|
||||||
"uuid",
|
|
||||||
"web-time",
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "cfg-if"
|
name = "cfg-if"
|
||||||
version = "1.0.4"
|
version = "1.0.4"
|
||||||
@@ -492,6 +533,12 @@ dependencies = [
|
|||||||
"cc",
|
"cc",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "cmov"
|
||||||
|
version = "0.5.4"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "colorchoice"
|
name = "colorchoice"
|
||||||
version = "1.0.5"
|
version = "1.0.5"
|
||||||
@@ -590,6 +637,35 @@ version = "0.8.23"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
|
checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "crypto-common"
|
||||||
|
version = "0.2.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453"
|
||||||
|
dependencies = [
|
||||||
|
"hybrid-array",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "cssparser"
|
||||||
|
version = "0.37.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "8c9cdaae01d5ed7882b04d795e7f752f46ff52d2fa3b50a20d28c464510bba98"
|
||||||
|
dependencies = [
|
||||||
|
"dtoa-short",
|
||||||
|
"itoa",
|
||||||
|
"smallvec",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "ctutils"
|
||||||
|
version = "0.4.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e"
|
||||||
|
dependencies = [
|
||||||
|
"cmov",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "darling"
|
name = "darling"
|
||||||
version = "0.20.11"
|
version = "0.20.11"
|
||||||
@@ -751,6 +827,17 @@ dependencies = [
|
|||||||
"syn 2.0.119",
|
"syn 2.0.119",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "digest"
|
||||||
|
version = "0.11.3"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
|
||||||
|
dependencies = [
|
||||||
|
"block-buffer",
|
||||||
|
"crypto-common",
|
||||||
|
"ctutils",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "diligent-date-parser"
|
name = "diligent-date-parser"
|
||||||
version = "0.1.5"
|
version = "0.1.5"
|
||||||
@@ -769,15 +856,6 @@ dependencies = [
|
|||||||
"dirs-sys",
|
"dirs-sys",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "dirs"
|
|
||||||
version = "7.0.0"
|
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
||||||
checksum = "8d57d423b3c82e89b9a24ca3091fee61f456a26edbd28d26c65906f4bc1dcd8f"
|
|
||||||
dependencies = [
|
|
||||||
"dirs-sys",
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dirs-sys"
|
name = "dirs-sys"
|
||||||
version = "0.5.0"
|
version = "0.5.0"
|
||||||
@@ -813,6 +891,21 @@ dependencies = [
|
|||||||
"windows-sys 0.52.0",
|
"windows-sys 0.52.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "dtoa"
|
||||||
|
version = "1.0.11"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "4c3cf4824e2d5f025c7b531afcb2325364084a16806f6d47fbc1f5fbd9960590"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "dtoa-short"
|
||||||
|
version = "0.3.5"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "cd1511a7b6a56299bd043a9c167a6d2bfb37bf84a6dfceaba651168adfb43c87"
|
||||||
|
dependencies = [
|
||||||
|
"dtoa",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dunce"
|
name = "dunce"
|
||||||
version = "1.0.5"
|
version = "1.0.5"
|
||||||
@@ -1206,6 +1299,16 @@ version = "0.4.3"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
|
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "html5ever"
|
||||||
|
version = "0.39.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "46a1761807faccc9a19e86944bbf40610014066306f96edcdedc2fb714bcb7b8"
|
||||||
|
dependencies = [
|
||||||
|
"log",
|
||||||
|
"markup5ever",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "http"
|
name = "http"
|
||||||
version = "1.5.0"
|
version = "1.5.0"
|
||||||
@@ -1239,6 +1342,12 @@ dependencies = [
|
|||||||
"pin-project-lite",
|
"pin-project-lite",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "http-range-header"
|
||||||
|
version = "0.4.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "9171a2ea8a68358193d15dd5d70c1c10a2afc3e7e4c5bc92bc9f025cebd7359c"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "httparse"
|
name = "httparse"
|
||||||
version = "1.10.1"
|
version = "1.10.1"
|
||||||
@@ -1251,6 +1360,15 @@ version = "1.0.3"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
|
checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "hybrid-array"
|
||||||
|
version = "0.4.15"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17"
|
||||||
|
dependencies = [
|
||||||
|
"typenum",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "hyper"
|
name = "hyper"
|
||||||
version = "1.11.1"
|
version = "1.11.1"
|
||||||
@@ -1470,15 +1588,6 @@ dependencies = [
|
|||||||
"serde_core",
|
"serde_core",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "infer"
|
|
||||||
version = "0.22.0"
|
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
||||||
checksum = "f4200d433cbd5178df7797c9c2e75b348b728e39631cf14520d1e2fc424201f4"
|
|
||||||
dependencies = [
|
|
||||||
"cfb",
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "intervaltree"
|
name = "intervaltree"
|
||||||
version = "0.2.7"
|
version = "0.2.7"
|
||||||
@@ -1496,18 +1605,20 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "ipx"
|
name = "ipx"
|
||||||
version = "0.1.0"
|
version = "0.5.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
"ammonia",
|
||||||
"anyhow",
|
"anyhow",
|
||||||
|
"argon2",
|
||||||
"atom_syndication",
|
"atom_syndication",
|
||||||
|
"axum",
|
||||||
"chrono",
|
"chrono",
|
||||||
"clap",
|
"clap",
|
||||||
"dirs",
|
|
||||||
"futures-util",
|
"futures-util",
|
||||||
"infer",
|
|
||||||
"librqbit",
|
"librqbit",
|
||||||
"opml",
|
"opml",
|
||||||
"percent-encoding",
|
"percent-encoding",
|
||||||
|
"quick-xml 0.42.0",
|
||||||
"reqwest",
|
"reqwest",
|
||||||
"rss",
|
"rss",
|
||||||
"rusqlite",
|
"rusqlite",
|
||||||
@@ -1515,6 +1626,8 @@ dependencies = [
|
|||||||
"serde_json",
|
"serde_json",
|
||||||
"tokio",
|
"tokio",
|
||||||
"toml",
|
"toml",
|
||||||
|
"tower",
|
||||||
|
"tower-http 0.7.1",
|
||||||
"tracing",
|
"tracing",
|
||||||
"tracing-subscriber",
|
"tracing-subscriber",
|
||||||
"url",
|
"url",
|
||||||
@@ -1974,7 +2087,7 @@ dependencies = [
|
|||||||
"httparse",
|
"httparse",
|
||||||
"librqbit-dualstack-sockets",
|
"librqbit-dualstack-sockets",
|
||||||
"network-interface",
|
"network-interface",
|
||||||
"quick-xml",
|
"quick-xml 0.41.0",
|
||||||
"reqwest",
|
"reqwest",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_derive",
|
"serde_derive",
|
||||||
@@ -2045,6 +2158,23 @@ version = "0.1.2"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
|
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "maplit"
|
||||||
|
version = "1.0.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "3e2e65a1a2e43cfcb47a895c4c8b10d1f4a61097f9f254f183aee60cad9c651d"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "markup5ever"
|
||||||
|
version = "0.39.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "7122d987ec5f704ee56f6e5b41a7d93722e9aae27ae07cafa4036c4d3f9757de"
|
||||||
|
dependencies = [
|
||||||
|
"log",
|
||||||
|
"tendril",
|
||||||
|
"web_atoms",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "matchers"
|
name = "matchers"
|
||||||
version = "0.2.0"
|
version = "0.2.0"
|
||||||
@@ -2161,6 +2291,12 @@ dependencies = [
|
|||||||
"winapi",
|
"winapi",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "new_debug_unreachable"
|
||||||
|
version = "1.0.6"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nix"
|
name = "nix"
|
||||||
version = "0.30.1"
|
version = "0.30.1"
|
||||||
@@ -2314,12 +2450,72 @@ dependencies = [
|
|||||||
"windows-link",
|
"windows-link",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "password-hash"
|
||||||
|
version = "0.6.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "aab41826031698d6ffcd9cff78ef56ef998e39dc7e5067cdfebe373842d4723b"
|
||||||
|
dependencies = [
|
||||||
|
"getrandom 0.4.3",
|
||||||
|
"phc",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "percent-encoding"
|
name = "percent-encoding"
|
||||||
version = "2.3.2"
|
version = "2.3.2"
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
|
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "phc"
|
||||||
|
version = "0.6.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "44dc769b75f93afdddd8c7fa12d685292ddeff1e66f7f0f3a234cf1818afe892"
|
||||||
|
dependencies = [
|
||||||
|
"base64ct",
|
||||||
|
"ctutils",
|
||||||
|
"getrandom 0.4.3",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "phf"
|
||||||
|
version = "0.13.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf"
|
||||||
|
dependencies = [
|
||||||
|
"phf_shared",
|
||||||
|
"serde",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "phf_codegen"
|
||||||
|
version = "0.13.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "49aa7f9d80421bca176ca8dbfebe668cc7a2684708594ec9f3c0db0805d5d6e1"
|
||||||
|
dependencies = [
|
||||||
|
"phf_generator",
|
||||||
|
"phf_shared",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "phf_generator"
|
||||||
|
version = "0.13.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "135ace3a761e564ec88c03a77317a7c6b80bb7f7135ef2544dbe054243b89737"
|
||||||
|
dependencies = [
|
||||||
|
"fastrand",
|
||||||
|
"phf_shared",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "phf_shared"
|
||||||
|
version = "0.13.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "e57fef6bc5981e38c2ce2d63bfa546861309f875b8a75f092d1d54ae2d64f266"
|
||||||
|
dependencies = [
|
||||||
|
"siphasher",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pin-project-lite"
|
name = "pin-project-lite"
|
||||||
version = "0.2.17"
|
version = "0.2.17"
|
||||||
@@ -2371,6 +2567,12 @@ dependencies = [
|
|||||||
"zerocopy",
|
"zerocopy",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "precomputed-hash"
|
||||||
|
version = "0.1.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "proc-macro2"
|
name = "proc-macro2"
|
||||||
version = "1.0.107"
|
version = "1.0.107"
|
||||||
@@ -2406,6 +2608,15 @@ dependencies = [
|
|||||||
"serde",
|
"serde",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "quick-xml"
|
||||||
|
version = "0.42.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "41b1177fdf999d2321d3fb46ff47159d9c1fb9ad66a4879f8c50a0b504615e9b"
|
||||||
|
dependencies = [
|
||||||
|
"memchr",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "quinn"
|
name = "quinn"
|
||||||
version = "0.11.11"
|
version = "0.11.11"
|
||||||
@@ -2666,7 +2877,7 @@ dependencies = [
|
|||||||
"tokio-rustls",
|
"tokio-rustls",
|
||||||
"tokio-util",
|
"tokio-util",
|
||||||
"tower",
|
"tower",
|
||||||
"tower-http",
|
"tower-http 0.6.11",
|
||||||
"tower-service",
|
"tower-service",
|
||||||
"url",
|
"url",
|
||||||
"wasm-bindgen",
|
"wasm-bindgen",
|
||||||
@@ -2727,7 +2938,7 @@ checksum = "f505d3e5e7b06b4dc0245b13294f8ef9a1a0f70284708be1e11c5b7b7441034e"
|
|||||||
dependencies = [
|
dependencies = [
|
||||||
"atom_syndication",
|
"atom_syndication",
|
||||||
"derive_builder",
|
"derive_builder",
|
||||||
"quick-xml",
|
"quick-xml 0.41.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@@ -3092,6 +3303,12 @@ version = "0.1.5"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e"
|
checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "siphasher"
|
||||||
|
version = "1.0.3"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "size_format"
|
name = "size_format"
|
||||||
version = "1.0.2"
|
version = "1.0.2"
|
||||||
@@ -3151,6 +3368,30 @@ version = "1.2.1"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
|
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "string_cache"
|
||||||
|
version = "0.9.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "a18596f8c785a729f2819c0f6a7eae6ebeebdfffbfe4214ae6b087f690e31901"
|
||||||
|
dependencies = [
|
||||||
|
"new_debug_unreachable",
|
||||||
|
"parking_lot",
|
||||||
|
"phf_shared",
|
||||||
|
"precomputed-hash",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "string_cache_codegen"
|
||||||
|
version = "0.6.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "585635e46db231059f76c5849798146164652513eb9e8ab2685939dd90f29b69"
|
||||||
|
dependencies = [
|
||||||
|
"phf_generator",
|
||||||
|
"phf_shared",
|
||||||
|
"proc-macro2",
|
||||||
|
"quote",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "strsim"
|
name = "strsim"
|
||||||
version = "0.11.1"
|
version = "0.11.1"
|
||||||
@@ -3243,6 +3484,15 @@ version = "1.0.1"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
|
checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "tendril"
|
||||||
|
version = "0.5.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "5fed54709c5b3a53d09bb1c113ea4f5ceafd1e772ddcb0030a82e1d56c087b08"
|
||||||
|
dependencies = [
|
||||||
|
"new_debug_unreachable",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "thiserror"
|
name = "thiserror"
|
||||||
version = "1.0.69"
|
version = "1.0.69"
|
||||||
@@ -3474,6 +3724,7 @@ dependencies = [
|
|||||||
"tokio",
|
"tokio",
|
||||||
"tower-layer",
|
"tower-layer",
|
||||||
"tower-service",
|
"tower-service",
|
||||||
|
"tracing",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@@ -3499,6 +3750,31 @@ dependencies = [
|
|||||||
"url",
|
"url",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "tower-http"
|
||||||
|
version = "0.7.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "08a05a66a4fdd61cbbe0a1d755ffe0ca6aba159dd4820936a0ff8a8278245b9c"
|
||||||
|
dependencies = [
|
||||||
|
"bitflags 2.13.1",
|
||||||
|
"bytes",
|
||||||
|
"futures-core",
|
||||||
|
"futures-util",
|
||||||
|
"http",
|
||||||
|
"http-body",
|
||||||
|
"http-body-util",
|
||||||
|
"http-range-header",
|
||||||
|
"httpdate",
|
||||||
|
"mime",
|
||||||
|
"mime_guess",
|
||||||
|
"percent-encoding",
|
||||||
|
"pin-project-lite",
|
||||||
|
"tokio",
|
||||||
|
"tokio-util",
|
||||||
|
"tower-layer",
|
||||||
|
"tower-service",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "tower-layer"
|
name = "tower-layer"
|
||||||
version = "0.3.3"
|
version = "0.3.3"
|
||||||
@@ -3517,6 +3793,7 @@ version = "0.1.44"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
|
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
"log",
|
||||||
"pin-project-lite",
|
"pin-project-lite",
|
||||||
"tracing-attributes",
|
"tracing-attributes",
|
||||||
"tracing-core",
|
"tracing-core",
|
||||||
@@ -3790,6 +4067,18 @@ dependencies = [
|
|||||||
"wasm-bindgen",
|
"wasm-bindgen",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "web_atoms"
|
||||||
|
version = "0.2.6"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "ba8b815c1b593dc0baf78dd0f4fc8fdb2de53198fb1163738093e9a311c33fb3"
|
||||||
|
dependencies = [
|
||||||
|
"phf",
|
||||||
|
"phf_codegen",
|
||||||
|
"string_cache",
|
||||||
|
"string_cache_codegen",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "webpki-root-certs"
|
name = "webpki-root-certs"
|
||||||
version = "1.0.9"
|
version = "1.0.9"
|
||||||
|
|||||||
10
Cargo.toml
10
Cargo.toml
@@ -1,19 +1,21 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "ipx"
|
name = "ipx"
|
||||||
version = "0.1.0"
|
version = "0.5.0"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
|
ammonia = "4.1.4"
|
||||||
anyhow = "1.0.104"
|
anyhow = "1.0.104"
|
||||||
|
argon2 = "0.6.0"
|
||||||
atom_syndication = "0.12.10"
|
atom_syndication = "0.12.10"
|
||||||
|
axum = "0.8.9"
|
||||||
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
|
chrono = { version = "0.4.45", default-features = false, features = ["std", "clock"] }
|
||||||
clap = { version = "4.6.6", features = ["derive"] }
|
clap = { version = "4.6.6", features = ["derive"] }
|
||||||
dirs = "7.0.0"
|
|
||||||
futures-util = { version = "0.3.34", default-features = false, features = ["std"] }
|
futures-util = { version = "0.3.34", default-features = false, features = ["std"] }
|
||||||
infer = "0.22.0"
|
|
||||||
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
|
librqbit = { version = "9.0.1", default-features = false, features = ["rust-tls", "http-api-client"] }
|
||||||
opml = "1.1.6"
|
opml = "1.1.6"
|
||||||
percent-encoding = "2.3.2"
|
percent-encoding = "2.3.2"
|
||||||
|
quick-xml = "0.42.0"
|
||||||
reqwest = { version = "0.13.5", default-features = false, features = ["rustls", "http2", "gzip", "stream", "json", "charset", "system-proxy"] }
|
reqwest = { version = "0.13.5", default-features = false, features = ["rustls", "http2", "gzip", "stream", "json", "charset", "system-proxy"] }
|
||||||
rss = "2.1.1"
|
rss = "2.1.1"
|
||||||
rusqlite = { version = "0.40.2", features = ["bundled"] }
|
rusqlite = { version = "0.40.2", features = ["bundled"] }
|
||||||
@@ -21,6 +23,8 @@ serde = { version = "1.0.229", features = ["derive"] }
|
|||||||
serde_json = "1.0.151"
|
serde_json = "1.0.151"
|
||||||
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
|
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
|
||||||
toml = "1.1.5"
|
toml = "1.1.5"
|
||||||
|
tower = { version = "0.5.3", features = ["util"] }
|
||||||
|
tower-http = { version = "0.7.1", features = ["fs"] }
|
||||||
tracing = "0.1.44"
|
tracing = "0.1.44"
|
||||||
tracing-subscriber = { version = "0.3.23", features = ["env-filter"] }
|
tracing-subscriber = { version = "0.3.23", features = ["env-filter"] }
|
||||||
url = "2.5.8"
|
url = "2.5.8"
|
||||||
|
|||||||
39
Dockerfile
Normal file
39
Dockerfile
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
# Build. rusqlite is bundled (compiles SQLite from source) and librqbit needs a C
|
||||||
|
# toolchain, so the builder needs cc. TLS is rustls throughout, so no OpenSSL headers.
|
||||||
|
FROM rust:1-slim-bookworm AS build
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
build-essential \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
# Dependencies first, so editing the source does not rebuild librqbit every time.
|
||||||
|
COPY Cargo.toml Cargo.lock ./
|
||||||
|
RUN mkdir src && echo 'fn main(){}' > src/main.rs \
|
||||||
|
&& cargo build --release --locked \
|
||||||
|
&& rm -rf src
|
||||||
|
|
||||||
|
COPY src ./src
|
||||||
|
COPY web ./web
|
||||||
|
# cargo skips a rebuild if mtimes look untouched; make sure it does not.
|
||||||
|
RUN touch src/main.rs && cargo build --release --locked
|
||||||
|
|
||||||
|
FROM debian:bookworm-slim
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
ca-certificates gosu \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
COPY --from=build /src/target/release/ipx /usr/local/bin/ipx
|
||||||
|
COPY docker-entrypoint.sh /usr/local/bin/
|
||||||
|
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
|
||||||
|
|
||||||
|
ENV IPX_CONFIG=/config/config.toml \
|
||||||
|
IPX_DATA_DIR=/data \
|
||||||
|
IPX_LOG=ipx=info \
|
||||||
|
PUID=99 \
|
||||||
|
PGID=100
|
||||||
|
VOLUME ["/config", "/data", "/downloads"]
|
||||||
|
|
||||||
|
# Web UI, and the BitTorrent peer port (TCP and UDP -- DHT needs the UDP side).
|
||||||
|
EXPOSE 8099/tcp 6881/tcp 6881/udp
|
||||||
|
|
||||||
|
ENTRYPOINT ["docker-entrypoint.sh"]
|
||||||
|
CMD ["ipx", "daemon"]
|
||||||
262
PROGRESS.md
262
PROGRESS.md
@@ -1,262 +0,0 @@
|
|||||||
# Progress
|
|
||||||
|
|
||||||
Running record of what has actually landed. Newest entry first.
|
|
||||||
The full design and step list live in the plan file at
|
|
||||||
`/config/.claude/plans/i-want-to-create-playful-quiche.md`.
|
|
||||||
|
|
||||||
## Build order
|
|
||||||
|
|
||||||
- [x] **1. Repo skeleton** — git init (`main`), `cargo init --name ipx`, deps pinned, LICENSE,
|
|
||||||
README, this file.
|
|
||||||
- [x] **2. `config.rs` + `db.rs`** — TOML config structs + SQLite schema.
|
|
||||||
- [x] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries.
|
|
||||||
- [x] **4. `download.rs`** — downloads, filters, dedupe.
|
|
||||||
- [x] **5. `retention.rs`** — oldest-first quota + age reaper.
|
|
||||||
- [x] **6. `ipc.rs` + daemon** — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon.
|
|
||||||
- [x] **7. `torrent.rs`** — librqbit, seed to ratio/time, stall abort. (swarm download unverified —
|
|
||||||
see the step 7 entry)
|
|
||||||
- [x] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd units, README.
|
|
||||||
|
|
||||||
## Smoke tests
|
|
||||||
|
|
||||||
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
|
|
||||||
2. `ipx fetch` again → no re-download, feed skipped for TTL.
|
|
||||||
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
|
|
||||||
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
|
|
||||||
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
|
|
||||||
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
|
|
||||||
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
|
|
||||||
`reaped`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-09 — Step 8: OPML and polish
|
|
||||||
|
|
||||||
`ipx add <url>` fetches the feed to name it from its own title (`Accidental Tech Podcast` ->
|
|
||||||
`accidental-tech-podcast`); a feed that cannot be reached is still added, named from its URL, rather
|
|
||||||
than refused. `ipx rm` leaves downloads and history alone, so re-adding a feed does not re-pull its
|
|
||||||
back catalogue. `ipx import`/`export` walk nested OPML folder outlines and skip URLs already
|
|
||||||
subscribed. `tracing` logs to stderr, `IPX_LOG` sets the filter. `contrib/` has a systemd user unit
|
|
||||||
for the daemon, plus a timer and one-shot service for the no-daemon style (with the caveat that
|
|
||||||
without a daemon there is no socket for a UI).
|
|
||||||
|
|
||||||
Verified: `cargo test` 27/27. Round-trip — exported 3 feeds with real titles, removed one,
|
|
||||||
re-imported: exactly the missing one came back, no duplicates, config still mode 0600. Quickstart
|
|
||||||
from a genuinely empty home with no env overrides: `list` on a missing config, `add`, `fetch`
|
|
||||||
(1 downloaded, 1 explicit skipped, 1 torrent 404), `list`; every path resolved under `$HOME`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-09 — Step 7: torrent.rs
|
|
||||||
|
|
||||||
`src/torrent.rs`: a librqbit `Session` started lazily on first torrent (binding ports and starting a
|
|
||||||
DHT for a config that has never seen a torrent would be rude). `.torrent` URLs and magnets both go
|
|
||||||
through `AddTorrent::from_url`. Progress is polled once a second and reported through the same
|
|
||||||
`Progress` events HTTP downloads use.
|
|
||||||
|
|
||||||
Downloads go **straight into the feed folder** rather than staging and moving. The plan said move
|
|
||||||
then seed, which cannot work — seeding serves the files it downloaded, so moving them first breaks
|
|
||||||
it. In-place also removes a copy the original had to do.
|
|
||||||
|
|
||||||
Seeding stops at `seed_ratio` or `seed_time_mins`, whichever comes first, then the torrent is
|
|
||||||
released from the session (files kept).
|
|
||||||
|
|
||||||
**Bug the smoke test caught:** the stall budget only covered the download loop, but resolving a
|
|
||||||
magnet's metadata happens inside `add_torrent`, which against a dead swarm never returns — a
|
|
||||||
torrent nobody seeds wedged the scan indefinitely. `add_torrent` is now wrapped in the same budget.
|
|
||||||
|
|
||||||
Verified: `cargo test` 27/27 (ratio incl. the divide-by-zero case, stall_mins = 0 not meaning
|
|
||||||
"abort instantly"). Routing: with `enabled = false` a torrent enclosure is marked
|
|
||||||
`skipped/torrents disabled` and never attempted. Session startup works here. Stall abort measured
|
|
||||||
end to end: with `stall_mins = 1`, a dead magnet failed at 20:58:23 -> 20:59:24, exactly 61s, and
|
|
||||||
the row recorded `error / no metadata after 1 minutes, gave up`.
|
|
||||||
|
|
||||||
**Not verified: an actual successful swarm download.** This sandbox has no reachable peers, so
|
|
||||||
smoke 5's happy path — payload lands, seeding stops at the ratio — has not been run. The code paths
|
|
||||||
either side of it are tested; the swarm itself needs a real network. Worth running once against a
|
|
||||||
live torrent feed before trusting it.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-09 — Step 6: ipc.rs + daemon
|
|
||||||
|
|
||||||
`src/ipc.rs`: `Event` and `Command` as serde-tagged enums (`{"ev":...}` / `{"cmd":...}`), one JSON
|
|
||||||
object per line over a Unix socket. `Emitter` is the single output path — it broadcasts to socket
|
|
||||||
clients, prints the human rendering to a terminal, or both, so the scan code no longer knows how it
|
|
||||||
is being watched. That replaces `printMSG` and its `;;1;;1;;100.00;;42.31` sentinels.
|
|
||||||
|
|
||||||
`main.rs` restructured around a `Ctx` (config, db, client, emitter). `ipx daemon` binds the socket,
|
|
||||||
serves clients, and runs a 60s ticker that defers to each feed's TTL. Commands from every client
|
|
||||||
funnel through one mpsc queue into a single worker, which is what stops two scans overlapping.
|
|
||||||
Any CLI subcommand with a wire form probes the socket first and proxies to a running daemon;
|
|
||||||
`--local` forces the work to happen in-process. SIGINT/SIGTERM remove the socket on the way out.
|
|
||||||
|
|
||||||
**Bug the smoke test caught:** `fetch` runs a retention sweep first, and that sweep was emitting the
|
|
||||||
terminal `ReapDone`. A UI waiting for its fetch to finish would have stopped reading before the scan
|
|
||||||
started. Only a standalone `ipx reap` emits it now.
|
|
||||||
|
|
||||||
Verified: `cargo test` 24/24. Smoke 3 in full — daemon starts and binds; a raw socket client sending
|
|
||||||
`{"cmd":"fetch","force":true}` gets `feed_start` → 14 throttled `progress` events → `download_done`
|
|
||||||
→ `feed_done` → `scan_done`; `{"cmd":"status"}` answers `{"ev":"status","feeds":1,...}`. With the
|
|
||||||
daemon up, `ipx fetch` from the CLI logged `command from a client cmd=Fetch { .. }` in the daemon
|
|
||||||
and rendered the streamed events, so it proxied rather than downloading in parallel. Socket removed
|
|
||||||
on SIGTERM; with no daemon the same command runs locally.
|
|
||||||
|
|
||||||
Deferred: `{"cmd":"cancel","enclosure":N}` from the plan's protocol is **not implemented** —
|
|
||||||
downloads run sequentially in one worker, so there is nothing to cancel concurrently yet. It wants
|
|
||||||
a per-download cancellation token, which is worth doing when downloads go parallel. Say if you want
|
|
||||||
it sooner.
|
|
||||||
|
|
||||||
Note: the event stream is a broadcast, so a CLI client seeing a busy daemon also sees that other
|
|
||||||
work. Fine for a terminal; a UI wanting strict request/response would want per-request ids.
|
|
||||||
|
|
||||||
Also note: the daemon reads config once at startup — changing `config.toml` needs a restart.
|
|
||||||
|
|
||||||
Next: step 7 — `torrent.rs`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-09 — Step 5: retention.rs
|
|
||||||
|
|
||||||
`src/retention.rs`: reconcile pass (rows claiming a file that is gone become `reaped`, fixing the
|
|
||||||
step-4 wart), age sweep, quota sweep keeping the original's 50 MB headroom pad, and entry pruning.
|
|
||||||
`ipx reap [--dry-run]`; a sweep also runs before every `fetch`, as the Python did per download.
|
|
||||||
`pick()` and `aged()` are pure so the ordering rules are testable without touching a disk.
|
|
||||||
|
|
||||||
**Judgement call worth Ray's eye.** The Python meant to reap only `read = 1 AND flagged = 0` but
|
|
||||||
never managed it — a missing `plistlib` import and an `EntreiesData` typo made that filter throw on
|
|
||||||
every candidate, so with a `.ipxd` present nothing was ever deleted. Requiring `read = 1` here would
|
|
||||||
be equally dead, because nothing marks episodes read until a UI exists. So: **`flagged` is the
|
|
||||||
keep-forever marker, and `read` only decides what goes first** (`ORDER BY read DESC, downloaded_at
|
|
||||||
ASC`). Quota therefore actually reclaims space headless. Say the word if you would rather unread
|
|
||||||
episodes were never touched.
|
|
||||||
|
|
||||||
Second call: `max_age_days` deletes *files* older than the cutoff, not just fileless entries as the
|
|
||||||
plan's wording had it — "keep 30 days of episodes" is what the setting reads like on a NAS.
|
|
||||||
|
|
||||||
Verified: `cargo test` 21/21, including the two tests encoding the exact bug the Python had —
|
|
||||||
flagged files are never offered, and read sort ahead of unread. Smoke 6 with three 30 MB episodes
|
|
||||||
against a 0.1 GB quota (52.4 MB ceiling after the pad): dry run listed ep1+ep2 and deleted nothing
|
|
||||||
(3 files still on disk), the real run deleted exactly those two oldest, left ep3, flipped both rows
|
|
||||||
to `reaped` with `path = NULL`. A full re-parse with the conditional-GET headers cleared then
|
|
||||||
re-downloaded nothing.
|
|
||||||
|
|
||||||
Next: step 6 — `ipc.rs` + daemon.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-09 — Step 4: download.rs
|
|
||||||
|
|
||||||
`src/download.rs`: streaming download to `<download_dir>/.ipx-incomplete/` (same filesystem as the
|
|
||||||
destination, so filing it is a rename, not the original's copy-then-unlink), content sniffing,
|
|
||||||
then `place()`. Filename comes from the URL's last path segment, percent-decoded, unless
|
|
||||||
`Content-Disposition` names one (RFC 5987 `filename*=` preferred). The sanitizer keeps UTF-8 —
|
|
||||||
`latin1_to_ascii` existed because 2004 filesystems demanded ASCII — strips the same characters
|
|
||||||
`stringCleaning()` did plus control chars, and adds a real 255-byte cap the Python never had,
|
|
||||||
preserving the extension across truncation.
|
|
||||||
|
|
||||||
Sniffing replaces `detectFileType()`, which called a `typeFile` module that was already missing in
|
|
||||||
2008 and so always answered `'data'`. Two rules survive: an HTML body is a failed download (login
|
|
||||||
wall/error page), and a torrent body is a torrent whatever the MIME claimed.
|
|
||||||
|
|
||||||
Filters run once at discovery and are recorded in `enclosures.state`; the download queue is then
|
|
||||||
just "everything still `pending`", so an enclosure held back by `max_new_per_check` is picked up by
|
|
||||||
the next scan instead of being lost. Keywords are OR'd across keywords and AND'd within one — the
|
|
||||||
original's nested loop let a later keyword silently undo an earlier miss.
|
|
||||||
|
|
||||||
Verified: `cargo test` 16/16. Smoke against a local server, five enclosures, each filter path hit:
|
|
||||||
`ep1.mp3 -> done`, `ep2.mp3 -> skipped (explicit)`, `ep2.mp3?v=3 -> skipped (no keyword match)`,
|
|
||||||
`paywall.html -> error (HTML page, not media)`, `ep5.torrent -> torrent (deferred to step 7)`.
|
|
||||||
Smoke 2 and 4 pass, and because a plain rerun 304s before parsing, dedupe was proved separately by
|
|
||||||
clearing the stored etag/last-modified and re-parsing all five entries: 0 downloaded, hand-deleted
|
|
||||||
file not refetched, `.ipx-incomplete` left empty.
|
|
||||||
|
|
||||||
Known wart: `ipx list` counts `path IS NOT NULL`, so a hand-deleted file still reads as downloaded.
|
|
||||||
Reconciling rows against the filesystem belongs in step 5.
|
|
||||||
Next: step 5 — `retention.rs`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-09 — Step 3: feed.rs
|
|
||||||
|
|
||||||
`src/feed.rs`: conditional GET (`If-None-Match` + `If-Modified-Since`, optional basic auth) and a
|
|
||||||
RSS-first / Atom-fallback parser normalising both into `ParsedFeed`/`Entry`/`Enclosure`. Feed-level
|
|
||||||
`itunes:explicit` overrides the entry level, as the original did. `<ttl>` is captured. RSS
|
|
||||||
`content:encoded` wins over `description`. Atom enclosures come only from `rel="enclosure"` links.
|
|
||||||
|
|
||||||
GUID: the original hashed the title or description when no guid existed; here the chain is
|
|
||||||
guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5
|
|
||||||
dependency. An entry with none of them has nothing to download and is dropped.
|
|
||||||
|
|
||||||
`db.rs` gained `http_state`, `record_feed`, `touch_feed`, `set_feed_error`, `record_entry`,
|
|
||||||
`record_enclosure`. A changed title/description flips `read` back to 0 — what the original's
|
|
||||||
textDiff was ultimately for, minus the `<ins>`/`<del>` markup, which belongs in the UI.
|
|
||||||
|
|
||||||
`main.rs` gained `ipx fetch [FEED] [--force]`. A failing feed records its error and the scan
|
|
||||||
continues.
|
|
||||||
|
|
||||||
Verified: `cargo test` 10/10. Gate met against a local `python3 -m http.server` serving the
|
|
||||||
fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run
|
|
||||||
showed both skip paths, `atomcast: not modified` (304) and `testcast: not due for 45m` (the feed's
|
|
||||||
own ttl=45 beating `interval_mins = 0`).
|
|
||||||
|
|
||||||
Deferred: nothing downloads yet — enclosure rows land in state `pending`. That is step 4.
|
|
||||||
Next: step 4 — `download.rs`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-09 — Step 2: config.rs + db.rs
|
|
||||||
|
|
||||||
`src/config.rs`: serde structs for `[general]`, `[torrent]` and `[feeds.<id>]` with defaults, `~`
|
|
||||||
expansion, `IPX_CONFIG` / `IPX_DATA_DIR` overrides, `save()` at mode 0600, `Feed::password()`
|
|
||||||
(`password_env` beats a literal `password`), `Torrent::ports()` parsing `"6881-6889"`. A missing
|
|
||||||
config file loads as an empty one so a fresh install works. Retention defaults are 0/0
|
|
||||||
(unlimited, keep forever) — nothing gets deleted until Ray asks for it.
|
|
||||||
|
|
||||||
`src/db.rs`: schema exactly as planned, WAL + `busy_timeout`, `Db::open()` idempotent, connection
|
|
||||||
behind a `Mutex`, `feed_summary()` for `list`, `now()` helper. `enclosures.url` is UNIQUE — the
|
|
||||||
dedupe key that replaces `history.dat`.
|
|
||||||
|
|
||||||
`src/main.rs`: clap skeleton with `ipx list`. Only the subcommands that work exist; the rest arrive
|
|
||||||
with their steps.
|
|
||||||
|
|
||||||
Verified: `cargo test` 5/5 green (config defaults, port-range fallback incl. reversed range,
|
|
||||||
password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — `ipx --config
|
|
||||||
<scratch> list` printed both feeds and created `state.db` once across two runs.
|
|
||||||
|
|
||||||
Deferred: nothing. Two dead-code warnings (`Config::save`, `Feed::password`) are expected; steps 3
|
|
||||||
and 8 consume them.
|
|
||||||
Next: step 3 — `feed.rs`.
|
|
||||||
|
|
||||||
Also added `chrono` 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2026-09-09 — Step 1: repo skeleton
|
|
||||||
|
|
||||||
Repo created at `/src/ipodderx-rs`, default branch `main`, `cargo init --name ipx` (edition 2024,
|
|
||||||
rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this
|
|
||||||
file. Hello-world `main.rs` builds.
|
|
||||||
|
|
||||||
Dependency versions pinned today — the later steps are written against these APIs:
|
|
||||||
|
|
||||||
| crate | version | notes |
|
|
||||||
|---|---|---|
|
|
||||||
| tokio | 1.53.1 | rt-multi-thread, macros, fs, io-util, net, sync, time, signal |
|
|
||||||
| reqwest | 0.13.5 | `default-features = false`; features **`rustls`** (not `rustls-tls` — renamed in 0.13), http2, gzip, stream, json, charset, **`system-proxy`** (env-var proxy pickup is opt-in in 0.13) |
|
|
||||||
| rss | 2.1.1 | default features; the `with-syndication` feature name in my notes does not exist — Atom is handled by the separate crate |
|
|
||||||
| atom_syndication | 0.12.10 | |
|
|
||||||
| rusqlite | 0.40.2 | `bundled` |
|
|
||||||
| librqbit | 9.0.1 | `default-features = false`; features `rust-tls`, `http-api-client`. The default `default-tls` feature pulls reqwest/native-tls **and** an OpenSSL sha1 backend (`crypto-hash`), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7 |
|
|
||||||
| serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5 | | |
|
|
||||||
| clap | 4.6.6 | derive |
|
|
||||||
| infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6 | | |
|
|
||||||
| tracing 0.1.44 / tracing-subscriber 0.3.23 | | env-filter |
|
|
||||||
|
|
||||||
Deferred: nothing.
|
|
||||||
|
|
||||||
Gotcha worth keeping: three feature names in the plan were wrong against current crate versions —
|
|
||||||
`reqwest/rustls-tls` is now `rustls`, env-var proxy support moved behind `system-proxy`, and
|
|
||||||
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
|
|
||||||
is the fix. Whole tree is rustls-only now, no C TLS dependency.
|
|
||||||
Next: step 2 — `config.rs` + `db.rs`. (done)
|
|
||||||
150
README.md
150
README.md
@@ -1,111 +1,83 @@
|
|||||||
# ipodderx-rs
|
# ipodderx-rs
|
||||||
|
|
||||||
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent),
|
A self-hosted podcatcher for a household. It checks your feeds, downloads the episodes, and serves
|
||||||
files them into per-feed folders, and reaps old episodes to stay under a disk quota.
|
a web UI modelled on the 2004 Mac app **iPodderX**, for any number of people sharing one copy of
|
||||||
Runs as a one-shot CLI or as a daemon with a Unix-socket JSON event stream for a UI to attach to.
|
the files. One Rust binary, `ipx`, is both the daemon and the command line.
|
||||||
|
|
||||||
## Lineage
|
It is a rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python engine
|
||||||
|
behind iPodderX (2004-2008, Ray Slakinski & August Trometer).
|
||||||
|
|
||||||
This is a modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the
|
## What it does
|
||||||
Python 2 engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), which was
|
|
||||||
open-sourced under the MIT License in 2010.
|
|
||||||
|
|
||||||
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
|
- **The web UI.** It has a toolbar, and a feed list that opens with Directory, Popular and All
|
||||||
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
|
Subscriptions. Items sit in a sortable table with a Files pane, and there is a player bar. It
|
||||||
oldest-first disk quota reaper.
|
comes in Dark, Light and Classic themes, and works on a phone.
|
||||||
|
- **Several people, one copy.** Each person has their own subscriptions and their own read, kept
|
||||||
|
and playback state. There is one file on disk per episode, however many people want it. People
|
||||||
|
sign in with a password or through a proxy (Cloudflare Zero Trust or Authentik), and admins
|
||||||
|
manage accounts and settings.
|
||||||
|
- **Scanning.** Feeds are checked on a schedule, globally or per feed, and a feed's own TTL is
|
||||||
|
honoured. Keyword, explicit-content and media-type filters decide what is downloaded, with a cap
|
||||||
|
on new downloads per scan.
|
||||||
|
- **Downloads.** Files come over HTTP or BitTorrent and are filed into a folder per feed.
|
||||||
|
Retention deletes the oldest files to stay under a disk quota or an age limit, and never touches
|
||||||
|
an item someone has kept.
|
||||||
|
- **OPML.** You can import and export your own subscriptions. You can also subscribe to an OPML
|
||||||
|
URL, which keeps a whole list in step as a folder.
|
||||||
|
|
||||||
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
|
## Run it
|
||||||
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
|
|
||||||
3DES-encrypted preferences, and the `printMSG` stdout protocol (replaced by a JSON-lines socket).
|
|
||||||
|
|
||||||
## Quickstart
|
With Docker:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cargo install --path .
|
docker build -t ipodderx .
|
||||||
|
docker compose up -d
|
||||||
ipx add https://atp.fm/rss # names the feed from its own title
|
|
||||||
ipx list
|
|
||||||
ipx fetch # scan now
|
|
||||||
ipx daemon # or run continuously, honouring each feed's <ttl>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Config lives at `~/.config/ipx/config.toml` (mode 0600, since it may hold feed passwords);
|
`docker-compose.yml` is set up for the author's own server. Point its `image` and its three volumes
|
||||||
state at `~/.local/share/ipx/state.db`. Override with `IPX_CONFIG` and `IPX_DATA_DIR`.
|
(`/config`, `/data` and `/downloads`) at yours first. The UI is on port 8099. BitTorrent uses 6881
|
||||||
Set `IPX_LOG=ipx=debug` for verbose logging on stderr.
|
over TCP and UDP. Files are written as `PUID`/`PGID`, 99:100 by default.
|
||||||
|
|
||||||
## Commands
|
From source:
|
||||||
|
|
||||||
| command | what it does |
|
```sh
|
||||||
|
cargo build --release
|
||||||
|
./target/release/ipx daemon
|
||||||
|
```
|
||||||
|
|
||||||
|
The first start creates **admin / ipodderx**. Sign in at `/login`, then change it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
echo -n 'a good password' | ipx user passwd admin
|
||||||
|
```
|
||||||
|
|
||||||
|
The UI is plain HTTP, so put TLS in front of it if it is reachable from outside your network.
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `ipx add <url> [--folder X] [--keywords a,b]` | subscribe; the id comes from the feed title |
|
| [docs/configuration.md](docs/configuration.md) | Every config key, path and environment variable |
|
||||||
| `ipx rm <feed>` | unsubscribe; downloads and history are kept |
|
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
|
||||||
| `ipx list` / `ipx status` | subscriptions and their state |
|
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
|
||||||
| `ipx fetch [FEED] [--force]` | scan; `--force` ignores the TTL |
|
| [docs/sso.md](docs/sso.md) | Signing in through Cloudflare Zero Trust or Authentik |
|
||||||
| `ipx reap [--dry-run]` | run retention now |
|
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, control socket, HTTP API |
|
||||||
| `ipx import/export <file.opml>` | move subscriptions in or out |
|
| [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
|
||||||
| `ipx daemon` | scheduler plus the control socket |
|
| [docs/history.md](docs/history.md) | How it was built, with what was wrong and why |
|
||||||
|
| [CLAUDE.md](CLAUDE.md) | Notes for working on the code, including how production is deployed |
|
||||||
|
|
||||||
Any command with a wire form probes the socket first: if a daemon is running it does the work,
|
## Tests
|
||||||
and the CLI just renders the events it streams back. `--local` forces in-process execution.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[general]
|
|
||||||
download_dir = "~/Podcasts"
|
|
||||||
socket = "/run/user/1000/ipx.sock" # default: $XDG_RUNTIME_DIR/ipx.sock
|
|
||||||
interval_mins = 60 # default poll; a feed's own <ttl> wins when longer
|
|
||||||
organize = "feed" # "feed" | "date"
|
|
||||||
max_total_gb = 50 # 0 = unlimited
|
|
||||||
max_age_days = 30 # 0 = keep forever
|
|
||||||
|
|
||||||
[torrent]
|
|
||||||
enabled = true
|
|
||||||
seed_ratio = 1.0 # stop seeding at this ratio ...
|
|
||||||
seed_time_mins = 60 # ... or after this long, whichever comes first
|
|
||||||
port_range = "6881-6889"
|
|
||||||
stall_mins = 30 # give up on a torrent making no progress
|
|
||||||
|
|
||||||
[feeds.atp]
|
|
||||||
url = "https://atp.fm/rss"
|
|
||||||
folder = "Accidental Tech Podcast" # default: the feed title
|
|
||||||
keywords = ["deep dive"] # OR across keywords, AND within one
|
|
||||||
allow_explicit = false
|
|
||||||
auto_download = true
|
|
||||||
max_new_per_check = 3 # the rest wait for the next scan
|
|
||||||
username = "ray" # optional HTTP basic auth
|
|
||||||
password_env = "IPX_ATP_PASS" # or a literal `password`
|
|
||||||
```
|
|
||||||
|
|
||||||
Retention keeps files that are `flagged` in the database, and deletes read episodes before unread
|
|
||||||
ones, oldest first.
|
|
||||||
|
|
||||||
## Socket protocol
|
|
||||||
|
|
||||||
Newline-delimited JSON over a Unix socket, both directions.
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
$ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
|
||||||
{"ev":"feed_start","feed":"atp"}
|
node tests/page-smoke.js # the page script loads without throwing
|
||||||
{"ev":"progress","feed":"atp","url":"...","file":"ep1.mp3","done":8192,"total":3000000}
|
npx playwright test # a real browser against a real daemon on fixture feeds
|
||||||
{"ev":"download_done","feed":"atp","url":"...","path":"...","bytes":3000000}
|
|
||||||
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
|
|
||||||
{"ev":"scan_done","feeds":1}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Commands: `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `status`.
|
`npm install` gets the test runner, and `npx playwright install --with-deps chromium` gets the
|
||||||
Events: `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
browser.
|
||||||
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
|
||||||
`scan_done`, `reap_done` and `status` are terminal -- a client that asked for work stops there.
|
|
||||||
|
|
||||||
Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy
|
|
||||||
daemon also sees that daemon's other work.
|
|
||||||
|
|
||||||
## Running it as a service
|
|
||||||
|
|
||||||
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
|
|
||||||
rather run periodic scans with no daemon (in which case there is no socket for a UI to attach to).
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
MIT. See [LICENSE](LICENSE).
|
MIT, see [LICENSE](LICENSE). The icons are [Font Awesome Free](https://fontawesome.com) 7.3.1 by
|
||||||
|
@fontawesome, under [CC BY 4.0](https://fontawesome.com/license/free), embedded as SVG.
|
||||||
|
|||||||
30
TODO.md
Normal file
30
TODO.md
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
# To do
|
||||||
|
|
||||||
|
## Cut what is no longer needed
|
||||||
|
|
||||||
|
From a whole-repo audit for over-engineering on 2026-09-12. Biggest cut first.
|
||||||
|
|
||||||
|
- [x] **Pre-accounts adoption and the dead `entries` columns.** The copy of the old read state into
|
||||||
|
`entry_state` and the `entries.read`, `flagged` and `position` columns are gone. Its other half,
|
||||||
|
subscribing the first admin to the catalogue, was not dead and stays as `adopt_catalogue`.
|
||||||
|
(`src/db.rs`, `src/main.rs`)
|
||||||
|
- [x] **`contrib/` systemd units.** From before the container; nothing points at them.
|
||||||
|
- [x] **`migrate_opml_children`.** A one-time move of OPML children out of `config.toml` that has
|
||||||
|
run. Delete it and its call. (`src/main.rs`)
|
||||||
|
- [x] **The legacy `interval_mins` key.** Production uses `schedule`. Delete the field, the fallback
|
||||||
|
in `General::interval` and its test. (`src/config.rs`)
|
||||||
|
- [x] **`Db::entries` and `Db::count_entries`.** One-line wrappers only the tests call; the tests
|
||||||
|
call `entries_in` and `count_in` instead. (`src/db.rs`)
|
||||||
|
- [x] **`web::generate_token`.** Repeats `auth::new_session_token`. Use that. (`src/web.rs`)
|
||||||
|
- [x] **Page leftovers.** `globalEvery`, `S.busy`, `S.limit`, `unitOptions`' `firstLabel`, `--r`,
|
||||||
|
`.ep.open`, the phone `.ep .art`, the duplicate phone `.fhead.slim{flex-wrap}`, the second
|
||||||
|
`#sidebar{z-index}`, and the `on()` helper. (`web/index.html`)
|
||||||
|
- [x] **`logbuf` visitors.** `record_i64`, `record_u64` and `record_bool` repeat what `Visit`'s
|
||||||
|
defaults already do through `record_debug`. (`src/logbuf.rs`)
|
||||||
|
- [x] **The `infer` dependency.** Its torrent check is the `d8:announce` test on the next line.
|
||||||
|
- [x] **The `dirs` dependency.** `XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME` from `std::env`.
|
||||||
|
- [x] **The `tokio-stream` dependency.** `futures_util::stream::unfold` over the broadcast receiver.
|
||||||
|
- [x] **The icon inlined four times.** About 94 KB of base64 across both pages; serve it once as
|
||||||
|
`/icon.png` from `include_bytes!`, open without signing in like `/login`.
|
||||||
|
|
||||||
|
After these: `cargo test`, `node tests/page-smoke.js`, `npx playwright test`.
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=ipx feed scan (one shot)
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=oneshot
|
|
||||||
ExecStart=%h/.cargo/bin/ipx fetch
|
|
||||||
Environment=IPX_LOG=ipx=info
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
# User unit: install to ~/.config/systemd/user/ipx.service, then
|
|
||||||
# systemctl --user enable --now ipx
|
|
||||||
# The socket lands in $XDG_RUNTIME_DIR/ipx.sock by default, so a UI running as the
|
|
||||||
# same user can attach without extra configuration.
|
|
||||||
[Unit]
|
|
||||||
Description=ipx podcatcher
|
|
||||||
After=network-online.target
|
|
||||||
Wants=network-online.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=simple
|
|
||||||
ExecStart=%h/.cargo/bin/ipx daemon
|
|
||||||
Restart=on-failure
|
|
||||||
RestartSec=30
|
|
||||||
Environment=IPX_LOG=ipx=info
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=default.target
|
|
||||||
@@ -1,16 +0,0 @@
|
|||||||
# Alternative to the daemon: a periodic one-shot scan, closer to how the original
|
|
||||||
# iPodderX agent was driven. Use this OR ipx.service, not both -- with no daemon
|
|
||||||
# running there is no socket, so a UI cannot attach.
|
|
||||||
#
|
|
||||||
# Install ipx-scan.service and ipx.timer to ~/.config/systemd/user/, then
|
|
||||||
# systemctl --user enable --now ipx.timer
|
|
||||||
[Unit]
|
|
||||||
Description=Periodic ipx feed scan
|
|
||||||
|
|
||||||
[Timer]
|
|
||||||
OnBootSec=5min
|
|
||||||
OnUnitActiveSec=1h
|
|
||||||
Persistent=true
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=timers.target
|
|
||||||
24
docker-compose.yml
Normal file
24
docker-compose.yml
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
services:
|
||||||
|
ipodderx:
|
||||||
|
image: 192.168.1.130:5000/ipodderx:latest
|
||||||
|
container_name: iPodderX
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
PUID: "99"
|
||||||
|
PGID: "100"
|
||||||
|
TZ: "America/Toronto"
|
||||||
|
IPX_LOG: "ipx=info"
|
||||||
|
ports:
|
||||||
|
- "8099:8099" # web UI
|
||||||
|
- "6881:6881/tcp" # BitTorrent peers
|
||||||
|
- "6881:6881/udp" # DHT
|
||||||
|
volumes:
|
||||||
|
- /mnt/fast/appdata/ipodderx:/config # config.toml, and the web token
|
||||||
|
- /mnt/user/ipodderx/:/data # state.db
|
||||||
|
- /mnt/user/ipodderx/downloads:/downloads
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "ipx", "status"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 3
|
||||||
|
start_period: 10s
|
||||||
40
docker-entrypoint.sh
Executable file
40
docker-entrypoint.sh
Executable file
@@ -0,0 +1,40 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# A container's loopback is not reachable from outside it, so the default bind of
|
||||||
|
# 127.0.0.1 would leave the UI unreachable. Write a starter config that binds 0.0.0.0
|
||||||
|
# on first run; after that the file is yours and is never rewritten.
|
||||||
|
if [ ! -f "$IPX_CONFIG" ]; then
|
||||||
|
mkdir -p "$(dirname "$IPX_CONFIG")"
|
||||||
|
cat > "$IPX_CONFIG" <<TOML
|
||||||
|
[general]
|
||||||
|
download_dir = "/downloads"
|
||||||
|
schedule = "every 60m"
|
||||||
|
max_total_gb = 0
|
||||||
|
max_age_days = 0
|
||||||
|
|
||||||
|
[torrent]
|
||||||
|
enabled = true
|
||||||
|
port_range = "6881-6889"
|
||||||
|
|
||||||
|
[web]
|
||||||
|
enabled = true
|
||||||
|
bind = "0.0.0.0:8099"
|
||||||
|
token = ""
|
||||||
|
TOML
|
||||||
|
echo "ipx: wrote a starter config to $IPX_CONFIG"
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$IPX_DATA_DIR" /downloads
|
||||||
|
|
||||||
|
# Unraid shares expect 99:100. Running as root would leave root-owned downloads.
|
||||||
|
if [ "$(id -u)" = "0" ] && [ -n "$PUID" ] && [ -n "$PGID" ]; then
|
||||||
|
if ! getent group ipx >/dev/null 2>&1; then addgroup --gid "$PGID" ipx 2>/dev/null || true; fi
|
||||||
|
if ! getent passwd ipx >/dev/null 2>&1; then
|
||||||
|
adduser --uid "$PUID" --gid "$PGID" --disabled-password --gecos "" ipx 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
chown -R "$PUID:$PGID" "$IPX_DATA_DIR" "$(dirname "$IPX_CONFIG")" 2>/dev/null || true
|
||||||
|
exec gosu "$PUID:$PGID" "$@"
|
||||||
|
fi
|
||||||
|
|
||||||
|
exec "$@"
|
||||||
142
docs/architecture.md
Normal file
142
docs/architecture.md
Normal file
@@ -0,0 +1,142 @@
|
|||||||
|
# How it works
|
||||||
|
|
||||||
|
One binary, `ipx`. `ipx daemon` runs three things in one process: a scheduler, a Unix-socket
|
||||||
|
control server, and the web UI. Everything else is a CLI that either does the work itself or hands
|
||||||
|
it to a running daemon.
|
||||||
|
|
||||||
|
## Modules
|
||||||
|
|
||||||
|
| File | Responsibility | What it replaced in the Python |
|
||||||
|
|---|---|---|
|
||||||
|
| `src/main.rs` | CLI, dispatch, scan loop, download policy | `iPXAgent.py` |
|
||||||
|
| `src/config.rs` | TOML load/save, `General`/`Feed`/`Web`, intervals, slugs | `iPXSettings.py`, `feeds.plist` |
|
||||||
|
| `src/db.rs` | SQLite schema, migrations, every query | `.ipxd` plists, `history.dat`, `qmcache.dat` |
|
||||||
|
| `src/feed.rs` | Conditional GET, RSS/Atom/OPML parsing | `FeedData.__getFeed/__getEntries` |
|
||||||
|
| `src/download.rs` | Streaming download, naming, type sniffing, placement | `iPXDownloader.getFile` |
|
||||||
|
| `src/torrent.rs` | librqbit session, seeding limits, stall abort | vendored BitTorrent 4.2.1 |
|
||||||
|
| `src/retention.rs` | Quota and age sweeps | `iPXQuotaManager.py` |
|
||||||
|
| `src/ipc.rs` | Event and command types, the socket server | `printMSG` on stdout |
|
||||||
|
| `src/auth.rs` | Argon2id hashing, session tokens, header names | — |
|
||||||
|
| `src/web.rs` | axum: HTTP API, auth, SSE, media streaming | — |
|
||||||
|
| `src/logbuf.rs` | Ring buffer behind the UI's Log view | — |
|
||||||
|
| `web/index.html` | The whole front end, `include_str!`d into the binary | — |
|
||||||
|
|
||||||
|
The page is compiled in, so **editing `web/index.html` needs a rebuild**.
|
||||||
|
|
||||||
|
## A scan
|
||||||
|
|
||||||
|
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
|
||||||
|
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
|
||||||
|
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
|
||||||
|
re-derived into the database each scan, never written to config.toml. A Patreon creator link
|
||||||
|
(a token, no `show=`) with more than one show is treated the same way, before any fetch: its
|
||||||
|
shows come from Patreon's web API and each becomes a derived feed.
|
||||||
|
4. Record entries. A changed title or description flips the item back to unread.
|
||||||
|
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
|
||||||
|
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
|
||||||
|
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
|
||||||
|
a reason. What a filter skipped is judged again every scan, so a change of settings brings it
|
||||||
|
back. A feed in a group takes your settings on the group for anything you have not set on it.
|
||||||
|
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
|
||||||
|
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
|
||||||
|
or an error page — and is deleted.
|
||||||
|
|
||||||
|
## Data model
|
||||||
|
|
||||||
|
```
|
||||||
|
feeds id, url, title, image, etag, last_modified, last_checked, ttl_mins,
|
||||||
|
last_error, orphaned, group_id, managed
|
||||||
|
entries feed_id, guid, title, link, published, description, first_seen,
|
||||||
|
image, duration, episode, season PK (feed_id, guid)
|
||||||
|
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
|
||||||
|
bytes_done, downloaded_at, last_error
|
||||||
|
users id, name, pass_hash, is_admin, created
|
||||||
|
sessions token, user_id, created, seen
|
||||||
|
subscriptions user_id, feed_id, keywords, auto_download, allow_explicit,
|
||||||
|
max_new_per_check, created PK (user_id, feed_id)
|
||||||
|
entry_state user_id, feed_id, guid, read, flagged, position
|
||||||
|
PK (user_id, feed_id, guid)
|
||||||
|
```
|
||||||
|
|
||||||
|
Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position` columns from
|
||||||
|
before accounts; two bugs came from queries still reading them, and `migrate()` drops them from an
|
||||||
|
older database.
|
||||||
|
|
||||||
|
Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in
|
||||||
|
`migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the
|
||||||
|
same path as `Db::open`, so a migration-only column cannot pass tests while missing in production.
|
||||||
|
|
||||||
|
## Control socket
|
||||||
|
|
||||||
|
Newline-delimited JSON, both directions, over `$XDG_RUNTIME_DIR/ipx.sock`.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
||||||
|
{"ev":"feed_start","feed":"atp"}
|
||||||
|
{"ev":"progress","feed":"atp","enclosure":42,"file":"ep1.mp3","done":8192,"total":3000000}
|
||||||
|
{"ev":"download_done","feed":"atp","enclosure":42,"path":"…","bytes":3000000}
|
||||||
|
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
|
||||||
|
{"ev":"scan_done","feeds":1}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Commands** — `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `download`
|
||||||
|
(`enclosure`), `status`.
|
||||||
|
|
||||||
|
**Events** — `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
||||||
|
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
||||||
|
`scan_done`, `reap_done` and `status` are terminal: a client that asked for work stops reading
|
||||||
|
there.
|
||||||
|
|
||||||
|
Progress carries the enclosure id, without which a UI cannot tell one download from another and
|
||||||
|
ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast,
|
||||||
|
so a client attached to a busy daemon also sees that daemon's other work.
|
||||||
|
|
||||||
|
Inside the process the same events go over a `tokio::broadcast`; commands arrive on an `mpsc` and
|
||||||
|
are handled by a single worker, so nothing races over the same download. Shutdown is a `watch`
|
||||||
|
channel raced *inside* each job — `tokio::select!` only races branches at the point of selection,
|
||||||
|
so a long download had to be able to notice the signal itself.
|
||||||
|
|
||||||
|
## HTTP API
|
||||||
|
|
||||||
|
Everything below `/api` needs a signed-in user; the browser gets a redirect to `/login`, anything
|
||||||
|
else a `401`.
|
||||||
|
|
||||||
|
| Route | |
|
||||||
|
|---|---|
|
||||||
|
| `GET /` | the app |
|
||||||
|
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
|
||||||
|
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
|
||||||
|
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
|
||||||
|
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable, sortable (`sort` = kept, title, feed, type, size or published; `dir` = asc or desc) |
|
||||||
|
| `GET /api/entries` | the same, across every feed you subscribe to (All Subscriptions) |
|
||||||
|
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
|
||||||
|
| `POST /api/read-all` | everything read in every feed you subscribe to (All Subscriptions) |
|
||||||
|
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, kept, position |
|
||||||
|
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
|
||||||
|
| `POST /api/fetch` | |
|
||||||
|
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
|
||||||
|
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
|
||||||
|
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
|
||||||
|
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
|
||||||
|
| `GET /api/events` | SSE, the same broadcast the socket carries |
|
||||||
|
| `GET /api/logs` | admin-only; the ring buffer, with a sequence cursor |
|
||||||
|
| `GET /media/{id}` | the file, with Range support so seeking works |
|
||||||
|
|
||||||
|
Show notes are feed-supplied HTML from an untrusted source, sanitized with `ammonia` server-side
|
||||||
|
before they reach the page.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test # parsing, filters, retention, schedules, SQL, per-user isolation
|
||||||
|
node tests/page-smoke.js # the page script loads and every selector it wires at load exists
|
||||||
|
npx playwright test # a real browser against a real daemon on fixture feeds
|
||||||
|
```
|
||||||
|
|
||||||
|
The Rust tests cannot see a wrong selector, a handler that runs and does nothing, or a page that
|
||||||
|
renders empty — which is what has actually reached users. Each Playwright case maps to a bug that
|
||||||
|
did.
|
||||||
|
|
||||||
|
The suite starts its own daemon and database under `/tmp/ipx-ui-test`, wiped once per run. Tests
|
||||||
|
share that daemon and run in order, so a test that marks something read changes what later tests
|
||||||
|
see — make assertions that do not depend on earlier ones.
|
||||||
91
docs/cli.md
Normal file
91
docs/cli.md
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
# Command line
|
||||||
|
|
||||||
|
```
|
||||||
|
ipx [--config PATH] [--local] <command>
|
||||||
|
```
|
||||||
|
|
||||||
|
Every command that has a wire form probes the control socket first: if a daemon is running, the
|
||||||
|
daemon does the work and the CLI just renders the events it streams back. That is deliberate — two
|
||||||
|
processes must never download the same thing. `--local` forces the work to happen in-process.
|
||||||
|
|
||||||
|
| Command | What it does |
|
||||||
|
|---|---|
|
||||||
|
| `ipx list` | Subscriptions and their state |
|
||||||
|
| `ipx status` | Counts: feeds, pending, downloaded |
|
||||||
|
| `ipx fetch [FEED] [--force]` | Scan everything, or one feed. `--force` ignores the TTL |
|
||||||
|
| `ipx add <url> [--folder X] [--keywords a,b]` | Subscribe; the id comes from the feed title |
|
||||||
|
| `ipx rm <feed>` | Unsubscribe; downloads and history are kept |
|
||||||
|
| `ipx import <file.opml>` / `ipx export <file.opml>` | Move subscriptions in or out. Import subscribes the first admin, as the shared web token does; in the web UI it subscribes whoever is signed in |
|
||||||
|
| `ipx reap [--dry-run]` | Run retention now |
|
||||||
|
| `ipx user <add\|list\|passwd\|rm>` | Accounts for the web UI |
|
||||||
|
| `ipx daemon [--web ADDR]` | Scheduler, control socket and web UI |
|
||||||
|
|
||||||
|
## Accounts
|
||||||
|
|
||||||
|
Passwords are read from **stdin**, so they miss the shell history and any `ps` listing.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
echo -n 'a good password' | ipx user add ray # local account
|
||||||
|
ipx user add ray@example.com --no-password # signs in through the proxy only
|
||||||
|
echo -n 'a good password' | ipx user passwd admin # change a password
|
||||||
|
ipx user list # who exists, and how each signs in
|
||||||
|
ipx user rm sam # account, subscriptions and read state
|
||||||
|
```
|
||||||
|
|
||||||
|
The first account created is an admin; later ones are ordinary users. A database with no accounts
|
||||||
|
at all gets **admin / ipodderx** on the next daemon start, announced in the log — change it.
|
||||||
|
|
||||||
|
To avoid even the command line, read it interactively:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
read -s PW && echo -n "$PW" | ipx user passwd admin
|
||||||
|
```
|
||||||
|
|
||||||
|
## Scanning
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx fetch # everything due
|
||||||
|
ipx fetch atp --force # one feed, ignoring its TTL and schedule
|
||||||
|
```
|
||||||
|
|
||||||
|
A scan: conditional GET (`If-None-Match` / `If-Modified-Since`), parse, record new entries, apply
|
||||||
|
the filters, then download up to the per-scan cap, newest first. A feed nothing has changed in
|
||||||
|
answers `304` and costs one request.
|
||||||
|
|
||||||
|
## Retention
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx reap --dry-run # what would go, oldest first
|
||||||
|
ipx reap # actually delete
|
||||||
|
```
|
||||||
|
|
||||||
|
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
|
||||||
|
with no file are pruned from the database. **An item anyone kept keeps its file**, and one only counts
|
||||||
|
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
|
||||||
|
stops the next scan fetching it again.
|
||||||
|
|
||||||
|
## The daemon
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx daemon # scheduler + socket + web UI
|
||||||
|
ipx daemon --web 0.0.0.0:8099 # override the configured bind for one run
|
||||||
|
```
|
||||||
|
|
||||||
|
One daemon per socket; a second refuses to start rather than fight over the database. It shuts down
|
||||||
|
cleanly on SIGTERM, including mid-download.
|
||||||
|
|
||||||
|
To kill it, match the binary exactly:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pkill -x ipx
|
||||||
|
```
|
||||||
|
|
||||||
|
`pkill -f ipx` matches the shell running the command too, and kills your own session.
|
||||||
|
|
||||||
|
## Talking to it directly
|
||||||
|
|
||||||
|
```sh
|
||||||
|
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
||||||
|
```
|
||||||
|
|
||||||
|
See [architecture.md](architecture.md#control-socket) for the protocol.
|
||||||
116
docs/configuration.md
Normal file
116
docs/configuration.md
Normal file
@@ -0,0 +1,116 @@
|
|||||||
|
# Configuration
|
||||||
|
|
||||||
|
One TOML file, read at startup and re-read whenever the web UI writes to it — most changes take
|
||||||
|
effect without a restart. Default location `$XDG_CONFIG_HOME/ipx/config.toml`
|
||||||
|
(`~/.config/ipx/config.toml`), overridden with `--config` or `$IPX_CONFIG`.
|
||||||
|
|
||||||
|
| What | Where | Override |
|
||||||
|
|---|---|---|
|
||||||
|
| Config | `~/.config/ipx/config.toml` | `--config`, `$IPX_CONFIG` |
|
||||||
|
| Database | `~/.local/share/ipx/state.db` | `$IPX_DATA_DIR` |
|
||||||
|
| Control socket | `$XDG_RUNTIME_DIR/ipx.sock` | `[general] socket` |
|
||||||
|
| Downloads | `[general] download_dir` | — |
|
||||||
|
|
||||||
|
`~` is expanded in paths. The database is SQLite in WAL mode; back it up by copying `state.db`
|
||||||
|
while the daemon is stopped, or with `sqlite3 state.db .backup`.
|
||||||
|
|
||||||
|
## `[general]`
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[general]
|
||||||
|
download_dir = "~/Podcasts"
|
||||||
|
socket = "/run/user/1000/ipx.sock"
|
||||||
|
schedule = "every 1h" # "every 30m", "every 4h", "2d", "90" (minutes)
|
||||||
|
organize = "feed" # "feed" | "date"
|
||||||
|
max_total_gb = 50 # 0 = unlimited
|
||||||
|
max_age_days = 30 # 0 = keep forever
|
||||||
|
max_new_per_check = 3 # per feed, per scan. 0 = unlimited
|
||||||
|
media_types = ["audio", "video"]
|
||||||
|
```
|
||||||
|
|
||||||
|
* **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
|
||||||
|
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
|
||||||
|
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
|
||||||
|
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
|
||||||
|
pad. Kept items are never deleted, and a file only counts as read once every subscriber has
|
||||||
|
read it. `0` disables it entirely.
|
||||||
|
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
|
||||||
|
Kept ones stay. `0` disables it.
|
||||||
|
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
|
||||||
|
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
|
||||||
|
fetched 216 files and 22 GB in one scan.
|
||||||
|
* **`media_types`** — top-level MIME types taken automatically. Anything else is still listed and
|
||||||
|
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
|
||||||
|
without this the disk fills with artwork. Empty takes everything.
|
||||||
|
|
||||||
|
## `[torrent]`
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[torrent]
|
||||||
|
enabled = true
|
||||||
|
seed_ratio = 1.0 # stop seeding at this ratio ...
|
||||||
|
seed_time_mins = 60 # ... or after this long, whichever comes first
|
||||||
|
port_range = "6881-6889"
|
||||||
|
stall_mins = 30 # give up on a torrent making no progress
|
||||||
|
```
|
||||||
|
|
||||||
|
A `.torrent` body is handed to the torrent path whatever MIME type it was advertised as. Torrents
|
||||||
|
run on their own tasks (two at a time) so a slow swarm never blocks a scan.
|
||||||
|
|
||||||
|
## `[web]`
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[web]
|
||||||
|
enabled = true
|
||||||
|
bind = "0.0.0.0:8099" # 127.0.0.1:8080 by default
|
||||||
|
token = "" # generated and saved on first run
|
||||||
|
trusted_header = "" # e.g. "Cf-Access-Authenticated-User-Email"
|
||||||
|
trusted_proxies = ["127.0.0.1", "::1"]
|
||||||
|
auto_create_users = true
|
||||||
|
session_days = 30
|
||||||
|
```
|
||||||
|
|
||||||
|
* **`token`** — the shared secret, which signs in as the **admin**. `?token=…` sets a cookie, so
|
||||||
|
you paste it once per browser. It is what the Docker healthcheck and any scripts use.
|
||||||
|
* **`trusted_header`** — a header naming the signed-in user, set by whatever fronts ipx. Empty
|
||||||
|
disables that path. See [sso.md](sso.md).
|
||||||
|
* **`trusted_proxies`** — addresses allowed to assert that header, and the entire security boundary
|
||||||
|
for it. Name the proxy, never a subnet.
|
||||||
|
* **`auto_create_users`** — create an account the first time the proxy vouches for a new name.
|
||||||
|
* **`session_days`** — sign a session out after this long without a request.
|
||||||
|
|
||||||
|
It is plain HTTP. On a LAN bind everything crosses the network in the clear — and a feed URL can
|
||||||
|
itself carry a credential. Put TLS in front of it if that matters.
|
||||||
|
|
||||||
|
## `[feeds.<id>]`
|
||||||
|
|
||||||
|
The table key is the feed id: stable, human-readable, and used in paths and the API. `ipx add`
|
||||||
|
derives it from the feed title.
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[feeds.atp]
|
||||||
|
url = "https://atp.fm/rss"
|
||||||
|
folder = "Accidental Tech Podcast" # default: the feed title
|
||||||
|
schedule = "every 6h" # overrides [general] for this feed
|
||||||
|
media_types = ["audio"] # overrides [general] for this feed
|
||||||
|
username = "ray" # HTTP basic auth
|
||||||
|
password_env = "IPX_ATP_PASS" # preferred over a literal `password`
|
||||||
|
```
|
||||||
|
|
||||||
|
With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
|
||||||
|
`max_new_per_check` live on each person's subscription in the database**, not here — the values in
|
||||||
|
config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
|
||||||
|
and are the same for everyone. See [users.md](users.md).
|
||||||
|
|
||||||
|
Feeds derived from a subscribed OPML are **not** written here: the OPML is the source of truth and
|
||||||
|
they are re-derived on every scan. Editing one in the UI promotes it to a real config entry.
|
||||||
|
|
||||||
|
## Environment
|
||||||
|
|
||||||
|
| Variable | Effect |
|
||||||
|
|---|---|
|
||||||
|
| `IPX_CONFIG` | Config file path |
|
||||||
|
| `IPX_DATA_DIR` | Directory holding `state.db` |
|
||||||
|
| `IPX_LOG` | What reaches stderr (`ipx=debug`, `ipx::scan=debug`, …) |
|
||||||
|
| `IPX_UI_LOG` | What the in-process log buffer captures for the UI's Log view |
|
||||||
|
| `http_proxy` / `https_proxy` | Honoured for feed and enclosure fetches |
|
||||||
1549
docs/history.md
Normal file
1549
docs/history.md
Normal file
File diff suppressed because it is too large
Load Diff
228
docs/sso.md
Normal file
228
docs/sso.md
Normal file
@@ -0,0 +1,228 @@
|
|||||||
|
# Signing in through Cloudflare Zero Trust or Authentik
|
||||||
|
|
||||||
|
ipx can take the signed-in identity from whatever sits in front of it, instead of asking for a
|
||||||
|
password itself. Both products below do the same thing in the end: they authenticate the person and
|
||||||
|
pass the result to the origin in a **header**. ipx reads that header, finds (or creates) the
|
||||||
|
matching account, and gets on with it.
|
||||||
|
|
||||||
|
Read [How this is secured](#how-this-is-secured) before exposing anything. The short version: a
|
||||||
|
header is worth exactly as much as the hop that set it, so ipx only believes one from an address you
|
||||||
|
list.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The ipx side (both setups)
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[web]
|
||||||
|
enabled = true
|
||||||
|
bind = "0.0.0.0:8099"
|
||||||
|
token = "…" # keep it: it is the admin, used by the healthcheck
|
||||||
|
|
||||||
|
# The header your proxy sets. Empty (the default) disables this whole path.
|
||||||
|
trusted_header = "Cf-Access-Authenticated-User-Email" # Authentik: "X-authentik-username"
|
||||||
|
|
||||||
|
# Addresses allowed to assert that header -- the proxy, and nothing else.
|
||||||
|
trusted_proxies = ["127.0.0.1", "::1"]
|
||||||
|
|
||||||
|
# Create an account the first time the proxy vouches for a name ipx has not seen.
|
||||||
|
auto_create_users = true
|
||||||
|
|
||||||
|
session_days = 30
|
||||||
|
```
|
||||||
|
|
||||||
|
Restart the daemon after editing. Accounts made this way have **no password**: they can only ever
|
||||||
|
arrive through the proxy. `ipx user list` marks them `proxy only`.
|
||||||
|
|
||||||
|
The first account created is an admin. Every later one is an ordinary user, and an ordinary user
|
||||||
|
cannot change global settings, a feed's URL or folder, or how often feeds are scanned: the API
|
||||||
|
refuses those with a `403`, not just the UI. Everything else about a feed (which items they want,
|
||||||
|
whether to fetch them, how many at a time) is theirs alone; see [users.md](users.md).
|
||||||
|
|
||||||
|
Somebody arriving through the proxy for the first time starts with **no feeds**, because
|
||||||
|
subscriptions are per person. Adding a feed someone else already reads costs no second fetch and no
|
||||||
|
second copy on disk.
|
||||||
|
|
||||||
|
Promote someone with:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx user list
|
||||||
|
echo -n 'a good password' | ipx user passwd <name> # optional: also lets them sign in directly
|
||||||
|
```
|
||||||
|
|
||||||
|
Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN
|
||||||
|
when the tunnel is down. So does the shared `[web] token`, which signs in as the admin: that is
|
||||||
|
what the Docker healthcheck uses, and the way back in if you lock yourself out. A brand new database
|
||||||
|
starts with **admin / ipodderx** — change it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Cloudflare Zero Trust
|
||||||
|
|
||||||
|
This is what runs `ipodderx.sdf1.net`: a `cloudflared` tunnel to the origin, with an Access
|
||||||
|
application in front of it. Cloudflare authenticates the visitor and adds
|
||||||
|
`Cf-Access-Authenticated-User-Email` to every request it forwards.
|
||||||
|
|
||||||
|
### 1. The tunnel
|
||||||
|
|
||||||
|
In **Zero Trust → Networks → Tunnels**, either use the existing tunnel or create one, then add a
|
||||||
|
public hostname:
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|---|---|
|
||||||
|
| Subdomain / domain | `ipodderx` / `sdf1.net` |
|
||||||
|
| Type | HTTP |
|
||||||
|
| URL | `localhost:8099` (or the LAN address of the box) |
|
||||||
|
|
||||||
|
Use `localhost` when `cloudflared` runs on the same machine as ipx — that keeps the origin request
|
||||||
|
coming from `127.0.0.1`, which is already in `trusted_proxies`. If `cloudflared` runs elsewhere (its
|
||||||
|
own container, another host), put **its** address in `trusted_proxies` instead, and make sure
|
||||||
|
nothing else can reach port 8099.
|
||||||
|
|
||||||
|
### 2. The Access application
|
||||||
|
|
||||||
|
**Zero Trust → Access → Applications → Add an application → Self-hosted**:
|
||||||
|
|
||||||
|
- Application domain: `ipodderx.sdf1.net`
|
||||||
|
- Session duration: whatever suits; ipx keeps its own 30-day session on top.
|
||||||
|
- Add a policy — *Allow*, with a rule such as `Emails` → your address, or `Emails ending in` →
|
||||||
|
your domain. Anyone this policy admits gets an ipx account when `auto_create_users` is on, so keep
|
||||||
|
the policy as narrow as the people you actually want reading your feeds.
|
||||||
|
|
||||||
|
### 3. Point ipx at the header
|
||||||
|
|
||||||
|
```toml
|
||||||
|
trusted_header = "Cf-Access-Authenticated-User-Email"
|
||||||
|
trusted_proxies = ["127.0.0.1", "::1"]
|
||||||
|
```
|
||||||
|
|
||||||
|
The username becomes the email address, lower-cased (`ray@example.com`). That is what shows in the
|
||||||
|
sidebar and what `ipx user list` prints.
|
||||||
|
|
||||||
|
### 4. Check it
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# From the box itself: no header, no session -> the sign-in page.
|
||||||
|
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' -H 'Accept: text/html' http://127.0.0.1:8099/
|
||||||
|
|
||||||
|
# Pretending to be the tunnel (only works because 127.0.0.1 is trusted):
|
||||||
|
curl -s -H 'Cf-Access-Authenticated-User-Email: you@example.com' http://127.0.0.1:8099/api/me
|
||||||
|
```
|
||||||
|
|
||||||
|
Then load `https://ipodderx.sdf1.net` in a browser: Cloudflare should ask who you are, and ipx
|
||||||
|
should show your address in the sidebar footer without ever asking for a password.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Authentik
|
||||||
|
|
||||||
|
Authentik does this with a **Proxy Provider** plus an **outpost**, which sits in the request path and
|
||||||
|
adds `X-authentik-username` (also `X-authentik-email`, `X-authentik-name`, `X-authentik-groups`).
|
||||||
|
|
||||||
|
### 1. Provider
|
||||||
|
|
||||||
|
**Applications → Providers → Create → Proxy Provider**:
|
||||||
|
|
||||||
|
- Name: `ipx`
|
||||||
|
- Authorization flow: your usual (`default-provider-authorization-implicit-consent`)
|
||||||
|
- Mode: **Forward auth (single application)** if an existing reverse proxy fronts ipx, or
|
||||||
|
**Proxy** to let the outpost talk to ipx directly.
|
||||||
|
- External host: `https://ipodderx.example.net`
|
||||||
|
- Internal host (Proxy mode): `http://<ip of the ipx box>:8099`
|
||||||
|
|
||||||
|
### 2. Application and outpost
|
||||||
|
|
||||||
|
**Applications → Create**, bind it to that provider, and give it a policy so only the people you
|
||||||
|
mean are let through. Then add the provider to an outpost (**Applications → Outposts**, the embedded
|
||||||
|
one is fine).
|
||||||
|
|
||||||
|
### 3. Forward auth, if you use nginx/SWAG in front
|
||||||
|
|
||||||
|
In the server block for ipx:
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
location /outpost.goauthentik.io {
|
||||||
|
proxy_pass http://authentik-server:9000/outpost.goauthentik.io;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
|
||||||
|
add_header Set-Cookie $auth_cookie;
|
||||||
|
auth_request_set $auth_cookie $upstream_http_set_cookie;
|
||||||
|
}
|
||||||
|
|
||||||
|
location / {
|
||||||
|
auth_request /outpost.goauthentik.io/auth/nginx;
|
||||||
|
error_page 401 = @goauthentik_proxy_signin;
|
||||||
|
auth_request_set $auth_cookie $upstream_http_set_cookie;
|
||||||
|
add_header Set-Cookie $auth_cookie;
|
||||||
|
|
||||||
|
# This is the line that matters to ipx.
|
||||||
|
auth_request_set $authentik_username $upstream_http_x_authentik_username;
|
||||||
|
proxy_set_header X-authentik-username $authentik_username;
|
||||||
|
|
||||||
|
proxy_pass http://ipx:8099;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Point ipx at the header
|
||||||
|
|
||||||
|
```toml
|
||||||
|
trusted_header = "X-authentik-username"
|
||||||
|
trusted_proxies = ["172.18.0.5"] # the outpost or nginx container, NOT a whole subnet
|
||||||
|
```
|
||||||
|
|
||||||
|
Usernames arrive as Authentik knows them (`ray`), lower-cased.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How this is secured
|
||||||
|
|
||||||
|
**The header is only believed from `trusted_proxies`.** Every other source is ignored, and the
|
||||||
|
request falls through to a session cookie or the shared token. This is the whole security boundary,
|
||||||
|
so:
|
||||||
|
|
||||||
|
- List the **proxy's own address**, not a range. `["127.0.0.1"]` when the tunnel runs beside ipx;
|
||||||
|
the container's IP when it does not.
|
||||||
|
- Never list a LAN subnet. Anyone on your network could then send
|
||||||
|
`Cf-Access-Authenticated-User-Email: admin@…` and be your admin.
|
||||||
|
- Make sure the origin port is not reachable *around* the proxy by anyone you would not admit
|
||||||
|
through it. If it is, bind ipx to `127.0.0.1` and let only the proxy reach it.
|
||||||
|
|
||||||
|
Verify the refusal, don't assume it — set `trusted_proxies = ["10.9.9.9"]` briefly and confirm a
|
||||||
|
header from your machine gets a `401`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -s -o /dev/null -w '%{http_code}\n' \
|
||||||
|
-H 'Cf-Access-Authenticated-User-Email: someone@example.com' http://127.0.0.1:8099/api/me
|
||||||
|
```
|
||||||
|
|
||||||
|
**What ipx does not do:** it does not verify Cloudflare's `Cf-Access-Jwt-Assertion` signature or
|
||||||
|
Authentik's session. It trusts the hop. That is a deliberate trade — it keeps the configuration to
|
||||||
|
three lines — and it is sound exactly as long as the point above holds.
|
||||||
|
|
||||||
|
**Turning it off:** clear `trusted_header`. Existing proxy-only accounts stay, but nobody can sign
|
||||||
|
in with them until they are given a password (`ipx user passwd <name>`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Everyday administration
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx user list # who exists, and how each one signs in
|
||||||
|
echo -n 'secret123' | ipx user add sam # local account, password on stdin
|
||||||
|
ipx user add sam --no-password # proxy-only account, created ahead of time
|
||||||
|
echo -n 'newsecret' | ipx user passwd sam # change a password
|
||||||
|
ipx user rm sam # remove the account
|
||||||
|
```
|
||||||
|
|
||||||
|
Set `auto_create_users = false` once everyone who should have an account has one. After that the
|
||||||
|
proxy vouching for an unknown name is logged and refused, rather than quietly making an account.
|
||||||
|
Pre-create people instead with `ipx user add <name> --no-password`, using exactly the name the
|
||||||
|
header will carry (Cloudflare sends the email address, lower-cased).
|
||||||
|
|
||||||
|
Scanning intervals, the disk quota, retention, the download folder and a feed's URL are
|
||||||
|
**admin-only**: the Settings button is hidden for everyone else, and the API refuses the change even
|
||||||
|
if the request is made by hand. Everyone controls their own keywords, auto-download, explicit
|
||||||
|
setting and per-scan cap, along with their own read state and which feeds they see.
|
||||||
|
|
||||||
|
See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
|
||||||
|
for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.
|
||||||
97
docs/users.md
Normal file
97
docs/users.md
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
# Accounts, and what several people share
|
||||||
|
|
||||||
|
ipx serves any number of people from one copy of the data. The rule that decides everything else:
|
||||||
|
**there is one file on disk per enclosure URL.** Two people subscribed to the same show cost one
|
||||||
|
fetch, one parse and one file.
|
||||||
|
|
||||||
|
## What is yours, what is everyone's
|
||||||
|
|
||||||
|
| Yours alone | The same for everyone |
|
||||||
|
|---|---|
|
||||||
|
| Read, kept, playback position | The feed's URL |
|
||||||
|
| Which feeds you see at all | Its download folder |
|
||||||
|
| Keywords, auto-download, explicit, per-scan cap | When it is scanned |
|
||||||
|
| | The file on disk |
|
||||||
|
|
||||||
|
The right-hand column describes the feed and the file rather than a preference — two people wanting
|
||||||
|
different folders would mean two copies. Those three are **admin-only**, and the API returns `403`
|
||||||
|
for anyone else rather than merely hiding the controls.
|
||||||
|
|
||||||
|
## How the scanner merges everyone's wants
|
||||||
|
|
||||||
|
One fetch serves every subscriber, so the policy is a union:
|
||||||
|
|
||||||
|
* an item is downloaded if **anyone** wants it — one person's keyword set matching is enough, and
|
||||||
|
one person with no keywords removes the filter for that feed entirely
|
||||||
|
* auto-download is on if **anyone** has it on
|
||||||
|
* the per-scan cap is the **largest** anyone asked for
|
||||||
|
|
||||||
|
So "auto-download off" means *I don't cause downloads*, not *I never see them*. If someone else's
|
||||||
|
subscription pulls an item, you see it listed as downloaded and can play it, because the enclosure
|
||||||
|
is shared.
|
||||||
|
|
||||||
|
## Deleting
|
||||||
|
|
||||||
|
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
|
||||||
|
for everyone** and names them in the confirmation, and the server has the last word: if anyone else
|
||||||
|
has kept the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
|
||||||
|
reason, and only `?force=true` goes through.
|
||||||
|
|
||||||
|
Retention follows the same rule: an item anyone kept keeps its file, and it counts as read only once
|
||||||
|
every subscriber has read it.
|
||||||
|
|
||||||
|
## Signing in
|
||||||
|
|
||||||
|
Three ways, tried in order of how specific the claim is:
|
||||||
|
|
||||||
|
1. **A proxy header** naming the user — Cloudflare Zero Trust or Authentik. Honoured only from an
|
||||||
|
address in `trusted_proxies`. See [sso.md](sso.md).
|
||||||
|
2. **A session cookie** from signing in at `/login`. Argon2id hashes, sessions in the database,
|
||||||
|
idle timeout `session_days`.
|
||||||
|
3. **The shared `[web] token`**, which signs in as the admin — this is what the Docker healthcheck
|
||||||
|
and any scripts use.
|
||||||
|
|
||||||
|
A database with no accounts creates **admin / ipodderx** on the next daemon start and says so in
|
||||||
|
the log. Change it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
echo -n 'a good password' | ipx user passwd admin
|
||||||
|
```
|
||||||
|
|
||||||
|
## Adding someone
|
||||||
|
|
||||||
|
```sh
|
||||||
|
echo -n 'their password' | ipx user add sam
|
||||||
|
```
|
||||||
|
|
||||||
|
They sign in at `/login` and start with **no feeds**: subscriptions are per person. Adding a feed
|
||||||
|
someone else already has costs nothing — no second fetch, no second copy — it just appears on their
|
||||||
|
list with their own read state. Unsubscribing removes it from their list alone; only when the last
|
||||||
|
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
|
||||||
|
re-subscribing does not pull the back catalogue again.
|
||||||
|
|
||||||
|
**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
|
||||||
|
shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
|
||||||
|
included. Directory lists every one of them A to Z. Your own feeds are marked Subscribed.
|
||||||
|
It shows a title, artwork and a count, never a URL or who reads it. Feeds from an
|
||||||
|
OPML subscription are left out, since they come with the OPML. So is anything that looks private: a
|
||||||
|
login configured for the feed, credentials in its URL, or a key such as `auth=` or `token=` in the
|
||||||
|
query, or a feed from a paid-feed service such as Patreon or Supercast, which put the key in the
|
||||||
|
path. Those are someone's paid subscriptions, and listing them would let anyone here read what they
|
||||||
|
pay for.
|
||||||
|
|
||||||
|
An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none
|
||||||
|
for someone the proxy signs in), tick or untick Admin, or remove an account. Removing one takes its
|
||||||
|
subscriptions and read state with it; downloaded files stay. The only admin cannot be demoted or
|
||||||
|
removed there, so there is always someone who can manage the rest.
|
||||||
|
|
||||||
|
## Admin
|
||||||
|
|
||||||
|
The first account is an admin. An admin can change global settings (scanning interval, quota,
|
||||||
|
retention, media types, download folder), a feed's URL, folder and schedule, and who has an account
|
||||||
|
and who else is an admin, and read the log, which names everyone's feeds and sign-ins. Everyone else
|
||||||
|
gets the Settings and Log buttons hidden and a `403` if they ask anyway.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipx user list # the admin column says who
|
||||||
|
```
|
||||||
58
package-lock.json
generated
Normal file
58
package-lock.json
generated
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
{
|
||||||
|
"name": "ipx-ui-tests",
|
||||||
|
"lockfileVersion": 3,
|
||||||
|
"requires": true,
|
||||||
|
"packages": {
|
||||||
|
"": {
|
||||||
|
"name": "ipx-ui-tests",
|
||||||
|
"devDependencies": {
|
||||||
|
"@playwright/test": "^1.56.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/@playwright/test": {
|
||||||
|
"version": "1.63.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.63.0.tgz",
|
||||||
|
"integrity": "sha512-oxMK4vllB9RK5NQ2l1pq1IfOf2AvnEuj/vYGDj0H2nMtmtZpKtCwt/l00GEO6xjGfpBNAvjovvYdCm50dRQkpQ==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"playwright": "1.63.0"
|
||||||
|
},
|
||||||
|
"bin": {
|
||||||
|
"playwright": "cli.js"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=20"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/playwright": {
|
||||||
|
"version": "1.63.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.63.0.tgz",
|
||||||
|
"integrity": "sha512-+7ziBLidS4NaNCdt57SUDT+wYmmd5fmiQejUic/kb+YsYSCPyOOE9sebzMjNmQrsnNpDJqd4WHvV/8lfKfUDUg==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"playwright-core": "1.63.0"
|
||||||
|
},
|
||||||
|
"bin": {
|
||||||
|
"playwright": "cli.js"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=20"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/playwright-core": {
|
||||||
|
"version": "1.63.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.63.0.tgz",
|
||||||
|
"integrity": "sha512-rYCsBF/M5HjUch52bbtVONEFjv6Xu8sm8h72dNlR5bzIE1fvC/bxgspzkjSfU+MweEMmPM8KJebG6nnyxo5mCg==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"bin": {
|
||||||
|
"playwright-core": "cli.js"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=20"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
13
package.json
Normal file
13
package.json
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"name": "ipx-ui-tests",
|
||||||
|
"private": true,
|
||||||
|
"description": "Browser tests for the ipx web UI. The Rust tests cover the server; these cover the page.",
|
||||||
|
"scripts": {
|
||||||
|
"test": "playwright test",
|
||||||
|
"test:headed": "playwright test --headed",
|
||||||
|
"smoke": "node tests/page-smoke.js"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@playwright/test": "^1.56.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
43
playwright.config.js
Normal file
43
playwright.config.js
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
const { defineConfig } = require('@playwright/test');
|
||||||
|
const setup = require('./tests/ui/global-setup');
|
||||||
|
|
||||||
|
// Before anything else, including the servers below.
|
||||||
|
setup.prepare();
|
||||||
|
|
||||||
|
// Real browser against a real daemon. The stub-DOM smoke test catches a script that
|
||||||
|
// fails to load; it cannot catch a wrong selector, a handler that runs but does nothing,
|
||||||
|
// or a page that renders empty -- which is exactly what has slipped through before.
|
||||||
|
module.exports = defineConfig({
|
||||||
|
testDir: './tests/ui',
|
||||||
|
timeout: 30_000,
|
||||||
|
expect: { timeout: 10_000 },
|
||||||
|
fullyParallel: false, // one daemon, one database
|
||||||
|
workers: 1,
|
||||||
|
reporter: process.env.CI ? 'line' : [['list']],
|
||||||
|
use: {
|
||||||
|
baseURL: 'http://127.0.0.1:8791',
|
||||||
|
trace: 'retain-on-failure',
|
||||||
|
screenshot: 'only-on-failure',
|
||||||
|
},
|
||||||
|
webServer: [
|
||||||
|
{
|
||||||
|
command: 'node tests/ui/fixtures/serve.js',
|
||||||
|
port: 8792,
|
||||||
|
reuseExistingServer: false,
|
||||||
|
stdout: 'ignore',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// Build first so the tests always run against current source.
|
||||||
|
command: 'cargo build -q && exec ./target/debug/ipx daemon',
|
||||||
|
port: 8791,
|
||||||
|
reuseExistingServer: false,
|
||||||
|
timeout: 180_000,
|
||||||
|
stdout: 'pipe',
|
||||||
|
env: {
|
||||||
|
IPX_CONFIG: `${setup.root}/config/config.toml`,
|
||||||
|
IPX_DATA_DIR: `${setup.root}/data`,
|
||||||
|
IPX_LOG: 'ipx=info',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
],
|
||||||
|
});
|
||||||
92
src/auth.rs
Normal file
92
src/auth.rs
Normal file
@@ -0,0 +1,92 @@
|
|||||||
|
//! Who is asking. Sign-in is either a local password or a header set by whatever fronts
|
||||||
|
//! this -- Cloudflare Zero Trust on `ipodderx.sdf1.net`, which puts the authenticated
|
||||||
|
//! address in `Cf-Access-Authenticated-User-Email`.
|
||||||
|
|
||||||
|
use anyhow::{Result, bail};
|
||||||
|
use argon2::Argon2;
|
||||||
|
use argon2::password_hash::{PasswordHasher, PasswordVerifier, phc::PasswordHash};
|
||||||
|
|
||||||
|
/// Argon2id with the crate's defaults, which are the OWASP-recommended parameters. The
|
||||||
|
/// salt is generated per password by the hasher itself.
|
||||||
|
pub fn hash_password(password: &str) -> Result<String> {
|
||||||
|
if password.len() < 8 {
|
||||||
|
bail!("password must be at least 8 characters");
|
||||||
|
}
|
||||||
|
Argon2::default()
|
||||||
|
.hash_password(password.as_bytes())
|
||||||
|
.map(|h| h.to_string())
|
||||||
|
.map_err(|e| anyhow::anyhow!("could not hash the password: {e}"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// False for a wrong password *and* for a stored hash this build cannot parse; either way
|
||||||
|
/// the answer is no.
|
||||||
|
pub fn verify_password(password: &str, stored: &str) -> bool {
|
||||||
|
let Ok(parsed) = PasswordHash::new(stored) else {
|
||||||
|
tracing::warn!("stored password hash is unreadable; refusing the sign-in");
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
Argon2::default()
|
||||||
|
.verify_password(password.as_bytes(), &parsed)
|
||||||
|
.is_ok()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A session id: 256 bits of urandom, hex. Long enough that guessing is not a strategy.
|
||||||
|
pub fn new_session_token() -> String {
|
||||||
|
let mut bytes = [0u8; 32];
|
||||||
|
if getrandom(&mut bytes).is_err() {
|
||||||
|
// Falling back to the clock would be a predictable session id. Better to fail.
|
||||||
|
panic!("no source of randomness for a session token");
|
||||||
|
}
|
||||||
|
bytes.iter().map(|b| format!("{b:02x}")).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn getrandom(buf: &mut [u8]) -> std::io::Result<()> {
|
||||||
|
use std::io::Read;
|
||||||
|
std::fs::File::open("/dev/urandom")?.read_exact(buf)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A username taken from a proxy header. Cloudflare sends an email address; the local part
|
||||||
|
/// is what a person recognises, and the whole thing stays unique enough for one household.
|
||||||
|
pub fn name_from_header(raw: &str) -> Option<String> {
|
||||||
|
let name = raw.trim();
|
||||||
|
if name.is_empty() || name.len() > 190 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
// Anything that could confuse a lookup or a log line is not a name.
|
||||||
|
if name.chars().any(|c| c.is_control() || c == ',' || c == ';') {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
Some(name.to_ascii_lowercase())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_password_verifies_only_against_itself() {
|
||||||
|
let h = hash_password("correct horse battery").unwrap();
|
||||||
|
assert!(verify_password("correct horse battery", &h));
|
||||||
|
assert!(!verify_password("Correct horse battery", &h));
|
||||||
|
assert!(!verify_password("", &h));
|
||||||
|
// A hash from a different scheme, or a truncated one, must not authenticate.
|
||||||
|
assert!(!verify_password("correct horse battery", "not-a-hash"));
|
||||||
|
assert!(hash_password("short").is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn session_tokens_are_long_and_distinct() {
|
||||||
|
let a = new_session_token();
|
||||||
|
let b = new_session_token();
|
||||||
|
assert_eq!(a.len(), 64);
|
||||||
|
assert_ne!(a, b);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_header_name_is_cleaned_or_refused() {
|
||||||
|
assert_eq!(name_from_header(" Ray@Example.COM "), Some("ray@example.com".into()));
|
||||||
|
assert_eq!(name_from_header(""), None);
|
||||||
|
assert_eq!(name_from_header("ray\nadmin"), None);
|
||||||
|
assert_eq!(name_from_header("ray;admin"), None);
|
||||||
|
}
|
||||||
|
}
|
||||||
218
src/config.rs
218
src/config.rs
@@ -5,29 +5,40 @@ use serde::{Deserialize, Serialize};
|
|||||||
use std::collections::BTreeMap;
|
use std::collections::BTreeMap;
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
#[derive(Debug, Default, Deserialize, Serialize)]
|
#[derive(Debug, Default, Clone, Deserialize, Serialize)]
|
||||||
pub struct Config {
|
pub struct Config {
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub general: General,
|
pub general: General,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub torrent: Torrent,
|
pub torrent: Torrent,
|
||||||
|
#[serde(default)]
|
||||||
|
pub web: Web,
|
||||||
/// Keyed by feed id: the TOML table name, which replaces the old genHash(feedURL).
|
/// Keyed by feed id: the TOML table name, which replaces the old genHash(feedURL).
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub feeds: BTreeMap<String, Feed>,
|
pub feeds: BTreeMap<String, Feed>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Deserialize, Serialize)]
|
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub struct General {
|
pub struct General {
|
||||||
pub download_dir: PathBuf,
|
pub download_dir: PathBuf,
|
||||||
pub socket: PathBuf,
|
pub socket: PathBuf,
|
||||||
/// Default poll interval; a feed's own <ttl> wins when it is longer.
|
/// How often to re-check feeds: "every 30m", "every 4h", "90" (minutes), "1d".
|
||||||
pub interval_mins: u64,
|
/// A feed's own `schedule` overrides this.
|
||||||
|
pub schedule: String,
|
||||||
pub organize: Organize,
|
pub organize: Organize,
|
||||||
/// 0 = unlimited.
|
/// 0 = unlimited.
|
||||||
pub max_total_gb: f64,
|
pub max_total_gb: f64,
|
||||||
/// 0 = keep forever.
|
/// 0 = keep forever.
|
||||||
pub max_age_days: u64,
|
pub max_age_days: u64,
|
||||||
|
/// How many new enclosures a single scan may take, when a feed does not say.
|
||||||
|
/// Unlimited by default was a trap: subscribing to an OPML of 80 feeds then pulled
|
||||||
|
/// every back-catalogue episode at once. 0 means unlimited, deliberately chosen.
|
||||||
|
pub max_new_per_check: usize,
|
||||||
|
/// Top-level media types worth downloading. Blog feeds put each article's header
|
||||||
|
/// image in an <enclosure>, so taking everything filled the disk with artwork and
|
||||||
|
/// counted it as episodes. Empty means take anything.
|
||||||
|
pub media_types: Vec<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
|
||||||
@@ -39,7 +50,7 @@ pub enum Organize {
|
|||||||
Date,
|
Date,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Deserialize, Serialize)]
|
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub struct Torrent {
|
pub struct Torrent {
|
||||||
pub enabled: bool,
|
pub enabled: bool,
|
||||||
@@ -51,6 +62,48 @@ pub struct Torrent {
|
|||||||
pub stall_mins: u64,
|
pub stall_mins: u64,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||||
|
#[serde(default)]
|
||||||
|
pub struct Web {
|
||||||
|
pub enabled: bool,
|
||||||
|
/// Use 0.0.0.0 to reach it from the LAN. Anything but loopback needs the token.
|
||||||
|
pub bind: String,
|
||||||
|
/// Shared secret. Generated and written back on first run when left empty. It signs
|
||||||
|
/// in as the admin, which is what keeps the healthcheck and any scripts working.
|
||||||
|
pub token: String,
|
||||||
|
/// A header naming the signed-in user, set by whatever fronts this -- Cloudflare Zero
|
||||||
|
/// Trust sends `Cf-Access-Authenticated-User-Email`. Empty disables the whole path.
|
||||||
|
pub trusted_header: String,
|
||||||
|
/// Addresses allowed to assert that header. A header is only as trustworthy as the
|
||||||
|
/// hop that set it, so an empty list means nobody: on a LAN-bound port anyone could
|
||||||
|
/// otherwise claim to be anyone. Loopback covers a tunnel running beside the daemon.
|
||||||
|
pub trusted_proxies: Vec<String>,
|
||||||
|
/// Create an account the first time the proxy vouches for a name it has not seen.
|
||||||
|
pub auto_create_users: bool,
|
||||||
|
/// Sign a session out after this long without a request.
|
||||||
|
pub session_days: i64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for Web {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self {
|
||||||
|
enabled: false,
|
||||||
|
bind: "127.0.0.1:8080".into(),
|
||||||
|
token: String::new(),
|
||||||
|
trusted_header: String::new(),
|
||||||
|
trusted_proxies: vec!["127.0.0.1".into(), "::1".into()],
|
||||||
|
auto_create_users: true,
|
||||||
|
session_days: 30,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Web {
|
||||||
|
pub fn binds_publicly(&self) -> bool {
|
||||||
|
!self.bind.starts_with("127.") && !self.bind.starts_with("localhost")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone, Deserialize, Serialize)]
|
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||||
pub struct Feed {
|
pub struct Feed {
|
||||||
pub url: String,
|
pub url: String,
|
||||||
@@ -65,7 +118,17 @@ pub struct Feed {
|
|||||||
pub allow_explicit: bool,
|
pub allow_explicit: bool,
|
||||||
#[serde(default = "yes")]
|
#[serde(default = "yes")]
|
||||||
pub auto_download: bool,
|
pub auto_download: bool,
|
||||||
/// Cap on new downloads per scan. None = unlimited.
|
/// Set on feeds that came from a subscribed OPML: the id of the OPML feed they
|
||||||
|
/// belong to. The OPML is re-read on every scan and this list kept in step.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub group: Option<String>,
|
||||||
|
/// Overrides the global schedule for this feed. Same forms: "every 6h", "2d".
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub schedule: Option<String>,
|
||||||
|
/// Media types for this feed. None follows `[general]`; an empty list takes anything.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub media_types: Option<Vec<String>>,
|
||||||
|
/// Cap on new downloads per scan for this feed. None follows `[general]`.
|
||||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
pub max_new_per_check: Option<usize>,
|
pub max_new_per_check: Option<usize>,
|
||||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
@@ -86,10 +149,12 @@ impl Default for General {
|
|||||||
Self {
|
Self {
|
||||||
download_dir: home().join("Podcasts"),
|
download_dir: home().join("Podcasts"),
|
||||||
socket: default_socket(),
|
socket: default_socket(),
|
||||||
interval_mins: 60,
|
schedule: "every 60m".into(),
|
||||||
organize: Organize::Feed,
|
organize: Organize::Feed,
|
||||||
max_total_gb: 0.0,
|
max_total_gb: 0.0,
|
||||||
max_age_days: 0,
|
max_age_days: 0,
|
||||||
|
max_new_per_check: 3,
|
||||||
|
media_types: vec!["audio".into(), "video".into()],
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -106,6 +171,49 @@ impl Default for Torrent {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl General {
|
||||||
|
/// Minutes between checks, or an hour when `schedule` is empty or unreadable. A malformed
|
||||||
|
/// value warns rather than stopping the daemon.
|
||||||
|
pub fn interval(&self) -> u64 {
|
||||||
|
if let Some(n) = parse_interval(&self.schedule) {
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
if !self.schedule.trim().is_empty() {
|
||||||
|
tracing::warn!(schedule = %self.schedule, "unrecognised schedule; using the default");
|
||||||
|
}
|
||||||
|
60
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parses a check interval into minutes.
|
||||||
|
///
|
||||||
|
/// Accepts "every 30m", "30m", "4h", "1d", "2w", "every 4 hours", or a bare number of
|
||||||
|
/// minutes.
|
||||||
|
/// Returns None for anything it cannot read, or for zero.
|
||||||
|
pub fn parse_interval(s: &str) -> Option<u64> {
|
||||||
|
let s = s.trim().to_lowercase();
|
||||||
|
let s = s.strip_prefix("every").unwrap_or(&s).trim();
|
||||||
|
if s.is_empty() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
|
||||||
|
let digits: String = s.chars().take_while(|c| c.is_ascii_digit()).collect();
|
||||||
|
if digits.is_empty() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let n: u64 = digits.parse().ok()?;
|
||||||
|
let unit = s[digits.len()..].trim();
|
||||||
|
|
||||||
|
let mins = match unit {
|
||||||
|
"" | "m" | "min" | "mins" | "minute" | "minutes" => n,
|
||||||
|
"h" | "hr" | "hrs" | "hour" | "hours" => n.checked_mul(60)?,
|
||||||
|
"d" | "day" | "days" => n.checked_mul(1440)?,
|
||||||
|
"w" | "week" | "weeks" => n.checked_mul(10080)?,
|
||||||
|
_ => return None,
|
||||||
|
};
|
||||||
|
(mins > 0).then_some(mins)
|
||||||
|
}
|
||||||
|
|
||||||
impl Torrent {
|
impl Torrent {
|
||||||
/// Inclusive listen port range. Falls back to the BitTorrent default on garbage input.
|
/// Inclusive listen port range. Falls back to the BitTorrent default on garbage input.
|
||||||
pub fn ports(&self) -> (u16, u16) {
|
pub fn ports(&self) -> (u16, u16) {
|
||||||
@@ -166,9 +274,7 @@ pub fn config_path() -> PathBuf {
|
|||||||
if let Ok(p) = std::env::var("IPX_CONFIG") {
|
if let Ok(p) = std::env::var("IPX_CONFIG") {
|
||||||
return PathBuf::from(p);
|
return PathBuf::from(p);
|
||||||
}
|
}
|
||||||
dirs::config_dir()
|
xdg("XDG_CONFIG_HOME", ".config").join("ipx/config.toml")
|
||||||
.unwrap_or_else(|| home().join(".config"))
|
|
||||||
.join("ipx/config.toml")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`.
|
/// `$IPX_DATA_DIR`, else `$XDG_DATA_HOME/ipx`.
|
||||||
@@ -176,9 +282,7 @@ pub fn data_dir() -> PathBuf {
|
|||||||
if let Ok(p) = std::env::var("IPX_DATA_DIR") {
|
if let Ok(p) = std::env::var("IPX_DATA_DIR") {
|
||||||
return PathBuf::from(p);
|
return PathBuf::from(p);
|
||||||
}
|
}
|
||||||
dirs::data_dir()
|
xdg("XDG_DATA_HOME", ".local/share").join("ipx")
|
||||||
.unwrap_or_else(|| home().join(".local/share"))
|
|
||||||
.join("ipx")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fn default_socket() -> PathBuf {
|
fn default_socket() -> PathBuf {
|
||||||
@@ -188,6 +292,25 @@ fn default_socket() -> PathBuf {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Whether an enclosure's type is one we want.
|
||||||
|
///
|
||||||
|
/// An unknown type is allowed: the real type is only known after downloading, and
|
||||||
|
/// refusing everything untyped would drop feeds that simply omit the attribute.
|
||||||
|
pub fn wanted_media(mime: Option<&str>, wanted: &[String]) -> bool {
|
||||||
|
if wanted.is_empty() {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
let Some(mime) = mime.map(str::trim).filter(|m| !m.is_empty()) else {
|
||||||
|
return true;
|
||||||
|
};
|
||||||
|
let top = mime.split('/').next().unwrap_or(mime).to_ascii_lowercase();
|
||||||
|
// A .torrent is a container for media, not media itself; judge it once unpacked.
|
||||||
|
if mime.to_ascii_lowercase().contains("torrent") {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
wanted.iter().any(|w| w.trim().eq_ignore_ascii_case(&top) || w.trim().eq_ignore_ascii_case(mime))
|
||||||
|
}
|
||||||
|
|
||||||
/// Feed ids are the TOML table key, so they must be readable and punctuation-free.
|
/// Feed ids are the TOML table key, so they must be readable and punctuation-free.
|
||||||
pub fn slug(text: &str) -> String {
|
pub fn slug(text: &str) -> String {
|
||||||
let mut out = String::new();
|
let mut out = String::new();
|
||||||
@@ -215,8 +338,16 @@ pub fn unique_slug(text: &str, taken: &BTreeMap<String, Feed>) -> String {
|
|||||||
(2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap()
|
(2..).map(|n| format!("{base}-{n}")).find(|s| !taken.contains_key(s)).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// `$var`, or `~/fallback` when it is unset or empty, as the XDG base directory spec says.
|
||||||
|
fn xdg(var: &str, fallback: &str) -> PathBuf {
|
||||||
|
std::env::var_os(var)
|
||||||
|
.filter(|v| !v.is_empty())
|
||||||
|
.map(PathBuf::from)
|
||||||
|
.unwrap_or_else(|| home().join(fallback))
|
||||||
|
}
|
||||||
|
|
||||||
fn home() -> PathBuf {
|
fn home() -> PathBuf {
|
||||||
dirs::home_dir().unwrap_or_else(|| PathBuf::from("."))
|
std::env::var_os("HOME").map(PathBuf::from).unwrap_or_else(|| PathBuf::from("."))
|
||||||
}
|
}
|
||||||
|
|
||||||
fn expand_tilde(p: &Path) -> PathBuf {
|
fn expand_tilde(p: &Path) -> PathBuf {
|
||||||
@@ -236,6 +367,8 @@ mod tests {
|
|||||||
r#"
|
r#"
|
||||||
[general]
|
[general]
|
||||||
download_dir = "/tmp/pods"
|
download_dir = "/tmp/pods"
|
||||||
|
# A key older versions read. An old config that still has it has to load.
|
||||||
|
interval_mins = 45
|
||||||
|
|
||||||
[feeds.example]
|
[feeds.example]
|
||||||
url = "https://example.com/feed.xml"
|
url = "https://example.com/feed.xml"
|
||||||
@@ -245,7 +378,7 @@ mod tests {
|
|||||||
.unwrap();
|
.unwrap();
|
||||||
|
|
||||||
assert_eq!(cfg.general.download_dir, PathBuf::from("/tmp/pods"));
|
assert_eq!(cfg.general.download_dir, PathBuf::from("/tmp/pods"));
|
||||||
assert_eq!(cfg.general.interval_mins, 60);
|
assert_eq!(cfg.general.interval(), 60);
|
||||||
assert_eq!(cfg.general.organize, Organize::Feed);
|
assert_eq!(cfg.general.organize, Organize::Feed);
|
||||||
assert!(cfg.torrent.enabled);
|
assert!(cfg.torrent.enabled);
|
||||||
|
|
||||||
@@ -256,6 +389,36 @@ mod tests {
|
|||||||
assert_eq!(feed.keywords, vec!["deep dive"]);
|
assert_eq!(feed.keywords, vec!["deep dive"]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn intervals_parse_from_the_forms_people_actually_type() {
|
||||||
|
for (input, want) in [
|
||||||
|
("every 30m", 30), ("30m", 30), ("30", 30), ("every 30 minutes", 30),
|
||||||
|
("every 4h", 240), ("4h", 240), ("4 hours", 240), ("EVERY 4H", 240),
|
||||||
|
("1d", 1440), ("every 2 days", 2880), (" every 90m ", 90),
|
||||||
|
("1w", 10080), ("every 2 weeks", 20160), ("2 w", 20160),
|
||||||
|
] {
|
||||||
|
assert_eq!(parse_interval(input), Some(want), "{input:?}");
|
||||||
|
}
|
||||||
|
for bad in ["", " ", "every", "soon", "-5m", "0", "0h", "every 0 minutes", "5 fortnights"] {
|
||||||
|
assert_eq!(parse_interval(bad), None, "{bad:?} should not parse");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn interval_falls_back_to_an_hour() {
|
||||||
|
let mut g = General::default();
|
||||||
|
assert_eq!(g.interval(), 60, "the default schedule");
|
||||||
|
|
||||||
|
g.schedule = "every 15m".into();
|
||||||
|
assert_eq!(g.interval(), 15);
|
||||||
|
|
||||||
|
// Empty or garbage must not stop the daemon.
|
||||||
|
g.schedule = String::new();
|
||||||
|
assert_eq!(g.interval(), 60);
|
||||||
|
g.schedule = "whenever".into();
|
||||||
|
assert_eq!(g.interval(), 60);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn port_range_falls_back_when_malformed() {
|
fn port_range_falls_back_when_malformed() {
|
||||||
let mut t = Torrent::default();
|
let mut t = Torrent::default();
|
||||||
@@ -268,6 +431,26 @@ mod tests {
|
|||||||
assert_eq!(t.ports(), (6881, 6889), "reversed range is not a range");
|
assert_eq!(t.ports(), (6881, 6889), "reversed range is not a range");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn media_types_keep_article_artwork_out() {
|
||||||
|
let want = vec!["audio".to_string(), "video".to_string()];
|
||||||
|
assert!(wanted_media(Some("audio/mpeg"), &want));
|
||||||
|
assert!(wanted_media(Some("audio/mp4"), &want));
|
||||||
|
assert!(wanted_media(Some("video/quicktime"), &want));
|
||||||
|
assert!(!wanted_media(Some("image/jpeg"), &want), "a blog header image is not an episode");
|
||||||
|
assert!(!wanted_media(Some("text/html"), &want));
|
||||||
|
|
||||||
|
// A torrent is a container; what is inside is judged after unpacking.
|
||||||
|
assert!(wanted_media(Some("application/x-bittorrent"), &want));
|
||||||
|
// Unknown type: only discoverable by downloading, so do not refuse it outright.
|
||||||
|
assert!(wanted_media(None, &want));
|
||||||
|
assert!(wanted_media(Some(""), &want));
|
||||||
|
// An empty list means take anything, which is how it behaved before.
|
||||||
|
assert!(wanted_media(Some("image/jpeg"), &[]));
|
||||||
|
// A full type can be named exactly.
|
||||||
|
assert!(wanted_media(Some("image/jpeg"), &["image/jpeg".to_string()]));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn slugs_are_readable_and_unique() {
|
fn slugs_are_readable_and_unique() {
|
||||||
assert_eq!(slug("Accidental Tech Podcast"), "accidental-tech-podcast");
|
assert_eq!(slug("Accidental Tech Podcast"), "accidental-tech-podcast");
|
||||||
@@ -279,7 +462,7 @@ mod tests {
|
|||||||
|
|
||||||
let mut taken = BTreeMap::new();
|
let mut taken = BTreeMap::new();
|
||||||
taken.insert("the-daily".to_string(), Feed {
|
taken.insert("the-daily".to_string(), Feed {
|
||||||
url: "u".into(), folder: None, keywords: vec![], allow_explicit: false,
|
url: "u".into(), folder: None, group: None, media_types: None, schedule: None, keywords: vec![], allow_explicit: false,
|
||||||
auto_download: true, max_new_per_check: None, username: None,
|
auto_download: true, max_new_per_check: None, username: None,
|
||||||
password: None, password_env: None,
|
password: None, password_env: None,
|
||||||
});
|
});
|
||||||
@@ -291,6 +474,9 @@ mod tests {
|
|||||||
let mut f = Feed {
|
let mut f = Feed {
|
||||||
url: "https://x/y".into(),
|
url: "https://x/y".into(),
|
||||||
folder: None,
|
folder: None,
|
||||||
|
group: None,
|
||||||
|
media_types: None,
|
||||||
|
schedule: None,
|
||||||
keywords: vec![],
|
keywords: vec![],
|
||||||
allow_explicit: false,
|
allow_explicit: false,
|
||||||
auto_download: true,
|
auto_download: true,
|
||||||
|
|||||||
105
src/download.rs
105
src/download.rs
@@ -7,19 +7,56 @@ use tokio::io::AsyncWriteExt;
|
|||||||
|
|
||||||
use crate::config::{Config, Feed as FeedCfg, Organize};
|
use crate::config::{Config, Feed as FeedCfg, Organize};
|
||||||
|
|
||||||
/// Characters the original's stringCleaning() stripped, plus the control range and the
|
/// Forbidden characters that were separating words: they become "-" so the words stay
|
||||||
/// trailing dots/spaces it left in. A real length cap is new -- the Python had none.
|
/// apart. The original's stringCleaning() deleted them, turning "Show | Series" into
|
||||||
const FORBIDDEN: &[char] = &['/', '\\', '?', '*', ':', '<', '>', '|', '"', '\''];
|
/// "Show Series".
|
||||||
|
const SEPARATORS: &[char] = &['/', '\\', '|', ':'];
|
||||||
|
/// Forbidden characters that were never separators: they just go.
|
||||||
|
const STRIPPED: &[char] = &['?', '*', '<', '>', '"', '\''];
|
||||||
|
/// A real length cap is new -- the Python had none.
|
||||||
const MAX_NAME_BYTES: usize = 255;
|
const MAX_NAME_BYTES: usize = 255;
|
||||||
|
|
||||||
/// Keeps UTF-8: the original transliterated to ASCII via latin1_to_ascii because 2004
|
/// Keeps UTF-8: the original transliterated to ASCII via latin1_to_ascii because 2004
|
||||||
/// filesystems demanded it. Ours do not.
|
/// filesystems demanded it. Ours do not.
|
||||||
pub fn sanitize(name: &str) -> String {
|
pub fn sanitize(name: &str) -> String {
|
||||||
let mut out: String = name
|
let mapped: String = name
|
||||||
.chars()
|
.chars()
|
||||||
.filter(|c| !c.is_control() && !FORBIDDEN.contains(c))
|
.map(|c| if c.is_control() { ' ' } else { c })
|
||||||
|
.filter(|c| !STRIPPED.contains(c))
|
||||||
|
.map(|c| if SEPARATORS.contains(&c) { '-' } else { c })
|
||||||
.collect();
|
.collect();
|
||||||
out = out.trim().trim_matches('.').trim().to_owned();
|
|
||||||
|
// Collapse each run of dashes and spaces into one thing. A run containing a dash
|
||||||
|
// becomes " - " when it also had whitespace ("Show | Series" -> "Show - Series",
|
||||||
|
// "Ep 12: One" -> "Ep 12 - One") and a bare "-" when it did not ("AC/DC" -> "AC-DC").
|
||||||
|
// A run of plain whitespace collapses to a single space.
|
||||||
|
let mut out = String::with_capacity(mapped.len());
|
||||||
|
let mut chars = mapped.chars().peekable();
|
||||||
|
while let Some(c) = chars.next() {
|
||||||
|
if !(c == '-' || c.is_whitespace()) {
|
||||||
|
out.push(c);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let mut has_dash = c == '-';
|
||||||
|
let mut has_space = c.is_whitespace();
|
||||||
|
while let Some(&next) = chars.peek() {
|
||||||
|
if next == '-' {
|
||||||
|
has_dash = true;
|
||||||
|
} else if next.is_whitespace() {
|
||||||
|
has_space = true;
|
||||||
|
} else {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
chars.next();
|
||||||
|
}
|
||||||
|
match (has_dash, has_space) {
|
||||||
|
(true, true) => out.push_str(" - "),
|
||||||
|
(true, false) => out.push('-'),
|
||||||
|
_ => out.push(' '),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Leading/trailing separators and dots are noise, and a leading "-" trips up CLI tools.
|
||||||
|
out = out.trim().trim_matches(|c| c == '.' || c == '-').trim().to_owned();
|
||||||
|
|
||||||
if out.len() > MAX_NAME_BYTES {
|
if out.len() > MAX_NAME_BYTES {
|
||||||
// Truncate on a char boundary, keeping the extension if there is a plausible one.
|
// Truncate on a char boundary, keeping the extension if there is a plausible one.
|
||||||
@@ -186,7 +223,7 @@ enum Sniffed {
|
|||||||
/// 2008 and so always answered 'data'.
|
/// 2008 and so always answered 'data'.
|
||||||
async fn sniff(path: &Path) -> Result<Sniffed> {
|
async fn sniff(path: &Path) -> Result<Sniffed> {
|
||||||
let head = read_head(path, 512).await?;
|
let head = read_head(path, 512).await?;
|
||||||
if infer::is(&head, "torrent") || head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
|
if head.starts_with(b"d8:announce") || head.starts_with(b"d7:") {
|
||||||
return Ok(Sniffed::Torrent);
|
return Ok(Sniffed::Torrent);
|
||||||
}
|
}
|
||||||
let text = String::from_utf8_lossy(&head);
|
let text = String::from_utf8_lossy(&head);
|
||||||
@@ -231,17 +268,27 @@ fn unique_path(dir: &Path, name: &str) -> PathBuf {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Download folder for a feed: per-feed name, or per-day when organize = "date".
|
/// Download folder for a feed: per-feed name, or per-day when organize = "date".
|
||||||
|
///
|
||||||
|
/// A folder may name more than one level ("Subscriptions/Some Show") -- feeds from a
|
||||||
|
/// subscribed OPML nest under it -- so each segment is sanitized separately rather than
|
||||||
|
/// letting the sanitizer eat the separator.
|
||||||
pub fn folder_for(cfg: &Config, id: &str, feed_cfg: &FeedCfg, title: Option<&str>) -> String {
|
pub fn folder_for(cfg: &Config, id: &str, feed_cfg: &FeedCfg, title: Option<&str>) -> String {
|
||||||
match cfg.general.organize {
|
match cfg.general.organize {
|
||||||
Organize::Date => chrono::Local::now().format("%m-%d-%Y").to_string(),
|
Organize::Date => chrono::Local::now().format("%m-%d-%Y").to_string(),
|
||||||
Organize::Feed => sanitize(
|
Organize::Feed => {
|
||||||
feed_cfg
|
let raw = feed_cfg
|
||||||
.folder
|
.folder
|
||||||
.as_deref()
|
.as_deref()
|
||||||
.or(title)
|
.or(title)
|
||||||
.filter(|s| !s.trim().is_empty())
|
.filter(|s| !s.trim().is_empty())
|
||||||
.unwrap_or(id),
|
.unwrap_or(id);
|
||||||
),
|
raw.split('/')
|
||||||
|
.map(str::trim)
|
||||||
|
.filter(|seg| !seg.is_empty() && *seg != "." && *seg != "..")
|
||||||
|
.map(sanitize)
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join("/")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -264,12 +311,26 @@ mod tests {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn sanitize_strips_path_and_control_characters() {
|
fn sanitize_strips_path_and_control_characters() {
|
||||||
assert_eq!(sanitize("../../etc/passwd"), "etcpasswd");
|
assert_eq!(sanitize("../../etc/passwd"), "etc-passwd");
|
||||||
assert_eq!(sanitize("Ep 12: The \"Best\" One?"), "Ep 12 The Best One");
|
assert_eq!(sanitize("Ep 12: The \"Best\" One?"), "Ep 12 - The Best One");
|
||||||
assert_eq!(sanitize("bad\u{0}name\u{7}.mp3"), "badname.mp3");
|
assert_eq!(sanitize("bad\u{0}name\u{7}.mp3"), "bad name .mp3");
|
||||||
assert_eq!(sanitize(" spaced.mp3 "), "spaced.mp3");
|
assert_eq!(sanitize(" spaced.mp3 "), "spaced.mp3");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn sanitize_turns_separators_into_dashes() {
|
||||||
|
// A real Patreon feed title; the pipes are forbidden characters.
|
||||||
|
assert_eq!(
|
||||||
|
sanitize("Get in the Trunk | Anthology Series | Delta Green"),
|
||||||
|
"Get in the Trunk - Anthology Series - Delta Green"
|
||||||
|
);
|
||||||
|
assert_eq!(sanitize("Ep 12: The One"), "Ep 12 - The One");
|
||||||
|
assert_eq!(sanitize("a b"), "a b", "plain whitespace stays whitespace");
|
||||||
|
assert_eq!(sanitize("AC/DC"), "AC-DC", "no spaces around it, so no spaces added");
|
||||||
|
assert_eq!(sanitize("well-known.mp3"), "well-known.mp3", "existing dashes survive");
|
||||||
|
assert_eq!(sanitize("Show -- Thing"), "Show - Thing");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn sanitize_never_yields_an_empty_or_dot_name() {
|
fn sanitize_never_yields_an_empty_or_dot_name() {
|
||||||
assert_eq!(sanitize(""), "download");
|
assert_eq!(sanitize(""), "download");
|
||||||
@@ -319,6 +380,22 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_folder_can_nest_without_the_sanitizer_eating_the_separator() {
|
||||||
|
let mut cfg = Config::default();
|
||||||
|
cfg.general.download_dir = "/tmp".into();
|
||||||
|
let mut f = crate::config::Feed {
|
||||||
|
url: "u".into(), folder: Some("Subscriptions/Some | Show".into()), group: None, media_types: None,
|
||||||
|
schedule: None, keywords: vec![], allow_explicit: false, auto_download: true,
|
||||||
|
max_new_per_check: None, username: None, password: None, password_env: None,
|
||||||
|
};
|
||||||
|
assert_eq!(folder_for(&cfg, "id", &f, None), "Subscriptions/Some - Show");
|
||||||
|
|
||||||
|
// A traversal in a folder name must not climb out of the download directory.
|
||||||
|
f.folder = Some("../../etc/Show".into());
|
||||||
|
assert_eq!(folder_for(&cfg, "id", &f, None), "etc/Show");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn keyword_matching_is_or_across_keywords_and_and_within_one() {
|
fn keyword_matching_is_or_across_keywords_and_and_within_one() {
|
||||||
let kws = vec!["deep dive".to_string(), "interview".to_string()];
|
let kws = vec!["deep dive".to_string(), "interview".to_string()];
|
||||||
|
|||||||
459
src/feed.rs
459
src/feed.rs
@@ -10,6 +10,7 @@ use crate::config::Feed as FeedCfg;
|
|||||||
pub struct ParsedFeed {
|
pub struct ParsedFeed {
|
||||||
pub title: Option<String>,
|
pub title: Option<String>,
|
||||||
pub ttl_mins: Option<u64>,
|
pub ttl_mins: Option<u64>,
|
||||||
|
pub image: Option<String>,
|
||||||
pub entries: Vec<Entry>,
|
pub entries: Vec<Entry>,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -22,10 +23,16 @@ pub struct Entry {
|
|||||||
pub description: Option<String>,
|
pub description: Option<String>,
|
||||||
pub categories: Vec<String>,
|
pub categories: Vec<String>,
|
||||||
pub explicit: bool,
|
pub explicit: bool,
|
||||||
|
/// Episode artwork; falls back to the feed's in the UI.
|
||||||
|
pub image: Option<String>,
|
||||||
|
/// Seconds.
|
||||||
|
pub duration: Option<i64>,
|
||||||
|
pub episode: Option<i64>,
|
||||||
|
pub season: Option<i64>,
|
||||||
pub enclosures: Vec<Enclosure>,
|
pub enclosures: Vec<Enclosure>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Default, PartialEq)]
|
#[derive(Debug, Default, Clone, PartialEq)]
|
||||||
pub struct Enclosure {
|
pub struct Enclosure {
|
||||||
pub url: String,
|
pub url: String,
|
||||||
pub mime: Option<String>,
|
pub mime: Option<String>,
|
||||||
@@ -80,10 +87,137 @@ pub async fn fetch(
|
|||||||
Ok(Fetched::Body { bytes, etag, last_modified })
|
Ok(Fetched::Body { bytes, etag, last_modified })
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// True when a body is an OPML document rather than a feed.
|
||||||
|
///
|
||||||
|
/// The original matched on the URL ending in ".opml" (iPXClass.py:34), which misses an
|
||||||
|
/// OPML served from a URL without that extension. Sniffing the body catches both.
|
||||||
|
pub fn is_opml(bytes: &[u8]) -> bool {
|
||||||
|
let head = &bytes[..bytes.len().min(1024)];
|
||||||
|
let text = String::from_utf8_lossy(head).to_lowercase();
|
||||||
|
text.contains("<opml")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The feeds listed in an OPML document, as (title, xml_url), walking nested folders.
|
||||||
|
pub fn parse_opml(bytes: &[u8]) -> Result<Vec<(String, String)>> {
|
||||||
|
let text = String::from_utf8_lossy(bytes);
|
||||||
|
let doc = opml::OPML::from_str(&text)
|
||||||
|
.map_err(|e| anyhow!("that does not parse as OPML: {e}"))?;
|
||||||
|
let mut out = vec![];
|
||||||
|
crate::collect_outlines(&doc.body.outlines, &mut out);
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The <head><title> of an OPML document.
|
||||||
|
pub fn opml_title(bytes: &[u8]) -> Option<String> {
|
||||||
|
let text = String::from_utf8_lossy(bytes);
|
||||||
|
let doc = opml::OPML::from_str(&text).ok()?;
|
||||||
|
doc.head
|
||||||
|
.and_then(|h| h.title)
|
||||||
|
.map(|t| t.trim().to_owned())
|
||||||
|
.filter(|t| !t.is_empty())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The token and show of a Patreon feed link, or None for any other URL.
|
||||||
|
///
|
||||||
|
/// Patreon gives each patron one token per creator. With no show it stands for the creator,
|
||||||
|
/// whose feed carries every show at once.
|
||||||
|
fn patreon_parts(url: &str) -> Option<(String, Option<String>)> {
|
||||||
|
let u = url::Url::parse(url).ok()?;
|
||||||
|
if !matches!(u.host_str()?, "patreon.com" | "www.patreon.com") || !u.path().starts_with("/rss") {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let param = |name: &str| u.query_pairs().find(|(k, _)| k == name).map(|(_, v)| v.into_owned());
|
||||||
|
Some((param("auth")?, param("show")))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A Patreon link naming a creator but no show.
|
||||||
|
pub fn is_patreon_creator(url: &str) -> bool {
|
||||||
|
matches!(patreon_parts(url), Some((_, None)))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What was typed into Add feed, as a URL. A bare Patreon token is taken as its creator's
|
||||||
|
/// feed, since the token alone says whose it is.
|
||||||
|
pub fn expand_input(input: &str) -> String {
|
||||||
|
let s = input.trim();
|
||||||
|
let token = s.len() >= 20 && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_');
|
||||||
|
if token { format!("https://www.patreon.com/rss?auth={s}") } else { s.to_owned() }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether two URLs are the same feed. One Patreon show has several spellings -- by the
|
||||||
|
/// creator's name, by number, or with no creator at all -- and the token and show are what
|
||||||
|
/// identify it.
|
||||||
|
pub fn same_feed(a: &str, b: &str) -> bool {
|
||||||
|
a == b || patreon_parts(a).is_some_and(|p| Some(p) == patreon_parts(b))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A Patreon creator's name and shows, each show as (title, feed URL).
|
||||||
|
///
|
||||||
|
/// ponytail: Patreon's own web API, undocumented, asked without signing in. If it changes,
|
||||||
|
/// finding shows stops and the show feeds already found keep working. The documented API
|
||||||
|
/// needs an OAuth client per install and does not list shows.
|
||||||
|
pub async fn patreon_shows(
|
||||||
|
client: &reqwest::Client,
|
||||||
|
url: &str,
|
||||||
|
) -> Result<(Option<String>, Vec<(String, String)>)> {
|
||||||
|
// The creator feed names its campaign by number in its self link, a few hundred bytes in.
|
||||||
|
// The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
|
||||||
|
let mut resp = client.get(url).send().await.context("connecting")?;
|
||||||
|
if !resp.status().is_success() {
|
||||||
|
return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
|
||||||
|
}
|
||||||
|
let mut head = Vec::new();
|
||||||
|
while patreon_campaign(&head).is_none() && head.len() < 64 * 1024 {
|
||||||
|
let Some(chunk) = resp.chunk().await.context("reading the feed")? else { break };
|
||||||
|
head.extend_from_slice(&chunk);
|
||||||
|
}
|
||||||
|
let campaign = patreon_campaign(&head)
|
||||||
|
.ok_or_else(|| anyhow!("the Patreon feed does not say whose it is"))?;
|
||||||
|
|
||||||
|
let api = format!(
|
||||||
|
"https://www.patreon.com/api/campaigns/{campaign}\
|
||||||
|
?include=shows&fields%5Bcampaign%5D=name&fields%5Bcollection%5D=title"
|
||||||
|
);
|
||||||
|
let resp = client.get(api).send().await.context("asking Patreon for the shows")?;
|
||||||
|
if !resp.status().is_success() {
|
||||||
|
return Err(anyhow!("Patreon would not list the shows: HTTP {}", resp.status()));
|
||||||
|
}
|
||||||
|
let (name, shows) = parse_patreon_shows(&resp.bytes().await.context("reading the shows")?)?;
|
||||||
|
Ok((name, shows.into_iter().map(|(id, title)| (title, format!("{url}&show={id}"))).collect()))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The campaign number in the start of a Patreon feed.
|
||||||
|
fn patreon_campaign(head: &[u8]) -> Option<String> {
|
||||||
|
let text = String::from_utf8_lossy(head);
|
||||||
|
text.match_indices("patreon.com/rss/").find_map(|(i, m)| {
|
||||||
|
let id: String = text[i + m.len()..].chars().take_while(char::is_ascii_digit).collect();
|
||||||
|
(!id.is_empty()).then_some(id)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A campaign's name and its shows as (id, title), from Patreon's JSON:API answer.
|
||||||
|
fn parse_patreon_shows(json: &[u8]) -> Result<(Option<String>, Vec<(String, String)>)> {
|
||||||
|
let v: serde_json::Value = serde_json::from_slice(json).context("Patreon's answer is not JSON")?;
|
||||||
|
// Missing is not the same as none. Read as no shows, the creator feed would be scanned as
|
||||||
|
// a plain feed, claim every show's files, and leave the shows empty once the list returned.
|
||||||
|
let ids = v["data"]["relationships"]["shows"]["data"]
|
||||||
|
.as_array()
|
||||||
|
.ok_or_else(|| anyhow!("Patreon's answer does not list the shows"))?;
|
||||||
|
let title = |id: &str| -> Option<String> {
|
||||||
|
let show = v["included"].as_array()?.iter().find(|x| x["type"] == "collection" && x["id"] == id)?;
|
||||||
|
show["attributes"]["title"].as_str().map(|t| t.trim().to_owned())
|
||||||
|
};
|
||||||
|
let shows = ids
|
||||||
|
.iter()
|
||||||
|
.filter_map(|s| s["id"].as_str())
|
||||||
|
.map(|id| (id.to_owned(), title(id).unwrap_or_else(|| format!("Show {id}"))))
|
||||||
|
.collect();
|
||||||
|
Ok((v["data"]["attributes"]["name"].as_str().map(str::to_owned), shows))
|
||||||
|
}
|
||||||
|
|
||||||
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
|
/// RSS first, then Atom -- the same split the original made on `parsedFeed.version`.
|
||||||
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
|
pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
|
||||||
match rss::Channel::read_from(bytes) {
|
match rss::Channel::read_from(bytes) {
|
||||||
Ok(ch) => Ok(from_rss(ch)),
|
Ok(ch) => Ok(from_rss(ch, bytes)),
|
||||||
Err(rss_err) => match atom_syndication::Feed::read_from(bytes) {
|
Err(rss_err) => match atom_syndication::Feed::read_from(bytes) {
|
||||||
Ok(feed) => Ok(from_atom(feed)),
|
Ok(feed) => Ok(from_atom(feed)),
|
||||||
Err(atom_err) => Err(anyhow!("not RSS ({rss_err}) and not Atom ({atom_err})")),
|
Err(atom_err) => Err(anyhow!("not RSS ({rss_err}) and not Atom ({atom_err})")),
|
||||||
@@ -91,7 +225,78 @@ pub fn parse(bytes: &[u8]) -> Result<ParsedFeed> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn from_rss(ch: rss::Channel) -> ParsedFeed {
|
/// Every `<enclosure>` of every `<item>`, in document order.
|
||||||
|
///
|
||||||
|
/// The `rss` crate models an item as having at most one enclosure -- which is what RSS 2.0
|
||||||
|
/// says -- and when a feed carries several it keeps only the *last*, silently losing the
|
||||||
|
/// rest. Feeds do ship several, so read them from the XML directly.
|
||||||
|
fn enclosures_by_item(bytes: &[u8]) -> Vec<Vec<Enclosure>> {
|
||||||
|
use quick_xml::events::Event;
|
||||||
|
|
||||||
|
let mut reader = quick_xml::Reader::from_reader(bytes);
|
||||||
|
reader.config_mut().trim_text(true);
|
||||||
|
let mut buf = Vec::new();
|
||||||
|
let mut out: Vec<Vec<Enclosure>> = Vec::new();
|
||||||
|
let mut current: Option<Vec<Enclosure>> = None;
|
||||||
|
|
||||||
|
let read_enclosure = |e: &quick_xml::events::BytesStart| -> Option<Enclosure> {
|
||||||
|
let (mut url, mut mime, mut length) = (String::new(), None, None);
|
||||||
|
for attr in e.attributes().flatten() {
|
||||||
|
// Values arrive escaped: a feed URL's "&" is "&" in the document.
|
||||||
|
let val = quick_xml::escape::unescape(&attr.value)
|
||||||
|
.map(|v| v.trim().to_string())
|
||||||
|
.unwrap_or_default();
|
||||||
|
match attr.key.local_name().as_ref() {
|
||||||
|
"url" => url = val,
|
||||||
|
"type" => mime = Some(val).filter(|v| !v.is_empty()),
|
||||||
|
"length" => length = val.parse().ok(),
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
(!url.is_empty()).then_some(Enclosure { url, mime, length })
|
||||||
|
};
|
||||||
|
|
||||||
|
loop {
|
||||||
|
match reader.read_event_into(&mut buf) {
|
||||||
|
Ok(Event::Start(e)) => match e.name().local_name().as_ref() {
|
||||||
|
"item" => current = Some(Vec::new()),
|
||||||
|
"enclosure" => {
|
||||||
|
if let (Some(list), Some(enc)) = (current.as_mut(), read_enclosure(&e)) {
|
||||||
|
list.push(enc);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
},
|
||||||
|
Ok(Event::Empty(e)) => match e.name().local_name().as_ref() {
|
||||||
|
// <item/> with no children still counts, so the indexes stay aligned.
|
||||||
|
"item" => out.push(Vec::new()),
|
||||||
|
"enclosure" => {
|
||||||
|
if let (Some(list), Some(enc)) = (current.as_mut(), read_enclosure(&e)) {
|
||||||
|
list.push(enc);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
},
|
||||||
|
Ok(Event::End(e)) => {
|
||||||
|
if e.name().local_name().as_ref() == "item"
|
||||||
|
&& let Some(list) = current.take()
|
||||||
|
{
|
||||||
|
out.push(list);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(Event::Eof) | Err(_) => break,
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
buf.clear();
|
||||||
|
}
|
||||||
|
if let Some(list) = current.take() {
|
||||||
|
out.push(list);
|
||||||
|
}
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
fn from_rss(ch: rss::Channel, bytes: &[u8]) -> ParsedFeed {
|
||||||
|
let per_item = enclosures_by_item(bytes);
|
||||||
let explicit = ch
|
let explicit = ch
|
||||||
.itunes_ext()
|
.itunes_ext()
|
||||||
.and_then(|it| it.explicit())
|
.and_then(|it| it.explicit())
|
||||||
@@ -100,9 +305,12 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
|
|||||||
let entries = ch
|
let entries = ch
|
||||||
.items()
|
.items()
|
||||||
.iter()
|
.iter()
|
||||||
.filter_map(|item| {
|
.enumerate()
|
||||||
let enclosures: Vec<Enclosure> = item
|
.filter_map(|(idx, item)| {
|
||||||
.enclosure()
|
// Straight from the XML, so an item with several keeps all of them. Falls
|
||||||
|
// back to the parsed one if the scan and the parser disagree on item count.
|
||||||
|
let enclosures: Vec<Enclosure> = per_item.get(idx).cloned().unwrap_or_else(|| {
|
||||||
|
item.enclosure()
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.map(|e| Enclosure {
|
.map(|e| Enclosure {
|
||||||
url: e.url().trim().to_owned(),
|
url: e.url().trim().to_owned(),
|
||||||
@@ -110,7 +318,8 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
|
|||||||
length: e.length().parse().ok(),
|
length: e.length().parse().ok(),
|
||||||
})
|
})
|
||||||
.filter(|e| !e.url.is_empty())
|
.filter(|e| !e.url.is_empty())
|
||||||
.collect();
|
.collect()
|
||||||
|
});
|
||||||
|
|
||||||
let guid = pick_guid(
|
let guid = pick_guid(
|
||||||
item.guid().map(|g| g.value()),
|
item.guid().map(|g| g.value()),
|
||||||
@@ -123,6 +332,7 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
|
|||||||
.itunes_ext()
|
.itunes_ext()
|
||||||
.and_then(|it| it.explicit())
|
.and_then(|it| it.explicit())
|
||||||
.is_some_and(is_yes);
|
.is_some_and(is_yes);
|
||||||
|
let it = item.itunes_ext();
|
||||||
|
|
||||||
Some(Entry {
|
Some(Entry {
|
||||||
guid,
|
guid,
|
||||||
@@ -138,6 +348,10 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
|
|||||||
.filter(|c| !c.is_empty() && !c.starts_with("http"))
|
.filter(|c| !c.is_empty() && !c.starts_with("http"))
|
||||||
.collect(),
|
.collect(),
|
||||||
explicit: explicit || entry_explicit,
|
explicit: explicit || entry_explicit,
|
||||||
|
image: item_image(item, &enclosures),
|
||||||
|
duration: it.and_then(|i| i.duration()).and_then(parse_duration),
|
||||||
|
episode: it.and_then(|i| i.episode()).and_then(|e| e.trim().parse().ok()),
|
||||||
|
season: it.and_then(|i| i.season()).and_then(|e| e.trim().parse().ok()),
|
||||||
enclosures,
|
enclosures,
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
@@ -146,6 +360,12 @@ fn from_rss(ch: rss::Channel) -> ParsedFeed {
|
|||||||
ParsedFeed {
|
ParsedFeed {
|
||||||
title: non_empty(Some(ch.title())),
|
title: non_empty(Some(ch.title())),
|
||||||
ttl_mins: ch.ttl().and_then(|t| t.trim().parse().ok()),
|
ttl_mins: ch.ttl().and_then(|t| t.trim().parse().ok()),
|
||||||
|
// itunes:image is the square artwork; <image><url> is the older, often smaller one.
|
||||||
|
image: ch
|
||||||
|
.itunes_ext()
|
||||||
|
.and_then(|i| i.image())
|
||||||
|
.map(str::to_owned)
|
||||||
|
.or_else(|| ch.image().map(|i| i.url().to_owned())),
|
||||||
entries,
|
entries,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -193,6 +413,10 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
|
|||||||
.map(str::to_owned),
|
.map(str::to_owned),
|
||||||
categories: e.categories().iter().map(|c| c.term().to_owned()).collect(),
|
categories: e.categories().iter().map(|c| c.term().to_owned()).collect(),
|
||||||
explicit: false,
|
explicit: false,
|
||||||
|
image: None,
|
||||||
|
duration: None,
|
||||||
|
episode: None,
|
||||||
|
season: None,
|
||||||
enclosures,
|
enclosures,
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
@@ -201,6 +425,7 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
|
|||||||
ParsedFeed {
|
ParsedFeed {
|
||||||
title: non_empty(Some(feed.title().as_str())),
|
title: non_empty(Some(feed.title().as_str())),
|
||||||
ttl_mins: None,
|
ttl_mins: None,
|
||||||
|
image: feed.logo().or_else(|| feed.icon()).map(str::to_owned),
|
||||||
entries,
|
entries,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -230,6 +455,55 @@ fn non_empty(s: Option<&str>) -> Option<String> {
|
|||||||
s.map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned)
|
s.map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The picture to show beside an item, in order of how deliberate it is:
|
||||||
|
/// `itunes:image`, then Media RSS `media:thumbnail`, then a `media:content` that is an
|
||||||
|
/// image, and finally an image enclosure -- which is how a blog's article picture arrives
|
||||||
|
/// (Substack puts it there), so those entries get artwork rather than a blank square.
|
||||||
|
fn item_image(item: &rss::Item, enclosures: &[Enclosure]) -> Option<String> {
|
||||||
|
if let Some(url) = item.itunes_ext().and_then(|i| i.image()) {
|
||||||
|
return non_empty(Some(url));
|
||||||
|
}
|
||||||
|
|
||||||
|
let media = item.extensions().get("media");
|
||||||
|
let attr = |name: &str, want_image: bool| -> Option<String> {
|
||||||
|
media?.get(name)?.iter().find_map(|e| {
|
||||||
|
if want_image {
|
||||||
|
// media:content carries anything; only take it when it says it is a picture.
|
||||||
|
let is_image = e.attrs.get("type").is_some_and(|t| t.starts_with("image/"))
|
||||||
|
|| e.attrs.get("medium").is_some_and(|m| m == "image");
|
||||||
|
if !is_image {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
non_empty(e.attrs.get("url").map(String::as_str))
|
||||||
|
})
|
||||||
|
};
|
||||||
|
attr("thumbnail", false)
|
||||||
|
.or_else(|| attr("content", true))
|
||||||
|
.or_else(|| {
|
||||||
|
enclosures
|
||||||
|
.iter()
|
||||||
|
.find(|e| e.mime.as_deref().is_some_and(|m| m.starts_with("image/")))
|
||||||
|
.map(|e| e.url.clone())
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// itunes:duration is either plain seconds ("5649") or a clock ("1:34:09", "23:45").
|
||||||
|
fn parse_duration(s: &str) -> Option<i64> {
|
||||||
|
let s = s.trim();
|
||||||
|
if s.is_empty() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
if !s.contains(':') {
|
||||||
|
return s.parse().ok().filter(|n| *n > 0);
|
||||||
|
}
|
||||||
|
let mut total: i64 = 0;
|
||||||
|
for part in s.split(':') {
|
||||||
|
total = total * 60 + part.trim().parse::<i64>().ok()?;
|
||||||
|
}
|
||||||
|
Some(total).filter(|n| *n > 0)
|
||||||
|
}
|
||||||
|
|
||||||
/// RSS pubDate is RFC 2822; some feeds ship RFC 3339 instead.
|
/// RSS pubDate is RFC 2822; some feeds ship RFC 3339 instead.
|
||||||
fn parse_date(s: &str) -> Option<i64> {
|
fn parse_date(s: &str) -> Option<i64> {
|
||||||
let s = s.trim();
|
let s = s.trim();
|
||||||
@@ -312,6 +586,176 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_rss_title_always_wins_and_episode_numbers_stay_metadata() {
|
||||||
|
// Some feeds set a different itunes:title. The displayed title is always the RSS
|
||||||
|
// <title>, verbatim -- separators and all -- and season/episode are stored
|
||||||
|
// alongside it rather than folded into it.
|
||||||
|
let xml = br#"<?xml version="1.0"?>
|
||||||
|
<rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
|
||||||
|
<channel><title>Show</title><link>https://x</link><description>d</description>
|
||||||
|
<item>
|
||||||
|
<title>Music from a Darkened Room | Session Zero</title>
|
||||||
|
<itunes:title>Session Zero</itunes:title>
|
||||||
|
<guid>sz</guid>
|
||||||
|
<itunes:season>8</itunes:season>
|
||||||
|
<itunes:duration>6720</itunes:duration>
|
||||||
|
<enclosure url="https://x/sz.mp3" length="1" type="audio/mpeg"/>
|
||||||
|
</item>
|
||||||
|
<item>
|
||||||
|
<title>Music from a Darkened Room Part 1 | Murphy's Drawer</title>
|
||||||
|
<guid>p1</guid>
|
||||||
|
<itunes:season>8</itunes:season><itunes:episode>1</itunes:episode>
|
||||||
|
<enclosure url="https://x/p1.mp3" length="1" type="audio/mpeg"/>
|
||||||
|
</item>
|
||||||
|
</channel></rss>"#;
|
||||||
|
let feed = parse(xml).unwrap();
|
||||||
|
|
||||||
|
let sz = &feed.entries[0];
|
||||||
|
assert_eq!(
|
||||||
|
sz.title.as_deref(),
|
||||||
|
Some("Music from a Darkened Room | Session Zero"),
|
||||||
|
"itunes:title must not override the RSS title"
|
||||||
|
);
|
||||||
|
assert_eq!(sz.season, Some(8));
|
||||||
|
assert_eq!(sz.episode, None, "a missing episode number stays missing");
|
||||||
|
assert_eq!(sz.duration, Some(6720));
|
||||||
|
|
||||||
|
let p1 = &feed.entries[1];
|
||||||
|
assert_eq!(p1.title.as_deref(), Some("Music from a Darkened Room Part 1 | Murphy's Drawer"));
|
||||||
|
assert_eq!((p1.season, p1.episode), (Some(8), Some(1)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn opml_is_recognised_and_its_feeds_listed() {
|
||||||
|
let xml = br#"<opml version="2.0"><head><title>My Subscriptions</title></head><body>
|
||||||
|
<outline text="Folder">
|
||||||
|
<outline type="rss" text="Alpha" xmlUrl="https://a.example/rss"/>
|
||||||
|
<outline type="rss" text="Beta" xmlUrl="https://b.example/rss"/>
|
||||||
|
</outline>
|
||||||
|
<outline text="Not a feed"/>
|
||||||
|
</body></opml>"#;
|
||||||
|
assert!(is_opml(xml));
|
||||||
|
assert_eq!(opml_title(xml).as_deref(), Some("My Subscriptions"));
|
||||||
|
|
||||||
|
let feeds = parse_opml(xml).unwrap();
|
||||||
|
assert_eq!(feeds.len(), 2, "nested folders are walked, non-feed outlines skipped");
|
||||||
|
assert_eq!(feeds[0], ("Alpha".into(), "https://a.example/rss".into()));
|
||||||
|
|
||||||
|
// A feed must never be mistaken for a subscription list.
|
||||||
|
assert!(!is_opml(include_bytes!("../tests/data/rss2.xml")));
|
||||||
|
assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_patreon_creator_is_a_list_of_its_shows() {
|
||||||
|
let tok = "AbCdEfGhIjKlMnOpQrStUvWxYz012_-9";
|
||||||
|
assert_eq!(expand_input(&format!(" {tok} ")), format!("https://www.patreon.com/rss?auth={tok}"));
|
||||||
|
assert_eq!(expand_input("https://example.com/rss"), "https://example.com/rss");
|
||||||
|
|
||||||
|
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss/glasscannon?auth={tok}")));
|
||||||
|
assert!(is_patreon_creator(&format!("https://www.patreon.com/rss?auth={tok}")));
|
||||||
|
assert!(!is_patreon_creator(&format!("https://www.patreon.com/rss/x?auth={tok}&show=1")), "one show is a feed");
|
||||||
|
assert!(!is_patreon_creator(&format!("https://example.com/rss?auth={tok}")));
|
||||||
|
|
||||||
|
// The show you already have by name is the one a bare token would add by number.
|
||||||
|
assert!(same_feed(
|
||||||
|
&format!("https://www.patreon.com/rss/glasscannon?auth={tok}&show=2073588"),
|
||||||
|
&format!("https://www.patreon.com/rss?auth={tok}&show=2073588"),
|
||||||
|
));
|
||||||
|
assert!(!same_feed(
|
||||||
|
&format!("https://www.patreon.com/rss?auth={tok}&show=1"),
|
||||||
|
&format!("https://www.patreon.com/rss?auth={tok}&show=2"),
|
||||||
|
));
|
||||||
|
|
||||||
|
// The self link carries the campaign by number, whichever spelling was asked for.
|
||||||
|
let head = br#"<rss><channel><link>https://www.patreon.com/glasscannon</link>
|
||||||
|
<atom:link href="https://www.patreon.com/rss/369921?auth=t" rel="self"/>"#;
|
||||||
|
assert_eq!(patreon_campaign(head).as_deref(), Some("369921"));
|
||||||
|
assert_eq!(patreon_campaign(b"<rss><channel><title>T"), None);
|
||||||
|
|
||||||
|
let json = br#"{"data":{"id":"369921","type":"campaign","attributes":{"name":"The Glass Cannon Network"},
|
||||||
|
"relationships":{"shows":{"data":[{"id":"2073588","type":"collection"},{"id":"2073636","type":"collection"}]}}},
|
||||||
|
"included":[{"id":"2073588","type":"collection","attributes":{"title":"Get in the Trunk "}},
|
||||||
|
{"id":"2073636","type":"collection","attributes":{"title":"Shadowdark"}}]}"#;
|
||||||
|
let (name, shows) = parse_patreon_shows(json).unwrap();
|
||||||
|
assert_eq!(name.as_deref(), Some("The Glass Cannon Network"));
|
||||||
|
assert_eq!(shows, [("2073588".into(), "Get in the Trunk".into()), ("2073636".into(), "Shadowdark".into())]);
|
||||||
|
|
||||||
|
// An answer that stops naming the shows is an error, never "this creator has none".
|
||||||
|
assert!(parse_patreon_shows(br#"{"data":{"attributes":{"name":"X"}}}"#).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_item_may_carry_several_enclosures() {
|
||||||
|
// The rss crate keeps only one per item -- the last -- so these come from the XML.
|
||||||
|
let xml = br#"<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>M</title><link>https://x</link><description>d</description>
|
||||||
|
<item><title>Two files</title><guid>m1</guid>
|
||||||
|
<enclosure url="https://x/a.mp3?v=1&t=2" length="111" type="audio/mpeg"/>
|
||||||
|
<enclosure url="https://x/b.mp4" length="222" type="video/mp4"/>
|
||||||
|
</item>
|
||||||
|
<item><title>One file</title><guid>m2</guid>
|
||||||
|
<enclosure url="https://x/c.mp3" length="333" type="audio/mpeg"/></item>
|
||||||
|
<item><title>None</title><guid>m3</guid></item>
|
||||||
|
</channel></rss>"#;
|
||||||
|
let f = parse(xml).unwrap();
|
||||||
|
assert_eq!(f.entries.len(), 3);
|
||||||
|
|
||||||
|
let two = &f.entries[0].enclosures;
|
||||||
|
assert_eq!(two.len(), 2, "both enclosures survive");
|
||||||
|
assert_eq!(
|
||||||
|
two[0].url, "https://x/a.mp3?v=1&t=2",
|
||||||
|
"document order, and the escaped ampersand is decoded"
|
||||||
|
);
|
||||||
|
assert_eq!(two[0].length, Some(111));
|
||||||
|
assert_eq!(two[1].url, "https://x/b.mp4");
|
||||||
|
assert_eq!(two[1].mime.as_deref(), Some("video/mp4"));
|
||||||
|
|
||||||
|
assert_eq!(f.entries[1].enclosures.len(), 1);
|
||||||
|
assert_eq!(f.entries[2].enclosures.len(), 0, "an item may have none");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_items_picture_comes_from_the_most_deliberate_source() {
|
||||||
|
let xml = br#"<?xml version="1.0"?>
|
||||||
|
<rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
|
||||||
|
xmlns:media="http://search.yahoo.com/mrss/">
|
||||||
|
<channel><title>P</title><link>https://x</link><description>d</description>
|
||||||
|
<item><title>Has itunes</title><guid>a</guid>
|
||||||
|
<itunes:image href="https://x/itunes.jpg"/>
|
||||||
|
<media:thumbnail url="https://x/thumb.jpg"/>
|
||||||
|
<enclosure url="https://x/a.jpg" length="1" type="image/jpeg"/></item>
|
||||||
|
<item><title>Has thumbnail</title><guid>b</guid>
|
||||||
|
<media:thumbnail url="https://x/thumb.jpg"/>
|
||||||
|
<enclosure url="https://x/b.jpg" length="1" type="image/jpeg"/></item>
|
||||||
|
<item><title>Has media content</title><guid>c</guid>
|
||||||
|
<media:content url="https://x/pic.jpg" type="image/jpeg"/>
|
||||||
|
<media:content url="https://x/clip.mp4" type="video/mp4"/></item>
|
||||||
|
<item><title>Only an image enclosure</title><guid>d</guid>
|
||||||
|
<enclosure url="https://x/d.jpg" length="1" type="image/jpeg"/></item>
|
||||||
|
<item><title>Audio only</title><guid>e</guid>
|
||||||
|
<enclosure url="https://x/e.mp3" length="1" type="audio/mpeg"/></item>
|
||||||
|
</channel></rss>"#;
|
||||||
|
let f = parse(xml).unwrap();
|
||||||
|
let img = |i: usize| f.entries[i].image.as_deref();
|
||||||
|
assert_eq!(img(0), Some("https://x/itunes.jpg"), "itunes:image wins");
|
||||||
|
assert_eq!(img(1), Some("https://x/thumb.jpg"), "then media:thumbnail");
|
||||||
|
assert_eq!(img(2), Some("https://x/pic.jpg"), "media:content, and only the image one");
|
||||||
|
assert_eq!(img(3), Some("https://x/d.jpg"), "a blog's article picture arrives as an enclosure");
|
||||||
|
assert_eq!(img(4), None, "audio is not a picture");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn durations_parse_from_seconds_or_a_clock() {
|
||||||
|
assert_eq!(parse_duration("5649"), Some(5649));
|
||||||
|
assert_eq!(parse_duration("23:45"), Some(1425));
|
||||||
|
assert_eq!(parse_duration("1:34:09"), Some(5649));
|
||||||
|
assert_eq!(parse_duration("0"), None, "zero is not a duration");
|
||||||
|
assert_eq!(parse_duration(""), None);
|
||||||
|
assert_eq!(parse_duration("garbage"), None);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn rejects_html_masquerading_as_a_feed() {
|
fn rejects_html_masquerading_as_a_feed() {
|
||||||
assert!(parse(b"<html><body>nope</body></html>").is_err());
|
assert!(parse(b"<html><body>nope</body></html>").is_err());
|
||||||
@@ -324,3 +768,4 @@ mod tests {
|
|||||||
assert_eq!(pick_guid(None, None, None, None), None);
|
assert_eq!(pick_guid(None, None, None, None), None);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
69
src/ipc.rs
69
src/ipc.rs
@@ -17,14 +17,17 @@ pub enum Event {
|
|||||||
FeedError { feed: String, msg: String },
|
FeedError { feed: String, msg: String },
|
||||||
Progress {
|
Progress {
|
||||||
feed: String,
|
feed: String,
|
||||||
|
/// Which enclosure this is about. Without it a UI cannot tell one download's
|
||||||
|
/// progress from another's and ends up animating every pending row.
|
||||||
|
enclosure: i64,
|
||||||
url: String,
|
url: String,
|
||||||
file: String,
|
file: String,
|
||||||
done: u64,
|
done: u64,
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
total: Option<u64>,
|
total: Option<u64>,
|
||||||
},
|
},
|
||||||
DownloadDone { feed: String, url: String, path: String, bytes: u64 },
|
DownloadDone { feed: String, enclosure: i64, url: String, path: String, bytes: u64 },
|
||||||
DownloadError { feed: String, url: String, msg: String },
|
DownloadError { feed: String, enclosure: i64, url: String, msg: String },
|
||||||
TorrentDeferred { feed: String, url: String },
|
TorrentDeferred { feed: String, url: String },
|
||||||
Reaped { path: String, bytes: u64 },
|
Reaped { path: String, bytes: u64 },
|
||||||
/// Terminal: a client that asked for work stops reading here.
|
/// Terminal: a client that asked for work stops reading here.
|
||||||
@@ -59,10 +62,10 @@ impl Event {
|
|||||||
Event::DownloadDone { path, .. } => format!(" saved {path}"),
|
Event::DownloadDone { path, .. } => format!(" saved {path}"),
|
||||||
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
|
Event::DownloadError { url, msg, .. } => format!(" failed {url}: {msg}"),
|
||||||
Event::Reaped { path, bytes } => {
|
Event::Reaped { path, bytes } => {
|
||||||
format!("reap {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
|
format!("deleted {path} ({:.1} MB)", *bytes as f64 / 1_048_576.0)
|
||||||
}
|
}
|
||||||
Event::ReapDone { files, bytes } => format!(
|
Event::ReapDone { files, bytes } => format!(
|
||||||
"reaped {files} file(s), {:.1} MB",
|
"deleted {files} old file(s), {:.1} MB",
|
||||||
*bytes as f64 / 1_048_576.0
|
*bytes as f64 / 1_048_576.0
|
||||||
),
|
),
|
||||||
Event::Status { feeds, pending, downloaded } => {
|
Event::Status { feeds, pending, downloaded } => {
|
||||||
@@ -70,9 +73,9 @@ impl Event {
|
|||||||
}
|
}
|
||||||
Event::Error { msg } => format!("error: {msg}"),
|
Event::Error { msg } => format!("error: {msg}"),
|
||||||
// Noise in a terminal; a UI still gets them on the socket.
|
// Noise in a terminal; a UI still gets them on the socket.
|
||||||
Event::FeedStart { .. } | Event::TorrentDeferred { .. } | Event::ScanDone { .. } => {
|
Event::FeedStart { feed } => format!("{feed}: checking"),
|
||||||
return None;
|
Event::TorrentDeferred { feed, .. } => format!("{feed}: torrent deferred"),
|
||||||
}
|
Event::ScanDone { feeds } => format!("scan complete, {feeds} feed(s)"),
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -90,6 +93,12 @@ pub enum Command {
|
|||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
dry_run: bool,
|
dry_run: bool,
|
||||||
},
|
},
|
||||||
|
/// Fetch one specific enclosure now, ignoring max_new_per_check and the queue order.
|
||||||
|
/// A scan cannot express "this one, now": it takes the lowest-id pending rows up to
|
||||||
|
/// the per-scan cap, so an explicit request has to bypass both.
|
||||||
|
Download {
|
||||||
|
enclosure: i64,
|
||||||
|
},
|
||||||
Status,
|
Status,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -110,7 +119,45 @@ impl Emitter {
|
|||||||
}
|
}
|
||||||
|
|
||||||
pub fn emit(&self, e: Event) {
|
pub fn emit(&self, e: Event) {
|
||||||
|
// Also log it. Scans and downloads travel as events, not tracing calls, so
|
||||||
|
// without this the log view shows only startup and HTTP lines and none of the
|
||||||
|
// work the daemon is actually doing. Progress goes to debug: it fires on every
|
||||||
|
// whole percent and would otherwise crowd everything else out of the buffer.
|
||||||
|
// Level by how much it matters. With 80-odd feeds in an OPML subscription, one
|
||||||
|
// line per feed per tick for "not due yet" would push everything worth reading
|
||||||
|
// out of the buffer within a few minutes.
|
||||||
|
let routine = match &e {
|
||||||
|
Event::Progress { .. } | Event::FeedSkip { .. } | Event::FeedStart { .. } => true,
|
||||||
|
Event::FeedDone { new, downloaded, failed, torrents, .. } => {
|
||||||
|
*new == 0 && *downloaded == 0 && *failed == 0 && *torrents == 0
|
||||||
|
}
|
||||||
|
_ => false,
|
||||||
|
};
|
||||||
|
let bad = matches!(
|
||||||
|
&e,
|
||||||
|
Event::FeedError { .. } | Event::DownloadError { .. } | Event::Error { .. }
|
||||||
|
);
|
||||||
|
if let Some(line) = e.human() {
|
||||||
|
let line = line.trim();
|
||||||
|
if bad {
|
||||||
|
tracing::warn!(target: "ipx::scan", "{line}");
|
||||||
|
} else if routine {
|
||||||
|
tracing::debug!(target: "ipx::scan", "{line}");
|
||||||
|
} else {
|
||||||
|
tracing::info!(target: "ipx::scan", "{line}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if let Some(tx) = &self.tx {
|
if let Some(tx) = &self.tx {
|
||||||
|
// The outbound half of the protocol, as it goes on the wire. Progress is the
|
||||||
|
// high-volume one, so it sits at debug.
|
||||||
|
if let Ok(json) = serde_json::to_string(&e) {
|
||||||
|
if matches!(e, Event::Progress { .. }) {
|
||||||
|
tracing::debug!(target: "ipx::io", "<- {json}");
|
||||||
|
} else {
|
||||||
|
tracing::info!(target: "ipx::io", "<- {json}");
|
||||||
|
}
|
||||||
|
}
|
||||||
// An error here only means nobody is listening yet.
|
// An error here only means nobody is listening yet.
|
||||||
let _ = tx.send(e.clone());
|
let _ = tx.send(e.clone());
|
||||||
}
|
}
|
||||||
@@ -234,6 +281,11 @@ mod tests {
|
|||||||
let got: Command = serde_json::from_str(r#"{"cmd":"reap","dry_run":true}"#).unwrap();
|
let got: Command = serde_json::from_str(r#"{"cmd":"reap","dry_run":true}"#).unwrap();
|
||||||
assert!(matches!(got, Command::Reap { dry_run: true }));
|
assert!(matches!(got, Command::Reap { dry_run: true }));
|
||||||
|
|
||||||
|
// "Download this one now" is its own command precisely because a scan cannot
|
||||||
|
// express it: a scan takes the lowest-id pending rows up to max_new_per_check.
|
||||||
|
let got: Command = serde_json::from_str(r#"{"cmd":"download","enclosure":11}"#).unwrap();
|
||||||
|
assert!(matches!(got, Command::Download { enclosure: 11 }));
|
||||||
|
|
||||||
assert!(serde_json::from_str::<Command>(r#"{"cmd":"nope"}"#).is_err());
|
assert!(serde_json::from_str::<Command>(r#"{"cmd":"nope"}"#).is_err());
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -241,6 +293,7 @@ mod tests {
|
|||||||
fn events_serialise_to_the_documented_shape() {
|
fn events_serialise_to_the_documented_shape() {
|
||||||
let ev = Event::Progress {
|
let ev = Event::Progress {
|
||||||
feed: "atp".into(),
|
feed: "atp".into(),
|
||||||
|
enclosure: 42,
|
||||||
url: "https://x/ep.mp3".into(),
|
url: "https://x/ep.mp3".into(),
|
||||||
file: "ep.mp3".into(),
|
file: "ep.mp3".into(),
|
||||||
done: 10_485_760,
|
done: 10_485_760,
|
||||||
@@ -249,10 +302,12 @@ mod tests {
|
|||||||
let json = serde_json::to_string(&ev).unwrap();
|
let json = serde_json::to_string(&ev).unwrap();
|
||||||
assert!(json.starts_with(r#"{"ev":"progress""#), "got {json}");
|
assert!(json.starts_with(r#"{"ev":"progress""#), "got {json}");
|
||||||
assert!(json.contains(r#""done":10485760"#));
|
assert!(json.contains(r#""done":10485760"#));
|
||||||
|
assert!(json.contains(r#""enclosure":42"#), "a UI needs this to target one row");
|
||||||
|
|
||||||
// total is omitted rather than null when the server sent no length.
|
// total is omitted rather than null when the server sent no length.
|
||||||
let ev = Event::Progress {
|
let ev = Event::Progress {
|
||||||
feed: "a".into(),
|
feed: "a".into(),
|
||||||
|
enclosure: 1,
|
||||||
url: "u".into(),
|
url: "u".into(),
|
||||||
file: "f".into(),
|
file: "f".into(),
|
||||||
done: 1,
|
done: 1,
|
||||||
|
|||||||
147
src/logbuf.rs
Normal file
147
src/logbuf.rs
Normal file
@@ -0,0 +1,147 @@
|
|||||||
|
//! In-process ring buffer of log lines, so the UI can show what the daemon is doing.
|
||||||
|
//!
|
||||||
|
//! Tailing a file would not survive Docker, where logs go to stdout and there is no file
|
||||||
|
//! to read. Capturing inside the tracing pipeline works the same either way.
|
||||||
|
|
||||||
|
use std::collections::VecDeque;
|
||||||
|
use std::sync::{LazyLock, Mutex};
|
||||||
|
use tracing::field::{Field, Visit};
|
||||||
|
use tracing_subscriber::Layer;
|
||||||
|
use tracing_subscriber::layer::Context;
|
||||||
|
|
||||||
|
/// Kept small enough to be cheap to hold and to serialise in one response.
|
||||||
|
const CAPACITY: usize = 5000;
|
||||||
|
|
||||||
|
#[derive(Clone, Debug, serde::Serialize)]
|
||||||
|
pub struct LogLine {
|
||||||
|
/// Monotonic, so a client can ask for "everything after N" without duplicates.
|
||||||
|
pub seq: u64,
|
||||||
|
pub ts: i64,
|
||||||
|
pub level: String,
|
||||||
|
pub target: String,
|
||||||
|
pub msg: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Ring {
|
||||||
|
lines: VecDeque<LogLine>,
|
||||||
|
next_seq: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
static BUF: LazyLock<Mutex<Ring>> = LazyLock::new(|| {
|
||||||
|
Mutex::new(Ring { lines: VecDeque::with_capacity(CAPACITY), next_seq: 1 })
|
||||||
|
});
|
||||||
|
|
||||||
|
pub fn push(level: &str, target: &str, msg: String) {
|
||||||
|
let mut ring = match BUF.lock() {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(p) => p.into_inner(), // a poisoned log buffer must not take the process down
|
||||||
|
};
|
||||||
|
let seq = ring.next_seq;
|
||||||
|
ring.next_seq += 1;
|
||||||
|
if ring.lines.len() == CAPACITY {
|
||||||
|
ring.lines.pop_front();
|
||||||
|
}
|
||||||
|
ring.lines.push_back(LogLine {
|
||||||
|
seq,
|
||||||
|
ts: crate::db::now(),
|
||||||
|
level: level.to_owned(),
|
||||||
|
target: target.to_owned(),
|
||||||
|
msg,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Lines newer than `after`, oldest first, plus the highest seq now held.
|
||||||
|
pub fn since(after: u64, limit: usize) -> (Vec<LogLine>, u64) {
|
||||||
|
let ring = match BUF.lock() {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(p) => p.into_inner(),
|
||||||
|
};
|
||||||
|
let latest = ring.next_seq.saturating_sub(1);
|
||||||
|
let mut out: Vec<LogLine> = ring
|
||||||
|
.lines
|
||||||
|
.iter()
|
||||||
|
.filter(|l| l.seq > after)
|
||||||
|
.cloned()
|
||||||
|
.collect();
|
||||||
|
// On a first load (after = 0) the tail is what matters, not the head.
|
||||||
|
if out.len() > limit {
|
||||||
|
out.drain(..out.len() - limit);
|
||||||
|
}
|
||||||
|
(out, latest)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A tracing layer that mirrors every event into the ring.
|
||||||
|
pub struct RingLayer;
|
||||||
|
|
||||||
|
impl<S: tracing::Subscriber> Layer<S> for RingLayer {
|
||||||
|
fn on_event(&self, event: &tracing::Event<'_>, _ctx: Context<'_, S>) {
|
||||||
|
let mut v = Collect::default();
|
||||||
|
event.record(&mut v);
|
||||||
|
let meta = event.metadata();
|
||||||
|
push(meta.level().as_str(), meta.target(), v.finish());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct Collect {
|
||||||
|
message: String,
|
||||||
|
fields: Vec<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Collect {
|
||||||
|
fn finish(self) -> String {
|
||||||
|
if self.fields.is_empty() {
|
||||||
|
self.message
|
||||||
|
} else if self.message.is_empty() {
|
||||||
|
self.fields.join(" ")
|
||||||
|
} else {
|
||||||
|
format!("{} {}", self.message, self.fields.join(" "))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fn add(&mut self, field: &Field, value: String) {
|
||||||
|
if field.name() == "message" {
|
||||||
|
self.message = value;
|
||||||
|
} else {
|
||||||
|
self.fields.push(format!("{}={}", field.name(), value));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Visit for Collect {
|
||||||
|
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
|
||||||
|
self.add(field, format!("{value:?}"));
|
||||||
|
}
|
||||||
|
// Numbers and bools reach record_debug through the trait's defaults, which prints them the
|
||||||
|
// same way. A string would print quoted there, hence its own method.
|
||||||
|
fn record_str(&mut self, field: &Field, value: &str) {
|
||||||
|
self.add(field, value.to_owned());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_ring_drops_oldest_and_keeps_sequence_stable() {
|
||||||
|
for i in 0..(CAPACITY + 50) {
|
||||||
|
push("INFO", "t", format!("line {i}"));
|
||||||
|
}
|
||||||
|
let (all, latest) = since(0, CAPACITY * 2);
|
||||||
|
assert_eq!(all.len(), CAPACITY, "bounded");
|
||||||
|
assert!(latest >= (CAPACITY + 50) as u64);
|
||||||
|
assert!(
|
||||||
|
all.first().unwrap().seq < all.last().unwrap().seq,
|
||||||
|
"oldest first"
|
||||||
|
);
|
||||||
|
|
||||||
|
// "everything after the last one I saw" must return nothing new.
|
||||||
|
let (none, _) = since(latest, 100);
|
||||||
|
assert!(none.is_empty());
|
||||||
|
|
||||||
|
// A first load takes the tail, not the head.
|
||||||
|
let (tail, _) = since(0, 5);
|
||||||
|
assert_eq!(tail.len(), 5);
|
||||||
|
assert_eq!(tail.last().unwrap().seq, latest);
|
||||||
|
}
|
||||||
|
}
|
||||||
1146
src/main.rs
1146
src/main.rs
File diff suppressed because it is too large
Load Diff
@@ -150,33 +150,52 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn query_never_offers_flagged_files_and_prefers_read_ones() {
|
fn query_never_offers_a_file_anyone_starred_and_prefers_ones_everyone_read() {
|
||||||
|
// One file serves both subscribers, so it takes both of them to release it.
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO entries (feed_id, guid, first_seen, read, flagged) VALUES
|
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
|
||||||
('f', 'keep', 0, 1, 1),
|
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0);
|
||||||
('f', 'unread', 0, 0, 0),
|
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
||||||
('f', 'read', 0, 1, 0);
|
('f', 'keep', 0),
|
||||||
|
('f', 'half', 0),
|
||||||
|
('f', 'unread', 0),
|
||||||
|
('f', 'read', 0);
|
||||||
|
-- Starred by one of the two, so it stays whatever the other thinks.
|
||||||
|
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
|
||||||
|
(1, 'f', 'keep', 1, 1),
|
||||||
|
(2, 'f', 'keep', 1, 0),
|
||||||
|
(1, 'f', 'half', 1, 0),
|
||||||
|
(1, 'f', 'read', 1, 0),
|
||||||
|
(2, 'f', 'read', 1, 0);
|
||||||
INSERT INTO enclosures (id, feed_id, guid, url, path, bytes_done, state, downloaded_at) VALUES
|
INSERT INTO enclosures (id, feed_id, guid, url, path, bytes_done, state, downloaded_at) VALUES
|
||||||
(1, 'f', 'keep', 'u1', '/tmp/keep', 10, 'done', 10),
|
(1, 'f', 'keep', 'u1', '/tmp/keep', 10, 'done', 10),
|
||||||
(2, 'f', 'unread', 'u2', '/tmp/unread', 10, 'done', 20),
|
(2, 'f', 'half', 'u2', '/tmp/half', 10, 'done', 20),
|
||||||
(3, 'f', 'read', 'u3', '/tmp/read', 10, 'done', 30);",
|
(3, 'f', 'unread', 'u3', '/tmp/unread', 10, 'done', 30),
|
||||||
|
(4, 'f', 'read', 'u4', '/tmp/read', 10, 'done', 40);",
|
||||||
)
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
|
||||||
let got: Vec<i64> = db.reap_candidates().unwrap().iter().map(|c| c.id).collect();
|
let got: Vec<i64> = db.reap_candidates().unwrap().iter().map(|c| c.id).collect();
|
||||||
assert_eq!(got, vec![3, 2], "flagged excluded; read goes before unread");
|
assert_eq!(
|
||||||
|
got,
|
||||||
|
vec![4, 2, 3],
|
||||||
|
"starred by anyone is never offered; read by everyone goes first, and one \
|
||||||
|
person still having it unread keeps it back with the unread ones"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn prune_keeps_entries_that_still_have_a_file() {
|
fn prune_keeps_entries_that_still_have_a_file() {
|
||||||
let db = Db::memory().unwrap();
|
let db = Db::memory().unwrap();
|
||||||
db.exec_for_test(
|
db.exec_for_test(
|
||||||
"INSERT INTO entries (feed_id, guid, first_seen, read, flagged) VALUES
|
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
|
||||||
('f', 'has-file', 100, 1, 0),
|
INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',1);
|
||||||
('f', 'no-file', 100, 1, 0),
|
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
||||||
('f', 'flagged', 100, 1, 1),
|
('f', 'has-file', 100),
|
||||||
('f', 'recent', 900, 1, 0);
|
('f', 'no-file', 100),
|
||||||
|
('f', 'flagged', 100),
|
||||||
|
('f', 'recent', 900);
|
||||||
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
||||||
(1, 'f', 'has-file', 'u1', '/tmp/x', 'done');",
|
(1, 'f', 'has-file', 'u1', '/tmp/x', 'done');",
|
||||||
)
|
)
|
||||||
|
|||||||
1521
src/web.rs
Normal file
1521
src/web.rs
Normal file
File diff suppressed because it is too large
Load Diff
115
tests/page-smoke.js
Normal file
115
tests/page-smoke.js
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
// Executes web/index.html's script against a stub DOM and fails on anything thrown.
|
||||||
|
//
|
||||||
|
// This exists because a ReferenceError at load once blanked the whole UI: a patch
|
||||||
|
// anchored on a function that no longer existed, so `prefsModal` was referenced but
|
||||||
|
// never defined. `node --check` passes that happily -- it is a parse, not a run --
|
||||||
|
// and every server-side test passed too, because the server was fine.
|
||||||
|
//
|
||||||
|
// node tests/page-smoke.js
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
const vm = require('vm');
|
||||||
|
|
||||||
|
const html = fs.readFileSync(path.join(__dirname, '..', 'web', 'index.html'), 'utf8');
|
||||||
|
const script = html.split('<script>')[1].split('</script>')[0];
|
||||||
|
const ids = new Set([...html.matchAll(/id="([^"]+)"/g)].map(m => m[1]));
|
||||||
|
|
||||||
|
const missing = [];
|
||||||
|
const el = (name) => new Proxy({ style: { setProperty(){}, getPropertyValue(){ return ''; } }, dataset: {}, classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
|
||||||
|
value: '', textContent: '', innerHTML: '', hidden: false, children: [], firstElementChild: null,
|
||||||
|
appendChild(){}, removeChild(){}, remove(){}, insertAdjacentHTML(){}, addEventListener(){},
|
||||||
|
setAttribute(){}, getAttribute(){ return null; }, select(){}, setSelectionRange(){}, focus(){},
|
||||||
|
replaceWith(){}, querySelector(){ return el('nested'); }, querySelectorAll(){ return []; },
|
||||||
|
play(){ return Promise.resolve(); }, pause(){}, closest(){ return null; } },
|
||||||
|
{ get: (t, k) => k in t ? t[k] : undefined, set: (t, k, v) => (t[k] = v, true) });
|
||||||
|
|
||||||
|
const document = {
|
||||||
|
querySelector(sel) {
|
||||||
|
if (sel.startsWith('#') && !ids.has(sel.slice(1))) { missing.push(sel); return null; }
|
||||||
|
return el(sel);
|
||||||
|
},
|
||||||
|
querySelectorAll: () => [],
|
||||||
|
createElement: () => el('created'),
|
||||||
|
addEventListener(){}, body: el('body'),
|
||||||
|
documentElement: { dataset: {} },
|
||||||
|
};
|
||||||
|
|
||||||
|
const ctx = {
|
||||||
|
document, console,
|
||||||
|
window: { isSecureContext: false, addEventListener(){} },
|
||||||
|
localStorage: { getItem: () => null, setItem(){}, removeItem(){} },
|
||||||
|
navigator: { clipboard: undefined, sendBeacon(){}, mediaSession: undefined },
|
||||||
|
fetch: (url) => Promise.resolve({
|
||||||
|
ok: true, status: 200, text: () => Promise.resolve(''),
|
||||||
|
json: () => Promise.resolve(
|
||||||
|
String(url).includes('/api/settings')
|
||||||
|
? { schedule: 'every 60m', every_mins: 60, download_dir: '/tmp', max_total_gb: 0, max_age_days: 0 }
|
||||||
|
: String(url).includes('/api/users')
|
||||||
|
? [{ id: 1, name: 'admin', admin: true, password: true }, { id: 2, name: 'sam', admin: false, password: false }]
|
||||||
|
: /\/api\/(popular|directory)/.test(String(url))
|
||||||
|
? [{ id: 'f', title: 'A Feed', image: null, subscribers: 2, subscribed: true },
|
||||||
|
{ id: 'g', title: null, image: null, subscribers: 1, subscribed: false }]
|
||||||
|
: /entries/.test(String(url)) ? { total: 0, entries: [] } : []),
|
||||||
|
}),
|
||||||
|
EventSource: function () { this.close = () => {}; },
|
||||||
|
MediaMetadata: function () {},
|
||||||
|
Blob: function () {},
|
||||||
|
setTimeout, clearTimeout, setInterval, clearInterval,
|
||||||
|
confirm: () => false, prompt: () => null, alert(){},
|
||||||
|
Date, Math, JSON, Object, Array, String, Number, Promise, Error, FormData: function(){},
|
||||||
|
URLSearchParams, encodeURIComponent, decodeURIComponent, parseInt, parseFloat, isNaN,
|
||||||
|
};
|
||||||
|
ctx.globalThis = ctx;
|
||||||
|
ctx.window.location = { href: '' };
|
||||||
|
|
||||||
|
try {
|
||||||
|
vm.createContext(ctx);
|
||||||
|
vm.runInContext(script, ctx, { filename: 'index.html<script>', timeout: 5000 });
|
||||||
|
} catch (e) {
|
||||||
|
console.error('FAIL: the page script threw while loading\n ' + e.stack.split('\n').slice(0, 3).join('\n '));
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The modals are built on demand, so a load-time check never reaches them. Drive the
|
||||||
|
// ones that construct markup from live data, which is where a bad field reference hides.
|
||||||
|
const feed = {
|
||||||
|
id: 'f', url: 'https://x/rss', title: 'A Feed', image: null, folder: null,
|
||||||
|
keywords: ['a'], allow_explicit: false, auto_download: true, max_new_per_check: 3,
|
||||||
|
schedule: 'every 6h', schedule_mins: 360, every_mins: 360,
|
||||||
|
last_checked: 1, next_check: 2, entries: 1, downloaded: 0, unread: 1, last_error: null,
|
||||||
|
};
|
||||||
|
const drive = [
|
||||||
|
['settingsModal', () => ctx.settingsModal(feed)],
|
||||||
|
['settingsModal (no override)', () => ctx.settingsModal({ ...feed, schedule: null, schedule_mins: null })],
|
||||||
|
['downloadLatestModal', () => ctx.downloadLatestModal(feed)],
|
||||||
|
['removeFeed', () => ctx.removeFeed(feed)],
|
||||||
|
['prefsModal', () => ctx.prefsModal()],
|
||||||
|
['usersModal', () => ctx.usersModal()],
|
||||||
|
['opmlModal', () => ctx.opmlModal()],
|
||||||
|
['selectFeed (directory)', () => ctx.selectFeed(':directory')],
|
||||||
|
['selectFeed (popular)', () => ctx.selectFeed(':popular')],
|
||||||
|
['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
|
||||||
|
['logsModal', () => ctx.logsModal()],
|
||||||
|
// `const S` is not reachable from here: top-level const/let do not become properties
|
||||||
|
// of a vm context the way var and function declarations do.
|
||||||
|
['renderGroup', () => ctx.renderGroup(feed, [{ ...feed, id: 'child', group: 'f', orphaned: true }])],
|
||||||
|
];
|
||||||
|
for (const [name, fn] of drive) {
|
||||||
|
try {
|
||||||
|
const r = fn();
|
||||||
|
if (r && typeof r.catch === 'function') r.catch(e => {
|
||||||
|
console.error(`FAIL: ${name} rejected: ${e.message}`); process.exit(1);
|
||||||
|
});
|
||||||
|
} catch (e) {
|
||||||
|
console.error(`FAIL: ${name} threw: ${e.message}`);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (missing.length) {
|
||||||
|
console.error('FAIL: handlers wired to elements that do not exist: ' + [...new Set(missing)].join(', '));
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
console.log('OK: page script loads clean, every selector it wires at load exists');
|
||||||
|
// logsModal arms a poll timer; without this the pending interval keeps node alive.
|
||||||
|
process.exit(0);
|
||||||
786
tests/ui/app.spec.js
Normal file
786
tests/ui/app.spec.js
Normal file
@@ -0,0 +1,786 @@
|
|||||||
|
const { test, expect } = require('@playwright/test');
|
||||||
|
const { TOKEN } = require('./global-setup');
|
||||||
|
|
||||||
|
// The token sets a cookie, so every test starts by presenting it once.
|
||||||
|
test.beforeEach(async ({ page }) => {
|
||||||
|
await page.goto(`/?token=${TOKEN}`);
|
||||||
|
await expect(page.locator('#feedlist')).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the page loads and lists the configured feeds', async ({ page }) => {
|
||||||
|
// Regression: a ReferenceError in the script left the shell rendered and the sidebar
|
||||||
|
// empty, with every handler below the error dead. Server-side checks all passed.
|
||||||
|
// Four top-level feeds in the fixture config; the OPML's children are inside a closed folder.
|
||||||
|
await expect(page.locator('.feed')).toHaveCount(5, { timeout: 15_000 });
|
||||||
|
await expect(page.getByText('Test Show')).toBeVisible();
|
||||||
|
const errors = [];
|
||||||
|
page.on('pageerror', e => errors.push(e.message));
|
||||||
|
await page.reload();
|
||||||
|
await expect(page.locator('.feed').first()).toBeVisible();
|
||||||
|
expect(errors, 'the page script must not throw at load').toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the theme toggle actually changes the theme', async ({ page }) => {
|
||||||
|
// Regression: this button was wired after a line that threw, so it did nothing.
|
||||||
|
const before = await page.evaluate(() => document.documentElement.dataset.theme || 'system');
|
||||||
|
await page.locator('#theme').click();
|
||||||
|
await expect
|
||||||
|
.poll(() => page.evaluate(() => document.documentElement.dataset.theme))
|
||||||
|
.not.toBe(before);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the theme button steps through dark, light and classic, and remembers', async ({ page }) => {
|
||||||
|
const theme = () => page.evaluate(() => document.documentElement.dataset.theme);
|
||||||
|
for (let i = 0; i < 3 && (await theme()) !== 'classic'; i++) await page.locator('#theme').click();
|
||||||
|
expect(await theme()).toBe('classic');
|
||||||
|
await expect(page.locator('#theme')).toHaveAttribute('title', /Classic.*Click for Dark/);
|
||||||
|
|
||||||
|
await page.reload();
|
||||||
|
await expect.poll(theme).toBe('classic');
|
||||||
|
// The 2004 Mac app set its type in Lucida Grande.
|
||||||
|
expect(await page.evaluate(() => getComputedStyle(document.body).fontFamily)).toContain('Lucida Grande');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('settings opens and saves the global schedule', async ({ page }) => {
|
||||||
|
await page.locator('#prefs').click();
|
||||||
|
await expect(page.locator('#modal.on')).toBeVisible();
|
||||||
|
await expect(page.locator('#gnum')).toBeVisible();
|
||||||
|
await page.locator('#gnum').fill('4');
|
||||||
|
await page.locator('#gunit').selectOption('h');
|
||||||
|
await page.locator('#gsave').click();
|
||||||
|
await expect(page.locator('#modal.on')).toBeHidden();
|
||||||
|
|
||||||
|
// It must survive a reload, i.e. actually reach the config.
|
||||||
|
await page.locator('#prefs').click();
|
||||||
|
await expect(page.locator('#gnum')).toHaveValue('4');
|
||||||
|
await expect(page.locator('#gunit')).toHaveValue('h');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('episodes show with their metadata, and the text opens below', async ({ page }) => {
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
|
||||||
|
await expect(page.getByText('First Episode')).toBeVisible();
|
||||||
|
// Newest first, so target the episode by name rather than by position.
|
||||||
|
const first = page.locator('.ep', { hasText: 'First Episode' });
|
||||||
|
await expect(first).toContainText('S1E1');
|
||||||
|
await expect(first).toContainText('30:30'); // itunes:duration 1830
|
||||||
|
await expect(page.locator('.ep', { hasText: 'Second Episode' })).toContainText('15:00');
|
||||||
|
|
||||||
|
// Selecting an item shows its text in the pane below, not inline in the row.
|
||||||
|
await first.click();
|
||||||
|
await expect(first).toHaveClass(/sel/);
|
||||||
|
await expect(page.locator('#detail')).toContainText('Show notes for the first one');
|
||||||
|
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the three panes are there and the item text lands in the bottom one', async ({ page }) => {
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
await expect(page.locator('#list')).toBeVisible();
|
||||||
|
await expect(page.locator('#grab')).toBeVisible(); // the draggable divider
|
||||||
|
await expect(page.locator('#detail')).toContainText('Pick an item');
|
||||||
|
|
||||||
|
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
||||||
|
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||||
|
// The enclosure goes to the Files pane beside the list, as the original's did.
|
||||||
|
await expect(page.locator('#files')).toBeVisible();
|
||||||
|
await expect(page.locator('#files .encbox')).toHaveCount(1);
|
||||||
|
|
||||||
|
// Only the downloaded one gets a player, and max_new_per_check is 1, so find it by
|
||||||
|
// its chip rather than assuming which episode the daemon happened to fetch.
|
||||||
|
const downloaded = page.locator('.ep', { has: page.locator('.kind.here') }).first();
|
||||||
|
await downloaded.click();
|
||||||
|
await expect(page.locator('#files [data-a="play"]')).toBeVisible();
|
||||||
|
await expect(page.locator('#files .encbox [title="Save to this computer"]')).toBeVisible();
|
||||||
|
|
||||||
|
// Selecting another item replaces the pane rather than stacking.
|
||||||
|
await page.locator('.ep', { hasText: 'First Episode' }).click();
|
||||||
|
await expect(page.locator('#detail .dt')).toHaveText('First Episode');
|
||||||
|
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a downloaded file that is not audio gets no player', async ({ page }) => {
|
||||||
|
// Regression: anything with a file got an <audio> element and a play button, so a blog's
|
||||||
|
// header image rendered as a broken player.
|
||||||
|
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
|
||||||
|
const row = page.locator('.ep', { hasText: 'An Article' });
|
||||||
|
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
|
await expect(row.locator('[data-a="play"]')).toHaveCount(0);
|
||||||
|
await row.click();
|
||||||
|
|
||||||
|
await expect(page.locator('#detail .dt')).toHaveText('An Article');
|
||||||
|
await expect(page.locator('#files [data-a="play"]')).toHaveCount(0);
|
||||||
|
// What it is and that it is here: one icon, green, with the words in its tooltip.
|
||||||
|
await expect(page.locator('#files .encbox .kind.here')).toHaveAttribute('title', 'image, downloaded');
|
||||||
|
// Still offered as a file, just not as an episode: viewable and keepable.
|
||||||
|
await expect(page.locator('#files [title="Save to this computer"]')).toBeVisible();
|
||||||
|
const view = page.locator('#files a[title="View in a new tab"]');
|
||||||
|
await expect(view).toHaveAttribute('target', '_blank');
|
||||||
|
await expect(view).toHaveAttribute('rel', /noopener/);
|
||||||
|
await expect(view).toHaveAttribute('href', /\/media\/\d+/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an item with several enclosures lists them all', async ({ page }) => {
|
||||||
|
await page.locator('.feed', { hasText: 'Multi Show' }).click();
|
||||||
|
const row = page.locator('.ep', { hasText: 'Two Files' });
|
||||||
|
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||||
|
// The row says there is more than one without listing them.
|
||||||
|
await expect(row).toContainText('+1 more file');
|
||||||
|
|
||||||
|
await row.click();
|
||||||
|
// The Files pane lists every one: the audio and the image.
|
||||||
|
await expect(page.locator('#files .encbox')).toHaveCount(2);
|
||||||
|
await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the filter tabs change what is listed', async ({ page }) => {
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
|
||||||
|
const all = await page.locator('.ep').count(); // All is the default tab
|
||||||
|
await expect(page.locator('#count')).toContainText('item');
|
||||||
|
|
||||||
|
await page.locator('.tabs button', { hasText: 'Unread' }).first().click();
|
||||||
|
expect(await page.locator('.ep').count()).toBeLessThanOrEqual(all);
|
||||||
|
|
||||||
|
await page.locator('.tabs button', { hasText: 'Kept' }).first().click();
|
||||||
|
await expect(page.locator('#count')).toContainText('0 items');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a feed URL is editable and has a copy button', async ({ page }) => {
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
await page.locator('#content .acts [data-a="settings"]').click();
|
||||||
|
await expect(page.locator('#surl')).toHaveValue(/show\.xml/);
|
||||||
|
await expect(page.locator('#scopy')).toBeVisible();
|
||||||
|
|
||||||
|
// navigator.clipboard is absent over plain http, so the button must not throw.
|
||||||
|
const errors = [];
|
||||||
|
page.on('pageerror', e => errors.push(e.message));
|
||||||
|
await page.locator('#scopy').click();
|
||||||
|
await expect(page.locator('#scopy')).toHaveText(/Copied|Failed/);
|
||||||
|
expect(errors).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the log view has tabs and shows daemon traffic', async ({ page }) => {
|
||||||
|
await page.locator('#logs').click();
|
||||||
|
await expect(page.locator('#logbox')).toBeVisible();
|
||||||
|
await expect(page.locator('#logtabs button')).toHaveCount(4);
|
||||||
|
|
||||||
|
// Generate traffic, then check the Daemon I/O tab shows both directions.
|
||||||
|
await page.locator('#logtabs button', { hasText: 'Daemon I/O' }).click();
|
||||||
|
await page.evaluate(() =>
|
||||||
|
fetch('/api/fetch', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ force: true }),
|
||||||
|
}));
|
||||||
|
await expect(page.locator('#logbox .l').first()).toBeVisible({ timeout: 15_000 });
|
||||||
|
await expect(page.locator('#logbox')).toContainText('"cmd":"fetch"', { timeout: 15_000 });
|
||||||
|
await expect(page.locator('#logbox')).toContainText('"ev":', { timeout: 15_000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an OPML subscription is a collapsible folder', async ({ page }) => {
|
||||||
|
// Read the subscription so its feeds exist.
|
||||||
|
await page.evaluate(() =>
|
||||||
|
fetch('/api/fetch', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// Only a folder has a triangle, and it is a button that says whether the folder is open.
|
||||||
|
const chev = page.locator('.feed.group .chev');
|
||||||
|
await expect(chev).toBeVisible({ timeout: 20_000 });
|
||||||
|
await expect(page.locator('.feed:not(.group) .chev')).toHaveCount(0);
|
||||||
|
await expect(chev).toHaveAttribute('aria-expanded', 'false');
|
||||||
|
|
||||||
|
// Closed by default: the children are not listed until the folder is opened.
|
||||||
|
const before = await page.locator('.feed').count();
|
||||||
|
await chev.click();
|
||||||
|
await expect(page.locator('.feed')).toHaveCount(before + 2);
|
||||||
|
// Scoped to the sidebar: the name also appears as the page heading once selected.
|
||||||
|
await expect(page.locator('#feedlist').getByText('Grouped Show')).toBeVisible();
|
||||||
|
|
||||||
|
// The subscription's own page lists what is inside it.
|
||||||
|
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
|
||||||
|
await expect(page.locator('.childrow')).toHaveCount(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('inside an OPML, feeds with unread items are listed first', async ({ page }) => {
|
||||||
|
await page.evaluate(() =>
|
||||||
|
fetch('/api/fetch', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
|
||||||
|
}));
|
||||||
|
const chev = page.locator('.feed.group .chev');
|
||||||
|
await expect(chev).toBeVisible({ timeout: 20_000 });
|
||||||
|
await chev.click();
|
||||||
|
const side = page.locator('#feedlist');
|
||||||
|
await expect(side.getByText('Aardvark Radio')).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
|
// Other tests change read state, so set it here: Aardvark Radio read, Grouped Show not.
|
||||||
|
// Opening an item reads it; the toggle in the pane below flips it back.
|
||||||
|
await side.getByText('Aardvark Radio').click();
|
||||||
|
const aa = page.locator('.ep', { hasText: 'Aardvark Ep' });
|
||||||
|
await aa.click();
|
||||||
|
await expect(aa).toHaveClass(/read/);
|
||||||
|
await side.getByText('Grouped Show').click();
|
||||||
|
const gs = page.locator('.ep', { hasText: 'Grouped Ep' });
|
||||||
|
await gs.click();
|
||||||
|
await page.locator('#detail button[title="Mark unread"]').click();
|
||||||
|
await expect(gs).not.toHaveClass(/read/);
|
||||||
|
|
||||||
|
// Aardvark comes first alphabetically and in the OPML, so only the unread sort puts
|
||||||
|
// Grouped Show above it. The folder stays open across the reload (localStorage).
|
||||||
|
await page.reload();
|
||||||
|
const want = ['Grouped Show', 'Aardvark Radio'];
|
||||||
|
await expect(page.locator('#feedlist .feed.child b')).toHaveText(want, { timeout: 20_000 });
|
||||||
|
await page.locator('.feed', { hasText: 'Test Subscriptions' }).first().click();
|
||||||
|
await expect(page.locator('.childrow b')).toHaveText(want);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('marking an OPML subscription read covers the feeds inside it', async ({ page }) => {
|
||||||
|
await page.evaluate(() =>
|
||||||
|
fetch('/api/fetch', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ feed: 'test-subscriptions', force: true }),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// The folder's own row has no entries, so anything it marks read came from its child.
|
||||||
|
const folder = page.locator('.feed', { hasText: 'Test Subscriptions' }).first();
|
||||||
|
await expect(folder).toBeVisible({ timeout: 20_000 });
|
||||||
|
await expect(folder.locator('.badge')).not.toHaveText('0');
|
||||||
|
await folder.click();
|
||||||
|
|
||||||
|
await page.locator('#content .acts [data-a="read"]').click();
|
||||||
|
await expect(folder.locator('.badge')).toHaveText('0');
|
||||||
|
});
|
||||||
|
|
||||||
|
test.describe('on a phone', () => {
|
||||||
|
test.use({ viewport: { width: 390, height: 844 } });
|
||||||
|
|
||||||
|
test('the feed list is reachable and an item reads full screen', async ({ page }) => {
|
||||||
|
// The burger used to live in the player bar, which is hidden until something plays --
|
||||||
|
// leaving no way to reach the feeds at all.
|
||||||
|
await expect(page.locator('#burger')).toBeVisible();
|
||||||
|
await expect(page.locator('#player')).not.toBeVisible();
|
||||||
|
|
||||||
|
await page.locator('#burger').click();
|
||||||
|
await page.locator('.feed', { hasText: 'Test Show' }).first().click();
|
||||||
|
await expect(page.locator('#sidebar')).not.toHaveClass(/open/);
|
||||||
|
|
||||||
|
// Nothing may push the page sideways at this width.
|
||||||
|
const over = await page.evaluate(() =>
|
||||||
|
document.documentElement.scrollWidth - window.innerWidth);
|
||||||
|
expect(over).toBeLessThanOrEqual(0);
|
||||||
|
|
||||||
|
// One pane at a time: the list, then the item over it, then back.
|
||||||
|
await expect(page.locator('#detail')).not.toBeVisible();
|
||||||
|
await page.locator('.ep').first().click();
|
||||||
|
await expect(page.locator('#detail')).toBeVisible();
|
||||||
|
await page.locator('#dback').click();
|
||||||
|
await expect(page.locator('#detail')).not.toBeVisible();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test('opening an item marks it read, and the toggle flips it back', async ({ page }) => {
|
||||||
|
const errors = [];
|
||||||
|
page.on('pageerror', e => errors.push(e.message));
|
||||||
|
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
|
||||||
|
await expect(row()).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
|
// Another test may have opened this item already, so start from a known state: the
|
||||||
|
// toggle in the text below flips it back -- which used to recurse until the stack blew.
|
||||||
|
await row().click();
|
||||||
|
await page.locator('#detail button[title="Mark unread"]').click();
|
||||||
|
await expect(row()).not.toHaveClass(/read/);
|
||||||
|
await expect(page.locator('#detail button[title="Mark read"]')).toBeVisible();
|
||||||
|
|
||||||
|
// Opening it is reading it.
|
||||||
|
await row().click();
|
||||||
|
await expect(row()).toHaveClass(/read/);
|
||||||
|
|
||||||
|
expect(errors).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the toolbar acts on the selected item', async ({ page }) => {
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
const row = () => page.locator('.ep', { hasText: 'Second Episode' });
|
||||||
|
await expect(row()).toBeVisible({ timeout: 20_000 });
|
||||||
|
// Nothing selected, nothing to act on.
|
||||||
|
await expect(page.locator('#tbRead')).toBeDisabled();
|
||||||
|
|
||||||
|
await row().click(); // opening it reads it
|
||||||
|
await expect(row()).toHaveClass(/read/);
|
||||||
|
await page.locator('#tbRead').click();
|
||||||
|
await expect(row()).not.toHaveClass(/read/);
|
||||||
|
|
||||||
|
await page.locator('#tbFlag').click();
|
||||||
|
await expect(row().locator('.fl')).toHaveClass(/on/);
|
||||||
|
await page.locator('#tbFlag').click(); // and back, so later tests see it unkept
|
||||||
|
await expect(row().locator('.fl')).not.toHaveClass(/on/);
|
||||||
|
|
||||||
|
// Second Episode is the one the daemon downloaded, so it plays from the toolbar.
|
||||||
|
await expect(page.locator('#tbPlay')).toBeEnabled();
|
||||||
|
await page.locator('#tbPlay').click();
|
||||||
|
await expect(page.locator('#player')).toBeVisible();
|
||||||
|
await page.locator('#pclose').click();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a second person has their own feeds and their own read state', async ({ browser }) => {
|
||||||
|
const { execFileSync } = require('child_process');
|
||||||
|
const setup = require('./global-setup');
|
||||||
|
const env = {
|
||||||
|
...process.env,
|
||||||
|
IPX_CONFIG: `${setup.root}/config/config.toml`,
|
||||||
|
IPX_DATA_DIR: `${setup.root}/data`,
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
execFileSync('./target/debug/ipx', ['user', 'add', 'sam'], { input: 'sampassword', env });
|
||||||
|
} catch (e) {
|
||||||
|
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
|
||||||
|
}
|
||||||
|
|
||||||
|
// A fresh context, so none of the admin's cookies come along.
|
||||||
|
const ctx = await browser.newContext();
|
||||||
|
const page = await ctx.newPage();
|
||||||
|
await page.goto('/login');
|
||||||
|
// The sign-in page shows the icon, so it has to load before anyone has signed in.
|
||||||
|
const icon = await page.request.get('/icon.png');
|
||||||
|
expect(icon.status()).toBe(200);
|
||||||
|
expect(icon.headers()['content-type']).toBe('image/png');
|
||||||
|
await page.locator('#name').fill('sam');
|
||||||
|
await page.locator('#pw').fill('sampassword');
|
||||||
|
await page.locator('button[type=submit]').click();
|
||||||
|
await expect(page.locator('#feedlist')).toBeVisible();
|
||||||
|
|
||||||
|
// Sam subscribes to nothing yet, so sees nothing -- the admin's feeds are not theirs.
|
||||||
|
await expect(page.locator('#feedlist')).toContainText('No feeds.');
|
||||||
|
await expect(page.locator('#prefs')).toBeHidden(); // not an admin
|
||||||
|
// Hiding the button is not the guard; the server is.
|
||||||
|
expect((await page.request.get('/api/users')).status()).toBe(403);
|
||||||
|
await expect(page.locator('#logs')).toBeHidden();
|
||||||
|
expect((await page.request.get('/api/logs')).status()).toBe(403);
|
||||||
|
|
||||||
|
// Subscribing to a feed the admin already has costs no second fetch: same feed, same
|
||||||
|
// files, but Sam's own read state.
|
||||||
|
await page.locator('#addFeed').click();
|
||||||
|
await page.locator('#nurl').fill('http://127.0.0.1:8792/show.xml');
|
||||||
|
await page.locator('#nsave').click();
|
||||||
|
await expect(page.locator('.feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
|
await page.locator('.feed', { hasText: 'Test Show' }).click();
|
||||||
|
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
|
||||||
|
// The admin read these earlier in this file; for Sam they are all still unread.
|
||||||
|
const rows = await page.locator('.ep').count();
|
||||||
|
await page.locator('.tabs button', { hasText: 'Unread' }).click();
|
||||||
|
await expect(page.locator('.ep')).toHaveCount(rows);
|
||||||
|
|
||||||
|
// Two people now share this feed, so the page says so and Delete is honest about it.
|
||||||
|
await expect(page.locator('#content .sub').first()).toContainText('shared with 1 other person');
|
||||||
|
|
||||||
|
await ctx.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('deleting a shared file warns that it is everyone\'s copy', async ({ page }) => {
|
||||||
|
// Admin and Sam both subscribe to Test Show by now, and the daemon downloaded a file.
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
await page.locator('.tabs button', { hasText: 'Downloaded' }).click();
|
||||||
|
const row = page.locator('.ep').first();
|
||||||
|
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||||
|
await row.click();
|
||||||
|
|
||||||
|
// An icon now; what it does, and to whom, is in its tooltip.
|
||||||
|
const del = page.locator('#files button[data-a="del"]');
|
||||||
|
await expect(del).toHaveAttribute('title', /^Delete for everyone \(shared with 1 other person/);
|
||||||
|
|
||||||
|
// Two prompts: the page's own, then the server's, because someone else has not played
|
||||||
|
// it. Accept the first, decline the second, and the file must survive.
|
||||||
|
const seen = [];
|
||||||
|
page.on('dialog', d => {
|
||||||
|
seen.push(d.message());
|
||||||
|
if (seen.length === 1) d.accept();
|
||||||
|
else d.dismiss();
|
||||||
|
});
|
||||||
|
await del.click();
|
||||||
|
await expect.poll(() => seen.length, { timeout: 10_000 }).toBe(2);
|
||||||
|
expect(seen[0]).toContain('shared with 1 other person');
|
||||||
|
expect(seen[1]).toContain('one copy of this file');
|
||||||
|
|
||||||
|
await page.reload();
|
||||||
|
await page.getByText('Test Show').click();
|
||||||
|
await page.locator('.tabs button', { hasText: 'Downloaded' }).click();
|
||||||
|
await expect(page.locator('.ep').first()).toBeVisible({ timeout: 20_000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
// Every row says "Admin" on its checkbox, so match the name exactly.
|
||||||
|
const userRow = (page, name) =>
|
||||||
|
page.locator('#modalCard [data-id]').filter({ has: page.locator('b', { hasText: new RegExp(`^${name}$`) }) });
|
||||||
|
|
||||||
|
async function openUsers(page) {
|
||||||
|
await page.locator('#prefs').click();
|
||||||
|
await page.locator('#gusers').click();
|
||||||
|
await expect(userRow(page, 'admin')).toBeVisible();
|
||||||
|
}
|
||||||
|
|
||||||
|
test('an admin adds someone, makes them an admin, and removes them', async ({ page }) => {
|
||||||
|
await openUsers(page);
|
||||||
|
await page.locator('#uname').fill('pat');
|
||||||
|
await page.locator('#upass').fill('patpassword');
|
||||||
|
await page.locator('#uadd').click();
|
||||||
|
const row = userRow(page, 'pat');
|
||||||
|
await expect(row).toBeVisible();
|
||||||
|
await expect(row.locator('[data-a="admin"]')).not.toBeChecked();
|
||||||
|
|
||||||
|
await row.locator('[data-a="admin"]').check();
|
||||||
|
// Not just the box: it has to have reached the database.
|
||||||
|
await expect.poll(async () =>
|
||||||
|
(await (await page.request.get('/api/users')).json()).find(u => u.name === 'pat')?.admin
|
||||||
|
).toBe(true);
|
||||||
|
|
||||||
|
page.once('dialog', d => d.accept());
|
||||||
|
await row.locator('[data-a="rm"]').click();
|
||||||
|
await expect(row).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the only admin cannot be demoted or removed', async ({ page }) => {
|
||||||
|
await openUsers(page);
|
||||||
|
await userRow(page, 'admin').locator('[data-a="admin"]').click();
|
||||||
|
await expect(page.locator('.toast.bad')).toContainText('only admin');
|
||||||
|
// Redrawn from the server, so the box is back.
|
||||||
|
await expect(userRow(page, 'admin').locator('[data-a="admin"]')).toBeChecked();
|
||||||
|
|
||||||
|
const me = (await (await page.request.get('/api/users')).json()).find(u => u.name === 'admin');
|
||||||
|
expect((await page.request.delete(`/api/users/${me.id}`)).status()).toBe(400);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Settings exports your OPML and imports a pasted one', async ({ page }) => {
|
||||||
|
await page.locator('#prefs').click();
|
||||||
|
const [dl] = await Promise.all([
|
||||||
|
page.waitForEvent('download'),
|
||||||
|
page.locator('#modalCard a[title="Export OPML"]').click(),
|
||||||
|
]);
|
||||||
|
expect(dl.suggestedFilename()).toBe('ipx-subscriptions.opml');
|
||||||
|
const out = require('fs').readFileSync(await dl.path(), 'utf8');
|
||||||
|
for (const f of ['show.xml', 'pics.xml', 'multi.xml', 'subs.opml']) expect(out).toContain(f);
|
||||||
|
// A feed from an OPML subscription comes back with the OPML itself, not on its own.
|
||||||
|
expect(out).not.toContain('other.xml');
|
||||||
|
|
||||||
|
// One feed new to everyone, one the admin already has.
|
||||||
|
await page.locator('#gopml').click();
|
||||||
|
await page.locator('#opmlText').fill('<opml version="2.0"><head><title>t</title></head><body>' +
|
||||||
|
'<outline text="Imported Show" xmlUrl="http://127.0.0.1:8792/imported.xml"/>' +
|
||||||
|
'<outline text="Test Show" xmlUrl="http://127.0.0.1:8792/show.xml"/></body></opml>');
|
||||||
|
await page.locator('#oimp').click();
|
||||||
|
await expect(page.locator('.toast', { hasText: 'Subscribed to' }))
|
||||||
|
.toHaveText('Subscribed to 1 feed(s), 1 you already had');
|
||||||
|
// Named from the OPML's id until the first scan reads the feed's own title.
|
||||||
|
await expect(page.locator('#feedlist .feed', { hasText: /Imported Show|imported-show/ }))
|
||||||
|
.toBeVisible({ timeout: 20_000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an uploaded OPML file imports, and a file that is not OPML is refused', async ({ page }) => {
|
||||||
|
await page.locator('#prefs').click();
|
||||||
|
await page.locator('#gopml').click();
|
||||||
|
const pick = page.locator('#opmlFile');
|
||||||
|
|
||||||
|
// An RSS feed is XML but not OPML: refused in the page, and the picker lets go of it.
|
||||||
|
await pick.setInputFiles({
|
||||||
|
name: 'feed.xml', mimeType: 'application/xml',
|
||||||
|
buffer: require('fs').readFileSync(require('path').join(__dirname, 'fixtures', 'show.xml')),
|
||||||
|
});
|
||||||
|
await page.locator('#oimp').click();
|
||||||
|
await expect(page.locator('.toast.bad', { hasText: 'feed.xml is not an OPML file' })).toBeVisible();
|
||||||
|
expect(await pick.evaluate(i => i.files.length)).toBe(0);
|
||||||
|
|
||||||
|
// Something that gets past the page's quick look is still refused by the server, untouched.
|
||||||
|
const sneaky = await page.request.post('/api/opml', {
|
||||||
|
data: { xml: '<rss version="2.0"><channel><title><opml></title></channel></rss>' },
|
||||||
|
});
|
||||||
|
expect(sneaky.status()).toBe(400);
|
||||||
|
expect(await sneaky.text()).toContain('not an OPML file');
|
||||||
|
|
||||||
|
// A real one. Multi Show is already the admin's, so it counts as already had.
|
||||||
|
await pick.setInputFiles({
|
||||||
|
name: 'subs.opml', mimeType: 'text/x-opml',
|
||||||
|
buffer: Buffer.from('<opml version="2.0"><head><title>t</title></head><body>' +
|
||||||
|
'<outline text="Multi Show" xmlUrl="http://127.0.0.1:8792/multi.xml"/></body></opml>'),
|
||||||
|
});
|
||||||
|
await page.locator('#oimp').click();
|
||||||
|
await expect(page.locator('.toast', { hasText: 'Subscribed to' }))
|
||||||
|
.toHaveText('Subscribed to 0 feed(s), 1 you already had');
|
||||||
|
await expect(page.locator('#modal.on')).toBeHidden();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an export from one account imports into another', async ({ page, browser }) => {
|
||||||
|
// Regression: import only added URLs the catalogue lacked and subscribed nobody, so importing
|
||||||
|
// the admin's export into a second account did nothing at all.
|
||||||
|
const { execFileSync } = require('child_process');
|
||||||
|
const setup = require('./global-setup');
|
||||||
|
const env = {
|
||||||
|
...process.env,
|
||||||
|
IPX_CONFIG: `${setup.root}/config/config.toml`,
|
||||||
|
IPX_DATA_DIR: `${setup.root}/data`,
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
execFileSync('./target/debug/ipx', ['user', 'add', 'opal'], { input: 'opalpassword', env });
|
||||||
|
} catch (e) {
|
||||||
|
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
|
||||||
|
}
|
||||||
|
const ctx = await browser.newContext();
|
||||||
|
const opal = await ctx.newPage();
|
||||||
|
await opal.goto('/login');
|
||||||
|
await opal.locator('#name').fill('opal');
|
||||||
|
await opal.locator('#pw').fill('opalpassword');
|
||||||
|
await opal.locator('button[type=submit]').click();
|
||||||
|
await expect(opal.locator('#feedlist')).toContainText('No feeds.');
|
||||||
|
|
||||||
|
// Export used to hand anyone the whole catalogue. Opal has nothing yet, so gets nothing.
|
||||||
|
const empty = await opal.request.get('/api/opml');
|
||||||
|
expect(empty.status()).toBe(200);
|
||||||
|
expect(await empty.text()).not.toContain('xmlUrl');
|
||||||
|
|
||||||
|
const urlsIn = xml => [...xml.matchAll(/xmlUrl="([^"]+)"/g)].map(m => m[1]).sort();
|
||||||
|
const exported = await (await page.request.get('/api/opml')).text(); // the admin's
|
||||||
|
const urls = urlsIn(exported);
|
||||||
|
expect(urls.length).toBeGreaterThan(2);
|
||||||
|
expect(await (await opal.request.post('/api/opml', { data: { xml: exported } })).json())
|
||||||
|
.toEqual({ added: urls.length, already: 0 });
|
||||||
|
expect(await (await opal.request.post('/api/opml', { data: { xml: exported } })).json())
|
||||||
|
.toEqual({ added: 0, already: urls.length });
|
||||||
|
|
||||||
|
await opal.reload();
|
||||||
|
await expect(opal.locator('.feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
|
// The round trip closes: opal's own export now lists what the admin's did.
|
||||||
|
expect(urlsIn(await (await opal.request.get('/api/opml')).text())).toEqual(urls);
|
||||||
|
await ctx.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('ipx import subscribes the admin, and ipx export writes the feeds out', async () => {
|
||||||
|
// Its own config and database. The CLI works in-process, and the suite's running daemon
|
||||||
|
// reads config.toml once at start, so it would not see what the CLI added anyway.
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
const { execFileSync } = require('child_process');
|
||||||
|
const setup = require('./global-setup');
|
||||||
|
const dir = path.join(setup.root, 'cli');
|
||||||
|
fs.rmSync(dir, { recursive: true, force: true });
|
||||||
|
fs.mkdirSync(path.join(dir, 'data'), { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(dir, 'config.toml'),
|
||||||
|
`[general]\ndownload_dir = "${dir}/downloads"\nsocket = "${dir}/ipx.sock"\n`);
|
||||||
|
const env = { ...process.env, IPX_CONFIG: path.join(dir, 'config.toml'), IPX_DATA_DIR: path.join(dir, 'data') };
|
||||||
|
const ipx = (args, input) => execFileSync('./target/debug/ipx', args, { env, input, encoding: 'utf8' });
|
||||||
|
|
||||||
|
ipx(['user', 'add', 'boss'], 'bosspassword'); // the first account is the admin
|
||||||
|
const opml = path.join(dir, 'in.opml');
|
||||||
|
fs.writeFileSync(opml, '<opml version="2.0"><head><title>t</title></head><body>' +
|
||||||
|
'<outline text="One" xmlUrl="http://127.0.0.1:8792/one.xml"/>' +
|
||||||
|
'<outline text="Two" xmlUrl="http://127.0.0.1:8792/two.xml"/></body></opml>');
|
||||||
|
expect(ipx(['import', opml])).toContain('subscribed boss to 2 feed(s); 0 already there');
|
||||||
|
expect(ipx(['import', opml])).toContain('subscribed boss to 0 feed(s); 2 already there');
|
||||||
|
|
||||||
|
const out = path.join(dir, 'out.opml');
|
||||||
|
ipx(['export', out]);
|
||||||
|
const xml = fs.readFileSync(out, 'utf8');
|
||||||
|
expect(xml).toContain('http://127.0.0.1:8792/one.xml');
|
||||||
|
expect(xml).toContain('http://127.0.0.1:8792/two.xml');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
|
||||||
|
const { execFileSync } = require('child_process');
|
||||||
|
const setup = require('./global-setup');
|
||||||
|
const env = {
|
||||||
|
...process.env,
|
||||||
|
IPX_CONFIG: `${setup.root}/config/config.toml`,
|
||||||
|
IPX_DATA_DIR: `${setup.root}/data`,
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
execFileSync('./target/debug/ipx', ['user', 'add', 'piper'], { input: 'piperpassword', env });
|
||||||
|
} catch (e) {
|
||||||
|
if (!String(e.stderr || e.stdout).includes('already exists')) throw e;
|
||||||
|
}
|
||||||
|
const ctx = await browser.newContext();
|
||||||
|
const piper = await ctx.newPage();
|
||||||
|
await piper.goto('/login');
|
||||||
|
await piper.locator('#name').fill('piper');
|
||||||
|
await piper.locator('#pw').fill('piperpassword');
|
||||||
|
await piper.locator('button[type=submit]').click();
|
||||||
|
await expect(piper.locator('#feedlist')).toContainText('No feeds.');
|
||||||
|
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||||
|
const offered = piper.locator('#popular .childrow');
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
|
// An OPML's own feeds ride on the OPML, and a key in a URL marks someone's paid feed.
|
||||||
|
await expect(offered.filter({ hasText: /Grouped Show|grouped-show/ })).toHaveCount(0);
|
||||||
|
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||||
|
|
||||||
|
// No URL reaches the page at all, so neither can a key, and the server holds the same line.
|
||||||
|
const listed = await (await piper.request.get('/api/popular')).text();
|
||||||
|
expect(listed).not.toContain('secret123');
|
||||||
|
expect(listed).not.toContain('.xml');
|
||||||
|
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
|
||||||
|
|
||||||
|
// Popular is the top ten of the directory, and the directory is every listed feed, A to Z.
|
||||||
|
const dir = await (await piper.request.get('/api/directory')).json();
|
||||||
|
const top = await (await piper.request.get('/api/popular')).json();
|
||||||
|
const names = dir.map(p => (p.title || p.id).toLowerCase());
|
||||||
|
expect(names).toEqual([...names].sort());
|
||||||
|
expect(top.length).toBe(Math.min(10, dir.length));
|
||||||
|
expect(top.every(t => dir.some(d => d.id === t.id))).toBe(true);
|
||||||
|
expect(dir.map(p => p.id)).not.toContain('paid-show');
|
||||||
|
|
||||||
|
// Subscribe from the directory this time; the popular list shares the same rows.
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||||
|
await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible();
|
||||||
|
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||||
|
|
||||||
|
const row = async () =>
|
||||||
|
(await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
|
||||||
|
const before = await row();
|
||||||
|
expect(before.subscribed).toBe(false);
|
||||||
|
await offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
|
||||||
|
await expect(piper.locator('#feedlist .feed', { hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||||
|
|
||||||
|
// Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
|
||||||
|
expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
|
||||||
|
await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
|
||||||
|
|
||||||
|
// All Subscriptions is every item from piper's feeds and only those: the admin's Picture
|
||||||
|
// Blog is not among them.
|
||||||
|
await piper.locator('#feedlist .place', { hasText: 'All Subscriptions' }).click();
|
||||||
|
const first = piper.locator('.ep', { hasText: 'First Episode' });
|
||||||
|
await expect(first).toBeVisible({ timeout: 20_000 });
|
||||||
|
await expect(first.locator('.fd')).toHaveText('Test Show');
|
||||||
|
await expect(piper.locator('.ep', { hasText: 'An Article' })).toHaveCount(0);
|
||||||
|
await expect(piper.locator('#count')).toContainText('All Subscriptions:');
|
||||||
|
await ctx.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('adding a feed scans it straight away', async ({ page }) => {
|
||||||
|
await page.locator('#addFeed').click();
|
||||||
|
await page.locator('#nurl').fill('http://127.0.0.1:8792/fresh.xml');
|
||||||
|
await page.locator('#nsave').click();
|
||||||
|
// Nobody pressed Scan. The scheduler's tick is a minute, so this is the add scanning it.
|
||||||
|
await expect(page.locator('.ep', { hasText: 'Fresh Ep' })).toBeVisible({ timeout: 10_000 });
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a deleted file looks as if it was never downloaded', async ({ page }) => {
|
||||||
|
// Other people subscribe to Picture Blog by now, so both prompts come; take them.
|
||||||
|
page.on('dialog', d => d.accept());
|
||||||
|
await page.locator('.feed', { hasText: 'Picture Blog' }).click();
|
||||||
|
const row = page.locator('.ep', { hasText: 'An Article' });
|
||||||
|
await expect(row).toBeVisible({ timeout: 20_000 });
|
||||||
|
await row.click();
|
||||||
|
await page.locator('#files button[data-a="del"]').click();
|
||||||
|
|
||||||
|
// No "reaped", no chip at all: just the way to get it again.
|
||||||
|
await expect(row.locator('.kind.here')).toHaveCount(0);
|
||||||
|
await expect(row).not.toContainText(/reaped/i);
|
||||||
|
await row.click();
|
||||||
|
await expect(page.locator('#files')).not.toContainText(/reaped/i);
|
||||||
|
await expect(page.locator('#files button[title="Download to the server"]')).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('one action, one icon: the toolbar, the page and every dialog agree', async ({ page }) => {
|
||||||
|
const icon = loc => loc.locator('svg path').first().getAttribute('d');
|
||||||
|
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
|
||||||
|
|
||||||
|
// Unsubscribe is a minus in the toolbar and the feed header, never the x that closes things.
|
||||||
|
expect(await icon(page.locator('#content .acts [data-a="rm"]'))).toBe(await icon(page.locator('#tbRemove')));
|
||||||
|
|
||||||
|
// The toolbar's read and keep show the selected item's state, as its own buttons do, and follow
|
||||||
|
// a change made from the toolbar.
|
||||||
|
await page.locator('.ep').first().click();
|
||||||
|
const pair = async a => [await icon(page.locator(a === 'read' ? '#tbRead' : '#tbFlag')),
|
||||||
|
await icon(page.locator(`#detail [data-a="${a}"]`))];
|
||||||
|
for (const a of ['read', 'flag']) { const [tb, own] = await pair(a); expect(tb).toBe(own); }
|
||||||
|
const [kept] = await pair('flag');
|
||||||
|
await page.locator('#tbFlag').click();
|
||||||
|
await expect.poll(async () => { const [tb, own] = await pair('flag'); return tb === own && tb !== kept; }).toBe(true);
|
||||||
|
await page.locator('#tbFlag').click(); // leave it as it was
|
||||||
|
await expect.poll(async () => (await pair('flag'))[0]).toBe(kept);
|
||||||
|
|
||||||
|
// Every button in every dialog is an icon with its words in the tooltip.
|
||||||
|
const dialogs = [
|
||||||
|
() => page.locator('#addFeed').click(),
|
||||||
|
() => page.locator('#prefs').click(),
|
||||||
|
async () => { await page.locator('#prefs').click(); await page.locator('#gusers').click(); },
|
||||||
|
async () => { await page.locator('#prefs').click(); await page.locator('#gopml').click(); },
|
||||||
|
() => page.locator('#logs').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="settings"]').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="dl"]').click(),
|
||||||
|
() => page.locator('#content .acts [data-a="rm"]').click(),
|
||||||
|
];
|
||||||
|
for (const open of dialogs) {
|
||||||
|
await open();
|
||||||
|
const btns = page.locator('#modalCard .btn');
|
||||||
|
await expect(btns.first()).toBeVisible();
|
||||||
|
for (const b of await btns.all()) {
|
||||||
|
await expect(b.locator('svg')).toHaveCount(1);
|
||||||
|
await expect(b).toHaveAttribute('title', /\S/);
|
||||||
|
}
|
||||||
|
await page.keyboard.press('Escape');
|
||||||
|
await expect(page.locator('#modal.on')).toBeHidden();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('All Subscriptions marks everything read, across every feed', async ({ page }) => {
|
||||||
|
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
|
||||||
|
await all.click();
|
||||||
|
// Earlier tests read things; make sure something is unread. Opening an item reads it, and
|
||||||
|
// its own button makes it unread again.
|
||||||
|
await page.locator('.ep').first().click();
|
||||||
|
await page.locator('#detail [data-a="read"][title="Mark unread"]').click();
|
||||||
|
await expect(all.locator('.badge')).not.toHaveText('0');
|
||||||
|
|
||||||
|
page.once('dialog', d => d.accept());
|
||||||
|
await page.locator('#content .acts [data-a="readall"]').click();
|
||||||
|
await expect(all.locator('.badge')).toHaveText('0');
|
||||||
|
await page.locator('.tabs button', { hasText: 'Unread' }).click();
|
||||||
|
await expect(page.locator('.ep')).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the item table sorts by any column, both ways, and remembers', async ({ page }) => {
|
||||||
|
const all = page.locator('#feedlist .place', { hasText: 'All Subscriptions' });
|
||||||
|
await all.click();
|
||||||
|
const head = k => page.locator(`#list .ephead [data-sort="${k}"]`);
|
||||||
|
const titles = () => page.locator('#eps .ep .t').allTextContents();
|
||||||
|
// Byte order on lower case, which is what SQLite gives for lower(...).
|
||||||
|
const cmp = (a, b) => (a.toLowerCase() < b.toLowerCase() ? -1 : a.toLowerCase() > b.toLowerCase() ? 1 : 0);
|
||||||
|
const sorted = (t, dir) => JSON.stringify(t) === JSON.stringify([...t].sort((a, b) => cmp(a, b) * dir));
|
||||||
|
await expect(page.locator('#eps .ep').nth(2)).toBeVisible({ timeout: 20_000 });
|
||||||
|
expect(new Set(await titles()).size).toBeGreaterThan(2); // or both orders would prove nothing
|
||||||
|
|
||||||
|
await expect(head('title')).toHaveText('Title');
|
||||||
|
await head('title').click();
|
||||||
|
await expect.poll(async () => sorted(await titles(), 1)).toBe(true);
|
||||||
|
await head('title').click();
|
||||||
|
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
|
||||||
|
|
||||||
|
// Kept across a reload.
|
||||||
|
await page.reload();
|
||||||
|
await all.click();
|
||||||
|
await expect(head('title').locator('.arr.desc')).toBeVisible();
|
||||||
|
await expect.poll(async () => sorted(await titles(), -1)).toBe(true);
|
||||||
|
|
||||||
|
// Size has its own column; the file column is just what the file is.
|
||||||
|
await expect(page.locator('#eps .ep .size', { hasText: /\d/ }).first()).toBeVisible();
|
||||||
|
await expect(page.locator('#eps .ep .file', { hasText: /\d/ })).toHaveCount(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('play in the Files pane plays once, in the player bar', async ({ page }) => {
|
||||||
|
// Regression: the pane had an <audio> of its own, and playing it started the player bar too,
|
||||||
|
// so the same file played twice at once.
|
||||||
|
await page.locator('#feedlist .feed', { hasText: 'Test Show' }).first().click();
|
||||||
|
await page.locator('.ep', { has: page.locator('.kind.here') }).first().click();
|
||||||
|
await page.locator('#files [data-a="play"]').click();
|
||||||
|
await expect(page.locator('#player')).toBeVisible();
|
||||||
|
await expect(page.locator('audio')).toHaveCount(1); // the player bar's, and nothing else
|
||||||
|
await page.locator('#pclose').click();
|
||||||
|
});
|
||||||
5
tests/ui/fixtures/aardvark.xml
Normal file
5
tests/ui/fixtures/aardvark.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Aardvark Radio</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Inside the OPML, and first in it and alphabetically.</description>
|
||||||
|
<item><title>Aardvark Ep</title><guid>aa-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
BIN
tests/ui/fixtures/art.jpg
Normal file
BIN
tests/ui/fixtures/art.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 3.0 KiB |
BIN
tests/ui/fixtures/art2.jpg
Normal file
BIN
tests/ui/fixtures/art2.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.0 KiB |
BIN
tests/ui/fixtures/ep1.mp3
Normal file
BIN
tests/ui/fixtures/ep1.mp3
Normal file
Binary file not shown.
BIN
tests/ui/fixtures/ep2.mp3
Normal file
BIN
tests/ui/fixtures/ep2.mp3
Normal file
Binary file not shown.
5
tests/ui/fixtures/fresh.xml
Normal file
5
tests/ui/fixtures/fresh.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Fresh Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Added in the browser suite, and scanned by adding it.</description>
|
||||||
|
<item><title>Fresh Ep</title><guid>fresh-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
5
tests/ui/fixtures/imported.xml
Normal file
5
tests/ui/fixtures/imported.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Imported Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Only ever arrives through an OPML import.</description>
|
||||||
|
<item><title>Imported Ep</title><guid>imp-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
9
tests/ui/fixtures/multi.xml
Normal file
9
tests/ui/fixtures/multi.xml
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Multi Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>An item with more than one file.</description>
|
||||||
|
<item><title>Two Files</title><guid>mu-1</guid>
|
||||||
|
<pubDate>Mon, 01 Sep 2026 10:00:00 +0000</pubDate>
|
||||||
|
<description>Audio and a picture.</description>
|
||||||
|
<enclosure url="http://127.0.0.1:8792/ep2.mp3" length="40000" type="audio/mpeg"/>
|
||||||
|
<enclosure url="http://127.0.0.1:8792/art2.jpg" length="3020" type="image/jpeg"/>
|
||||||
|
</item></channel></rss>
|
||||||
5
tests/ui/fixtures/other.xml
Normal file
5
tests/ui/fixtures/other.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Grouped Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Inside the OPML.</description>
|
||||||
|
<item><title>Grouped Ep</title><guid>g-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
5
tests/ui/fixtures/paid.xml
Normal file
5
tests/ui/fixtures/paid.xml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Paid Show</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>Subscribed with a key in its URL, so it must never be offered to anyone else.</description>
|
||||||
|
<item><title>Paid Ep</title><guid>paid-1</guid><description>x</description></item>
|
||||||
|
</channel></rss>
|
||||||
8
tests/ui/fixtures/pics.xml
Normal file
8
tests/ui/fixtures/pics.xml
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0"><channel><title>Picture Blog</title><link>http://127.0.0.1:8792/</link>
|
||||||
|
<description>A text blog whose entries carry a header image, as Substack does.</description>
|
||||||
|
<item><title>An Article</title><guid>pic-1</guid>
|
||||||
|
<pubDate>Mon, 01 Sep 2026 10:00:00 +0000</pubDate>
|
||||||
|
<description><p>Words, not audio.</p></description>
|
||||||
|
<enclosure url="http://127.0.0.1:8792/art.jpg" length="3020" type="image/jpeg"/></item>
|
||||||
|
</channel></rss>
|
||||||
19
tests/ui/fixtures/serve.js
Normal file
19
tests/ui/fixtures/serve.js
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
// Serves the fixture feeds so the daemon under test has something real to scan.
|
||||||
|
const http = require('http');
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
|
||||||
|
const dir = __dirname;
|
||||||
|
const port = Number(process.env.FIXTURE_PORT || 8792);
|
||||||
|
|
||||||
|
http.createServer((req, res) => {
|
||||||
|
const name = decodeURIComponent(req.url.split('?')[0].replace(/^\//, '')) || 'index';
|
||||||
|
const file = path.join(dir, path.basename(name));
|
||||||
|
fs.readFile(file, (err, body) => {
|
||||||
|
if (err) { res.writeHead(404).end('no'); return; }
|
||||||
|
const type = file.endsWith('.mp3') ? 'audio/mpeg'
|
||||||
|
: file.endsWith('.opml') ? 'text/x-opml' : 'application/xml';
|
||||||
|
res.writeHead(200, { 'content-type': type, 'content-length': body.length });
|
||||||
|
res.end(body);
|
||||||
|
});
|
||||||
|
}).listen(port, '127.0.0.1', () => console.log(`fixtures on ${port}`));
|
||||||
15
tests/ui/fixtures/show.xml
Normal file
15
tests/ui/fixtures/show.xml
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
|
||||||
|
<channel><title>Test Show</title><link>http://127.0.0.1:8792/</link><description>A fixture feed.</description>
|
||||||
|
<itunes:image href="http://127.0.0.1:8792/art.png"/>
|
||||||
|
<item><title>First Episode</title><guid>ui-1</guid>
|
||||||
|
<pubDate>Mon, 01 Sep 2026 10:00:00 +0000</pubDate>
|
||||||
|
<description><p>Show notes for the first one.</p></description>
|
||||||
|
<itunes:duration>1830</itunes:duration><itunes:season>1</itunes:season><itunes:episode>1</itunes:episode>
|
||||||
|
<enclosure url="http://127.0.0.1:8792/ep1.mp3" length="40000" type="audio/mpeg"/></item>
|
||||||
|
<item><title>Second Episode</title><guid>ui-2</guid>
|
||||||
|
<pubDate>Mon, 08 Sep 2026 10:00:00 +0000</pubDate>
|
||||||
|
<description>Notes for the second.</description>
|
||||||
|
<itunes:duration>900</itunes:duration>
|
||||||
|
<enclosure url="http://127.0.0.1:8792/ep1.mp3?2" length="40000" type="audio/mpeg"/></item>
|
||||||
|
</channel></rss>
|
||||||
5
tests/ui/fixtures/subs.opml
Normal file
5
tests/ui/fixtures/subs.opml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
<opml version="2.0"><head><title>Test Subscriptions</title></head>
|
||||||
|
<body><outline text="Folder">
|
||||||
|
<outline type="rss" text="Aardvark Radio" xmlUrl="http://127.0.0.1:8792/aardvark.xml"/>
|
||||||
|
<outline type="rss" text="Grouped Show" xmlUrl="http://127.0.0.1:8792/other.xml"/>
|
||||||
|
</outline></body></opml>
|
||||||
66
tests/ui/global-setup.js
Normal file
66
tests/ui/global-setup.js
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
// Builds a scratch config and data dir so the browser tests drive a real daemon with
|
||||||
|
// known feeds, rather than whatever happens to be on the machine.
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
const os = require('os');
|
||||||
|
|
||||||
|
const root = path.join(os.tmpdir(), 'ipx-ui-test');
|
||||||
|
const TOKEN = 'testtokentesttokentesttoken12345'; // fixed, so tests need not scrape a log
|
||||||
|
|
||||||
|
// Called from playwright.config.js at load time, NOT as globalSetup: Playwright starts
|
||||||
|
// webServer *before* globalSetup, so a config written there does not exist yet when the
|
||||||
|
// daemon launches -- it would fall back to the real config and fight the live daemon.
|
||||||
|
// Playwright imports this config again in every worker process, so prepare() runs more
|
||||||
|
// than once per suite. Wiping on the second call deleted the data directory out from under
|
||||||
|
// the running daemon: it kept serving from the unlinked inode, while anything else opening
|
||||||
|
// that path -- the CLI, a query -- got a brand new empty database and disagreed with it.
|
||||||
|
function prepare() {
|
||||||
|
// Only the process that launches the run may wipe. A worker gets TEST_WORKER_INDEX.
|
||||||
|
if (process.env.TEST_WORKER_INDEX !== undefined || process.env.PW_WORKER_INDEX !== undefined) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
fs.rmSync(root, { recursive: true, force: true });
|
||||||
|
for (const d of ['config', 'data', 'downloads']) {
|
||||||
|
fs.mkdirSync(path.join(root, d), { recursive: true });
|
||||||
|
}
|
||||||
|
fs.writeFileSync(path.join(root, 'config', 'config.toml'), `
|
||||||
|
[general]
|
||||||
|
download_dir = "${path.join(root, 'downloads')}"
|
||||||
|
socket = "${path.join(root, 'ipx.sock')}"
|
||||||
|
schedule = "every 60m"
|
||||||
|
max_new_per_check = 1
|
||||||
|
|
||||||
|
[torrent]
|
||||||
|
enabled = false
|
||||||
|
|
||||||
|
[web]
|
||||||
|
enabled = true
|
||||||
|
bind = "127.0.0.1:8791"
|
||||||
|
token = "${TOKEN}"
|
||||||
|
|
||||||
|
[feeds.test-show]
|
||||||
|
url = "http://127.0.0.1:8792/show.xml"
|
||||||
|
auto_download = true
|
||||||
|
|
||||||
|
# Downloads its image, so the UI has a file that is not playable to deal with.
|
||||||
|
[feeds.picture-blog]
|
||||||
|
url = "http://127.0.0.1:8792/pics.xml"
|
||||||
|
auto_download = true
|
||||||
|
media_types = ["image"]
|
||||||
|
|
||||||
|
[feeds.multi-show]
|
||||||
|
url = "http://127.0.0.1:8792/multi.xml"
|
||||||
|
auto_download = true
|
||||||
|
|
||||||
|
[feeds.test-subscriptions]
|
||||||
|
url = "http://127.0.0.1:8792/subs.opml"
|
||||||
|
auto_download = false
|
||||||
|
|
||||||
|
# A key in its URL, like a Patreon feed: someone's paid subscription, never offered to others.
|
||||||
|
[feeds.paid-show]
|
||||||
|
url = "http://127.0.0.1:8792/paid.xml?auth=secret123"
|
||||||
|
auto_download = false
|
||||||
|
`);
|
||||||
|
}
|
||||||
|
|
||||||
|
module.exports = { prepare, root, TOKEN };
|
||||||
1950
web/index.html
Normal file
1950
web/index.html
Normal file
File diff suppressed because it is too large
Load Diff
BIN
web/ipodderx-icon.jpg
Normal file
BIN
web/ipodderx-icon.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 9.3 KiB |
BIN
web/ipodderx-icon.png
Normal file
BIN
web/ipodderx-icon.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 17 KiB |
93
web/login.html
Normal file
93
web/login.html
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
<title>Sign in — iPodderX</title>
|
||||||
|
<link rel="icon" href="/icon.png">
|
||||||
|
<style>
|
||||||
|
:root {
|
||||||
|
--bg:#0e131b; /* the screen's navy (#314B74), taken right down */
|
||||||
|
--panel:#151c27;
|
||||||
|
--panel2:#1c2431;
|
||||||
|
--raise:#25303f;
|
||||||
|
--line:#2c3849;
|
||||||
|
--fg:#f5f5f5; /* #F5F5F5 device highlight */
|
||||||
|
--dim:#95a0b1; /* #95A0B1 straight from the icon's blue-grey */
|
||||||
|
--faint:#7a8799; /* lifted from the icon ramp until it clears AA at small sizes */
|
||||||
|
--accent:#92b2e6; /* #92B2E6 the screen blue */
|
||||||
|
--accent2:#f49e2c; /* #F49E2C the EQ bars */
|
||||||
|
--ink:#0e131b; /* text on an accent fill */
|
||||||
|
--good:#6fbf8b;
|
||||||
|
--warn:#f49e2c; /* the amber doubles as the pending colour */
|
||||||
|
--bad:#e2705f;
|
||||||
|
--shadow:0 8px 28px rgba(6,10,16,.55);
|
||||||
|
--r:10px;
|
||||||
|
}
|
||||||
|
:root[data-theme="light"] {
|
||||||
|
--bg:#f2f4f7;
|
||||||
|
--panel:#ffffff; /* #FFFFFF device body */
|
||||||
|
--panel2:#e9edf3;
|
||||||
|
--raise:#dde3ec;
|
||||||
|
--line:#d6d6d6; /* #D6D6D6 device edge */
|
||||||
|
--fg:#1a1a1a; /* #1A1A1A icon outline */
|
||||||
|
--dim:#606060; /* #606060 */
|
||||||
|
--faint:#767676; /* between the icon's #929292 and #606060, to clear AA */
|
||||||
|
--accent:#2d5391; /* #2D5391 the deep screen blue reads better on white */
|
||||||
|
--accent2:#b06f10;
|
||||||
|
--ink:#ffffff;
|
||||||
|
--good:#2f7d4f;
|
||||||
|
--warn:#b06f10;
|
||||||
|
--bad:#b3402f;
|
||||||
|
--shadow:0 8px 28px rgba(45,83,145,.14);
|
||||||
|
}
|
||||||
|
*{box-sizing:border-box}
|
||||||
|
html,body{height:100%}
|
||||||
|
body{
|
||||||
|
margin:0;display:grid;place-items:center;background:var(--bg);color:var(--fg);
|
||||||
|
font:14.5px/1.55 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif;padding:20px;
|
||||||
|
}
|
||||||
|
form{
|
||||||
|
width:min(360px,100%);background:var(--panel);border:1px solid var(--line);
|
||||||
|
border-radius:14px;padding:22px;box-shadow:var(--shadow);
|
||||||
|
}
|
||||||
|
/* The one place the 2004 icon is shown at the size it was drawn for. */
|
||||||
|
.brand{display:flex;flex-direction:column;align-items:center;gap:6px;margin-bottom:20px}
|
||||||
|
.brand img{width:96px;height:auto}
|
||||||
|
h1{font-size:21px;margin:0;font-weight:650;letter-spacing:-.01em}
|
||||||
|
label{display:block;font-size:12px;color:var(--dim);margin:0 0 4px}
|
||||||
|
input{
|
||||||
|
width:100%;background:var(--bg);border:1px solid var(--line);color:var(--fg);
|
||||||
|
border-radius:8px;padding:9px 11px;font:inherit;margin-bottom:13px;
|
||||||
|
}
|
||||||
|
input:focus{outline:0;border-color:var(--accent)}
|
||||||
|
button{
|
||||||
|
width:100%;font:inherit;font-weight:600;cursor:pointer;color:var(--ink);
|
||||||
|
background:var(--accent);border:0;border-radius:8px;padding:10px;
|
||||||
|
}
|
||||||
|
.msg{color:var(--bad);font-size:13px;min-height:19px;margin:10px 0 0;text-align:center}
|
||||||
|
button:focus-visible{outline:2px solid var(--accent);outline-offset:2px}
|
||||||
|
</style>
|
||||||
|
|
||||||
|
<form id="f">
|
||||||
|
<div class="brand"><img src="/icon.png" alt=""><h1>iPodderX</h1></div>
|
||||||
|
<label for="name">Name</label>
|
||||||
|
<input id="name" name="name" autocomplete="username" autofocus required>
|
||||||
|
<label for="pw">Password</label>
|
||||||
|
<input id="pw" name="password" type="password" autocomplete="current-password" required>
|
||||||
|
<button type="submit">Sign in</button>
|
||||||
|
<p class="msg" id="msg"></p>
|
||||||
|
</form>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
document.getElementById('f').onsubmit = async e => {
|
||||||
|
e.preventDefault();
|
||||||
|
const msg = document.getElementById('msg');
|
||||||
|
msg.textContent = '';
|
||||||
|
const r = await fetch('/api/login', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
name: document.getElementById('name').value,
|
||||||
|
password: document.getElementById('pw').value,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
if (r.ok) location.href = '/';
|
||||||
|
else msg.textContent = await r.text() || 'Sign in failed';
|
||||||
|
};
|
||||||
|
</script>
|
||||||
Reference in New Issue
Block a user