Documentation: docs/, a changelog, and CLAUDE.md
PROGRESS.md becomes CHANGELOG.md with the finished step lists moved to an appendix. The README is an overview pointing at docs/: configuration, cli, users, sso (refreshed for accounts and admin-only settings), and architecture. CLAUDE.md collects what working on this code actually requires -- pkill -x not -f, the page being compiled in, the dead columns on entries, the Playwright worker that deleted its own database. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
This commit is contained in:
@@ -1,58 +1,29 @@
|
||||
# Progress
|
||||
# Changelog
|
||||
|
||||
Running record of what has actually landed. Newest entry first.
|
||||
The full design and step list live in the plan file at
|
||||
`/config/.claude/plans/i-want-to-create-playful-quiche.md`.
|
||||
What has actually landed in ipodderx-rs, newest first. Each entry says what changed and, where it
|
||||
matters, what was wrong before -- the reasoning is the point, not the diff.
|
||||
|
||||
## Build order
|
||||
See [README.md](README.md) for what the thing is, and [docs/](docs/) for how to run it.
|
||||
|
||||
- [x] **1. Repo skeleton** — git init (`main`), `cargo init --name ipx`, deps pinned, LICENSE,
|
||||
README, this file.
|
||||
- [x] **2. `config.rs` + `db.rs`** — TOML config structs + SQLite schema.
|
||||
- [x] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries.
|
||||
- [x] **4. `download.rs`** — downloads, filters, dedupe.
|
||||
- [x] **5. `retention.rs`** — oldest-first quota + age reaper.
|
||||
- [x] **6. `ipc.rs` + daemon** — UDS JSON-lines server, TTL scheduler, CLI-proxies-to-daemon.
|
||||
- [x] **7. `torrent.rs`** — librqbit, seed to ratio/time, stall abort. (swarm download unverified —
|
||||
see the step 7 entry)
|
||||
- [x] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd units, README.
|
||||
## 2026-09-11 — Documentation
|
||||
|
||||
### Phase 2 — web front end
|
||||
`PROGRESS.md` became this changelog; the finished step lists moved to an appendix. The README is now
|
||||
an overview that points at `docs/`: [configuration](docs/configuration.md) (every key, path and
|
||||
environment variable), [cli](docs/cli.md), [users](docs/users.md) (what several people share),
|
||||
[sso](docs/sso.md) (refreshed for accounts and the admin-only settings), and
|
||||
[architecture](docs/architecture.md) (modules, the scan, the schema, the socket protocol, the HTTP
|
||||
API). `CLAUDE.md` collects what someone working on the code needs to know and has already been
|
||||
learned the hard way here -- `pkill -x` not `-f`, the page being compiled in, the dead columns on
|
||||
`entries`, the Playwright worker that deleted its own database.
|
||||
|
||||
Decided with Ray: axum serving plain HTML/JS (no WASM toolchain), running **inside the daemon**
|
||||
process so it reads SQLite and the event bus directly, LAN-bindable with a shared token.
|
||||
---
|
||||
|
||||
- [x] **9. Config hot-reload + web skeleton.** `Ctx.cfg` becomes `RwLock<Arc<Config>>` so the UI can
|
||||
edit feeds without a daemon restart. `[web]` config section (enabled/bind/token, token
|
||||
auto-generated and saved on first run). axum server started by `ipx daemon`, token checked by
|
||||
middleware, `?token=` sets a cookie so `<audio>` requests authenticate too.
|
||||
*Done when:* `ipx daemon` serves a page on the configured bind, and a wrong token gets 401.
|
||||
- [x] **10. Browsing.** `/api/feeds`, `/api/feeds/:id/entries`, entry detail. Descriptions are
|
||||
untrusted feed HTML — sanitized with `ammonia` before they reach the page.
|
||||
*Done when:* the Glass Cannon feed's 131 entries browse and read correctly.
|
||||
- [x] **11. Media actions.** Range-request audio streaming (`tower-http` ServeFile) so seeking
|
||||
works, download-on-demand for a pending enclosure, delete a file, mark read/flagged.
|
||||
*Done when:* an episode plays and seeks in a browser, and delete reaps the row.
|
||||
- [x] **12. Feed configuration.** Add/remove feeds and edit folder, keywords, allow_explicit,
|
||||
auto_download, max_new_per_check from the UI, written back to config.toml and hot-reloaded.
|
||||
*Done when:* flipping allow_explicit in the UI takes effect on the next scan with no restart.
|
||||
- [x] **13. Live progress + polish.** SSE from the existing broadcast bus so downloads show live.
|
||||
README section, screenshot-free usage notes.
|
||||
*Done when:* starting a fetch from the UI shows progress advancing without a reload.
|
||||
## 2026-09-11 — Pruning respects a star from anyone
|
||||
|
||||
Note: `read`/`flagged` finally get a writer here. Retention orders by them (see the step 5 entry),
|
||||
and until now nothing set them.
|
||||
|
||||
## Smoke tests
|
||||
|
||||
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
|
||||
2. `ipx fetch` again → no re-download, feed skipped for TTL.
|
||||
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
|
||||
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
|
||||
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
|
||||
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
|
||||
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
|
||||
`reaped`.
|
||||
Same defect as the reaper, one function along: `prune_entries` guarded on `entries.flagged`, which
|
||||
nothing has written since read state moved to `entry_state`. Starring a text item with no file would
|
||||
not have saved it from the age sweep. It follows the reaper's rule now, and takes orphaned read
|
||||
state with whatever it deletes.
|
||||
|
||||
---
|
||||
|
||||
@@ -173,34 +144,6 @@ who is signed in above the sidebar footer, with a sign-out, and a 401 sends the
|
||||
Nothing is per-user *yet*: everyone still sees the same feeds and read state. That is step B.
|
||||
|
||||
---
|
||||
|
||||
## Multi-user — the plan
|
||||
|
||||
Decided with Ray: **stay on SQLite** (Postgres was considered and dropped -- it is a deployment
|
||||
choice, not a capability one, and nothing here contends for writes). Sign-in is either a local
|
||||
username and password or the Cloudflare Zero Trust that already fronts `ipodderx.sdf1.net`, which
|
||||
puts the authenticated identity in `Cf-Access-Authenticated-User-Email`. Feeds, items and files are **shared**; read state and subscriptions are **per user**.
|
||||
|
||||
The point of sharing: two people subscribed to the same show cost one fetch, one parse, and one file
|
||||
on disk. `enclosures.url` is already globally UNIQUE, so the file half is nearly free.
|
||||
|
||||
- [x] **A. Users, sessions, sign-in.** `users` + `sessions` tables, Argon2 hashing, session cookie,
|
||||
`ipx user add|list|passwd|rm`. A proxy header (`trusted_header` in `[web]`) signs in and
|
||||
optionally creates a user -- honoured only from a `trusted_proxies` address, so a LAN client
|
||||
cannot simply assert it. The existing shared token keeps working and resolves to the admin, so
|
||||
the healthcheck and any scripts survive. Login page for direct access.
|
||||
- [x] **B. Per-user read state.** `entry_state(user_id, feed_id, guid, read, flagged, position)`;
|
||||
the current columns on `entries` migrate into the first user's rows. Unread counts, filters and
|
||||
playback position all become per user.
|
||||
- [x] **C. Per-user subscriptions.** `subscriptions(user_id, feed_id)`. config.toml stays the feed
|
||||
catalogue; the UI lists only what you subscribe to. Adding a feed someone else already has costs
|
||||
nothing. A feed nobody subscribes to stops being scanned but keeps its files.
|
||||
- [x] **D. One file, many users.** Auto-download when *any* subscriber wants it; retention never
|
||||
deletes a file another user has starred or not yet played; deleting a download says so when
|
||||
someone else still has it.
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-10 — Items, not episodes
|
||||
|
||||
Half the library is text feeds, so the UI no longer calls everything an episode: counts, the search
|
||||
@@ -1251,3 +1194,37 @@ Gotcha worth keeping: three feature names in the plan were wrong against current
|
||||
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
|
||||
is the fix. Whole tree is rustls-only now, no C TLS dependency.
|
||||
Next: step 2 — `config.rs` + `db.rs`. (done)
|
||||
|
||||
---
|
||||
|
||||
## How this was built
|
||||
|
||||
Three planned phases, all complete. The dated entries above are the record of each.
|
||||
|
||||
**The engine (steps 1-8)** — repo skeleton with pinned dependencies; `config.rs` + `db.rs` (TOML
|
||||
config, SQLite schema); `feed.rs` (conditional GET, RSS-then-Atom parsing); `download.rs`
|
||||
(streaming downloads, filters, URL dedupe); `retention.rs` (oldest-first quota and age reaper);
|
||||
`ipc.rs` plus the daemon (Unix-socket JSON lines, TTL scheduler, CLI proxying to a running daemon);
|
||||
`torrent.rs` (librqbit, seeding to a ratio or a time, stall abort); OPML import/export and polish.
|
||||
|
||||
**The web front end (steps 9-13)** — axum serving plain HTML and JS, no WASM toolchain, running
|
||||
*inside* the daemon so it reads SQLite and the event bus directly: config hot-reload and the `[web]`
|
||||
section; browsing feeds and entries with `ammonia`-sanitized show notes; media actions (Range
|
||||
streaming, download on demand, delete, read/starred); feed configuration written back to
|
||||
config.toml; live progress over SSE from the existing broadcast bus.
|
||||
|
||||
**Multi-user (steps A-D)** — accounts, sessions and sign-in (Argon2id, session cookie, a trusted
|
||||
proxy header for Cloudflare Zero Trust); per-user read state; per-user subscriptions; and the rules
|
||||
that follow from one file serving everyone. SQLite stayed: Postgres was considered and dropped, as
|
||||
nothing here contends for writes.
|
||||
|
||||
### The live checks each step had to pass
|
||||
|
||||
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
|
||||
2. `ipx fetch` again → no re-download, feed skipped for TTL.
|
||||
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
|
||||
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
|
||||
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
|
||||
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
|
||||
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
|
||||
`reaped`.
|
||||
119
CLAUDE.md
Normal file
119
CLAUDE.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# Working on ipodderx-rs
|
||||
|
||||
Notes for whoever picks this up next. Read [docs/architecture.md](docs/architecture.md) for how the
|
||||
thing is built; this file is about working on it without repeating mistakes that have already been
|
||||
made here.
|
||||
|
||||
## Where things are
|
||||
|
||||
The live install on this machine:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Binary | `/config/.cargo/bin/ipx` |
|
||||
| Config | `/config/.config/ipx/config.toml` |
|
||||
| Database | `/config/.local/share/ipx/state.db` |
|
||||
| Downloads | `/mnt/user/audio/ipx` |
|
||||
| Web UI | `0.0.0.0:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel |
|
||||
|
||||
Deploying a change is: build, stop, copy, start.
|
||||
|
||||
```sh
|
||||
cargo build --release
|
||||
pkill -x ipx; sleep 2
|
||||
cp target/release/ipx /config/.cargo/bin/ipx
|
||||
setsid nohup /config/.cargo/bin/ipx --config /config/.config/ipx/config.toml daemon \
|
||||
>/tmp/ipx.log 2>&1 </dev/null &
|
||||
```
|
||||
|
||||
**`pkill -x ipx`, never `pkill -f ipx`.** `-f` matches the shell running the command and kills the
|
||||
session (exit 144). This has happened more than once.
|
||||
|
||||
The daemon is not supervised: it will not survive a reboot. `contrib/` has a systemd unit nobody
|
||||
has installed.
|
||||
|
||||
## Before you touch the page
|
||||
|
||||
`web/index.html` is `include_str!`d into the binary, so **every page change needs a rebuild** before
|
||||
it is visible. It is one file: markup, CSS and script.
|
||||
|
||||
After any edit to it:
|
||||
|
||||
```sh
|
||||
node tests/page-smoke.js
|
||||
```
|
||||
|
||||
That loads the script against a stub DOM and checks every selector it wires at load actually
|
||||
exists. It exists because a patch once anchored on a deleted function, `String.replace` silently
|
||||
matched nothing, and the whole UI died with a `ReferenceError` while every server-side test passed.
|
||||
|
||||
Patching that file by guessing an anchor string has failed repeatedly. Read the exact block first
|
||||
(`sed -n 'START,ENDp'`), match it verbatim, and assert the replacement happened rather than hoping.
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
cargo test # ~51 tests: parsing, filters, retention, schedules, SQL, per-user state
|
||||
node tests/page-smoke.js
|
||||
npx playwright test # 16 browser tests against a real daemon on fixture feeds
|
||||
```
|
||||
|
||||
Things about the browser suite that have cost time:
|
||||
|
||||
* It starts **its own daemon and database** under `/tmp/ipx-ui-test`, wiped once per run. Playwright
|
||||
re-imports the config in every worker, so `prepare()` guards on `TEST_WORKER_INDEX` — without
|
||||
that guard a worker deleted the database out from under the running daemon, which then kept
|
||||
serving from the unlinked inode while everything else saw an empty file.
|
||||
* Tests **share that daemon and run in order**. A test that opens an item marks it read and changes
|
||||
what later tests see. Write assertions that do not depend on what ran before, or normalise the
|
||||
state first.
|
||||
* Fixture feeds must not share an enclosure URL, because `enclosures.url` is globally unique and
|
||||
whichever feed is scanned first claims it.
|
||||
* `webServer` starts **before** `globalSetup`, which is why the fixture config is written at
|
||||
config-load time instead.
|
||||
|
||||
Non-trivial logic leaves one runnable check behind. Pure functions (`merge_policy`, `pick`,
|
||||
`matches_keywords`, `parse_interval`) are the easiest place to put it.
|
||||
|
||||
## Things that are true and easy to get wrong
|
||||
|
||||
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
|
||||
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
|
||||
* **`entries.read`, `entries.flagged` and `entries.position` are dead columns.** Read state lives in
|
||||
`entry_state` per user. Two bugs have already come from queries still reading the old ones
|
||||
(retention, and the entry pruner) — grep before adding a third.
|
||||
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
|
||||
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
|
||||
derived and never written to config.
|
||||
* **One fetch serves everyone**, so scan policy is a union of subscribers' wants (`merge_policy`).
|
||||
Anyone wanting an item is enough to fetch it.
|
||||
* **The UI hiding a control is not enforcement.** Admin-only actions check `user.is_admin` in the
|
||||
handler and return `403`.
|
||||
* **A `tokio::select!` only races its branches at the point of selection.** A long download has to
|
||||
watch the shutdown channel itself; the daemon ignored SIGTERM for exactly this reason.
|
||||
* Only one daemon per socket. Removing the socket file defeats the guard and you get two daemons
|
||||
fighting over the database, with the stale one still holding the port.
|
||||
* `/api/settings` answering `200` does **not** mean the worker is alive — it is a different task.
|
||||
Probe the control socket (`ipx status`) to check that.
|
||||
|
||||
## House style
|
||||
|
||||
Comments explain **why**, not what. If a line looks odd, the comment says what went wrong without
|
||||
it. No emoji, no exclamation marks, no "obviously". Prose in the UI and docs is plain English and
|
||||
addressed to the person using it.
|
||||
|
||||
Every change gets an entry at the top of [CHANGELOG.md](CHANGELOG.md), dated, saying what landed and
|
||||
what was wrong before. That record has been more useful than the git log more than once.
|
||||
|
||||
Deliberate simplifications get a `ponytail:` comment naming the ceiling and the upgrade path, e.g.
|
||||
`// ponytail: global connection mutex, move to a pool if feed count makes it contend`.
|
||||
|
||||
## Known gaps
|
||||
|
||||
* No user administration in the web UI; `ipx user` on the box only.
|
||||
* Cloudflare's `Cf-Access-Jwt-Assertion` is not verified — ipx trusts the hop plus `trusted_proxies`
|
||||
(documented in [docs/sso.md](docs/sso.md)).
|
||||
* The Docker image predates multi-user; `docker compose build` before relying on it.
|
||||
* Downloads land root-owned; Unraid shares want `99:100`.
|
||||
* A feed's `<description>` subtitle is dropped whenever `content:encoded` exists, which loses
|
||||
Substack-style subtitles.
|
||||
251
README.md
251
README.md
@@ -1,14 +1,14 @@
|
||||
# ipodderx-rs
|
||||
|
||||
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent),
|
||||
files them into per-feed folders, and reaps old episodes to stay under a disk quota.
|
||||
Runs as a one-shot CLI or as a daemon with a Unix-socket JSON event stream for a UI to attach to.
|
||||
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent), files them
|
||||
into per-feed folders, and reaps old files to stay under a disk quota. Runs as a one-shot CLI or as
|
||||
a daemon with a web UI, serving any number of people from one copy of the data.
|
||||
|
||||
## Lineage
|
||||
|
||||
This is a modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the
|
||||
Python 2 engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), which was
|
||||
open-sourced under the MIT License in 2010.
|
||||
A modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python 2
|
||||
engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), open-sourced under the MIT
|
||||
License in 2010.
|
||||
|
||||
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
|
||||
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
|
||||
@@ -16,193 +16,71 @@ oldest-first disk quota reaper.
|
||||
|
||||
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
|
||||
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
|
||||
3DES-encrypted preferences, and the `printMSG` stdout protocol (replaced by a JSON-lines socket).
|
||||
3DES-encrypted preferences, and the `printMSG` stdout protocol -- replaced by a JSON-lines socket.
|
||||
|
||||
## Quick start
|
||||
|
||||
```sh
|
||||
cargo install --path .
|
||||
cargo build --release
|
||||
install -m755 target/release/ipx ~/.cargo/bin/
|
||||
|
||||
ipx add https://atp.fm/rss # names the feed from its own title
|
||||
ipx list
|
||||
ipx fetch # scan now
|
||||
ipx daemon # or run continuously, honouring each feed's <ttl>
|
||||
ipx add https://atp.fm/rss # subscribe
|
||||
ipx fetch # scan and download
|
||||
ipx daemon # scheduler, control socket and web UI
|
||||
```
|
||||
|
||||
Config lives at `~/.config/ipx/config.toml` (mode 0600, since it may hold feed passwords);
|
||||
state at `~/.local/share/ipx/state.db`. Override with `IPX_CONFIG` and `IPX_DATA_DIR`.
|
||||
Set `IPX_LOG=ipx=debug` for verbose logging on stderr.
|
||||
On first start with `[web] enabled = true` the daemon mints a token, writes it to config.toml and
|
||||
prints the URL to open. A database with no accounts starts with **admin / ipodderx** at `/login` --
|
||||
change it with `echo -n '<password>' | ipx user passwd admin`.
|
||||
|
||||
## Commands
|
||||
## Documentation
|
||||
|
||||
| command | what it does |
|
||||
| | |
|
||||
|---|---|
|
||||
| `ipx add <url> [--folder X] [--keywords a,b]` | subscribe; the id comes from the feed title |
|
||||
| `ipx rm <feed>` | unsubscribe; downloads and history are kept |
|
||||
| `ipx list` / `ipx status` | subscriptions and their state |
|
||||
| `ipx fetch [FEED] [--force]` | scan; `--force` ignores the TTL |
|
||||
| `ipx reap [--dry-run]` | run retention now |
|
||||
| `ipx import/export <file.opml>` | move subscriptions in or out |
|
||||
| `ipx daemon` | scheduler plus the control socket |
|
||||
| [docs/configuration.md](docs/configuration.md) | Every config key, paths, environment variables |
|
||||
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
|
||||
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
|
||||
| [docs/sso.md](docs/sso.md) | Cloudflare Zero Trust or Authentik in front |
|
||||
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, socket, HTTP API |
|
||||
| [CHANGELOG.md](CHANGELOG.md) | What landed, and why |
|
||||
| [CLAUDE.md](CLAUDE.md) | Notes for anyone (or anything) working on the code |
|
||||
|
||||
Any command with a wire form probes the socket first: if a daemon is running it does the work,
|
||||
and the CLI just renders the events it streams back. `--local` forces in-process execution.
|
||||
## The web UI
|
||||
|
||||
## Configuration
|
||||
`ipx daemon` serves it in the same process, so it reads SQLite and the event bus directly.
|
||||
|
||||
```toml
|
||||
[general]
|
||||
download_dir = "~/Podcasts"
|
||||
socket = "/run/user/1000/ipx.sock" # default: $XDG_RUNTIME_DIR/ipx.sock
|
||||
interval_mins = 60 # default poll; a feed's own <ttl> wins when longer
|
||||
organize = "feed" # "feed" | "date"
|
||||
max_total_gb = 50 # 0 = unlimited
|
||||
max_age_days = 30 # 0 = keep forever
|
||||
max_new_per_check = 3 # per feed, per scan. 0 = unlimited (pulls whole back catalogues)
|
||||
media_types = ["audio", "video"] # what downloads automatically. Anything else is still
|
||||
# listed and can be fetched by hand -- blog feeds put article
|
||||
# images in enclosures. Empty takes everything.
|
||||
Feeds down the side; the selected feed's items across the top; the selected item's text and its
|
||||
enclosures below, which is where you play, download or delete them. The divider drags and its
|
||||
position is remembered. Playback serves Range requests, so seeking works. An OPML subscription is a
|
||||
collapsible folder whose page lists the feeds inside it.
|
||||
|
||||
[torrent]
|
||||
enabled = true
|
||||
seed_ratio = 1.0 # stop seeding at this ratio ...
|
||||
seed_time_mins = 60 # ... or after this long, whichever comes first
|
||||
port_range = "6881-6889"
|
||||
stall_mins = 30 # give up on a torrent making no progress
|
||||
An item may carry several enclosures; all of them appear below, and anything that is not audio or
|
||||
video gets a View link rather than a player -- the publisher's copy until it is downloaded, the
|
||||
local one after. Opening an item marks it read. Show notes are untrusted feed HTML, sanitized with
|
||||
`ammonia` server-side before they reach the page.
|
||||
|
||||
[feeds.atp]
|
||||
url = "https://atp.fm/rss"
|
||||
folder = "Accidental Tech Podcast" # default: the feed title
|
||||
keywords = ["deep dive"] # OR across keywords, AND within one
|
||||
allow_explicit = false
|
||||
auto_download = true
|
||||
max_new_per_check = 3 # the rest wait for the next scan
|
||||
username = "ray" # optional HTTP basic auth
|
||||
password_env = "IPX_ATP_PASS" # or a literal `password`
|
||||
```
|
||||
The **Log** button shows the running daemon live in four tabs: *Daemon I/O* is the control protocol
|
||||
itself, every command in and event out; *Scans* is feed and download activity; *HTTP* is web
|
||||
requests; *All* is everything, with level and text filters and a copy button. It reads a ring buffer
|
||||
held in the process, not a file, so it works the same under Docker.
|
||||
|
||||
Retention keeps files that are `flagged` in the database, and deletes read episodes before unread
|
||||
ones, oldest first.
|
||||
|
||||
## Socket protocol
|
||||
|
||||
Newline-delimited JSON over a Unix socket, both directions.
|
||||
|
||||
```sh
|
||||
$ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
||||
{"ev":"feed_start","feed":"atp"}
|
||||
{"ev":"progress","feed":"atp","url":"...","file":"ep1.mp3","done":8192,"total":3000000}
|
||||
{"ev":"download_done","feed":"atp","url":"...","path":"...","bytes":3000000}
|
||||
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
|
||||
{"ev":"scan_done","feeds":1}
|
||||
```
|
||||
|
||||
Commands: `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `status`.
|
||||
Events: `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
||||
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
||||
`scan_done`, `reap_done` and `status` are terminal -- a client that asked for work stops there.
|
||||
|
||||
Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy
|
||||
daemon also sees that daemon's other work.
|
||||
It is plain HTTP. On a LAN bind the token and everything else cross the network in the clear, and a
|
||||
feed URL can itself carry a credential. Put TLS in front of it if that matters.
|
||||
|
||||
## OPML
|
||||
|
||||
Two different things, both supported:
|
||||
|
||||
**Importing and exporting** a file copies subscriptions in or out once — `ipx import subs.opml`,
|
||||
`ipx export subs.opml`, or the OPML button in the UI.
|
||||
**Importing and exporting** a file copies subscriptions in or out once: `ipx import subs.opml`,
|
||||
`ipx export subs.opml`, or Settings → Subscriptions in the UI.
|
||||
|
||||
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
|
||||
other feed; every scan re-reads it and keeps your feed list in step. An OPML is recognised by its
|
||||
content, so a URL without a `.opml` extension still works.
|
||||
other feed; every scan re-reads it and keeps your list in step. The feeds inside are not written to
|
||||
config.toml -- the OPML is the source of truth, so they are re-derived each scan and held in the
|
||||
database. They show as a folder, download into one nested folder, and inherit the subscription's
|
||||
settings until you change one, which gives it its own entry.
|
||||
|
||||
The feeds inside it are **not written to `config.toml`** — the OPML is the source of truth, so they
|
||||
are re-derived each scan and held in the database. Your config keeps only what you chose. They show
|
||||
as a collapsible folder in the sidebar, download into one nested folder, and newly listed ones are
|
||||
scanned in the same run rather than waiting for the next interval. They inherit the subscription's
|
||||
settings; change anything on one and it gets its own config entry from then on.
|
||||
|
||||
When a feed drops out of the OPML upstream:
|
||||
|
||||
| it has downloads | what happens |
|
||||
|---|---|
|
||||
| no | unsubscribed and removed from the config |
|
||||
| yes | kept, flagged in the UI as no longer listed |
|
||||
|
||||
A downloaded file is never left behind with nothing explaining where it came from.
|
||||
|
||||
## Web UI
|
||||
|
||||
```toml
|
||||
[web]
|
||||
enabled = true
|
||||
bind = "0.0.0.0:8080" # 127.0.0.1:8080 by default
|
||||
token = "" # generated and written back on first run
|
||||
```
|
||||
|
||||
`ipx daemon` then serves it in the same process (`ipx daemon --web ADDR` overrides the bind for one
|
||||
run). On first start it mints a token, saves it to config.toml, and prints the URL to open:
|
||||
|
||||
```
|
||||
web ui token generated. Open:
|
||||
http://0.0.0.0:8080/?token=1f4c…
|
||||
```
|
||||
|
||||
`?token=` sets a year-long cookie, so you only paste it once per browser. Everything is behind that
|
||||
token, including `/media/...` — a cookie rather than a header precisely because an `<audio>` element
|
||||
cannot send headers.
|
||||
|
||||
Browse feeds, read show notes, play episodes in the browser (Range requests are served, so seeking
|
||||
works), download or delete individual files, mark episodes read or flag them to keep, and edit a
|
||||
feed's folder/keywords/explicit/auto-download/limit settings. Config edits are written to
|
||||
config.toml and hot-reloaded — no daemon restart.
|
||||
|
||||
Show notes are feed-supplied HTML from an untrusted source; they are sanitized with `ammonia`
|
||||
server-side before they reach the page.
|
||||
|
||||
**It is plain HTTP.** On a LAN bind, the token and everything else crosses the network in the
|
||||
clear — and a feed URL can itself contain a credential (Patreon's, for one, carries an auth token).
|
||||
Put it behind a reverse proxy with TLS if that matters to you.
|
||||
|
||||
## Layout
|
||||
|
||||
Feeds down the side; the selected feed's episodes across the top; the selected episode's text and
|
||||
its enclosures in the pane below, which is where you play, download or delete them. The divider
|
||||
between the two panes drags and the position is remembered. An OPML subscription's page instead
|
||||
lists the feeds inside it.
|
||||
|
||||
An item may carry several enclosures. All of them appear in the pane below; the row summarises the
|
||||
one you would act on and notes how many others there are. Anything that is not audio or video gets a
|
||||
View link — opening the publisher's copy, or the local one once downloaded — rather than a player.
|
||||
|
||||
## Log view
|
||||
|
||||
The **Log** button in the sidebar shows the running daemon's output live, in four tabs:
|
||||
**Daemon I/O** is the control protocol itself — every command in and every event out, as JSON;
|
||||
**Scans** is feed and download activity; **HTTP** is web requests; **All** is everything: feed scans, downloads,
|
||||
torrent activity and every HTTP request, with level and text filters and a copy button. It reads a
|
||||
2000-line ring buffer held inside the process (`/api/logs`), not a file — so it works the same under
|
||||
Docker, where logs go to stdout and there is no file to tail. The buffer keeps `debug` even when the terminal does not, so protocol traffic and routine
|
||||
skips are there without making stderr unreadable — `IPX_UI_LOG` changes what it captures and
|
||||
`IPX_LOG` what reaches stderr.
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
cargo test # the server: parsing, filters, retention, schedules, SQL
|
||||
node tests/page-smoke.js # the page script loads without throwing
|
||||
npx playwright test # a real browser against a real daemon
|
||||
```
|
||||
|
||||
The Rust tests cover the server and the smoke test catches a script that fails to load, but neither
|
||||
can see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which
|
||||
is what has actually slipped through. The Playwright suite drives a headless browser against a
|
||||
daemon started on fixture feeds, and each test maps to a bug that reached a user: the page rendering
|
||||
empty, a dead theme toggle, settings not persisting, episode metadata, filter tabs, the feed URL
|
||||
field and its copy button, the log tabs, and OPML folders.
|
||||
|
||||
`npm install` gets the test runner; the browser itself comes from
|
||||
`npx playwright install --with-deps chromium` (in `install.sh`).
|
||||
When a feed drops out of the OPML upstream, it is unsubscribed and removed -- unless it has
|
||||
downloads, in which case it is kept and flagged in the UI as no longer listed. A downloaded file is
|
||||
never left behind with nothing explaining where it came from.
|
||||
|
||||
## Docker
|
||||
|
||||
@@ -212,27 +90,26 @@ docker compose logs -f ipx # the token is printed on first start
|
||||
```
|
||||
|
||||
`docker-compose.yml` mounts `./config`, `./data` and a downloads directory, publishes 8099 for the
|
||||
UI and 6881 (TCP **and** UDP — DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
|
||||
land owned the way Unraid shares expect. On first start the entrypoint writes a config bound to
|
||||
`0.0.0.0`, since a container's loopback is not reachable from outside it, and prints the URL with
|
||||
its generated token.
|
||||
|
||||
The healthcheck runs `ipx status`, which goes through the control socket to the command worker — so
|
||||
it catches a daemon that is alive but wedged, not merely one that has died.
|
||||
UI and 6881 (TCP **and** UDP -- DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
|
||||
land owned the way Unraid shares expect. The healthcheck runs `ipx status` through the control
|
||||
socket, so it catches a daemon that is alive but wedged rather than merely one that has died.
|
||||
|
||||
## Running it as a service
|
||||
|
||||
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
|
||||
rather run periodic scans with no daemon (in which case there is no socket for a UI to attach to).
|
||||
rather run periodic scans with no daemon -- in which case there is no socket for a UI to attach to.
|
||||
|
||||
## Tests
|
||||
|
||||
```sh
|
||||
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
|
||||
node tests/page-smoke.js # the page script loads without throwing
|
||||
npx playwright test # a real browser against a real daemon
|
||||
```
|
||||
|
||||
`npm install` gets the test runner; the browser comes from
|
||||
`npx playwright install --with-deps chromium` (in `install.sh`).
|
||||
|
||||
## License
|
||||
|
||||
MIT. See [LICENSE](LICENSE).
|
||||
|
||||
## Signing in
|
||||
|
||||
A new database starts with **admin / ipodderx** at `/login`; change it with
|
||||
`echo -n <password> | ipx user passwd admin`. Accounts are managed with `ipx user add|list|passwd|rm`.
|
||||
|
||||
To let Cloudflare Zero Trust or Authentik do the authenticating instead, see
|
||||
[docs/sso.md](docs/sso.md).
|
||||
|
||||
134
docs/architecture.md
Normal file
134
docs/architecture.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# How it works
|
||||
|
||||
One binary, `ipx`. `ipx daemon` runs three things in one process: a scheduler, a Unix-socket
|
||||
control server, and the web UI. Everything else is a CLI that either does the work itself or hands
|
||||
it to a running daemon.
|
||||
|
||||
## Modules
|
||||
|
||||
| File | Responsibility | What it replaced in the Python |
|
||||
|---|---|---|
|
||||
| `src/main.rs` | CLI, dispatch, scan loop, download policy | `iPXAgent.py` |
|
||||
| `src/config.rs` | TOML load/save, `General`/`Feed`/`Web`, intervals, slugs | `iPXSettings.py`, `feeds.plist` |
|
||||
| `src/db.rs` | SQLite schema, migrations, every query | `.ipxd` plists, `history.dat`, `qmcache.dat` |
|
||||
| `src/feed.rs` | Conditional GET, RSS/Atom/OPML parsing | `FeedData.__getFeed/__getEntries` |
|
||||
| `src/download.rs` | Streaming download, naming, type sniffing, placement | `iPXDownloader.getFile` |
|
||||
| `src/torrent.rs` | librqbit session, seeding limits, stall abort | vendored BitTorrent 4.2.1 |
|
||||
| `src/retention.rs` | Quota and age sweeps | `iPXQuotaManager.py` |
|
||||
| `src/ipc.rs` | Event and command types, the socket server | `printMSG` on stdout |
|
||||
| `src/auth.rs` | Argon2id hashing, session tokens, header names | — |
|
||||
| `src/web.rs` | axum: HTTP API, auth, SSE, media streaming | — |
|
||||
| `src/logbuf.rs` | Ring buffer behind the UI's Log view | — |
|
||||
| `web/index.html` | The whole front end, `include_str!`d into the binary | — |
|
||||
|
||||
The page is compiled in, so **editing `web/index.html` needs a rebuild**.
|
||||
|
||||
## A scan
|
||||
|
||||
1. Skip the feed unless `last_checked + max(schedule, ttl)` has passed (`--force` ignores this).
|
||||
2. Conditional GET with the stored `ETag` / `Last-Modified`. `304` ends it there.
|
||||
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are
|
||||
re-derived into the database each scan, never written to config.toml.
|
||||
4. Record entries. A changed title or description flips the item back to unread.
|
||||
5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
|
||||
original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
|
||||
6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
|
||||
a reason.
|
||||
7. Download what is still pending, newest first, up to the per-scan cap. A `.torrent` body goes to
|
||||
the torrent path whatever its advertised type; an HTML body is a failed download — a login wall
|
||||
or an error page — and is deleted.
|
||||
|
||||
## Data model
|
||||
|
||||
```
|
||||
feeds id, url, title, image, etag, last_modified, last_checked, ttl_mins,
|
||||
last_error, orphaned, group_id, managed
|
||||
entries feed_id, guid, title, link, published, description, first_seen,
|
||||
image, duration, episode, season PK (feed_id, guid)
|
||||
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
|
||||
bytes_done, downloaded_at, last_error
|
||||
users id, name, pass_hash, is_admin, created
|
||||
sessions token, user_id, created, seen
|
||||
subscriptions user_id, feed_id, keywords, auto_download, allow_explicit,
|
||||
max_new_per_check, created PK (user_id, feed_id)
|
||||
entry_state user_id, feed_id, guid, read, flagged, position
|
||||
PK (user_id, feed_id, guid)
|
||||
```
|
||||
|
||||
`entries` still has `read`, `flagged` and `position` columns from before accounts existed. They are
|
||||
**dead** — the migration copied them into `entry_state` and nothing reads them now. Anything found
|
||||
querying them is a bug; two were.
|
||||
|
||||
Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in
|
||||
`migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the
|
||||
same path as `Db::open`, so a migration-only column cannot pass tests while missing in production.
|
||||
|
||||
## Control socket
|
||||
|
||||
Newline-delimited JSON, both directions, over `$XDG_RUNTIME_DIR/ipx.sock`.
|
||||
|
||||
```sh
|
||||
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
||||
{"ev":"feed_start","feed":"atp"}
|
||||
{"ev":"progress","feed":"atp","enclosure":42,"file":"ep1.mp3","done":8192,"total":3000000}
|
||||
{"ev":"download_done","feed":"atp","enclosure":42,"path":"…","bytes":3000000}
|
||||
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
|
||||
{"ev":"scan_done","feeds":1}
|
||||
```
|
||||
|
||||
**Commands** — `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `download`
|
||||
(`enclosure`), `status`.
|
||||
|
||||
**Events** — `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
||||
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
||||
`scan_done`, `reap_done` and `status` are terminal: a client that asked for work stops reading
|
||||
there.
|
||||
|
||||
Progress carries the enclosure id, without which a UI cannot tell one download from another and
|
||||
ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast,
|
||||
so a client attached to a busy daemon also sees that daemon's other work.
|
||||
|
||||
Inside the process the same events go over a `tokio::broadcast`; commands arrive on an `mpsc` and
|
||||
are handled by a single worker, so nothing races over the same download. Shutdown is a `watch`
|
||||
channel raced *inside* each job — `tokio::select!` only races branches at the point of selection,
|
||||
so a long download had to be able to notice the signal itself.
|
||||
|
||||
## HTTP API
|
||||
|
||||
Everything below `/api` needs a signed-in user; the browser gets a redirect to `/login`, anything
|
||||
else a `401`.
|
||||
|
||||
| Route | |
|
||||
|---|---|
|
||||
| `GET /` | the app |
|
||||
| `GET /login`, `POST /api/login`, `POST /api/logout`, `GET /api/me` | sign-in |
|
||||
| `GET /api/feeds`, `POST /api/feeds` | your subscriptions; subscribe |
|
||||
| `PATCH /api/feeds/{id}`, `DELETE /api/feeds/{id}` | your settings or (admin) the feed's; unsubscribe |
|
||||
| `GET /api/feeds/{id}/entries` | paged, filtered, searchable |
|
||||
| `POST /api/feeds/{id}/read-all`, `POST /api/feeds/{id}/download-latest` | |
|
||||
| `POST /api/entries/{feed}/{guid}/flags`, `…/position` | your read, starred, position |
|
||||
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
|
||||
| `POST /api/fetch`, `GET /api/opml`, `POST /api/opml` | |
|
||||
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
|
||||
| `GET /api/events` | SSE, the same broadcast the socket carries |
|
||||
| `GET /api/logs` | the ring buffer, with a sequence cursor |
|
||||
| `GET /media/{id}` | the file, with Range support so seeking works |
|
||||
|
||||
Show notes are feed-supplied HTML from an untrusted source, sanitized with `ammonia` server-side
|
||||
before they reach the page.
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
cargo test # parsing, filters, retention, schedules, SQL, per-user isolation
|
||||
node tests/page-smoke.js # the page script loads and every selector it wires at load exists
|
||||
npx playwright test # a real browser against a real daemon on fixture feeds
|
||||
```
|
||||
|
||||
The Rust tests cannot see a wrong selector, a handler that runs and does nothing, or a page that
|
||||
renders empty — which is what has actually reached users. Each Playwright case maps to a bug that
|
||||
did.
|
||||
|
||||
The suite starts its own daemon and database under `/tmp/ipx-ui-test`, wiped once per run. Tests
|
||||
share that daemon and run in order, so a test that marks something read changes what later tests
|
||||
see — make assertions that do not depend on earlier ones.
|
||||
91
docs/cli.md
Normal file
91
docs/cli.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# Command line
|
||||
|
||||
```
|
||||
ipx [--config PATH] [--local] <command>
|
||||
```
|
||||
|
||||
Every command that has a wire form probes the control socket first: if a daemon is running, the
|
||||
daemon does the work and the CLI just renders the events it streams back. That is deliberate — two
|
||||
processes must never download the same thing. `--local` forces the work to happen in-process.
|
||||
|
||||
| Command | What it does |
|
||||
|---|---|
|
||||
| `ipx list` | Subscriptions and their state |
|
||||
| `ipx status` | Counts: feeds, pending, downloaded |
|
||||
| `ipx fetch [FEED] [--force]` | Scan everything, or one feed. `--force` ignores the TTL |
|
||||
| `ipx add <url> [--folder X] [--keywords a,b]` | Subscribe; the id comes from the feed title |
|
||||
| `ipx rm <feed>` | Unsubscribe; downloads and history are kept |
|
||||
| `ipx import <file.opml>` / `ipx export <file.opml>` | Move subscriptions in or out |
|
||||
| `ipx reap [--dry-run]` | Run retention now |
|
||||
| `ipx user <add\|list\|passwd\|rm>` | Accounts for the web UI |
|
||||
| `ipx daemon [--web ADDR]` | Scheduler, control socket and web UI |
|
||||
|
||||
## Accounts
|
||||
|
||||
Passwords are read from **stdin**, so they miss the shell history and any `ps` listing.
|
||||
|
||||
```sh
|
||||
echo -n 'a good password' | ipx user add ray # local account
|
||||
ipx user add ray@example.com --no-password # signs in through the proxy only
|
||||
echo -n 'a good password' | ipx user passwd admin # change a password
|
||||
ipx user list # who exists, and how each signs in
|
||||
ipx user rm sam # account, subscriptions and read state
|
||||
```
|
||||
|
||||
The first account created is an admin; later ones are ordinary users. A database with no accounts
|
||||
at all gets **admin / ipodderx** on the next daemon start, announced in the log — change it.
|
||||
|
||||
To avoid even the command line, read it interactively:
|
||||
|
||||
```sh
|
||||
read -s PW && echo -n "$PW" | ipx user passwd admin
|
||||
```
|
||||
|
||||
## Scanning
|
||||
|
||||
```sh
|
||||
ipx fetch # everything due
|
||||
ipx fetch atp --force # one feed, ignoring its TTL and schedule
|
||||
```
|
||||
|
||||
A scan: conditional GET (`If-None-Match` / `If-Modified-Since`), parse, record new entries, apply
|
||||
the filters, then download up to the per-scan cap, newest first. A feed nothing has changed in
|
||||
answers `304` and costs one request.
|
||||
|
||||
## Retention
|
||||
|
||||
```sh
|
||||
ipx reap --dry-run # what would go, oldest first
|
||||
ipx reap # actually delete
|
||||
```
|
||||
|
||||
Files are deleted to get back under `max_total_gb`, oldest first, and items past `max_age_days`
|
||||
with no file are pruned from the database. **Starred by anyone keeps a file**, and one only counts
|
||||
as read when everyone subscribed has read it. The enclosure row survives as `reaped`, which is what
|
||||
stops the next scan fetching it again.
|
||||
|
||||
## The daemon
|
||||
|
||||
```sh
|
||||
ipx daemon # scheduler + socket + web UI
|
||||
ipx daemon --web 0.0.0.0:8099 # override the configured bind for one run
|
||||
```
|
||||
|
||||
One daemon per socket; a second refuses to start rather than fight over the database. It shuts down
|
||||
cleanly on SIGTERM, including mid-download.
|
||||
|
||||
To kill it, match the binary exactly:
|
||||
|
||||
```sh
|
||||
pkill -x ipx
|
||||
```
|
||||
|
||||
`pkill -f ipx` matches the shell running the command too, and kills your own session.
|
||||
|
||||
## Talking to it directly
|
||||
|
||||
```sh
|
||||
printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
||||
```
|
||||
|
||||
See [architecture.md](architecture.md#control-socket) for the protocol.
|
||||
118
docs/configuration.md
Normal file
118
docs/configuration.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# Configuration
|
||||
|
||||
One TOML file, read at startup and re-read whenever the web UI writes to it — most changes take
|
||||
effect without a restart. Default location `$XDG_CONFIG_HOME/ipx/config.toml`
|
||||
(`~/.config/ipx/config.toml`), overridden with `--config` or `$IPX_CONFIG`.
|
||||
|
||||
| What | Where | Override |
|
||||
|---|---|---|
|
||||
| Config | `~/.config/ipx/config.toml` | `--config`, `$IPX_CONFIG` |
|
||||
| Database | `~/.local/share/ipx/state.db` | `$IPX_DATA_DIR` |
|
||||
| Control socket | `$XDG_RUNTIME_DIR/ipx.sock` | `[general] socket` |
|
||||
| Downloads | `[general] download_dir` | — |
|
||||
|
||||
`~` is expanded in paths. The database is SQLite in WAL mode; back it up by copying `state.db`
|
||||
while the daemon is stopped, or with `sqlite3 state.db .backup`.
|
||||
|
||||
## `[general]`
|
||||
|
||||
```toml
|
||||
[general]
|
||||
download_dir = "~/Podcasts"
|
||||
socket = "/run/user/1000/ipx.sock"
|
||||
schedule = "every 1h" # "every 30m", "every 4h", "2d", "90" (minutes)
|
||||
organize = "feed" # "feed" | "date"
|
||||
max_total_gb = 50 # 0 = unlimited
|
||||
max_age_days = 30 # 0 = keep forever
|
||||
max_new_per_check = 3 # per feed, per scan. 0 = unlimited
|
||||
media_types = ["audio", "video"]
|
||||
```
|
||||
|
||||
* **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
|
||||
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
|
||||
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
|
||||
* **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
|
||||
pad. Starred items are never deleted, and a file only counts as read once every subscriber has
|
||||
read it. `0` disables it entirely.
|
||||
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
|
||||
Starred ones stay. `0` disables it.
|
||||
* **`max_new_per_check`** — the cap that stops a new subscription pulling a whole back catalogue.
|
||||
`0` means unlimited, which is rarely what you want: subscribing to an OPML of 80 feeds with no cap
|
||||
fetched 216 files and 22 GB in one scan.
|
||||
* **`media_types`** — top-level MIME types taken automatically. Anything else is still listed and
|
||||
can be fetched by hand; blog feeds put each article's header image in an `<enclosure>`, and
|
||||
without this the disk fills with artwork. Empty takes everything.
|
||||
|
||||
`interval_mins` from older configs is still read, and `schedule` supersedes it.
|
||||
|
||||
## `[torrent]`
|
||||
|
||||
```toml
|
||||
[torrent]
|
||||
enabled = true
|
||||
seed_ratio = 1.0 # stop seeding at this ratio ...
|
||||
seed_time_mins = 60 # ... or after this long, whichever comes first
|
||||
port_range = "6881-6889"
|
||||
stall_mins = 30 # give up on a torrent making no progress
|
||||
```
|
||||
|
||||
A `.torrent` body is handed to the torrent path whatever MIME type it was advertised as. Torrents
|
||||
run on their own tasks (two at a time) so a slow swarm never blocks a scan.
|
||||
|
||||
## `[web]`
|
||||
|
||||
```toml
|
||||
[web]
|
||||
enabled = true
|
||||
bind = "0.0.0.0:8099" # 127.0.0.1:8080 by default
|
||||
token = "" # generated and saved on first run
|
||||
trusted_header = "" # e.g. "Cf-Access-Authenticated-User-Email"
|
||||
trusted_proxies = ["127.0.0.1", "::1"]
|
||||
auto_create_users = true
|
||||
session_days = 30
|
||||
```
|
||||
|
||||
* **`token`** — the shared secret, which signs in as the **admin**. `?token=…` sets a cookie, so
|
||||
you paste it once per browser. It is what the Docker healthcheck and any scripts use.
|
||||
* **`trusted_header`** — a header naming the signed-in user, set by whatever fronts ipx. Empty
|
||||
disables that path. See [sso.md](sso.md).
|
||||
* **`trusted_proxies`** — addresses allowed to assert that header, and the entire security boundary
|
||||
for it. Name the proxy, never a subnet.
|
||||
* **`auto_create_users`** — create an account the first time the proxy vouches for a new name.
|
||||
* **`session_days`** — sign a session out after this long without a request.
|
||||
|
||||
It is plain HTTP. On a LAN bind everything crosses the network in the clear — and a feed URL can
|
||||
itself carry a credential. Put TLS in front of it if that matters.
|
||||
|
||||
## `[feeds.<id>]`
|
||||
|
||||
The table key is the feed id: stable, human-readable, and used in paths and the API. `ipx add`
|
||||
derives it from the feed title.
|
||||
|
||||
```toml
|
||||
[feeds.atp]
|
||||
url = "https://atp.fm/rss"
|
||||
folder = "Accidental Tech Podcast" # default: the feed title
|
||||
schedule = "every 6h" # overrides [general] for this feed
|
||||
media_types = ["audio"] # overrides [general] for this feed
|
||||
username = "ray" # HTTP basic auth
|
||||
password_env = "IPX_ATP_PASS" # preferred over a literal `password`
|
||||
```
|
||||
|
||||
With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
|
||||
`max_new_per_check` live on each person's subscription in the database**, not here — the values in
|
||||
config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
|
||||
and are the same for everyone. See [users.md](users.md).
|
||||
|
||||
Feeds derived from a subscribed OPML are **not** written here: the OPML is the source of truth and
|
||||
they are re-derived on every scan. Editing one in the UI promotes it to a real config entry.
|
||||
|
||||
## Environment
|
||||
|
||||
| Variable | Effect |
|
||||
|---|---|
|
||||
| `IPX_CONFIG` | Config file path |
|
||||
| `IPX_DATA_DIR` | Directory holding `state.db` |
|
||||
| `IPX_LOG` | What reaches stderr (`ipx=debug`, `ipx::scan=debug`, …) |
|
||||
| `IPX_UI_LOG` | What the in-process log buffer captures for the UI's Log view |
|
||||
| `http_proxy` / `https_proxy` | Honoured for feed and enclosure fetches |
|
||||
26
docs/sso.md
26
docs/sso.md
@@ -35,7 +35,15 @@ Restart the daemon after editing. Accounts made this way have **no password**: t
|
||||
arrive through the proxy. `ipx user list` marks them `proxy only`.
|
||||
|
||||
The first account created is an admin. Every later one is an ordinary user, and an ordinary user
|
||||
cannot change global settings or how often feeds are scanned. Promote someone with:
|
||||
cannot change global settings, a feed's URL or folder, or how often feeds are scanned: the API
|
||||
refuses those with a `403`, not just the UI. Everything else about a feed (which items they want,
|
||||
whether to fetch them, how many at a time) is theirs alone; see [users.md](users.md).
|
||||
|
||||
Somebody arriving through the proxy for the first time starts with **no feeds**, because
|
||||
subscriptions are per person. Adding a feed someone else already reads costs no second fetch and no
|
||||
second copy on disk.
|
||||
|
||||
Promote someone with:
|
||||
|
||||
```sh
|
||||
ipx user list
|
||||
@@ -43,7 +51,9 @@ echo -n 'a good password' | ipx user passwd <name> # optional: also lets them
|
||||
```
|
||||
|
||||
Local sign-in at `/login` keeps working alongside all of this, which is how you get in from the LAN
|
||||
when the tunnel is down. A brand new database starts with **admin / ipodderx** — change it.
|
||||
when the tunnel is down. So does the shared `[web] token`, which signs in as the admin: that is
|
||||
what the Docker healthcheck uses, and the way back in if you lock yourself out. A brand new database
|
||||
starts with **admin / ipodderx** — change it.
|
||||
|
||||
---
|
||||
|
||||
@@ -206,7 +216,13 @@ ipx user rm sam # remove the account
|
||||
|
||||
Set `auto_create_users = false` once everyone who should have an account has one. After that the
|
||||
proxy vouching for an unknown name is logged and refused, rather than quietly making an account.
|
||||
Pre-create people instead with `ipx user add <name> --no-password`, using exactly the name the
|
||||
header will carry (Cloudflare sends the email address, lower-cased).
|
||||
|
||||
Scanning intervals, the disk quota, retention and the download folder are **admin-only** — the
|
||||
Settings button is hidden for everyone else, and the API refuses the change even if the request is
|
||||
made by hand. Ordinary users still control their own folders, keywords and downloads per feed.
|
||||
Scanning intervals, the disk quota, retention, the download folder and a feed's URL are
|
||||
**admin-only**: the Settings button is hidden for everyone else, and the API refuses the change even
|
||||
if the request is made by hand. Everyone controls their own keywords, auto-download, explicit
|
||||
setting and per-scan cap, along with their own read state and which feeds they see.
|
||||
|
||||
See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
|
||||
for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.
|
||||
|
||||
83
docs/users.md
Normal file
83
docs/users.md
Normal file
@@ -0,0 +1,83 @@
|
||||
# Accounts, and what several people share
|
||||
|
||||
ipx serves any number of people from one copy of the data. The rule that decides everything else:
|
||||
**there is one file on disk per enclosure URL.** Two people subscribed to the same show cost one
|
||||
fetch, one parse and one file.
|
||||
|
||||
## What is yours, what is everyone's
|
||||
|
||||
| Yours alone | The same for everyone |
|
||||
|---|---|
|
||||
| Read, starred, playback position | The feed's URL |
|
||||
| Which feeds you see at all | Its download folder |
|
||||
| Keywords, auto-download, explicit, per-scan cap | When it is scanned |
|
||||
| | The file on disk |
|
||||
|
||||
The right-hand column describes the feed and the file rather than a preference — two people wanting
|
||||
different folders would mean two copies. Those three are **admin-only**, and the API returns `403`
|
||||
for anyone else rather than merely hiding the controls.
|
||||
|
||||
## How the scanner merges everyone's wants
|
||||
|
||||
One fetch serves every subscriber, so the policy is a union:
|
||||
|
||||
* an item is downloaded if **anyone** wants it — one person's keyword set matching is enough, and
|
||||
one person with no keywords removes the filter for that feed entirely
|
||||
* auto-download is on if **anyone** has it on
|
||||
* the per-scan cap is the **largest** anyone asked for
|
||||
|
||||
So "auto-download off" means *I don't cause downloads*, not *I never see them*. If someone else's
|
||||
subscription pulls an item, you see it listed as downloaded and can play it, because the enclosure
|
||||
is shared.
|
||||
|
||||
## Deleting
|
||||
|
||||
Deleting a file deletes everyone's copy. A feed with other subscribers labels the button **Delete
|
||||
for everyone** and names them in the confirmation, and the server has the last word: if anyone else
|
||||
has starred the item or not played it yet, `DELETE /api/enclosures/{id}` answers `409` with the
|
||||
reason, and only `?force=true` goes through.
|
||||
|
||||
Retention follows the same rule: starred by anyone keeps a file, and it counts as read only once
|
||||
every subscriber has read it.
|
||||
|
||||
## Signing in
|
||||
|
||||
Three ways, tried in order of how specific the claim is:
|
||||
|
||||
1. **A proxy header** naming the user — Cloudflare Zero Trust or Authentik. Honoured only from an
|
||||
address in `trusted_proxies`. See [sso.md](sso.md).
|
||||
2. **A session cookie** from signing in at `/login`. Argon2id hashes, sessions in the database,
|
||||
idle timeout `session_days`.
|
||||
3. **The shared `[web] token`**, which signs in as the admin — this is what the Docker healthcheck
|
||||
and any scripts use.
|
||||
|
||||
A database with no accounts creates **admin / ipodderx** on the next daemon start and says so in
|
||||
the log. Change it:
|
||||
|
||||
```sh
|
||||
echo -n 'a good password' | ipx user passwd admin
|
||||
```
|
||||
|
||||
## Adding someone
|
||||
|
||||
```sh
|
||||
echo -n 'their password' | ipx user add sam
|
||||
```
|
||||
|
||||
They sign in at `/login` and start with **no feeds**: subscriptions are per person. Adding a feed
|
||||
someone else already has costs nothing — no second fetch, no second copy — it just appears on their
|
||||
list with their own read state. Unsubscribing removes it from their list alone; only when the last
|
||||
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
|
||||
re-subscribing does not pull the back catalogue again.
|
||||
|
||||
Accounts are managed from the command line only; there is no user administration in the web UI yet.
|
||||
|
||||
## Admin
|
||||
|
||||
The first account is an admin. An admin can change global settings (scanning interval, quota,
|
||||
retention, media types, download folder) and a feed's URL, folder and schedule. Everyone else gets
|
||||
the Settings button hidden and a `403` if they ask anyway.
|
||||
|
||||
```sh
|
||||
ipx user list # the admin column says who
|
||||
```
|
||||
Reference in New Issue
Block a user