rays/ipodderx-rs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Documentation: docs/, a changelog, and CLAUDE.md

Browse Source 
PROGRESS.md becomes CHANGELOG.md with the finished step lists moved to an
appendix. The README is an overview pointing at docs/: configuration,
cli, users, sso (refreshed for accounts and admin-only settings), and
architecture. CLAUDE.md collects what working on this code actually
requires -- pkill -x not -f, the page being compiled in, the dead columns
on entries, the Playwright worker that deleted its own database.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
This commit is contained in:
Ray Slakinski
2026-09-11 03:00:25 +00:00
parent c47c224372
commit 6114add4a6
8 changed files with 684 additions and 269 deletions
Download Patch File Download Diff File Expand all files Collapse all files

253
README.md
View File

@@ -1,14 +1,14 @@
# ipodderx-rs
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent),
files them into per-feed folders, and reaps old episodes to stay under a disk quota.
Runs as a one-shot CLI or as a daemon with a Unix-socket JSON event stream for a UI to attach to.
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent), files them
into per-feed folders, and reaps old files to stay under a disk quota. Runs as a one-shot CLI or as
a daemon with a web UI, serving any number of people from one copy of the data.
## Lineage
This is a modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the
Python 2 engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), which was
open-sourced under the MIT License in 2010.
A modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the Python 2
engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), open-sourced under the MIT
License in 2010.
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
@@ -16,193 +16,71 @@ oldest-first disk quota reaper.
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
3DES-encrypted preferences, and the `printMSG` stdout protocol (replaced by a JSON-lines socket).
3DES-encrypted preferences, and the `printMSG` stdout protocol -- replaced by a JSON-lines socket.
## Quickstart
## Quick start
```sh
cargo install --path .
cargo build --release
install -m755 target/release/ipx ~/.cargo/bin/
ipx add https://atp.fm/rss # names the feed from its own title
ipx list
ipx fetch # scan now
ipx daemon # or run continuously, honouring each feed's <ttl>
ipx add https://atp.fm/rss # subscribe
ipx fetch # scan and download
ipx daemon # scheduler, control socket and web UI
```
Config lives at `~/.config/ipx/config.toml` (mode 0600, since it may hold feed passwords);
state at `~/.local/share/ipx/state.db`. Override with `IPX_CONFIG` and `IPX_DATA_DIR`.
Set `IPX_LOG=ipx=debug` for verbose logging on stderr.
On first start with `[web] enabled = true` the daemon mints a token, writes it to config.toml and
prints the URL to open. A database with no accounts starts with **admin / ipodderx** at `/login` --
change it with `echo -n '<password>' | ipx user passwd admin`.
## Commands
## Documentation
| command | what it does |
| | |
|---|---|
| `ipx add <url> [--folder X] [--keywords a,b]` | subscribe; the id comes from the feed title |
| `ipx rm <feed>` | unsubscribe; downloads and history are kept |
| `ipx list` / `ipx status` | subscriptions and their state |
| `ipx fetch [FEED] [--force]` | scan; `--force` ignores the TTL |
| `ipx reap [--dry-run]` | run retention now |
| `ipx import/export <file.opml>` | move subscriptions in or out |
| `ipx daemon` | scheduler plus the control socket |
| [docs/configuration.md](docs/configuration.md) | Every config key, paths, environment variables |
| [docs/cli.md](docs/cli.md) | Every command, including `ipx user` |
| [docs/users.md](docs/users.md) | Accounts, and what several people share |
| [docs/sso.md](docs/sso.md) | Cloudflare Zero Trust or Authentik in front |
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, socket, HTTP API |
| [CHANGELOG.md](CHANGELOG.md) | What landed, and why |
| [CLAUDE.md](CLAUDE.md) | Notes for anyone (or anything) working on the code |
Any command with a wire form probes the socket first: if a daemon is running it does the work,
and the CLI just renders the events it streams back. `--local` forces in-process execution.
## The web UI
## Configuration
`ipx daemon` serves it in the same process, so it reads SQLite and the event bus directly.
```toml
[general]
download_dir = "~/Podcasts"
socket = "/run/user/1000/ipx.sock" # default: $XDG_RUNTIME_DIR/ipx.sock
interval_mins = 60 # default poll; a feed's own <ttl> wins when longer
organize = "feed" # "feed" | "date"
max_total_gb = 50 # 0 = unlimited
max_age_days = 30 # 0 = keep forever
max_new_per_check = 3 # per feed, per scan. 0 = unlimited (pulls whole back catalogues)
media_types = ["audio", "video"] # what downloads automatically. Anything else is still
# listed and can be fetched by hand -- blog feeds put article
# images in enclosures. Empty takes everything.
Feeds down the side; the selected feed's items across the top; the selected item's text and its
enclosures below, which is where you play, download or delete them. The divider drags and its
position is remembered. Playback serves Range requests, so seeking works. An OPML subscription is a
collapsible folder whose page lists the feeds inside it.
[torrent]
enabled = true
seed_ratio = 1.0 # stop seeding at this ratio ...
seed_time_mins = 60 # ... or after this long, whichever comes first
port_range = "6881-6889"
stall_mins = 30 # give up on a torrent making no progress
An item may carry several enclosures; all of them appear below, and anything that is not audio or
video gets a View link rather than a player -- the publisher's copy until it is downloaded, the
local one after. Opening an item marks it read. Show notes are untrusted feed HTML, sanitized with
`ammonia` server-side before they reach the page.
[feeds.atp]
url = "https://atp.fm/rss"
folder = "Accidental Tech Podcast" # default: the feed title
keywords = ["deep dive"] # OR across keywords, AND within one
allow_explicit = false
auto_download = true
max_new_per_check = 3 # the rest wait for the next scan
username = "ray" # optional HTTP basic auth
password_env = "IPX_ATP_PASS" # or a literal `password`
```
The **Log** button shows the running daemon live in four tabs: *Daemon I/O* is the control protocol
itself, every command in and event out; *Scans* is feed and download activity; *HTTP* is web
requests; *All* is everything, with level and text filters and a copy button. It reads a ring buffer
held in the process, not a file, so it works the same under Docker.
Retention keeps files that are `flagged` in the database, and deletes read episodes before unread
ones, oldest first.
## Socket protocol
Newline-delimited JSON over a Unix socket, both directions.
```sh
$ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
{"ev":"feed_start","feed":"atp"}
{"ev":"progress","feed":"atp","url":"...","file":"ep1.mp3","done":8192,"total":3000000}
{"ev":"download_done","feed":"atp","url":"...","path":"...","bytes":3000000}
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
{"ev":"scan_done","feeds":1}
```
Commands: `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `status`.
Events: `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
`scan_done`, `reap_done` and `status` are terminal -- a client that asked for work stops there.
Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy
daemon also sees that daemon's other work.
It is plain HTTP. On a LAN bind the token and everything else cross the network in the clear, and a
feed URL can itself carry a credential. Put TLS in front of it if that matters.
## OPML
Two different things, both supported:
**Importing and exporting** a file copies subscriptions in or out once — `ipx import subs.opml`,
`ipx export subs.opml`, or the OPML button in the UI.
**Importing and exporting** a file copies subscriptions in or out once: `ipx import subs.opml`,
`ipx export subs.opml`, or Settings → Subscriptions in the UI.
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
other feed; every scan re-reads it and keeps your feed list in step. An OPML is recognised by its
content, so a URL without a `.opml` extension still works.
other feed; every scan re-reads it and keeps your list in step. The feeds inside are not written to
config.toml -- the OPML is the source of truth, so they are re-derived each scan and held in the
database. They show as a folder, download into one nested folder, and inherit the subscription's
settings until you change one, which gives it its own entry.
The feeds inside it are **not written to `config.toml`** — the OPML is the source of truth, so they
are re-derived each scan and held in the database. Your config keeps only what you chose. They show
as a collapsible folder in the sidebar, download into one nested folder, and newly listed ones are
scanned in the same run rather than waiting for the next interval. They inherit the subscription's
settings; change anything on one and it gets its own config entry from then on.
When a feed drops out of the OPML upstream:
| it has downloads | what happens |
|---|---|
| no | unsubscribed and removed from the config |
| yes | kept, flagged in the UI as no longer listed |
A downloaded file is never left behind with nothing explaining where it came from.
## Web UI
```toml
[web]
enabled = true
bind = "0.0.0.0:8080" # 127.0.0.1:8080 by default
token = "" # generated and written back on first run
```
`ipx daemon` then serves it in the same process (`ipx daemon --web ADDR` overrides the bind for one
run). On first start it mints a token, saves it to config.toml, and prints the URL to open:
```
web ui token generated. Open:
http://0.0.0.0:8080/?token=1f4c…
```
`?token=` sets a year-long cookie, so you only paste it once per browser. Everything is behind that
token, including `/media/...` — a cookie rather than a header precisely because an `<audio>` element
cannot send headers.
Browse feeds, read show notes, play episodes in the browser (Range requests are served, so seeking
works), download or delete individual files, mark episodes read or flag them to keep, and edit a
feed's folder/keywords/explicit/auto-download/limit settings. Config edits are written to
config.toml and hot-reloaded — no daemon restart.
Show notes are feed-supplied HTML from an untrusted source; they are sanitized with `ammonia`
server-side before they reach the page.
**It is plain HTTP.** On a LAN bind, the token and everything else crosses the network in the
clear — and a feed URL can itself contain a credential (Patreon's, for one, carries an auth token).
Put it behind a reverse proxy with TLS if that matters to you.
## Layout
Feeds down the side; the selected feed's episodes across the top; the selected episode's text and
its enclosures in the pane below, which is where you play, download or delete them. The divider
between the two panes drags and the position is remembered. An OPML subscription's page instead
lists the feeds inside it.
An item may carry several enclosures. All of them appear in the pane below; the row summarises the
one you would act on and notes how many others there are. Anything that is not audio or video gets a
View link — opening the publisher's copy, or the local one once downloaded — rather than a player.
## Log view
The **Log** button in the sidebar shows the running daemon's output live, in four tabs:
**Daemon I/O** is the control protocol itself — every command in and every event out, as JSON;
**Scans** is feed and download activity; **HTTP** is web requests; **All** is everything: feed scans, downloads,
torrent activity and every HTTP request, with level and text filters and a copy button. It reads a
2000-line ring buffer held inside the process (`/api/logs`), not a file — so it works the same under
Docker, where logs go to stdout and there is no file to tail. The buffer keeps `debug` even when the terminal does not, so protocol traffic and routine
skips are there without making stderr unreadable — `IPX_UI_LOG` changes what it captures and
`IPX_LOG` what reaches stderr.
## Tests
```sh
cargo test # the server: parsing, filters, retention, schedules, SQL
node tests/page-smoke.js # the page script loads without throwing
npx playwright test # a real browser against a real daemon
```
The Rust tests cover the server and the smoke test catches a script that fails to load, but neither
can see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which
is what has actually slipped through. The Playwright suite drives a headless browser against a
daemon started on fixture feeds, and each test maps to a bug that reached a user: the page rendering
empty, a dead theme toggle, settings not persisting, episode metadata, filter tabs, the feed URL
field and its copy button, the log tabs, and OPML folders.
`npm install` gets the test runner; the browser itself comes from
`npx playwright install --with-deps chromium` (in `install.sh`).
When a feed drops out of the OPML upstream, it is unsubscribed and removed -- unless it has
downloads, in which case it is kept and flagged in the UI as no longer listed. A downloaded file is
never left behind with nothing explaining where it came from.
## Docker
@@ -212,27 +90,26 @@ docker compose logs -f ipx # the token is printed on first start
```
`docker-compose.yml` mounts `./config`, `./data` and a downloads directory, publishes 8099 for the
UI and 6881 (TCP **and** UDP — DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
land owned the way Unraid shares expect. On first start the entrypoint writes a config bound to
`0.0.0.0`, since a container's loopback is not reachable from outside it, and prints the URL with
its generated token.
The healthcheck runs `ipx status`, which goes through the control socket to the command worker — so
it catches a daemon that is alive but wedged, not merely one that has died.
UI and 6881 (TCP **and** UDP -- DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
land owned the way Unraid shares expect. The healthcheck runs `ipx status` through the control
socket, so it catches a daemon that is alive but wedged rather than merely one that has died.
## Running it as a service
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
rather run periodic scans with no daemon (in which case there is no socket for a UI to attach to).
rather run periodic scans with no daemon -- in which case there is no socket for a UI to attach to.
## Tests
```sh
cargo test # the engine: parsing, filters, retention, schedules, SQL, per-user state
node tests/page-smoke.js # the page script loads without throwing
npx playwright test # a real browser against a real daemon
```
`npm install` gets the test runner; the browser comes from
`npx playwright install --with-deps chromium` (in `install.sh`).
## License
MIT. See [LICENSE](LICENSE).
## Signing in
A new database starts with **admin / ipodderx** at `/login`; change it with
`echo -n <password> | ipx user passwd admin`. Accounts are managed with `ipx user add|list|passwd|rm`.
To let Cloudflare Zero Trust or Authentik do the authenticating instead, see
[docs/sso.md](docs/sso.md).