rays
dd9c0881ee
The rss crate keeps at most one enclosure per item and, when a feed ships several, silently keeps the last -- so a two-file item lost its first file. enclosures_by_item reads them from the XML in document order, unescaping attributes so a URL's & survives. The row summarises the one you would act on and counts the rest; the pane below lists them all. Non-media enclosures gain a View link opening in a new tab: the publisher's URL, or the local copy once downloaded. A direct link, not a proxy, so the daemon does not become a fetch-anything relay. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
231 lines
11 KiB
Markdown
231 lines
11 KiB
Markdown
# ipodderx-rs
|
|
|
|
A headless podcatcher: scans RSS/Atom feeds, downloads enclosures (HTTP and BitTorrent),
|
|
files them into per-feed folders, and reaps old episodes to stay under a disk quota.
|
|
Runs as a one-shot CLI or as a daemon with a Unix-socket JSON event stream for a UI to attach to.
|
|
|
|
## Lineage
|
|
|
|
This is a modern Rust rewrite of [ipodderx-core](https://git.sdf1.net/rays/ipodderx-core), the
|
|
Python 2 engine behind **iPodderX** (2004-2008, Ray Slakinski & August Trometer), which was
|
|
open-sourced under the MIT License in 2010.
|
|
|
|
What carries over: the feed scan and TTL handling, GUID/URL dedupe, per-feed and per-date download
|
|
folders, keyword filters, the explicit-content filter, torrent enclosures, and "SmartSpace" -- the
|
|
oldest-first disk quota reaper.
|
|
|
|
What does not: iTunes and iPhoto export via AppleScript, text-to-speech enclosures, the Windows
|
|
WMP/COM paths, XML plists and Python pickles for state, the `directory.iPodderX.com` survey ping,
|
|
3DES-encrypted preferences, and the `printMSG` stdout protocol (replaced by a JSON-lines socket).
|
|
|
|
## Quickstart
|
|
|
|
```sh
|
|
cargo install --path .
|
|
|
|
ipx add https://atp.fm/rss # names the feed from its own title
|
|
ipx list
|
|
ipx fetch # scan now
|
|
ipx daemon # or run continuously, honouring each feed's <ttl>
|
|
```
|
|
|
|
Config lives at `~/.config/ipx/config.toml` (mode 0600, since it may hold feed passwords);
|
|
state at `~/.local/share/ipx/state.db`. Override with `IPX_CONFIG` and `IPX_DATA_DIR`.
|
|
Set `IPX_LOG=ipx=debug` for verbose logging on stderr.
|
|
|
|
## Commands
|
|
|
|
| command | what it does |
|
|
|---|---|
|
|
| `ipx add <url> [--folder X] [--keywords a,b]` | subscribe; the id comes from the feed title |
|
|
| `ipx rm <feed>` | unsubscribe; downloads and history are kept |
|
|
| `ipx list` / `ipx status` | subscriptions and their state |
|
|
| `ipx fetch [FEED] [--force]` | scan; `--force` ignores the TTL |
|
|
| `ipx reap [--dry-run]` | run retention now |
|
|
| `ipx import/export <file.opml>` | move subscriptions in or out |
|
|
| `ipx daemon` | scheduler plus the control socket |
|
|
|
|
Any command with a wire form probes the socket first: if a daemon is running it does the work,
|
|
and the CLI just renders the events it streams back. `--local` forces in-process execution.
|
|
|
|
## Configuration
|
|
|
|
```toml
|
|
[general]
|
|
download_dir = "~/Podcasts"
|
|
socket = "/run/user/1000/ipx.sock" # default: $XDG_RUNTIME_DIR/ipx.sock
|
|
interval_mins = 60 # default poll; a feed's own <ttl> wins when longer
|
|
organize = "feed" # "feed" | "date"
|
|
max_total_gb = 50 # 0 = unlimited
|
|
max_age_days = 30 # 0 = keep forever
|
|
max_new_per_check = 3 # per feed, per scan. 0 = unlimited (pulls whole back catalogues)
|
|
media_types = ["audio", "video"] # what downloads automatically. Anything else is still
|
|
# listed and can be fetched by hand -- blog feeds put article
|
|
# images in enclosures. Empty takes everything.
|
|
|
|
[torrent]
|
|
enabled = true
|
|
seed_ratio = 1.0 # stop seeding at this ratio ...
|
|
seed_time_mins = 60 # ... or after this long, whichever comes first
|
|
port_range = "6881-6889"
|
|
stall_mins = 30 # give up on a torrent making no progress
|
|
|
|
[feeds.atp]
|
|
url = "https://atp.fm/rss"
|
|
folder = "Accidental Tech Podcast" # default: the feed title
|
|
keywords = ["deep dive"] # OR across keywords, AND within one
|
|
allow_explicit = false
|
|
auto_download = true
|
|
max_new_per_check = 3 # the rest wait for the next scan
|
|
username = "ray" # optional HTTP basic auth
|
|
password_env = "IPX_ATP_PASS" # or a literal `password`
|
|
```
|
|
|
|
Retention keeps files that are `flagged` in the database, and deletes read episodes before unread
|
|
ones, oldest first.
|
|
|
|
## Socket protocol
|
|
|
|
Newline-delimited JSON over a Unix socket, both directions.
|
|
|
|
```sh
|
|
$ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
|
|
{"ev":"feed_start","feed":"atp"}
|
|
{"ev":"progress","feed":"atp","url":"...","file":"ep1.mp3","done":8192,"total":3000000}
|
|
{"ev":"download_done","feed":"atp","url":"...","path":"...","bytes":3000000}
|
|
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
|
|
{"ev":"scan_done","feeds":1}
|
|
```
|
|
|
|
Commands: `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `status`.
|
|
Events: `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
|
|
`download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
|
|
`scan_done`, `reap_done` and `status` are terminal -- a client that asked for work stops there.
|
|
|
|
Progress is throttled to whole percents. The stream is a broadcast, so a client attached to a busy
|
|
daemon also sees that daemon's other work.
|
|
|
|
## OPML
|
|
|
|
Two different things, both supported:
|
|
|
|
**Importing and exporting** a file copies subscriptions in or out once — `ipx import subs.opml`,
|
|
`ipx export subs.opml`, or the OPML button in the UI.
|
|
|
|
**Subscribing to an OPML URL** is a live subscription, as iPodderX had. Add the OPML's URL like any
|
|
other feed; every scan re-reads it and keeps your feed list in step. An OPML is recognised by its
|
|
content, so a URL without a `.opml` extension still works.
|
|
|
|
The feeds inside it are **not written to `config.toml`** — the OPML is the source of truth, so they
|
|
are re-derived each scan and held in the database. Your config keeps only what you chose. They show
|
|
as a collapsible folder in the sidebar, download into one nested folder, and newly listed ones are
|
|
scanned in the same run rather than waiting for the next interval. They inherit the subscription's
|
|
settings; change anything on one and it gets its own config entry from then on.
|
|
|
|
When a feed drops out of the OPML upstream:
|
|
|
|
| it has downloads | what happens |
|
|
|---|---|
|
|
| no | unsubscribed and removed from the config |
|
|
| yes | kept, flagged in the UI as no longer listed |
|
|
|
|
A downloaded file is never left behind with nothing explaining where it came from.
|
|
|
|
## Web UI
|
|
|
|
```toml
|
|
[web]
|
|
enabled = true
|
|
bind = "0.0.0.0:8080" # 127.0.0.1:8080 by default
|
|
token = "" # generated and written back on first run
|
|
```
|
|
|
|
`ipx daemon` then serves it in the same process (`ipx daemon --web ADDR` overrides the bind for one
|
|
run). On first start it mints a token, saves it to config.toml, and prints the URL to open:
|
|
|
|
```
|
|
web ui token generated. Open:
|
|
http://0.0.0.0:8080/?token=1f4c…
|
|
```
|
|
|
|
`?token=` sets a year-long cookie, so you only paste it once per browser. Everything is behind that
|
|
token, including `/media/...` — a cookie rather than a header precisely because an `<audio>` element
|
|
cannot send headers.
|
|
|
|
Browse feeds, read show notes, play episodes in the browser (Range requests are served, so seeking
|
|
works), download or delete individual files, mark episodes read or flag them to keep, and edit a
|
|
feed's folder/keywords/explicit/auto-download/limit settings. Config edits are written to
|
|
config.toml and hot-reloaded — no daemon restart.
|
|
|
|
Show notes are feed-supplied HTML from an untrusted source; they are sanitized with `ammonia`
|
|
server-side before they reach the page.
|
|
|
|
**It is plain HTTP.** On a LAN bind, the token and everything else crosses the network in the
|
|
clear — and a feed URL can itself contain a credential (Patreon's, for one, carries an auth token).
|
|
Put it behind a reverse proxy with TLS if that matters to you.
|
|
|
|
## Layout
|
|
|
|
Feeds down the side; the selected feed's episodes across the top; the selected episode's text and
|
|
its enclosures in the pane below, which is where you play, download or delete them. The divider
|
|
between the two panes drags and the position is remembered. An OPML subscription's page instead
|
|
lists the feeds inside it.
|
|
|
|
An item may carry several enclosures. All of them appear in the pane below; the row summarises the
|
|
one you would act on and notes how many others there are. Anything that is not audio or video gets a
|
|
View link — opening the publisher's copy, or the local one once downloaded — rather than a player.
|
|
|
|
## Log view
|
|
|
|
The **Log** button in the sidebar shows the running daemon's output live, in four tabs:
|
|
**Daemon I/O** is the control protocol itself — every command in and every event out, as JSON;
|
|
**Scans** is feed and download activity; **HTTP** is web requests; **All** is everything: feed scans, downloads,
|
|
torrent activity and every HTTP request, with level and text filters and a copy button. It reads a
|
|
2000-line ring buffer held inside the process (`/api/logs`), not a file — so it works the same under
|
|
Docker, where logs go to stdout and there is no file to tail. The buffer keeps `debug` even when the terminal does not, so protocol traffic and routine
|
|
skips are there without making stderr unreadable — `IPX_UI_LOG` changes what it captures and
|
|
`IPX_LOG` what reaches stderr.
|
|
|
|
## Tests
|
|
|
|
```sh
|
|
cargo test # the server: parsing, filters, retention, schedules, SQL
|
|
node tests/page-smoke.js # the page script loads without throwing
|
|
npx playwright test # a real browser against a real daemon
|
|
```
|
|
|
|
The Rust tests cover the server and the smoke test catches a script that fails to load, but neither
|
|
can see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which
|
|
is what has actually slipped through. The Playwright suite drives a headless browser against a
|
|
daemon started on fixture feeds, and each test maps to a bug that reached a user: the page rendering
|
|
empty, a dead theme toggle, settings not persisting, episode metadata, filter tabs, the feed URL
|
|
field and its copy button, the log tabs, and OPML folders.
|
|
|
|
`npm install` gets the test runner; the browser itself comes from
|
|
`npx playwright install --with-deps chromium` (in `install.sh`).
|
|
|
|
## Docker
|
|
|
|
```sh
|
|
docker compose up -d # builds the image and starts it
|
|
docker compose logs -f ipx # the token is printed on first start
|
|
```
|
|
|
|
`docker-compose.yml` mounts `./config`, `./data` and a downloads directory, publishes 8099 for the
|
|
UI and 6881 (TCP **and** UDP — DHT needs the UDP side), and sets `PUID`/`PGID` to `99:100` so files
|
|
land owned the way Unraid shares expect. On first start the entrypoint writes a config bound to
|
|
`0.0.0.0`, since a container's loopback is not reachable from outside it, and prints the URL with
|
|
its generated token.
|
|
|
|
The healthcheck runs `ipx status`, which goes through the control socket to the command worker — so
|
|
it catches a daemon that is alive but wedged, not merely one that has died.
|
|
|
|
## Running it as a service
|
|
|
|
`contrib/` has a systemd user unit for the daemon, and a timer plus one-shot service if you would
|
|
rather run periodic scans with no daemon (in which case there is no socket for a UI to attach to).
|
|
|
|
## License
|
|
|
|
MIT. See [LICENSE](LICENSE).
|