ipodderx-rs/docs/architecture.md

How it works

One binary, ipx. ipx daemon runs three things in one process: a scheduler, a Unix-socket control server, and the web UI. Everything else is a CLI that either does the work itself or hands it to a running daemon.

Modules

File	Responsibility	What it replaced in the Python
src/main.rs	CLI, dispatch, scan loop, download policy	iPXAgent.py
src/config.rs	TOML load/save, General/Feed/Web, intervals, slugs	iPXSettings.py, feeds.plist
src/db.rs	SQLite schema, migrations, every query	.ipxd plists, history.dat, qmcache.dat
src/feed.rs	Conditional GET, RSS/Atom/OPML parsing	FeedData.__getFeed/__getEntries
src/download.rs	Streaming download, naming, type sniffing, placement	iPXDownloader.getFile
src/torrent.rs	librqbit session, seeding limits, stall abort	vendored BitTorrent 4.2.1
src/retention.rs	Quota and age sweeps	iPXQuotaManager.py
src/ipc.rs	Event and command types, the socket server	printMSG on stdout
src/auth.rs	Argon2id hashing, session tokens, header names	—
src/web.rs	axum: HTTP API, auth, SSE, media streaming	—
src/logbuf.rs	Ring buffer behind the UI's Log view	—
web/index.html	The app's markup	—
web/admin.html	The admin page's markup: server settings, accounts, the log. Sent to admins only	—
web/app.css	The stylesheet both pages share	—
web/src/*.ts	The page's script, one scope split across files, type-checked by npx tsc	—
web/build.mjs	swc: strips the types into app.js/login.js, named in the page by a hash of their contents, and minifies	—
build.rs	Runs web/build.mjs into OUT_DIR, where web.rs include_str!s the result	—

The page is compiled in, so editing web/index.html or web/src needs a rebuild, and a build needs node and npm ci run once.

A scan

1. Skip the feed unless last_checked + max(schedule, ttl) has passed (--force ignores this).
2. Conditional GET with the stored ETag / Last-Modified. 304 ends it there.
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are re-derived into the database each scan, never written to config.toml. A Patreon creator link (a token, no show=) with more than one show is treated the same way, before any fetch: its shows come from Patreon's web API and each becomes a derived feed.
4. Record entries. A changed title or description flips the item back to unread.
5. Record enclosures. enclosures.url is UNIQUE, which is the dedupe key and subsumes the original's history.dat pickle: a reaped file keeps its row so it is never fetched twice.
6. Apply the merged policy (see users.md) and mark anything rejected as skipped with a reason. What a filter skipped is judged again every scan, so a change of settings brings it back. A feed in a group takes your settings on the group for anything you have not set on it.
7. Download what is still pending, newest first, up to the per-scan cap. A .torrent body goes to the torrent path whatever its advertised type; an HTML body is a failed download — a login wall or an error page — and is deleted.

Data model

feeds       id, url, title, image, etag, last_modified, last_checked, ttl_mins,
            last_error, orphaned, group_id, managed
entries     feed_id, guid, title, link, published, description, first_seen,
            image, duration, episode, season            PK (feed_id, guid)
enclosures  id, feed_id, guid, url UNIQUE, mime, length, path, state,
            bytes_done, downloaded_at, last_error
users       id, name, pass_hash, is_admin, created, last_login
sessions    token, user_id, seen
subscriptions  user_id, feed_id, keywords, auto_download, allow_explicit,
               max_new_per_check                         PK (user_id, feed_id)
entry_state    user_id, feed_id, guid, read, flagged, position
                                                         PK (user_id, feed_id, guid)

Read state is entry_state alone. entries had read, flagged and position columns from before accounts; two bugs came from queries still reading them, and migrate() drops them from an older database.

Schema changes: add the table or column to SCHEMA. CREATE TABLE IF NOT EXISTS leaves a table that already exists alone, so a new column on one also goes in migrate()'s wanted list, and a retired one in its retired list; both are checked with PRAGMA table_info. Columns from before 0.3.0, the oldest version an upgrade may start from, need no entry. Db::memory() runs the same path as Db::open, so a migration cannot pass the tests while missing in production.

Control socket

Newline-delimited JSON, both directions, over $XDG_RUNTIME_DIR/ipx.sock.

printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
{"ev":"feed_start","feed":"atp"}
{"ev":"progress","feed":"atp","enclosure":42,"file":"ep1.mp3","done":8192,"total":3000000}
{"ev":"download_done","feed":"atp","enclosure":42,"path":"…","bytes":3000000}
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
{"ev":"scan_done","feeds":1}

Commands — fetch (optional feed, force), reap (optional dry_run), download (enclosure), status.

Events — feed_start, feed_skip, feed_done, feed_error, progress, download_done, download_error, torrent_deferred, reaped, reap_done, scan_done, status, error. scan_done, reap_done and status are terminal: a client that asked for work stops reading there. Commands run one at a time, in the order they arrive, except status: the socket answers it straight away, so the Docker healthcheck is never left waiting behind a scan or a download, and answers only the client that asked, since status would end any other client's session.

Progress carries the enclosure id, without which a UI cannot tell one download from another and ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast, so a client attached to a busy daemon also sees that daemon's other work.

Inside the process the same events go over a tokio::broadcast; commands arrive on an mpsc and are handled by a single worker, so nothing races over the same download. Shutdown is a watch channel raced inside each job — tokio::select! only races branches at the point of selection, so a long download had to be able to notice the signal itself.

HTTP API

Everything below /api needs a signed-in user; the browser gets a redirect to /login, anything else a 401.

Route
GET /	the app
GET /login, POST /api/login, POST /api/logout, GET /api/me	sign-in
GET /api/feeds, POST /api/feeds	your subscriptions; subscribe
PATCH /api/feeds/{id}, DELETE /api/feeds/{id}	your settings or (admin) the feed's; unsubscribe
GET /api/feeds/{id}/entries	paged, filtered, searchable, sortable (sort = kept, title, feed, type, size or published; dir = asc or desc)
GET /api/entries	the same, across every feed you subscribe to (All Subscriptions)
POST /api/feeds/{id}/read-all, POST /api/feeds/{id}/download-latest
POST /api/read-all	everything read in every feed you subscribe to (All Subscriptions)
POST /api/entries/{feed}/{guid}/flags, …/position	your read, kept, position
POST /api/enclosures/{id}/download, DELETE /api/enclosures/{id}	?force=true overrides the shared-file warning
POST /api/fetch
GET /api/opml, POST /api/opml	export your subscriptions; subscribe to every feed in an OPML
GET /api/popular, GET /api/directory, POST /api/popular/{id}	the ten most subscribed feeds, and every listable feed A to Z, with an OPML's feeds in place of the OPML and everyone counted (id, title, art, count, whether it is yours, the feed's iTunes category, whether it carries audio or video; never a URL, never a private feed); subscribe by id
GET /api/settings, PATCH /api/settings	admin-only to write
GET /api/users, POST /api/users, PATCH /api/users/{id}, DELETE /api/users/{id}	admin-only; the only admin cannot be demoted or removed
GET /api/events	SSE, the same broadcast the socket carries
GET /api/logs	admin-only; the ring buffer, with a sequence cursor
GET /media/{id}	the file, with Range support so seeking works

Show notes are feed-supplied HTML from an untrusted source, sanitized with ammonia server-side before they reach the page.

Testing

cargo test                # parsing, filters, retention, schedules, SQL, per-user isolation
node tests/page-smoke.js  # the page script loads and every selector it wires at load exists
npx playwright test       # a real browser against a real daemon on fixture feeds

The Rust tests cannot see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which is what has actually reached users. Each Playwright case maps to a bug that did.

The suite starts its own daemon and database under /tmp/ipx-ui-test, wiped once per run. Tests share that daemon and run in order, so a test that marks something read changes what later tests see — make assertions that do not depend on earlier ones.