ipx/docs/architecture.md

How it works

One binary, ipx. ipx daemon runs three things in one process: a scheduler, a Unix-socket control server, and the web UI. Everything else is a CLI that either does the work itself or hands it to a running daemon.

The repo is a Cargo workspace (#202): crates/ipx is the daemon and CLI, and crates/feed (ipx-feed) fetches and parses feeds, shared with crates/directory (ipx-directory), the directory of public podcasts that ipx servers share (directory.md). The version is the workspace's, in the root Cargo.toml.

Modules

File	Responsibility	What it replaced in the Python
crates/ipx/src/main.rs	CLI, dispatch, scan loop, download policy	iPXAgent.py
crates/ipx/src/config.rs	TOML load/save, General/Feed/Web, intervals, slugs	iPXSettings.py, feeds.plist
crates/ipx/src/db.rs	Every query, through SeaORM; creates missing tables	.ipxd plists, history.dat, qmcache.dat
crates/ipx/src/entity.rs	The tables, as SeaORM entities: the schema	—
crates/feed/src/lib.rs	Conditional GET, RSS/Atom/OPML parsing, url_looks_private	FeedData.__getFeed/__getEntries
crates/ipx/src/download.rs	Streaming download, naming, type sniffing, placement	iPXDownloader.getFile
crates/ipx/src/torrent.rs	librqbit session, seeding limits, stall abort	vendored BitTorrent 4.2.1
crates/ipx/src/retention.rs	Quota and age sweeps	iPXQuotaManager.py
crates/ipx/src/ipc.rs	Event and command types, the socket server	printMSG on stdout
crates/ipx/src/auth.rs	Argon2id hashing, session tokens, header names	—
crates/ipx/src/web.rs	axum: HTTP API, auth, SSE, media streaming	—
crates/ipx/src/logbuf.rs	Ring buffer behind the UI's Log view	—
web/index.html	The app's markup	—
web/admin.html	The admin page's markup: server settings, accounts, the log. Sent to admins only	—
web/app.css	The stylesheet both pages share	—
web/src/*.ts	The page's script, one scope split across files, type-checked by npx tsc	—
web/build.mjs	swc: strips the types into app.js/admin.js/login.js, named in the page by a hash of their contents, minifies, and writes the static site to web/dist	—
web/boot.js	In <head>: the theme from its cookie, and an admin's link, before the first frame	—
web/Caddyfile	ipx-web: serves web/dist, asks ipx before serving a page, passes /api and /media to ipx	—

The page is a separate image, ipx-web (Dockerfile.web, #203), so editing web/ needs ipx-web rebuilt, not ipx, and building it needs node and npm ci run once. ipx serves the API and the files, and no page.

A scan

1. Skip the feed unless last_checked + max(schedule, ttl) has passed (--force ignores this).
2. Conditional GET with the stored ETag / Last-Modified. 304 ends it there.
3. Sniff the body: RSS, then Atom, then OPML. An OPML is a live subscription — its feeds are re-derived into the database each scan, never written to config.toml. A Patreon creator link (a token, no show=) with more than one show is treated the same way, before any fetch: its shows come from Patreon's web API and each becomes a derived feed.
4. Record entries: the new ones, and those the feed now gives a different title, text, artwork, length or number, compared with what is stored for the items it lists (#141). A corrected item keeps its read state.
5. Record enclosures. enclosures.url is UNIQUE, which is the dedupe key and subsumes the original's history.dat pickle: a reaped file keeps its row so it is never fetched twice.
6. Apply the merged policy (see users.md) and mark anything rejected as skipped with a reason. What a filter skipped is judged again every scan, so a change of settings brings it back. A feed in a group takes your settings on the group for anything you have not set on it.
7. Download what is still pending, newest first, up to the per-scan cap. A .torrent body goes to the torrent path whatever its advertised type; an HTML body is a failed download — a login wall or an error page — and is deleted.

Data model

feeds       id, url, title, image, category, description, etag, last_modified,
            last_checked, ttl_mins, last_error, orphaned, group_id, managed
entries     feed_id, guid, title, link, published, description, first_seen,
            image, duration, episode, season            PK (feed_id, guid)
enclosures  id, feed_id, guid, url UNIQUE, mime, length, path, state,
            bytes_done, downloaded_at, last_error
users       id, name, pass_hash, is_admin, created, last_login
sessions    token, user_id, seen
subscriptions  user_id, feed_id, keywords, auto_download, allow_explicit,
               max_new_per_check                         PK (user_id, feed_id)
entry_state    user_id, feed_id, guid, read, flagged, position
                                                         PK (user_id, feed_id, guid)

Read state is entry_state alone. entries had read, flagged and position columns from before accounts; two bugs came from queries still reading them, and they were dropped in 0.5.

Schema changes: the tables are the entities in crates/ipx/src/entity.rs, and Db::open creates whatever table or index a database is missing from them (db::create_missing), with IF NOT EXISTS. It never alters a table that exists, so a new column on one needs its own ALTER in create_missing, looked for first and run only when it is missing, as feeds.description's is, or sea-orm-migration once there are several. Db::memory() builds its database the same way, so the tests run on the schema production gets. A database from before 0.7 takes its last columns from the old migrate(), so it upgrades through a 0.7 release first.

Control socket

Newline-delimited JSON, both directions, over $XDG_RUNTIME_DIR/ipx.sock.

printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/ipx.sock
{"ev":"feed_start","feed":"atp"}
{"ev":"progress","feed":"atp","enclosure":42,"file":"ep1.mp3","done":8192,"total":3000000}
{"ev":"download_done","feed":"atp","enclosure":42,"path":"…","bytes":3000000}
{"ev":"feed_done","feed":"atp","new":1,"downloaded":1,"failed":0,"torrents":0}
{"ev":"scan_done","feeds":1}

Commands — fetch (optional feed, force), reap (optional dry_run), download (enclosure), status, reload (read the catalogue and settings again from the database, which ipx add, rm and import send after changing them in a process of their own; answered with status).

Events — feed_start, feed_skip, feed_done, feed_error, progress, download_done, download_error, torrent_deferred, reaped, reap_done, scan_done, status, error. scan_done, reap_done and status are terminal: a client that asked for work stops reading there. Commands run one at a time, in the order they arrive, except status and reload: the socket answers them straight away, so the Docker healthcheck is never left waiting behind a scan or a download, and answers only the client that asked, since status would end any other client's session.

Progress carries the enclosure id, without which a UI cannot tell one download from another and ends up animating every pending row. It is throttled to whole percents. The stream is a broadcast, so a client attached to a busy daemon also sees that daemon's other work.

Inside the process the same events go over a tokio::broadcast; commands arrive on an mpsc and are handled by a single worker, so nothing races over the same download. Shutdown is a watch channel raced inside each job — tokio::select! only races branches at the point of selection, so a long download had to be able to notice the signal itself.

HTTP API

Everything below /api needs a signed-in user; the browser gets a redirect to /login, anything else a 401. A feed's items and files (its entries, download-latest, /api/enclosures/{id} and /media/{id}) answer only someone who subscribes to the feed; anyone else gets a 404.

Route
GET /api/page-check, GET /api/login-check, GET /api/signin	for Caddy (#203): whether the visitor may have the page in X-Forwarded-Uri (204), or where to send them; signing in with ?token=
POST /api/login, POST /api/logout, GET /api/me	sign-in; /api/me also gives the version and the account's theme from before
GET /api/feeds, POST /api/feeds	your subscriptions; subscribe
PATCH /api/feeds/{id}, DELETE /api/feeds/{id}	your settings or (admin) the feed's; unsubscribe
GET /api/feeds/{id}/entries	paged, filtered, searchable, sortable (sort = kept, title, feed, type, size or published; dir = asc or desc)
GET /api/entries	the same, across every feed you subscribe to (All Subscriptions)
POST /api/feeds/{id}/read-all, POST /api/feeds/{id}/download-latest
POST /api/read-all	everything read in every feed you subscribe to (All Subscriptions)
POST /api/entries/{feed}/{guid}/flags, …/position	your read, kept, position
POST /api/enclosures/{id}/download, DELETE /api/enclosures/{id}	?force=true overrides the shared-file warning
POST /api/fetch
GET /api/opml, POST /api/opml	export your subscriptions; subscribe to every feed in an OPML
GET /api/directory/{id}	a feed's page from the directory service (#205), by the directory's id: its description and latest twenty items, sanitized here, and why its last check failed, in words; never a file or its address
GET /api/popular, GET /api/directory, POST /api/popular/{id}	the directory service's ten most subscribed and every feed it lists, A to Z, asked by ipx with its token (directory id, title, art, subscribers across every sharing server, whether it is yours and its id here, category, whether it carries audio or video; never a URL); subscribe by the directory's id, which takes the feed's address from the directory. 503 with no directory set up
GET /api/directory-status	admin-only: the name and access the directory gives this server's token, or why it refused, and how the last snapshot went
GET /api/settings, PATCH /api/settings	admin-only to write. The directory's token is written, never read back: directory_token_set says whether there is one
GET /api/users, POST /api/users, PATCH /api/users/{id}, DELETE /api/users/{id}	admin-only; the only admin cannot be demoted or removed
GET /api/events	SSE, the same broadcast the socket carries
GET /api/logs	admin-only; the ring buffer, with a sequence cursor
GET /media/{id}	the file, with Range support so seeking works

Show notes are feed-supplied HTML from an untrusted source, sanitized with ammonia server-side before they reach the page.

Testing

cargo test                # parsing, filters, retention, schedules, SQL, per-user isolation
node tests/page-smoke.js  # the page script loads and every selector it wires at load exists
node tests/native-bridge.js  # the page hands playback to a native shell
npx playwright test       # a real browser against a real daemon on fixture feeds
node tests/load/run.js    # k6: many people at once against a daemon with 1,500 generated feeds

The Rust tests cannot see a wrong selector, a handler that runs and does nothing, or a page that renders empty — which is what has actually reached users. Each Playwright case maps to a bug that did.

The suite starts its own daemon and database under /tmp/ipx-ui-test, wiped once per run. Tests share that daemon and run in order, so a test that marks something read changes what later tests see — make assertions that do not depend on earlier ones.