rays/ipx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 18 Wiki Activity

docs: the three services, their versions and traces (#207, #213)

Browse Source 
architecture.md opens on the three services, how a trace crosses them, and the directory's
modules; sso.md's diagram and config show ipx-web in front, trusting Caddy and passing headers
only from TUNNEL_PEER; configuration.md has ipx-web's variables and the Directory as the
service's; cli.md and users.md stop describing the old local Directory; directory.md has its
traces and version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ray Slakinski
2026-10-12 16:40:08 +00:00
parent 38c1abba1d
commit 10a78efcc3
6 changed files with 76 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
docs/architecture.md
View File

@@ -1,13 +1,27 @@
# How it works
One binary, `ipx`. `ipx daemon` runs three things in one process: a scheduler, a Unix-socket
control server, and the web UI. Everything else is a CLI that either does the work itself or hands
it to a running daemon.
Three services, each a container of its own with its own version and changelog (#207):
The repo is a Cargo workspace (#202): `crates/ipx` is the daemon and CLI, and `crates/feed`
(`ipx-feed`) fetches and parses feeds, shared with `crates/directory` (`ipx-directory`), the
directory of public podcasts that ipx servers share ([directory.md](directory.md)).
The version is the workspace's, in the root `Cargo.toml`.
| Service | What it is | Source | Image |
|---|---|---|---|
| ipx-web | Caddy: serves the page, a static site built from `web/`, and passes `/api` and `/media` to ipx on the same address. It asks ipx before it serves the app or the admin page (`/api/page-check`) | `web/`, `web/Caddyfile` | `Dockerfile.web` |
| ipx | The daemon and CLI, one binary. `ipx daemon` runs a scheduler, a Unix-socket control server and the HTTP API in one process; everything else is a CLI that either does the work itself or hands it to a running daemon | `crates/ipx` | `Dockerfile` |
| ipx-directory | The directory of public podcasts that ipx servers share. ipx asks it on its people's behalf; the page never talks to it ([directory.md](directory.md)) | `crates/directory` | `Dockerfile.directory` |
The Rust source is a Cargo workspace (#202); `crates/feed` (`ipx-feed`) fetches and parses feeds
for both Rust services. Versions: ipx's in `crates/ipx/Cargo.toml`, the directory's in
`crates/directory/Cargo.toml`, the page's in `package.json`; CLAUDE.md has the release steps.
## Traces and logs
A request is one trace (#213). The page starts one for each tap and sends `traceparent` with its
requests (#190); Caddy's span continues it and passes it on to ipx; ipx's request span, its steps
and its calls to the directory follow, and the directory's request span joins it. Scans,
downloads, DeadAir's work, shares with the directory and the directory's refresher rounds are
traces of their own. All three log one JSON object a line, for Loki; ipx's lines carry their trace
id, and Caddy's request lines theirs. ipx also serves Prometheus metrics (`[web] metrics_bind`,
#194). The exporter and the `traceparent` handling both Rust services use are in
`crates/directory/src/otel.rs`.
## Modules
@@ -25,6 +39,12 @@ The version is the workspace's, in the root `Cargo.toml`.
| `crates/ipx/src/auth.rs` | Argon2id hashing, session tokens, header names | — |
| `crates/ipx/src/web.rs` | axum: HTTP API, auth, SSE, media streaming | — |
| `crates/ipx/src/logbuf.rs` | Ring buffer behind the UI's Log view | — |
| `crates/ipx/src/directory.rs` | Asking the directory service, and sharing this server's public feeds with it | — |
| `crates/ipx/src/metrics.rs` | `/metrics` for Prometheus | — |
| `crates/directory/src/lib.rs` | The directory's HTTP API, its token check, the refresher | — |
| `crates/directory/src/store.rs` | The directory's database: feeds, snapshots, blocked | — |
| `crates/directory/src/config.rs` | The directory's config file: its tokens and their access | — |
| `crates/directory/src/otel.rs` | Traces for both Rust services: the exporter, `traceparent` in and out | — |
| `web/index.html` | The app's markup | — |
| `web/admin.html` | The admin page's markup: server settings, accounts, the log. Sent to admins only | — |
| `web/app.css` | The stylesheet both pages share | — |

6
docs/cli.md
View File

@@ -13,12 +13,12 @@ processes must never download the same thing. `--local` forces the work to happe
| `ipx list` | Subscriptions and their state |
| `ipx status` | Counts: feeds, pending, downloaded |
| `ipx fetch [FEED] [--force]` | Scan everything, or one feed. `--force` ignores the TTL |
| `ipx add <url> [--folder X] [--keywords a,b] [--list] [--category C]` | Subscribe; the id comes from the feed title. `--list` puts it in the Directory for anyone to subscribe to instead, and keeps it there when its last subscriber leaves; `--category` files it under one of the Directory's categories. Run for a feed already in the catalogue, these list it or set its category |
| `ipx add <url> [--folder X] [--keywords a,b] [--list] [--category C]` | Subscribe; the id comes from the feed title. `--list` subscribes no one but keeps it in the catalogue, and so in what this server shares with the directory, when its last subscriber leaves; `--category` is the category the directory files it under when the feed names none. Run for a feed already in the catalogue, these list it or set its category |
| `ipx rm <feed>` | Unsubscribe; downloads and history are kept |
| `ipx import <file.opml>` / `ipx export <file.opml>` | Move subscriptions in or out. Import subscribes the first admin, as the shared web token does; in the web UI it subscribes whoever is signed in |
| `ipx reap [--dry-run]` | Run retention now |
| `ipx user <add\|list\|passwd\|rm>` | Accounts for the web UI |
| `ipx daemon [--web ADDR]` | Scheduler, control socket and web UI |
| `ipx daemon [--web ADDR]` | Scheduler, control socket and the HTTP API (the page is ipx-web's) |
`add`, `rm` and `import` change the catalogue in a process of their own. With a daemon running they
then tell it to read the catalogue again; without that it kept its own copy and wrote it back at
@@ -71,7 +71,7 @@ stops the next scan fetching it again.
## The daemon
```sh
ipx daemon # scheduler + socket + web UI
ipx daemon # scheduler + socket + HTTP API
ipx daemon --web 0.0.0.0:8099 # override the configured bind for one run
```

27
docs/configuration.md
View File

@@ -140,7 +140,7 @@ folder = "Accidental Tech Podcast" # default: the feed title
schedule = "every 6h" # overrides [general] for this feed
media_types = ["audio"] # overrides [general] for this feed
category = "Technology" # the Directory's, if the feed names none
listed = true # in the Directory with no subscribers
listed = true # shared with the directory with no subscribers
username = "ray" # HTTP basic auth
password_env = "IPX_ATP_PASS" # preferred over a literal `password`
```
@@ -150,9 +150,14 @@ With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
and are the same for everyone. See [users.md](users.md).
**The Directory** lists every feed in the catalogue, whether or not anyone subscribes to it yet.
To put one there for others to find, `ipx add --list --category News <url>`: it subscribes no one,
and a listed feed stays when its last subscriber leaves, where another is dropped. A feed nobody
**The Directory** is the directory service's ([directory.md](directory.md)), which an admin sets
up in Admin, Directory: its address, this server's token and whether to share. They are server
settings, kept in the database (`directory_url`, `directory_token`, `directory_share`), not here.
With sharing on, this server sends it every feed in the catalogue that is not private, with how
many people here subscribe, and the `category` above for a feed that names none. To offer one with
nobody subscribed, `ipx add --list --category News <url>`: it subscribes no one, and a listed feed
stays in the catalogue, and so in what is shared, when its last subscriber leaves, where another
is dropped. A feed nobody
subscribes to is checked once a day. One that is dead (failing for 30 days) or quiet (nothing
new in a year), with nobody subscribed and no file on disk, is removed from the catalogue.
@@ -172,3 +177,17 @@ and they are re-derived on every scan. Editing one in the UI promotes it to a ca
| `IPX_UI_LOG` | What the in-process log buffer captures for the UI's Log view |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | An OTLP/HTTP collector, such as Tempo at `http://host:4318`: the daemon sends it traces of scans, downloads and web requests. The other `OTEL_EXPORTER_OTLP_*` variables apply too |
| `http_proxy` / `https_proxy` | Honoured for feed and enclosure fetches |
## ipx-web
The page's container reads only its environment (`web/Caddyfile`):
| Variable | Effect |
|---|---|
| `IPX_UPSTREAM` | Where ipx is, `host:port`; `ipx:8099` by default |
| `WEB_BIND` | Where to listen; `:8080` by default |
| `WEB_ROOT` | The built page; `/srv` in the image |
| `TUNNEL_PEER` | The one address whose `Cf-Access-*` headers are passed on to ipx: where cloudflared's requests come from. Unset passes none. See [sso.md](sso.md) |
| `WEB_TRACING` | `on` sends a span for each request to the collector the `OTEL_*` variables name (`OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf`, `OTEL_SERVICE_NAME`), continuing the page's trace into ipx. Off by default |
ipx-directory's settings are in [directory.md](directory.md).

7
docs/directory.md
View File

@@ -71,6 +71,13 @@ token_sha256 = "…"
`DIRECTORY_LOG_FORMAT=json` logs one JSON object a line, for Loki, as ipx's `IPX_LOG_FORMAT` does.
Each request is a line with its `method`, `path`, `status`, `duration_ms` and the `token`'s name.
With `OTEL_EXPORTER_OTLP_ENDPOINT` set (`http://host:4318`, OTLP over HTTP), it sends traces: a
span a request, a child of the caller's when the request carries a `traceparent`, as ipx's do, and
a trace a refresher round with a span a feed (#213). `OTEL_RESOURCE_ATTRIBUTES` tags them, as
production does with `deployment.environment.name=production`. Its version is in
`crates/directory/Cargo.toml`, logged at start and answered by `ipx-directory --version`; its
changelog is `crates/directory/CHANGELOG.md`.
## Running one
```yaml

18
docs/sso.md
View File

@@ -18,7 +18,7 @@ real setup and pointed at the wrong address.
```
browser ─► Cloudflare Access, app "ipodderx" ─── sign in ───► Authentik (OpenID Connect)
─► tunnel "rays-unraid" (the cloudflared container on Tower)
─► http://192.168.1.130:8099 ─► ipx
─► http://192.168.1.130:8099, ipx-web (Caddy) ─► ipx
```
Authentik is not in the request path. It is the identity provider Cloudflare Access asks. Access
@@ -38,7 +38,7 @@ request it forwards through the tunnel, and ipx signs that person in.
enabled = true
bind = "0.0.0.0:8099"
trusted_header = "Cf-Access-Authenticated-User-Email"
trusted_proxies = ["127.0.0.1", "::1", "192.168.64.1"]
trusted_proxies = ["127.0.0.1", "::1", "192.168.64.10"] # ipx-web; see "Behind ipx-web"
access_team = "rays-sdf1.cloudflareaccess.com"
access_aud = "8bfe73dfbc8c548d1cb5dc11c6db6887bcaf4f5144840396f83a620a140e1c4f"
auto_create_users = true
@@ -67,7 +67,7 @@ Cloudflare and Authentik were already right. Three things on the ipx side were n
### The address to trust, and why it is 192.168.64.1
`cloudflared` runs in its own container and reaches ipx through the host's published port. Docker
`cloudflared` runs in its own container and reaches ipx-web (before #203, ipx) through the host's published port. Docker
(iptables firewall backend) masquerades traffic between its bridge networks, so the tunnel's
requests arrive from the **gateway of ipx's own network**, `ipx_default`:
@@ -81,8 +81,8 @@ lists them in hex.)
The `ipx` project's compose file pins that network's subnet to `192.168.64.0/20`, so its gateway
stays `192.168.64.1` when the network is recreated. It was `192.168.16.1`, `content_default`'s, while
ipx was part of the `content` project. If the subnet is ever changed, update `trusted_proxies` to
match.
ipx was part of the `content` project. If the subnet is ever changed, update `TUNNEL_PEER` in the
compose file to match (before #203, `trusted_proxies`).
### Behind ipx-web (#203)
@@ -185,10 +185,12 @@ itself, with a **Proxy Provider** and an **outpost** that adds `X-authentik-user
## How this is secured
**The header is only believed from `trusted_proxies`.** Every other source is ignored, and the
request falls through to a session cookie or the shared token. That is the whole security boundary.
**The header is only passed on from `TUNNEL_PEER`, and only believed from `trusted_proxies`.**
Caddy strips `Cf-Access-*` from every request that does not come from `TUNNEL_PEER`, and ipx
ignores the header from anyone but Caddy, its one trusted proxy; such a request falls through to a
session cookie or the shared token. That is the whole security boundary.
With the tunnel reaching ipx through the host's port, `192.168.64.1` means **any container on Tower
With the tunnel reaching ipx-web through the host's port, `192.168.64.1` means **any container on Tower
that connects to `192.168.1.130:8099`**, not only `cloudflared`. Machines on the LAN, and Tower
itself, arrive under their own addresses and cannot set the header; the checks above show both
sides. Never list a LAN address or range: anyone there could then send

10
docs/users.md
View File

@@ -70,8 +70,10 @@ list with their own read state. Unsubscribing removes it from their list alone;
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
re-subscribing does not pull the back catalogue again.
**Directory** sits at the top of the feed list, above your own feeds. It opens on the ten feeds
with the most subscribers on this server, you included, ranked, then the categories, each a tile
**Directory** sits at the top of the feed list, above your own feeds. It shows the directory
service an admin has set up ([directory.md](directory.md)), which iPX servers share, and says so
when there is none. It opens on the ten feeds with the most subscribers on every server sharing
with it, you included, ranked, then the categories, each a tile
showing a few of its shows' covers. A category has a page of its own: its most subscribed feeds,
then all of them as a grid of cover art, with its subcategories along the top (Tech News under
News, Video Games under Leisure) and a choice of order. See all lists every feed, and the search
@@ -81,7 +83,7 @@ Blogs (the rest). A show's category is the one it gives itself in iTunes, or for
none, one an admin or iPX picked for it. Your own feeds are marked Subscribed.
It shows a title, artwork and a count, never a URL or who reads it. An OPML subscription is listed
as the feeds inside it, one by one, and never the OPML itself, so you can take just the shows you
want. Anything that looks private is left out: a login configured for the feed, credentials in its URL,
want. Anything private is left out: a feed someone marked Private in its settings, a login configured for the feed, credentials in its URL,
or a key such as `auth=` or `token=` in the query, or a feed from a paid-feed service such as
Patreon or Supercast, which put the key in the path, and any feed inside an OPML that looks private
itself. Those are someone's paid subscriptions, and listing them would let anyone here read what
@@ -99,7 +101,7 @@ removed there, so there is always someone who can manage the rest.
## Admin
The first account is an admin. An admin can change global settings (scanning interval, quota,
retention, media types, download folder), a feed's URL, folder and schedule, and who has an account
retention, media types, download folder, the directory), a feed's URL, folder and schedule, and who has an account
and who else is an admin, and read the log, which names everyone's feeds and sign-ins. Everyone else
gets the Settings and Log buttons hidden and a `403` if they ask anyway.