rays
c47aa57909
ipx asks ipx-directory on its people's behalf, with the token an admin set in the new Admin, Directory tab, so the page never talks to it and never sees the token or a feed's address. /api/directory, /api/popular and a feed's Directory page answer from it; subscribing takes the address from the directory and goes through Add feed's steps. With no directory set up, the Directory says so. The local directory code is gone; its privacy checks are the snapshot builder's now, and their tests moved with them. With sharing on, ipx sends its snapshot after every scan, whenever the config changes (set_cfg) and every half hour: each public feed and how many here subscribe, never an OPML or Patreon creator itself or anything private. A feed can be marked private in its settings, for a paid feed whose address does not give it away; anyone subscribed may mark it, only an admin unmark it. The admin's category for a feed that names none goes with it, which the directory uses only while the feed names none, so the category an admin gives a blog still reaches the Directory. The directory lists only feeds it has read as a feed: the first snapshot after a start went before the first scan, when an OPML was not yet known to be one, and the directory listed the OPML. It now fetches what a snapshot brings in at once instead of within the minute. The browser suite runs its own ipx-directory on SQLite; the Directory's tests match feeds by title, since the directory names them by address. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
115 lines
5.5 KiB
Markdown
115 lines
5.5 KiB
Markdown
# The directory
|
|
|
|
`ipx-directory` is a catalogue of public podcasts and blogs that ipx servers share (#204). An ipx
|
|
server sends it the public feeds its people subscribe to, and how many of them do. The directory
|
|
fetches each feed's title, artwork, category, description and latest twenty items every six hours,
|
|
and any server holding a token can browse it. It never downloads a file, and it never stores
|
|
one's address: every ipx server still fetches its own feeds.
|
|
|
|
This server's is at `https://ipx-dir.sdf1.net`, the `ipx-directory` service of the Arcane project
|
|
`ipx`, with its database `ipx_directory` on the `databases` project's Postgres.
|
|
|
|
## In ipx
|
|
|
|
An admin sets it up in **Admin, Directory**: the directory's address, the token its operator gave
|
|
this server, and whether to share. ipx asks the directory on its people's behalf, so the page never
|
|
talks to it and never sees the token or a feed's address. With no directory set up, the Directory
|
|
says so and feeds are added by their address.
|
|
|
|
With sharing on, ipx sends its snapshot after every scan, whenever the catalogue or these settings
|
|
change, and every half hour: each feed checked at least once, with how many people here subscribe,
|
|
and the category an admin gave a feed that names none. It never sends an OPML or a Patreon creator
|
|
itself (their feeds go instead), or a private feed. A feed's settings have a **Private** switch for
|
|
a paid feed iPX does not recognise by its address; anyone subscribed may turn it on, and only an
|
|
admin off.
|
|
|
|
## Tokens
|
|
|
|
Everything but `/healthz` needs a token, sent as `Authorization: Bearer <token>`. Each one has a
|
|
name and one of three levels of access:
|
|
|
|
| | browse | send snapshots | remove, block, refresh feeds; drop a server's snapshot |
|
|
|---|---|---|---|
|
|
| `ro` | yes | | |
|
|
| `rw` | yes | yes | |
|
|
| `rwa` | yes | yes | yes |
|
|
|
|
Tokens are in the config file and nowhere else, and no level of access can make or revoke one
|
|
over the API. To make one:
|
|
|
|
```sh
|
|
docker exec ipx-directory ipx-directory token new <name> --access ro|rw|rwa
|
|
```
|
|
|
|
It prints the token, to give to whoever it is for, and the lines to add to the config file. The file
|
|
holds the token's SHA-256, never the token. The directory reads the file again within a minute of
|
|
a change, so adding a token and revoking one (deleting its entry) need no restart. A revoked token
|
|
also takes the counts its server sent. A file that does not load (a misspelled `access`, two tokens
|
|
with one name) leaves the tokens as they were, and the log says why.
|
|
|
|
## The config file
|
|
|
|
`/config/directory.toml` in the container (`DIRECTORY_CONFIG`, or `--config`), on Tower
|
|
`/mnt/fast/appdata/ipx-directory/directory.toml`:
|
|
|
|
```toml
|
|
bind = "0.0.0.0:8100"
|
|
# The environment variable holding the database URL: postgres://... or sqlite://...
|
|
database_url_env = "DIRECTORY_DATABASE_URL"
|
|
|
|
[[tokens]]
|
|
name = "tower"
|
|
access = "rwa"
|
|
token_sha256 = "…"
|
|
```
|
|
|
|
`DIRECTORY_LOG_FORMAT=json` logs one JSON object a line, for Loki, as ipx's `IPX_LOG_FORMAT` does.
|
|
Each request is a line with its `method`, `path`, `status`, `duration_ms` and the `token`'s name.
|
|
|
|
## Running one
|
|
|
|
```yaml
|
|
services:
|
|
ipx-directory:
|
|
image: ipx-directory # docker build -f Dockerfile.directory -t ipx-directory .
|
|
restart: unless-stopped
|
|
environment:
|
|
DIRECTORY_DATABASE_URL: "postgres://ipx_directory:password@db-host:5432/ipx_directory"
|
|
ports:
|
|
- "8100:8100"
|
|
volumes:
|
|
- ./directory:/config:ro # directory.toml
|
|
healthcheck:
|
|
test: ["CMD", "ipx-directory", "health"]
|
|
```
|
|
|
|
## What it will not hold
|
|
|
|
A private feed is never meant to reach it. ipx leaves out a feed marked private, one with a
|
|
username or password, and one whose address `ipx_feed::url_looks_private` catches: a paid-feed
|
|
host, or a key in the query. The directory checks every address it is sent with the same function
|
|
and refuses what it catches, so one server's miss does not publish someone's paid feed to every
|
|
other. What both miss, an `rwa` token can remove: `DELETE /v1/feeds/{id}` removes the feed and
|
|
blocks it, so no snapshot brings it back.
|
|
|
|
## API
|
|
|
|
| Route | Access | |
|
|
|---|---|---|
|
|
| `GET /healthz` | none | `ok` when it can read its database |
|
|
| `GET /v1/me` | ro | the token's `name` and `access` |
|
|
| `GET /v1/feeds` | ro | every feed A to Z: `id`, `url`, `title`, `image`, `category`, `subcategory`, `podcast`, `subscribers` |
|
|
| `GET /v1/popular?limit=10` | ro | the most subscribed, at most 100 |
|
|
| `GET /v1/feeds/{id}` | ro | one feed, with `description`, `failing` (why its last check failed, in words) and its latest twenty `items` (`guid`, `title`, `link`, `published`, `duration`, `description`), never their files |
|
|
| `POST /v1/snapshot` | rw | `[{"url", "subscribers"}]`, at most 20,000: replaces everything this token sent before; answers what it `accepted` and how many it refused as private, blocked or invalid |
|
|
| `DELETE /v1/feeds/{id}` | rwa | remove and block |
|
|
| `GET /v1/blocked`, `DELETE /v1/blocked/{id}` | rwa | what is blocked; unblock |
|
|
| `DELETE /v1/instances/{name}` | rwa | drop one token's snapshot, its counts with it |
|
|
| `POST /v1/feeds/{id}/refresh` | rwa | fetch the feed now, answering with its page |
|
|
|
|
A feed's `id` is the first 64 bits of the SHA-256 of its address, normalised (scheme and host in
|
|
lower case, no default port or fragment), so every server arrives at the same id. `subscribers`
|
|
is the sum over every token's latest snapshot. A feed is listed once the directory has read it as
|
|
a feed, so an OPML or an address that never worked is not, and goes when no snapshot has it any more. Descriptions
|
|
are the feed's own HTML, unsanitized: whoever shows them sanitizes them, as ipx does.
|