rays/ipx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 18 Wiki Activity

Split: the directory service at ipx-dir.sdf1.net #204

New Issue
Closed
opened 2026-10-11 20:01:39 -07:00 by rays · 1 comment
rays commented 2026-10-11 20:01:39 -07:00
Owner
Copy Link

Part of splitting ipx into a directory service, a backend and a frontend (see the tracking issue).

A new service, ipx-directory (crates/directory), at https://ipx-dir.sdf1.net: a catalogue of public podcasts that ipx servers share. It is metadata only: it never downloads a file, and each ipx server still fetches its own feeds.

Data: its own Postgres database (ipx_directory). Per feed: URL, title, art, category, description, whether it carries audio or video, and its latest twenty items (title, link, date, length, text). Feed id is a hash of the normalised URL, so every instance agrees on it. A slow refresher fetches each feed with conditional GET, using crates/feed.

Access: a config file, no admin UI, no token database:

bind = "0.0.0.0:8100"
database_url_env = "DIRECTORY_DATABASE_URL"

[[tokens]]
name = "tower"
access = "rwa"            # ro | rw | rwa, no default
token_env = "DIRECTORY_TOKEN_TOWER"
  • ro: browse. rw: browse and send snapshots. rwa: also admin (below).
  • No token or an unknown one: 401. Too little access: 403.
  • The file is re-read when it changes (mtime checked each minute). A token removed loses its access and its snapshot; a file that fails to load leaves the previous config in place and logs why. Duplicate names are refused.
  • ipx-directory token new <name> --access ro|rw|rwa prints a random token and the lines to paste.
  • Requests are logged with the token's name, never the token.
  • Tokens are only ever changed in the file: no access level can make or revoke one over the API.

API

access
GET /v1/me ro the token's name and access
GET /v1/feeds, GET /v1/popular, GET /v1/feeds/{id} ro A to Z with category; most subscribed; one feed with its items
POST /v1/snapshot rw [{url, subscribers}], replacing that token's last snapshot; popular is the sum of every token's latest
DELETE /v1/feeds/{id} rwa remove and block, so no snapshot brings it back
GET /v1/blocked, DELETE /v1/blocked/{id} rwa list, unblock
DELETE /v1/instances/{name} rwa drop one token's snapshot
POST /v1/feeds/{id}/refresh rwa refetch metadata now

A snapshot is checked with looks_private() again on arrival, as well as before it is sent, and anything private is refused.

Deploy: image 192.168.1.130:5000/ipx-directory, a service in Arcane, its database on the databases Postgres, the tunnel route for ipx-dir.sdf1.net, JSON logs for Loki.

Part of splitting ipx into a directory service, a backend and a frontend (see the tracking issue). A new service, `ipx-directory` (`crates/directory`), at https://ipx-dir.sdf1.net: a catalogue of public podcasts that ipx servers share. It is metadata only: it never downloads a file, and each ipx server still fetches its own feeds. **Data**: its own Postgres database (`ipx_directory`). Per feed: URL, title, art, category, description, whether it carries audio or video, and its latest twenty items (title, link, date, length, text). Feed id is a hash of the normalised URL, so every instance agrees on it. A slow refresher fetches each feed with conditional GET, using `crates/feed`. **Access**: a config file, no admin UI, no token database: ```toml bind = "0.0.0.0:8100" database_url_env = "DIRECTORY_DATABASE_URL" [[tokens]] name = "tower" access = "rwa" # ro | rw | rwa, no default token_env = "DIRECTORY_TOKEN_TOWER" ``` * `ro`: browse. `rw`: browse and send snapshots. `rwa`: also admin (below). * No token or an unknown one: 401. Too little access: 403. * The file is re-read when it changes (mtime checked each minute). A token removed loses its access and its snapshot; a file that fails to load leaves the previous config in place and logs why. Duplicate names are refused. * `ipx-directory token new <name> --access ro|rw|rwa` prints a random token and the lines to paste. * Requests are logged with the token's name, never the token. * Tokens are only ever changed in the file: no access level can make or revoke one over the API. **API** | | access | | |---|---|---| | `GET /v1/me` | ro | the token's name and access | | `GET /v1/feeds`, `GET /v1/popular`, `GET /v1/feeds/{id}` | ro | A to Z with category; most subscribed; one feed with its items | | `POST /v1/snapshot` | rw | `[{url, subscribers}]`, replacing that token's last snapshot; popular is the sum of every token's latest | | `DELETE /v1/feeds/{id}` | rwa | remove and block, so no snapshot brings it back | | `GET /v1/blocked`, `DELETE /v1/blocked/{id}` | rwa | list, unblock | | `DELETE /v1/instances/{name}` | rwa | drop one token's snapshot | | `POST /v1/feeds/{id}/refresh` | rwa | refetch metadata now | A snapshot is checked with `looks_private()` again on arrival, as well as before it is sent, and anything private is refused. **Deploy**: image `192.168.1.130:5000/ipx-directory`, a service in Arcane, its database on the `databases` Postgres, the tunnel route for ipx-dir.sdf1.net, JSON logs for Loki.
rays added the enhancement label 2026-10-11 20:01:39 -07:00
rays commented 2026-10-11 20:51:54 -07:00
Author
Owner
Copy Link

Done in 83bda97 (and the Dockerfile fix after it). Live at https://ipx-dir.sdf1.net: ipx-directory in the ipx Arcane project, database ipx_directory on the databases Postgres (covered by db-backup's ALL), tunnel route with no Access. One change from the plan: tokens are in directory.toml as SHA-256 hashes, not env vars, so adding one is a file edit too, not only revoking. Tower's rwa token is in /mnt/fast/appdata/ipx-directory/tower.token until #205 puts it in ipx's settings. Checked: a feed snapshot, a Supercast address refused as private, a refresh of The Daily with its category and 20 items and no file addresses; 401 without a token from the public address. Not done: Cloudflare rate limiting in front (tokens gate everything, so it is left until the log shows a need); following a feed that moved for good (ponytail comment in refresh).

Done in 83bda97 (and the Dockerfile fix after it). Live at https://ipx-dir.sdf1.net: ipx-directory in the ipx Arcane project, database ipx_directory on the databases Postgres (covered by db-backup's ALL), tunnel route with no Access. One change from the plan: tokens are in directory.toml as SHA-256 hashes, not env vars, so adding one is a file edit too, not only revoking. Tower's rwa token is in /mnt/fast/appdata/ipx-directory/tower.token until #205 puts it in ipx's settings. Checked: a feed snapshot, a Supercast address refused as private, a refresh of The Daily with its category and 20 items and no file addresses; 401 without a token from the public address. Not done: Cloudflare rate limiting in front (tokens gate everything, so it is left until the log shows a need); following a feed that moved for good (ponytail comment in refresh).
rays closed this issue 2026-10-11 20:51:55 -07:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 wontfix
This won't be fixed
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
rays
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rays/ipx#204
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?