ipx/docs/directory.md

The directory

ipx-directory is a catalogue of public podcasts and blogs that ipx servers share (#204). An ipx server sends it the public feeds its people subscribe to, and how many of them do. The directory fetches each feed's title, artwork, category, description and latest twenty items every six hours, and any server holding a token can browse it. It never downloads a file, and it never stores one's address: every ipx server still fetches its own feeds.

This server's is at https://ipx-dir.sdf1.net, the ipx-directory service of the Arcane project ipx, with its database ipx_directory on the databases project's Postgres.

In ipx

An admin sets it up in Admin, Directory: the directory's address, the token its operator gave this server, and whether to share. ipx asks the directory on its people's behalf, so the page never talks to it and never sees the token or a feed's address. With no directory set up, the Directory says so and feeds are added by their address.

With sharing on, ipx sends its snapshot after every scan, whenever the catalogue or these settings change, and every half hour: each feed checked at least once, with how many people here subscribe, and the category an admin gave a feed that names none. It never sends an OPML or a Patreon creator itself (their feeds go instead), or a private feed. A feed's settings have a Private switch for a paid feed iPX does not recognise by its address; anyone subscribed may turn it on, and only an admin off.

Tokens

Everything but /healthz needs a token, sent as Authorization: Bearer <token>. Each one has a name and one of three levels of access:

	browse	send snapshots	remove, block, refresh feeds; drop a server's snapshot
ro	yes
rw	yes	yes
rwa	yes	yes	yes

Tokens are in the config file and nowhere else, and no level of access can make or revoke one over the API. To make one:

docker exec ipx-directory ipx-directory token new <name> --access ro|rw|rwa

It prints the token, to give to whoever it is for, and the lines to add to the config file. The file holds the token's SHA-256, never the token. The directory reads the file again within a minute of a change, so adding a token and revoking one (deleting its entry) need no restart. A revoked token also takes the counts its server sent. A file that does not load (a misspelled access, two tokens with one name) leaves the tokens as they were, and the log says why.

The config file

/config/directory.toml in the container (DIRECTORY_CONFIG, or --config), on Tower /mnt/fast/appdata/ipx-directory/directory.toml:

bind = "0.0.0.0:8100"
# The environment variable holding the database URL: postgres://... or sqlite://...
database_url_env = "DIRECTORY_DATABASE_URL"

[[tokens]]
name = "tower"
access = "rwa"
token_sha256 = "…"

DIRECTORY_LOG_FORMAT=json logs one JSON object a line, for Loki, as ipx's IPX_LOG_FORMAT does. Each request is a line with its method, path, status, duration_ms and the token's name.

Running one

services:
  ipx-directory:
    image: ipx-directory          # docker build -f Dockerfile.directory -t ipx-directory .
    restart: unless-stopped
    environment:
      DIRECTORY_DATABASE_URL: "postgres://ipx_directory:password@db-host:5432/ipx_directory"
    ports:
      - "8100:8100"
    volumes:
      - ./directory:/config:ro    # directory.toml
    healthcheck:
      test: ["CMD", "ipx-directory", "health"]

What it will not hold

A private feed is never meant to reach it. ipx leaves out a feed marked private, one with a username or password, and one whose address ipx_feed::url_looks_private catches: a paid-feed host, or a key in the query. The directory checks every address it is sent with the same function and refuses what it catches, so one server's miss does not publish someone's paid feed to every other. What both miss, an rwa token can remove: DELETE /v1/feeds/{id} removes the feed and blocks it, so no snapshot brings it back.

API

Route	Access
GET /healthz	none	ok when it can read its database
GET /v1/me	ro	the token's name and access
GET /v1/feeds	ro	every feed A to Z: id, url, title, image, category, subcategory, podcast, subscribers
GET /v1/popular?limit=10	ro	the most subscribed, at most 100
GET /v1/feeds/{id}	ro	one feed, with description, failing (why its last check failed, in words) and its latest twenty items (guid, title, link, published, duration, description), never their files
POST /v1/snapshot	rw	[{"url", "subscribers"}], at most 20,000: replaces everything this token sent before; answers what it accepted and how many it refused as private, blocked or invalid
DELETE /v1/feeds/{id}	rwa	remove and block
GET /v1/blocked, DELETE /v1/blocked/{id}	rwa	what is blocked; unblock
DELETE /v1/instances/{name}	rwa	drop one token's snapshot, its counts with it
POST /v1/feeds/{id}/refresh	rwa	fetch the feed now, answering with its page

A feed's id is the first 64 bits of the SHA-256 of its address, normalised (scheme and host in lower case, no default port or fragment), so every server arrives at the same id. subscribers is the sum over every token's latest snapshot. A feed is listed once the directory has read it as a feed, so an OPML or an address that never worked is not, and goes when no snapshot has it any more. Descriptions are the feed's own HTML, unsanitized: whoever shows them sanitizes them, as ipx does.