The style guide imports types and uses them unqualified. 222 places
wrote a path such as std::collections::HashMap or std::time::Duration
inline; each file now imports the type once, one use a module, and
names it bare. Done with a script, file by file, only where the name is
unambiguous there:
- one path for the name throughout the file, and not a name the file
already binds otherwise (its own types, other imports, a function's
local use);
- not a name generic enough to mislead bare (Error, Result, Value,
Mutex, Client, Request, Response, Command, Event, Context, Map, Ok);
- only code a normal build compiles, so no import is unused outside
tests or names a crate only tests depend on.
193 paths stay, those cases above. No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The style guide asks for doc comments on a library's public API. Every
type, function, constant and static the two libraries export now has
one, saying what it is for or what it does that its name does not:
fetch follows redirects itself so credentials stay on the feed's host,
Config::parse refuses repeated token names, Store::due_one is the
admin's Refresh. Fields and variants whose names say it all have none,
as the guide allows.
category_path's comment had slid above categories(), which then had
two; each has its own again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured against /src/styleguide/rustguide.md:
- Each use statement imports from one module, and free functions are
called through theirs (db::step, db::now, jsonwebtoken::decode,
crate::feed::redact, unix::signal) rather than imported bare.
- unsafe_code is forbidden workspace-wide. The one unsafe was a test's
set_var, which is unsound while other tests' threads read the
environment; the test now reads PATH, which every run has.
- Locks say why they may panic: expect("a panic elsewhere poisoned this
lock") in place of 25 unwraps, and unique_slug's endless search says
it cannot end.
- No combinator chains in for headers: each is a named variable first.
- #[must_use] on the 27 functions clippy's must_use_candidate names.
- A literal bool passed where its meaning is not plain gets its name
(/* keep_open */ true, /* piece */ false, ...).
No behaviour changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3bfd8f8 gave the refresher the guard with no exceptions, which broke
the browser suite: its directory refreshes fixture feeds on 127.0.0.1,
and five Directory tests waited on listings that never filled. The
cargo tests passed because none of them fetches over HTTP.
directory.toml takes fetch_internal as config.toml does, empty by
default; the suite's lists 127.0.0.1. Production's has none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The refresher fetches every feed a snapshot names, and anyone with a
write token sends snapshots; a listed feed's redirect goes wherever its
publisher likes. Nothing checked the destination, so the directory
fetched loopback, its Docker network and the LAN, and what answered
became a listing's title and description on its public pages.
Its client now has ipx-feed's guard (#320) with nothing allowed: no
feed the directory lists is on this network.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/, /all, /search and /category are anyone's to ask for (#215), and each
called Store::list: the whole catalogue, joined with the snapshots and
grouped, on the 10-connection pool the refresher and every server's
/v1 calls share, on the Postgres that holds production ipx's database.
The pages' Cache-Control did nothing for a request with a query string
of its own, such as /search?q=<random>.
Directory::pages_list keeps the list for a minute behind a mutex held
through the read, so a burst of misses waits on one query rather than
each sending its own. Filtering and rendering still happen per request,
in memory. /v1/feeds, behind a token, still reads the database.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The code was formatted by hand, and `cargo fmt --check` differed in 866
places. rustfmt.toml keeps the house's long lines (120) and short
expressions on one line, which halves what changes; this commit is only
what rustfmt does with it, and is named in .git-blame-ignore-revs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured 2026-10-16, uncompressed until now: the directory's answers (the feed list ipx reads,
603 KB of JSON, ~134 ms of a 159 ms Directory request), ipx's snapshot upload (~100 KB each half
hour), both /metrics (4.7 and 0.9 KB every 15 s) and every trace export to Tempo. Already
compressed: ipx-web's answers (zstd, #239), the home page and the directory's public pages
through Cloudflare (brotli, zstd).
- The directory compresses with brotli, zstd or gzip as asked, and unpacks a gzipped request.
- ipx sends its snapshot gzipped; the directory is deployed first.
- Both /metrics answer gzipped, as Prometheus asks.
- opentelemetry-otlp's gzip-http: OTEL_EXPORTER_OTLP_COMPRESSION=gzip compresses the export, set
for all three in production's compose file; Caddy's exporter reads the same variable.
- web/build.mjs writes .br (quality 11) and .gz (level 9) beside each text file, and Caddy serves
them in place (file_server precompressed): app.js 30 KB where zstd on the fly gave 37, app.css
11 where 14. A .br or .gz asked for by name is refused, or /admin.js.br would be the admin
page's script without the check that keeps it to admins; in a handle of its own, since a bare
respond runs after the handles.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With apple_country set, the directory fetches Apple's top 100 podcasts and the top 50 of each of
its 19 categories for that country (itunes.apple.com/{cc}/rss/toppodcasts), looks up each
show's RSS address (/lookup, 100 ids a call, three seconds apart, as Apple asks for no more than
about twenty a minute), and stores them as the snapshot apple-podcasts with no subscribers. The
refresher fetches them like any feed; the most-subscribed chart is unchanged. Apple's primary
genre goes in as the snapshot's category, used only while a feed names none.
The same weekly run reads Apple's genre tree (ws/genres?id=26) and warns when it differs from
ipx_feed's CATEGORIES; on 2026-10-15 they were the same, all 19 and their subcategories.
A syncs table keeps when it last ran: at start if a week has passed, again weekly, an hour after
a failure. keep_instances spares apple-podcasts, which no token holds, and no token may take
the name. A trace a sync, a span each answer.
CLAUDE.md: builds one cargo command at a time, the load looked at before and after, after a
build chained with a test compile took Tower past 200.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on 2026-10-15: the feed list and item lists went out uncompressed at over 100 KB; ipx
read 2.6 GB of feed bodies a day and the directory 3.9 GB.
- ipx-web compresses with zstd or gzip, leaving out /api/events, which a compressor would hold
back, and /media (#239).
- A body identical to the last one (SHA-256 in feeds.body_hash) is taken as a 304, unless
something is missing a read would fill or the check is forced; and a feed without a schedule
of its own whose body has been the same waits a twelfth of that, up to four hours, beside the
failure backoff (#242). 35 Patreon show feeds sent ~300 KB each hourly, about 250 MB a day.
- The directory checks each feed every twelve hours, not six (#243).
- reqwest accepts brotli and zstd too (#244).
#240 (cache the feed list's counts) is left open: at production's size on the test Postgres
that count takes 12 ms and All Subscriptions 32 ms, against 366 and 300 ms in production, so
the cause is production's database, not the query (#241).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fetch that failed logged reqwest's words, which name the whole address: Patreon's auth key
went to Loki on 2026-10-15, and, since scan events go to every open page, to everyone's
browser too. #225 had taken queries out of the traces' address fields only.
Every event is redacted where it is emitted, for the log, the live stream and a terminal
alike: address fields lose their query and password, and so does any address in an error's
words (feed::redact_text). The artwork proxy's warnings and the directory's failed checks
the same. The error kept on the feed, shown to its own subscribers, keeps the detail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the slow traces of 2026-10-13: the directory checked a round's 25 feeds one after
another, so two libsyn feeds timing out at 30 s made a round 70 s; a forced scan looked up
62 site icons in turn, 22 s of 62; mjg59.dreamwidth.org takes the connection and never
answers, so it ran the full 30 s in every scan it was in; and since #225 strips queries,
every YouTube fetch span read the same address, outside any span naming its feed.
The directory now checks five at a time (SQLite stays on one connection, so its writes
queue). A forced scan looks up the icon in the fetch-ahead task, which already runs six
deep, and hands it to scan_one with the body. Both feed clients give up after 15 s of
silence (no successful fetch took over 8 s in a day of traces); FEED_TIMEOUT stays 30 s
for a feed still sending. The directory's client had no connect timeout; it has ipx's 10 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ipx's resident memory climbed from about 40 MB to 500-850 MB each night while scans ran five
times as often, and stayed there until a restart; the directory sat at 1.25 GiB idle. glibc's
allocator keeps what each thread's arena freed. mimalloc gives freed pages back, so resident
memory now follows what is in use. jemalloc was the first choice for its in-use figures, but its
upstream was archived in 2025.
Declared once in the directory crate, which ipx builds in, so both binaries get it.
What was missing to see it: the directory served no /metrics (now on metrics_bind, a port apart
from the tunnel's), nothing counted the feed bytes fetched, and the fetch span had no body size.
The feed crate's spans never reached Tempo either: both trace exporters filtered to their own
crate, so ipx_feed's fetch and parse spans were dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The directory renders its own pages from the store: the most subscribed, categories, A to Z,
search, and a page per feed with its address and a Copy button. No token, nothing that writes; /v1
keeps its token check. The publisher's HTML is shown as text, item links only when http(s), with
a strict Content-Security-Policy; covers load from their publishers with no referrer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Directory feed's page offers Check now and Remove to an admin whose token is rwa (the page
learns it from can_admin on the feed's page, asked of the directory only for admins). Admin,
Directory lists the servers sharing, from a new GET /v1/instances, and the blocked feeds by
title: a feed is often blocked because its address carries a key, so the address stays in the
directory. Unblocking or dropping a snapshot sends this server's snapshot again at once; since
directory's 403 reaches the page as 403, not 502.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The exporter setup and the traceparent parser move from ipx into ipx_directory::otel, which ipx
already depends on, so both binaries share them. The directory makes a span a request, a child of
the caller's traceparent, and a trace a refresher round with a span a feed; ipx sends the current
span's traceparent on every call to the directory and wraps each share in a span. The dashboard's
Traces row and the prod-check skill cover all three services.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Caddy logs one JSON line a request, filtered: no ?token= in the address and no request
headers, which hold the session cookie and Access's token. Its default log, errors included,
has the same filter, since an error line carries the request too. The directory logs a line a
refresher round, so the dashboard can show its progress.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ipx asks ipx-directory on its people's behalf, with the token an admin set
in the new Admin, Directory tab, so the page never talks to it and never
sees the token or a feed's address. /api/directory, /api/popular and a
feed's Directory page answer from it; subscribing takes the address from
the directory and goes through Add feed's steps. With no directory set
up, the Directory says so. The local directory code is gone; its privacy
checks are the snapshot builder's now, and their tests moved with them.
With sharing on, ipx sends its snapshot after every scan, whenever the
config changes (set_cfg) and every half hour: each public feed and how
many here subscribe, never an OPML or Patreon creator itself or anything
private. A feed can be marked private in its settings, for a paid feed
whose address does not give it away; anyone subscribed may mark it, only
an admin unmark it. The admin's category for a feed that names none goes
with it, which the directory uses only while the feed names none, so the
category an admin gives a blog still reaches the Directory.
The directory lists only feeds it has read as a feed: the first snapshot
after a start went before the first scan, when an OPML was not yet known
to be one, and the directory listed the OPML. It now fetches what a
snapshot brings in at once instead of within the minute.
The browser suite runs its own ipx-directory on SQLite; the Directory's
tests match feeds by title, since the directory names them by address.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new crate and service, crates/directory, live at ipx-dir.sdf1.net. ipx
servers send the public feeds their people subscribe to (POST /v1/snapshot,
replacing what that token sent before); the directory fetches each feed's
title, artwork, category, description and latest twenty items every six
hours with ipx-feed, and lists them A to Z and by subscribers summed over
every server. It holds metadata only, never a file or a file's address.
Tokens are in its config file as SHA-256 hashes, each ro, rw or rwa, and
the file is read again within a minute of a change, so adding and revoking
one is an edit and nothing else; a revoked token takes its server's counts.
At first they were to come from environment variables, which would have
meant recreating the container to add one. A file that does not load keeps
the tokens as they were.
Every address sent is checked with ipx_feed::url_looks_private, the check
ipx makes before sending, so one server's miss does not publish a paid feed
to every other; an rwa token can remove and block what both miss.
Deployed: ipx-directory in the ipx Arcane project, database ipx_directory
(its own login) on the databases project's Postgres, which db-backup
already covers, and a tunnel route with no Access in front. The store's
tests run on Postgres as well as SQLite when IPX_TEST_DATABASE_URL is set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
