ipx's clients had no destination policy and followed redirects
anywhere. Adding a feed returned the status or first line of whatever
answered at the address; an enclosure declared as audio was downloaded
whole from any URL and served back through /media; /api/art fetched
any image a feed named. Feeds come from publishers and anyone signed
in, so loopback, the Docker network and the LAN were all reachable.
ipx-feed's guard is a connector layer: after the name resolves and the
connection opens, before the request is written, it looks at the
peer's address and refuses anything off the public internet unless
config.toml's [general] fetch_internal lists it. Being on the
connection, it covers every request on ctx.client and feed_client,
each redirect hop, IP literals (which never reach a DNS resolver) and
a name that resolves differently the second time. reqwest's rustls
connection reports the peer address; a test run by hand checks HTTPS
gets through, since were it lost every https fetch would be refused.
The directory is on the server's own network by design
(http://ipx-directory:8100), so it moves to local_client, unguarded,
set by an admin. The browser suite and load tests serve their feeds on
loopback and list 127.0.0.1.
Not covered: librqbit's own connections (trackers, peers), which do
not go through these clients.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A description with Markdown and no HTML tag is rendered to HTML with
pulldown-cmark, and an item with no text but RSS's <source:markdown>
(https://source.scripting.com/, as Scripting News sends beside its HTML) uses
that. HTML is left alone, and plain text without Markdown in it stays as it
was, so a line like "1984. ..." is not made a list. ammonia cleans the result
as any feed's HTML. Stored items keep what they have: the scanner does not
rewrite an item it has.
Reference: http://scripting.com/2026/10/06/005559.html
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Linking ipx's 431 MB test binary took Tower, 31 GB with no swap and 4 GB free
beside its containers, into page-fault thrashing for 54 minutes: load 160 with
the CPU idle and 130 processes blocked. Line tables for our crates and no debug
info for dependencies make it 109 MB, built in 2m14s with load at 20 and 6.5 GB
left. CLAUDE.md says to watch memory beside the load.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured 2026-10-16, uncompressed until now: the directory's answers (the feed list ipx reads,
603 KB of JSON, ~134 ms of a 159 ms Directory request), ipx's snapshot upload (~100 KB each half
hour), both /metrics (4.7 and 0.9 KB every 15 s) and every trace export to Tempo. Already
compressed: ipx-web's answers (zstd, #239), the home page and the directory's public pages
through Cloudflare (brotli, zstd).
- The directory compresses with brotli, zstd or gzip as asked, and unpacks a gzipped request.
- ipx sends its snapshot gzipped; the directory is deployed first.
- Both /metrics answer gzipped, as Prometheus asks.
- opentelemetry-otlp's gzip-http: OTEL_EXPORTER_OTLP_COMPRESSION=gzip compresses the export, set
for all three in production's compose file; Caddy's exporter reads the same variable.
- web/build.mjs writes .br (quality 11) and .gz (level 9) beside each text file, and Caddy serves
them in place (file_server precompressed): app.js 30 KB where zstd on the fly gave 37, app.css
11 where 14. A .br or .gz asked for by name is refused, or /admin.js.br would be the admin
page's script without the check that keeps it to admins; in a handle of its own, since a bare
respond runs after the handles.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on 2026-10-15: the feed list and item lists went out uncompressed at over 100 KB; ipx
read 2.6 GB of feed bodies a day and the directory 3.9 GB.
- ipx-web compresses with zstd or gzip, leaving out /api/events, which a compressor would hold
back, and /media (#239).
- A body identical to the last one (SHA-256 in feeds.body_hash) is taken as a 304, unless
something is missing a read would fill or the check is forced; and a feed without a schedule
of its own whose body has been the same waits a twelfth of that, up to four hours, beside the
failure backoff (#242). 35 Patreon show feeds sent ~300 KB each hourly, about 250 MB a day.
- The directory checks each feed every twelve hours, not six (#243).
- reqwest accepts brotli and zstd too (#244).
#240 (cache the feed list's counts) is left open: at production's size on the test Postgres
that count takes 12 ms and All Subscriptions 32 ms, against 366 and 300 ms in production, so
the cause is production's database, not the query (#241).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ipx's resident memory climbed from about 40 MB to 500-850 MB each night while scans ran five
times as often, and stayed there until a restart; the directory sat at 1.25 GiB idle. glibc's
allocator keeps what each thread's arena freed. mimalloc gives freed pages back, so resident
memory now follows what is in use. jemalloc was the first choice for its in-use figures, but its
upstream was archived in 2025.
Declared once in the directory crate, which ipx builds in, so both binaries get it.
What was missing to see it: the directory served no /metrics (now on metrics_bind, a port apart
from the tunnel's), nothing counted the feed bytes fetched, and the fetch span had no body size.
The feed crate's spans never reached Tempo either: both trace exporters filtered to their own
crate, so ipx_feed's fetch and parse spans were dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The workspace stops holding one version. ipx keeps 0.12.1-dev and its vX.Y.Z tags; the directory
starts at 0.1.0-dev, tagged directory-vX.Y.Z, and the page at 0.1.0-dev in package.json, tagged
web-vX.Y.Z, which web/build.mjs gives the script for the logo's tooltip beside ipx's. crates/feed
is a library, not released. CLAUDE.md's release steps are for one service at a time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new crate and service, crates/directory, live at ipx-dir.sdf1.net. ipx
servers send the public feeds their people subscribe to (POST /v1/snapshot,
replacing what that token sent before); the directory fetches each feed's
title, artwork, category, description and latest twenty items every six
hours with ipx-feed, and lists them A to Z and by subscribers summed over
every server. It holds metadata only, never a file or a file's address.
Tokens are in its config file as SHA-256 hashes, each ro, rw or rwa, and
the file is read again within a minute of a change, so adding and revoking
one is an edit and nothing else; a revoked token takes its server's counts.
At first they were to come from environment variables, which would have
meant recreating the container to add one. A file that does not load keeps
the tokens as they were.
Every address sent is checked with ipx_feed::url_looks_private, the check
ipx makes before sending, so one server's miss does not publish a paid feed
to every other; an rwa token can remove and block what both miss.
Deployed: ipx-directory in the ipx Arcane project, database ipx_directory
(its own login) on the databases project's Postgres, which db-backup
already covers, and a tunnel route with no Access in front. The store's
tests run on Postgres as well as SQLite when IPX_TEST_DATABASE_URL is set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first step of the split (#207). The daemon moves to crates/ipx unchanged;
feed.rs becomes crates/feed (ipx-feed), which the directory service (#204)
will share for its metadata refresher and its privacy check.
To stand alone, fetch takes the feed's address and its credentials instead
of ipx's config::Feed (config::Feed::auth gives them), collect_outlines
moves in with the OPML parsing, and the address half of looks_private is
feed::url_looks_private. ipx imports the crate as `feed` at its root, so
every crate::feed path is unchanged. The version is the workspace's, in the
root Cargo.toml.
Checked: cargo test on Postgres (118 + 29), tsc, page smoke, native bridge,
contrast and Playwright (72) pass; the image built from the new Dockerfile
is deployed and healthy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For anyone who turns DeadAir on, a background task makes each episode a
copy with every silence quieter than -38 dB for over 0.35 s shortened to
0.2 s. Nothing is re-encoded: ffmpeg copies the original's frames and
leaves the silent ones out, about 11 s for an hour of MP3 in a release
build. On Brad & Will's episode 359 that is 520 cuts and 3.3 minutes
saved, the copy decodes clean, and it has no isolated spikes in its
silences that the original lacks.
Cuts are made from the middle of each silence, keeping 0.1 s at each
end, so the frames either side of a cut are silent: decoding blends a
frame with the one before it, and an MP3 frame can begin in the bytes of
the frame before, so the first frame after a cut decodes against the
wrong neighbour. Inside a silence nobody hears that; at a word they would.
Positions stay in the original's time and the server converts them both
ways. The iOS and Android apps save positions themselves from their own
player, even while the page is asleep, so converting in the page would
have stored the copy's times as the original's. /media/{id}/deadair
serves the copy at its own address and records that this person is
playing it; a position saved meanwhile is converted from the copy's
clock, as is the length a player measures.
The copy is made by a task of its own, not the worker: a first pass over
a library is hours of ffmpeg, and scans must not wait behind it. It
checks every two minutes, which covers new downloads, files already
downloaded, a switch turned on and a file downloaded again with one rule.
Episodes everyone who would hear the copy has finished get none.
download::remove deletes a copy with its original, and the quota counts
copies.
Settings: the switch is PATCH /api/me {deadair}, a feed kept out is
PATCH /api/feeds/{id} {deadair: false}. GET /api/me gives the switch and
deadair_saved, the seconds saved on episodes heard 75% or more on a copy,
each counted once. Three new tables, so no ALTER.
The image gains Debian's ffmpeg, about 480 MB, in a layer that stays
cached until the base image changes. The player's switch, the feed
setting and the stat come next, after mockups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API took a session cookie, a proxy's word or the shared admin token, so a script or an
agent working for one person had to sign in with their password and carry the cookie, or be
given the admin token. Settings now makes named tokens, ipx_ and 256 random bits, sent as
Authorization: Bearer. A token is its owner and no more. Only its SHA-256 is kept, in the
new api_tokens table, with when it was made and last used; it is shown once and revoked from
the same list. An unknown or revoked one gets a 401 rather than falling through to a cookie.
Cloudflare Access still stands in front of the tunnel, so from outside a token needs an
Access service token beside it; docs/sso.md says how.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The log was text, so the Grafana dashboard picked lines apart with
regular expressions, and a change of wording would have blanked its
panels. With IPX_LOG_FORMAT=json each line is one JSON object: the
access log carries method, path, route, status and ms as fields (the
route passed from the routing layer in the response's extensions), and
each wire event its ev, feed, new, downloaded, failed, bytes, msg and
the rest (log_wire), beside the old message. The two startup lines that
were println! are logged, so no line breaks the JSON. Text stays the
default, for a terminal. The dashboard reads the fields with Loki's json
parser, and groups requests by route rather than path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ipx had no spans, only log lines, so there was no way to see where a
slow scan, download or request spent its time. With
OTEL_EXPORTER_OTLP_ENDPOINT set, the daemon now exports traces over
OTLP/HTTP (Tempo on Tower): a scan, each feed in it, the feed fetch and
site icon lookup, downloads, torrents, reaps, and web requests. Log lines
inside a span ride along as its events.
Only the daemon exports: the healthcheck runs ipx status every 30s and
would bury everything else. The web event stream and the log view's
polling get no span, for the same reason. The exporter shares ipx's
reqwest 0.13, so no second HTTP stack comes in.
The stderr log now prefixes lines inside a span with it, as
tracing-subscriber's fmt layer does (scan{only=None force=false}: ...).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Production ran four commits past v0.9.0 while the logo's tooltip said
0.9.0, because Cargo.toml's version only moved at a release. It is now
0.9.1-dev, and CLAUDE.md's release steps end by moving to the next -dev
version. No commit hash: the name says there is newer work, and git says
which.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Glass theme (#43), playback handed to a native shell (#45), and the
bottom bar kept clear of the home indicator (#46), which had no changelog
entry of its own. The unreleased compare link had been left at v0.6.1 since
0.7.0; it points at the new tag now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The proxy sign-in believed Cf-Access-Authenticated-User-Email from any
address in trusted_proxies. On Tower that address is the Docker gateway,
so any container there could name itself anyone (docs/sso.md said as
much, and CLAUDE.md listed it as a known gap).
With [web] access_team and access_aud set, a proxied request must also
carry a Cf-Access-Jwt-Assertion that verifies against Cloudflare's keys
(RS256 only, this application's audience, the team's issuer, not
expired), and the name comes from its email claim. The keys are fetched
at start and again when a token names an unseen key, at most once a
minute, so made-up key ids cannot make every request a request to
Cloudflare. While the keys cannot be had, proxied sign-in is refused;
password and token sign-in are unaffected. Both settings empty, nothing
changes.
jsonwebtoken does the checking, on the aws-lc-rs backend already in the
tree through rustls. Tests sign with throwaway keys in tests/data: a
valid token, another app's audience, expired, a forged signature, HS256,
alg none, the refetch limit, and keys that cannot be fetched. Checked
live on a scratch daemon: the header alone and a forged token got 401,
the admin token still signed in.
vouched_name takes the peer and headers rather than the request: a
&Request held across the new await made the auth middleware's future
unsendable, as a body is not Sync.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last nineteen functions move to SeaORM: recording feeds, items and
enclosures, managed OPML feeds, folding WordPress's repeated files, and handing a
Patreon creator's files to its shows. Two SQLite-only forms go: GLOB becomes a
LIKE with the underscore escaped (broader, harmlessly: the fold still keys on
`_=` and digits), and UPDATE OR IGNORE becomes an UPDATE ... WHERE NOT EXISTS.
The two transactions are SeaORM transactions.
With nothing left on it, rusqlite goes, with the SQL schema and migrate(). The
entities are the schema: create_missing makes whatever tables and indexes a
database lacks, from them, with CREATE ... IF NOT EXISTS. Production's schema
already has every column migrate() added and none it dropped.
Not SeaORM's schema sync, used until now: despite its docs it drops a unique
index the entities do not describe, so it dropped users_name_lower on every open.
Every `ipx` command then took a write lock, and against a daemon busy writing,
`ipx status` -- the healthcheck -- failed 7 times in 15 where the old code
failed none. Now 15 in 15, as before. On Postgres it would not have started.
WAL is set only when a file is not already in it: setting it takes a lock that
cannot wait out a busy daemon.
Checked on copies of production: a forced scan of all 162 feeds against the real
feeds with no database errors; the feed list, filters, sorts, search and the
reaper's candidates against the old code on the same data, earlier in the
branch. The column comments from the SQL schema move to the entities.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first step of moving to SeaORM (#18, phase 1). Nothing a user sees changes.
- src/entity.rs: the seven tables as SeaORM entities, matching the SQLite schema.
Strings are Text, as the columns are; yes/no columns are bool, which is BOOLEAN
on Postgres and stays INTEGER in the existing SQLite file (sync notes the
difference and leaves it alone).
- Db holds a SeaORM connection to the same SQLite file beside the rusqlite one;
functions move to it one at a time, and rusqlite goes with the last of them.
- db::sync creates what a database is missing from the entities (SeaORM's
schema-sync, experimental, so sea-orm is pinned to ~2.0), plus the two indexes
an entity cannot express. Checked against a copy of production: it added the
lower(name) index and changed nothing else.
- Test databases are now built from the entities alone, in a temporary file
(two connections to one ":memory:" are two databases), so every test also
checks that the entities describe what the queries need. That caught the one
difference: finding a user by name relied on COLLATE NOCASE, which Postgres
lacks; it now compares lower() on both sides.
- rusqlite steps back to 0.39: 0.40's libsqlite3-sys is newer than sqlx accepts,
and only one may link SQLite. It goes away at the end of this phase.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An Atom title of type="html", and an RSS title in CDATA, reach the parser
with their entities intact, so The Verge's "Meta’s" showed as typed:
55 stored titles across 17 feeds. Titles are decoded one entity at a time
with quick-xml's HTML5 table, leaving an & that starts none ("Q&A") alone
rather than failing the whole title.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Remember the feed/place and tab across a reload or new visit; an
unknown or unsubscribed one lands on All Subscriptions instead of the
first feed alphabetically.
- Add an Auto theme that follows the system's light/dark setting, and
move Dark/Light/Classic/Auto into Settings as a dropdown alongside the
header button's toggle.
- Add Currently Listening below Popular: episodes started and not
finished, across every subscribed feed, one tap to resume. Reuses the
existing entries/filter machinery (Filter::InProgress) rather than a
new endpoint.
- Likely fix for the iOS bug where the topbar stopped responding to taps
until a hard refresh (100vh -> 100dvh); unverified on a real device.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DmQfE1eFPApnXWyPHBWqUA
- Stop scanning an OPML/Patreon feed's derived rows once nobody subscribes
to it; retire them (drop or orphan) the way sync_group already does when
the list itself drops one. This is what let 922 defunct davewiner feeds
keep scanning hourly after the OPML left config.
- Repair feed XML with a bare `&`, and give a plain reason (moved web page
with its new address when linked, or nothing yet for an empty body)
instead of a raw parser error.
- Show a failing feed's plain-English reason and next step (Unsubscribe /
Use the new address) in the sidebar and on its own page, once it has
been down a day.
- Fix four small UI bugs: show-note links open in a new tab, video files
play as video, an opened item no longer disappears from the Unread tab,
and Subscribe/Unsubscribe get their own icons.
- Fix Settings disappearing for non-admin accounts: it was hiding the
whole modal instead of just the admin-only parts (Users, the editable
schedule/quota, Save), which are the only parts the server actually
refuses them.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DmQfE1eFPApnXWyPHBWqUA
Works through TODO.md from the 2026-09-12 over-engineering audit. Drops the
entries.read/flagged/position columns (migrate() removes them from older
databases), migrate_opml_children, the legacy interval_mins key, the
contrib/ systemd units, test-only Db wrappers, a duplicate token generator,
redundant logbuf visitors, unused page state and CSS, and the infer, dirs
and tokio-stream dependencies. The icon is served once as /icon.png instead
of inlined four times, taking about 94 KB off the two pages.
The adoption's subscription half was not dead: it gives a fresh install's
first admin the config's feeds. It stays as adopt_catalogue, now tested.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
The original iPodderX layout (toolbar, places, item table, Files pane),
the Classic theme, Directory and Popular, All Subscriptions with mark
everything read, sortable columns and a Size column, one meaning per
icon across the UI, and fixes for the double play, Escape in dialogs,
the dark-theme password box and paid feeds listed in Popular. See
CHANGELOG.md [0.4.0] and docs/history.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173mGu6rK18Ne7UGTwAaVJV
- Add feed lists what other accounts subscribe to, most subscribers
first, and subscribes you by id (GET /api/popular, POST
/api/popular/{id}). Rows never carry a URL. Feeds from an OPML and
anything that looks private (a login, credentials in the URL, a key
such as auth= or token=) are never listed, and the subscribe route
checks the id against the same list.
- CHANGELOG.md follows Keep a Changelog 1.1.0: 0.1.0 (2026-09-09, the
CLI), 0.2.0 (2026-09-10, the web UI), 0.3.0 (2026-09-11, accounts and
sharing). The long-form entries moved unchanged to docs/history.md.
- Cargo.toml is 0.3.0. CLAUDE.md says how to add an entry and cut a
release.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HdTEWQNrzyFULijigkmMn
