All Subscriptions took 130-170 ms when used a minute before, and 0.5-1.9 s
the first time after a quiet spell. Its sort needs each item's date, which
was only in the item's row, so every one of a person's 13,700 rows was
read, show notes inline and all, and cold that came off disk; count_in,
which needs no date, took 20 ms throughout. entries_by_time holds the keys
and dates, the page's keys are sorted and cut from it, and only the rows
shown are read. On a copy of the shape in the test database: 619 pages
read against 4,174, with no visit to the table. Neither change alone
helped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing the page did in the browser reached the logs: a DeadAir playback
failure on the phone was a toast and nothing more. POST /api/client takes
page errors, unhandled rejections, hls.js and media errors, the time from
play to sound and the length of a stall, and logs them as ev lines under
ipx::client, inside the request's span. Only those kinds are taken, the
text cut to 500 characters, and the page sends each at most once a minute.
The page's API requests send traceparent, a new trace id for each tap or
key, and the server's request span takes it as its parent, so one action's
requests are one trace in Tempo, the report included. The stub DOM the
smoke and native-bridge tests run in gains crypto and performance, which
every browser the page runs in has.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Played from ipodderx.sdf1.net, DeadAir's audio reached ipx as one request
with no Range header: Cloudflare caches by extension, deadair.mp3 has one,
and /media said nothing about caching, so Cloudflare fetched the whole file
and served ranges from its own copy, to anyone Access let in, without the
subscriber check (#129). /media/{id} has no extension and was never cached.
Cache-Control: private on every media response, the save included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 47 s reap had a trace of one span, so whether the time went on checking
every file on disk, the database or deleting could not be told. The reap's
steps are spans now, with what it removed on the reap span; an episode
list's query, sanitising, DeadAir and count are steps; and DeadAir's work
on a file is a trace of its own (probe, silencedetect, frame_bytes,
store_plans), only for files it looks at, not the idle passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The save button fetched the original while the player played it shortened.
/media/{id}/deadair sends the original's ID3 tag and then the pieces of the
person's plan one after another, read as it goes: MP3 and ADTS frames each
stand alone, so that is a file any player takes, and nothing is stored, as
with the playlist. The tag stops at the first kept frame, since what can
follow it, the Xing frame, gives the original's length.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DeadAir made a shortened copy of each episode with ffmpeg: as much disk
again as the original, one pause length for everyone, and a minutes-long
wait after the download. Seeking past silences instead was tried on an
iPhone and left about 1.3 s of nothing at every cut (#178).
Now it finds a file's silences once (silencedetect, plus ffprobe for
each frame's byte position) and stores a plan for each pause a person
can choose, 0.2 to 0.5 s: the cuts, and the frames kept as byte ranges
of the original. /media/{id}/deadair.m3u8 hands this person's plan out
as an HLS playlist of ranges of /media/{id}/deadair.mp3, the original.
Tested on iOS Safari first (#180): no gap at the cuts, plays locked,
seeks exactly, and needs no ID3 timestamp in front of each piece.
Other browsers get hls.js (1.6.19, the light build), loaded only when
needed. Not where canPlayType says it can play HLS: Chromium answers
"maybe" and then fails on MP3 ranges with "stream parsing failed", so
only Apple's browsers play the playlist themselves.
The pause is per person and per show, on deadair_prefs.pause_ms, 0.3 s
unless chosen; 0.2 s, what the copies left, made speech sound rushed.
AAC in MP4 has no ranges a playlist can use, so only MP3 and ADTS AAC
are shortened (the library is 803 MP3s and no .m4a). The copies, their
table and #179's expiry go: the first start drops the table and deletes
the files it names, only those named name.deadair.*.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A copy is nearly the size of its original and was kept as long as the
original. Now `deadair.played_at` notes when one was last served (at
most once an hour, since a player asks again at every seek), and each
pass of the maker removes copies unplayed, or unmade, for 7 days. Only
the file goes: the row keeps its cuts with its path cleared, which
tells an expired copy apart from a file that cannot have one (no cuts)
and keeps the background queue from making it again straight away.
Serving the original of such an episode deletes the row, which puts it
back on the queue; the copy then comes back within a pass, though the
page does not switch to it mid-play (#178).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new download in a show with DeadAir on plays at full length until its
copy is made, and nothing said so. Each file now carries
deadair_pending while DeadAir is on for its show and the copy is still
to come (not for one heard through, which it passes over), and the
episode's line and details say "Cutting dead air…". The maker sends a
deadair_done event when it finishes a file, and the page takes the list
again, so the line becomes how much shorter the episode is without a
reload.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Production's ffmpeg, Debian's 5.1, refused the copy of Brad & Will's
359: "Error initializing bitstream filter: bsf_list". The frames to drop
went to it as between(...)+between(...)+..., which nests a level a term,
and 5.1 takes 100 terms and refuses 200; the 6.1 the tests ran on took
all 520. Summed as a balanced tree the same 520 nest about 10 deep, and
5.1 takes 4000 that way, checked in the production container.
The failure was also stored as the file having no copy, so nothing would
have tried it again after this fix. Failures are now remembered until
the next start instead. The cap on cuts drops to 3500, so the argument
stays under Linux's 128 KiB with frame numbers in the millions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The copy maker skips an episode everyone who would hear the copy has
finished, and took finished to be the read mark. Opening an item sets it
as well, so 359: Potatoes From the Sky, opened and downloaded with
DeadAir on for its show, got no copy. Finished is now a place at 90% of
the length or past it, as Currently Listening has it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A DeadAir switch in each show's settings, and the same switch beside
the speed for the show that is playing, drawn as a waveform with its gap
closed up, outlined when off and solid when on, with aria-pressed. A
file the server gives a copy for plays from /media/{id}/deadair. An
episode's line says how much shorter it is, and Settings says how much
DeadAir has saved you.
Turning it on or off mid-episode saves the place with a request it waits
for, not a beacon, then asks for the entry again from the new
GET /api/entries/{feed}/{guid}, whose position is in the clock of the
version it now plays, and carries on there. A copy not made yet plays
from the next time. Turning a show on wakes the copy maker, so the copy
is there in seconds rather than at its next pass.
The browser test plays a real MP3 with long silences, quiet.mp3, from a
show it adds and removes itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was one switch for everything, with a feed kept out by hand; any
feed already subscribed would have been in it the moment the switch was
turned on. Now each feed, or the group it came in, is turned on by
itself, and a feed of a group turned on can still be kept out. The
general switch and /api/me's deadair are gone; /api/me keeps
deadair_saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For anyone who turns DeadAir on, a background task makes each episode a
copy with every silence quieter than -38 dB for over 0.35 s shortened to
0.2 s. Nothing is re-encoded: ffmpeg copies the original's frames and
leaves the silent ones out, about 11 s for an hour of MP3 in a release
build. On Brad & Will's episode 359 that is 520 cuts and 3.3 minutes
saved, the copy decodes clean, and it has no isolated spikes in its
silences that the original lacks.
Cuts are made from the middle of each silence, keeping 0.1 s at each
end, so the frames either side of a cut are silent: decoding blends a
frame with the one before it, and an MP3 frame can begin in the bytes of
the frame before, so the first frame after a cut decodes against the
wrong neighbour. Inside a silence nobody hears that; at a word they would.
Positions stay in the original's time and the server converts them both
ways. The iOS and Android apps save positions themselves from their own
player, even while the page is asleep, so converting in the page would
have stored the copy's times as the original's. /media/{id}/deadair
serves the copy at its own address and records that this person is
playing it; a position saved meanwhile is converted from the copy's
clock, as is the length a player measures.
The copy is made by a task of its own, not the worker: a first pass over
a library is hours of ffmpeg, and scans must not wait behind it. It
checks every two minutes, which covers new downloads, files already
downloaded, a switch turned on and a file downloaded again with one rule.
Episodes everyone who would hear the copy has finished get none.
download::remove deletes a copy with its original, and the quota counts
copies.
Settings: the switch is PATCH /api/me {deadair}, a feed kept out is
PATCH /api/feeds/{id} {deadair: false}. GET /api/me gives the switch and
deadair_saved, the seconds saved on episodes heard 75% or more on a copy,
each counted once. Three new tables, so no ALTER.
The image gains Debian's ffmpeg, about 480 MB, in a layer that stays
cached until the base image changes. The player's switch, the feed
setting and the stat come next, after mockups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
axum 0.7 set TCP_NODELAY on every accepted connection; 0.8 stopped, and serve
never set it, so each response sent in parts held its last small segment
until the browser's delayed ACK came back. On production's light traffic
nothing showed: small responses go out in one write. Under load it did.
tests/load, Nagle on then off: media range requests median 41.8 ms to
10.3 ms, p95 44.4 to 13.2, throughput 39 to 149 MB/s. Browse and sign-in
unchanged within noise; flag saves vary 80-195 ms either way, behind the
scan's writes, not the network.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Five Glass Cannon Network feeds name Patreon covers of about 7 MB, over the
5 MB limit on /api/art. Each page load fetched them all again, refused them,
and logged a 502 per cover, about 84 a day, with nothing in the log to say
which image or why: the reason went only to the browser. The limit is now
20 MB, a failure is answered from memory for an hour, and the fetch failure
is logged with its address and reason.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The feed settings dialog says 0 means every episode, the whole back catalogue, and as the server's
default it does. But merge_policy's cap() made only the server-wide 0 unlimited: a 0 set on a feed,
or on one person's subscription, became a budget of 0, hold_back held every file, and nothing
downloaded. Now 0 is unlimited at every level, as the dialog has always said.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Counting a person's files through enclosures_entry, once the count was of their own feeds only, read
every file row of their 146 feeds to find the few with a file: 58 ms of a feed list, more than the
37-50 ms the count over every row of the table had taken. Production holds 15 files among many
thousand rows. A partial index on the rows with a path holds just those, and serves the healthcheck's
count and the Downloaded filter too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The traces #164 added showed feed_list as nearly all of GET /api/feeds, and in it the item count at
184-316 ms, about 70% of the request: it counted every item of every feed in the database, 1534
feeds, for someone reading 146, on every feed list and every live update of one. The file count did
the same over every file row, 37-50 ms.
Both now join the person's subscriptions, as the unread count already did, and so read only their
feeds' rows, through the entries key and the enclosures_entry index, both on feed_id first. The
Directory also reads feed_list, for every feed's title and artwork, and uses no count; a feed the
person does not read is still listed, with none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /api/feeds took 250-300 ms in production with nothing under it in its trace but feed_summary
spans of a millisecond, so there was no telling where the time went. feed_rows now wraps the
catalogue, the person's subscriptions, the subscriber counts, the pins and feed_list in a step
span each, and feed_list each of its five queries.
db::step makes a span only inside one already: the healthcheck's status every 30 s and the page's
live updates run the same code outside any trace, where each span would be a trace of its own, as
feed_summary's already are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
patch_feed asked whether you subscribe, then asked again for the subscription, with a fallback for
none that could never run, since it only got there when there was one. One if-let does both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /api/enclosures/{id}/download and DELETE /api/enclosures/{id} raised a missing row with
anyhow!, which ApiError makes a 500 and logs as an api error. A page left open after a feed was
forgotten asks for exactly that: the caller's mistake, a 404 like /media/{id} already answers, and
as a 500 it turned up as a server error in the production check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`ipx add` and the page's Add feed were the same steps written twice: find the feed, look for it in
the catalogue by same_feed, add_one, store. They had drifted, and #153's lock had to be put into
both by hand. add_to_catalogue() now does the shared part, under the lock, and answers the id and
whether the feed was there already; each caller still finds the feed itself, so the page can call a
bad address the caller's mistake (400), and then does its own part: the CLI lists or categorises,
the page subscribes and scans. add_one is private to it now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A feed's Directory page and subscribing from it checked the feed may be listed by building the whole
Directory, every feed's listing, every file row for Podcasts and Blogs, every subscription, and
looking for the id in it: 0.57 s for a feed's page in production, nearly all of it that.
The rule (not an OPML or creator itself, neither the feed nor its group private) is now listable(),
which the Directory filters by and the two ask of the catalogue directly. One rule, so the
Directory and the check cannot drift apart; a test covers it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
subscriptions() names each feed derived from an OPML or a Patreon creator after its group when the
group has no folder, and asked feed_summary, three queries, for the group's title once for every
feed in it: the same title each time. Production's 32 Glass Cannon Network shows made that about 96
queries each time it ran, which is every feed list, every live update the page receives, every scan
and every wait between scans. It now asks once a group per call.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A torrent of several files is recorded as its folder. /media could not serve a folder, and the
reaper's remove_file failed on it, warned "could not delete" on every pass and freed nothing, so it
counted against the quota for good; Delete in the page failed the same way.
download::remove deletes a file or a whole folder, for the reaper and Delete alike, and the reaper
counts a folder by its recorded size, as a directory's own size is not its contents'.
download::playable answers the path itself, or for a folder the largest file in it, which is the
episode, and /media serves that.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removing an account deleted the user, and the foreign keys took their subscriptions with them, but
nothing did what unsubscribing does next: a feed nobody subscribes to any more leaves the catalogue
(or stays, if listed, for the Directory). A feed only that person read stayed with no subscribers,
and a feed with none downloads by its own settings, so it went on fetching each new episode, daily,
for no one, until it had been quiet for a year.
remove_feed's body is now leave(), and remove_user() leaves each of the person's feeds with it before
deleting the account, from the page and from `ipx user rm` alike; a feed inside a group of theirs
goes with the group. `ipx user rm` now tells a running daemon to reload, as add and rm do (#120),
since it can change the catalogue.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reaper counted read marks and pins in entry_state from anyone and compared them with the feed's
subscribers. Unsubscribing leaves your rows behind, and the flags endpoint writes them for feeds you
do not subscribe to as well. So an episode read by someone who had left and by one subscriber of
two counted as heard by everyone, and went first under the quota though the other had not played
it; and a pin of a person who had left kept a file, and its item, for good, which can leave the
quota impossible to meet.
Both queries, the reaper's candidates and prune_entries, now count only the marks of people who
subscribe to the item's feed. The old prune test's pin had no subscriber behind it; it has one now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An enclosure advertised as audio whose body is a .torrent was handed to torrent_one from inside
fetch_one, which returned without mark_downloaded, and in a scan nothing else recorded it. It
stayed pending: the page never showed it downloaded, and every scan of its feed fetched the
.torrent again and ran the torrent again, on the worker, for up to the stall limit and then the
seeding time (30 and 60 minutes in production), with no other feed checked meanwhile.
It now goes the way an advertised torrent does: in the daemon, marked downloading and handed to
spawn_torrent, which records it when it finishes; in a one-off CLI run, run there and recorded.
fetch_one answers None for the first, which a scan counts as a torrent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The worker's queue held 64 commands, and a web request that queues work waits inside itself for a
free place. Download latest lets you ask for up to 100 and sends one command a file, so past 64
the request waited for the worker to finish whole downloads, one after another, an hour for 100;
Cloudflare gave up on it after 100 s and the page said it failed, though most were queued. Until
it finished, adding a feed, Refresh, Download and socket commands all waited for a place too. A
command is a few bytes; the queue now has room for any one request's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each change to the catalogue or the server settings stored the whole thing from a copy it took
earlier, and nothing kept two from overlapping. Adding a feed copied it first and then spent up to
a minute finding and reading the feed, so another person's new feed, an unsubscribe, an admin's
settings, or the scan's own changes (a moved feed's address, a category) made in that minute were
stored over when it finished, with no error. A feed dropped that way left its subscriber's list.
Every change now takes its copy through Ctx::edit_cfg, which holds a lock until store_cfg keeps it,
and store_cfg takes nothing else, so no writer can skip it. Adding a feed takes its copy once it
has found the feed, and before the duplicate check, so two people adding one feed add it once.
reload takes the lock too. `ipx add`, `rm` and `import`, in a process of their own, are outside it,
as the comment on the lock says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The download client had a connect timeout and nothing else, so that a long download is never cut
off (#108). But nothing bounded a read either: a server that stops sending partway without closing
the connection left the download waiting for good, on the one worker, with every scan and download
behind it, while the healthcheck, which the socket answers itself, went on reading healthy. A read
timeout fires only when no bytes arrive, so a long download is untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
205 of the Directory's 208 feeds without a category had never been read: on 2026-10-05 at 03:24
UTC every one failed with "dns error: ... Temporary failure in name resolution" within a minute,
during an outage on Tower that also stopped iPX's log reaching Loki until Tower rebooted at 11:56.
They are working podcasts, NPR's, Megaphone's, Acast's, which resolve now. A feed nobody
subscribes to is read once a day and a failing one backs off up to a day (#99), so each sat
failing, unread and so uncategorised (#117) until the next day, and explain_failure called every
DNS error "This address no longer resolves; the site is gone.", on its page and since #142 on its
Directory page. A forced check of the 208 read 205; each has its items, its description and its
own category now.
A temporary lookup failure, EAI_AGAIN, is not recorded on the feed at all: no last_error, no
error_since to back off from or call it gone by. The daemon notes it in Ctx::retry and the feed
is due again in ten minutes, which the scan's due check and the sleep until the next scan both
honour; a restart forgets it, which only makes the retry sooner. A name that does not exist is
still the feed's error, explained as "may be gone", and the Directory page, like the feed's own,
names a cause only once a feed has been failing a day; before that its last check failed.
The page also drops "Nobody here subscribes yet", which said nothing worth a line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
daily-quests, one subscriber and no items, showed "Nothing read from it yet. A feed nobody here
subscribes to is checked once a day." on its page in the Directory: the second sentence was
written whatever the count, and the feed had been failing since #119 said so, which the page
never showed, /api/directory/{id} sending no error.
It sends `failing` now, the reason a feed's own page gives (explain_failure), or that the last
check failed, never the error itself, which can name the feed's address, and the Directory names
none. The page shows it under the header in the error colour, and says a feed is checked once a
day only when nobody subscribes and nothing is wrong.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"APPOFENIACS - A Technology Movie about Human Behavior" in Ain't It Cool News showed 66,170
characters: the feed's own HTTP headers and then the whole feed, tags stripped, and every other
item of the feed carried the text of the one before it. The publisher's feed had served it that
way, as only its XML could make a shift like that, and has since been fixed; ipx had read the
fixed feed and kept the broken text. Since 57ab419 (#96) a scan called record_entry only for a
guid it had not stored, so its UPDATE of title, description, image and duration was never
reached, and no correction, retitled episode or fixed length arrived.
Db::changed_items reads back what is stored for the items the feed lists, not the archive, and
names those it now gives a different title, text, artwork, length or number, a field it leaves
out counting as unchanged; the scan writes those again too. A corrected item keeps its read
state. A test serves a feed broken and then fixed from a socket of its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SeaORM gives a SQLite pool one connection unless told otherwise, and connect() did not say, so
every request, every scan write and every read went through it in turn. Twenty-five people
browsing in the load tests (#133) got 50 requests a second out of 12 cores, the fastest answers
in 3-30ms and the medians ten times that.
SQLite now gets eight. In WAL readers run beside the one writer. Every transaction here writes
first, so it takes the write lock or waits out the busy timeout for it; SQLite refuses at once
only a transaction holding a snapshot a write has moved past, and none here does. Db::memory
switches its file to WAL too, as Db::open does, so the tests share it as the daemon does. A test
holds a write open and reads beside it, and fails with one connection.
The same browsing: a feed's items at p95 10ms where it was 133, a search 22 where 119, the feed
list 89 where 242. The load tests' SQLite budgets are tightened to match; the Rust tests take
2.8s where they took 7.5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`docker exec iPX ipx rm make` printed "removed make" and took it out of the catalogue table, but
the web UI went on listing it. add, rm and import run in a process of their own and write the
whole catalogue there; the daemon kept its own copy in memory, and its next change of its own --
an admin's edit, a feed categorised (#117), a moved feed followed (#115) -- wrote the removed
feed back and dropped any the command line had added.
A new socket command, reload, has the daemon read the catalogue and settings again from the
database. The socket answers it at once, as it answers status, with the counts after it: queued
behind a scan, the web UI could write the old copy back in the meantime. add, rm and import send
it when a daemon is live, and say so if it does not answer. User commands need nothing: the
daemon reads accounts from the database on every request.
A browser test runs the real `ipx add --list` and `ipx rm` against the test daemon, with an
admin's edit after each, and fails without this.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
daily-quests and hots-dots had 0 items and no error, so their subscribers saw shows that had not
posted yet rather than feeds that are gone. Both domains have lapsed, and this network's DNS
filter answers for a parked domain with its block page, over http a 200 with an empty body.
scan_one took a body of nothing but whitespace for "nothing yet" and recorded no error.
It still does for a feed that has stored items, which is what the Antarctic Survey's empty 202
means, but a feed that has never stored one now fails with "the feed's address answered with
nothing at all", which the feed's page explains as an address that answers with nothing, the
site maybe gone. The next real read clears it. A test serves both from a socket of its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Running the load tests on Postgres (#139) found a scan reading 1.2 feeds a second: after ten
minutes the first of 1,500 had reached 1,136, where SQLite does them in 27s since #135. On the
databases project's server, production's, 200 single-row inserts took 2,565ms as commits of their
own, 12.8ms each waiting for the WAL to reach the disk (synchronous_commit=on), 56ms with
synchronous_commit=off, and 67ms in one transaction. A feed's entries and enclosures are written
a row at a time, about fifty commits for 20 new items, and production's scans paid the same.
ipx's own sessions now set synchronous_commit=off, through sqlx's connect options, so it holds
on top of whatever options a URL sets, the tests' search_path included. A crash of the Postgres
server can lose the last commits, about the last 0.6s, and never corrupts anything; a crash of ipx
loses nothing; other databases on the server keep their setting. The same first scan takes 20s,
and the Rust tests on Postgres 3.3s instead of 7.1s. The test of SQLite's setting checks this one
too when it runs on Postgres.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The load tests (#133) sent forty clients' wrong passwords to /api/login, 54 attempts a second,
which takes no account. Everyone else's requests took 4s (median 3.96s for /api/feeds, about 20ms
otherwise) and `ipx status`, the healthcheck, 1.3s (#137): each attempt was an Argon2id check,
tens of milliseconds of CPU, run inside the handler on one of the runtime's workers, so a handful
at once held every worker the rest of the server answers on. And a wrong password for an
account's name was refused a median 31ms later than one for a made-up name (#138), since only a
name with a hash was checked: the answer read the same, the time said which names are accounts.
auth::check_password runs the check on the blocking pool, at most half the cores at once, so a
flood waits on itself, and checks a name with no account, or no password, against a fixed decoy
hash, false in the same time. Under the same flood the rest of the site answers at p95 57ms,
`ipx status` at most 90ms, the gap is 0.2ms, and twice as many attempts are answered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The load tests (#133) found a daemon on SQLite scanning 1.3s a feed of 20 items served from the
same machine: 1,500 feeds would have taken about 50 minutes. On Tower's /tmp, a loop device, one
synchronous 4 KB write takes 52ms, and SQLite was in WAL mode with its default synchronous=FULL,
which waits for the disk on every commit; nearly every statement is a commit of its own, a
feed's entries and enclosures being written a row at a time. Subscribing the first admin to an
imported catalogue of 1,500 feeds took 39s, before the web server listened.
Every SQLite connection now sets synchronous=NORMAL, SQLite's own advice for WAL: it syncs at
checkpoints. A crash of ipx loses nothing; a power cut can lose the last transactions, and never
corrupts the file. The same scan takes 27s and the same subscribing 0.75s. Postgres, which
production runs on, is unaffected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A feed's page in the Directory (#128) had its cover, category and latest items, but not the
feed's own description, which Apple's show page leads with: iPX read every item's description
and threw the channel's away, and the feeds table had nowhere to keep it.
It is parsed now, RSS's <description>, or iTunes' summary when that is empty, or Atom's
subtitle, kept in feeds.description (kept when a later read has none, as title and image are),
and sent, sanitized, with the items from /api/directory/{id}, which is now an object, not a
list. The page shows it as plain text under the header, three lines of it, with More when there
is more. A description that only repeats the title is left out.
feeds.description is the first column added to a table that already exists. create_missing
looks for it with a SELECT and runs the ALTER only when it is missing, since it runs on every
open, the healthcheck's included, and an ALTER's lock is what made that time out before. The
same once-only step forgets every feed's ETag and Last-Modified: a feed is read whole only when
it has changed, so its description would otherwise wait for its next item.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Routes that take a feed or an enclosure id did not check who was asking. Anyone signed in could
read any feed's items through GET /api/feeds/{id}/entries, a paid feed's included, with the
addresses of its files, which can carry the subscriber's key: the Directory leaves such feeds
out for that reason, and this route handed them back to whoever guessed the id, a slug of the
title. In production it answered 25 items of a feed the asking account does not subscribe to.
/media/{id} served any downloaded file by its sequential id, POST /api/enclosures/{id}/download
and /api/feeds/{id}/download-latest queued any feed's downloads, and DELETE
/api/enclosures/{id}?force=true deleted any file.
Each now answers 404, "you do not subscribe to that feed", unless the person subscribes to it.
A feed inside an OPML has a subscription row of its own for everyone subscribed to the OPML, so
that holds for those feeds too. Found while adding the Directory's feed page (#128), which has
its own route that answers only for listed feeds and carries no files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Directory was one grid of every listed feed, 1,472 in production, under Podcasts and Blogs
tabs, a wall of about 30 category buttons and a sort menu, with a picked category's
subcategories appearing as a second row of buttons that looked like the first. It read as a
list to scroll, and the two rows were hard to tell apart.
It opens now on the ten most subscribed feeds, ranked, and the categories as tiles, each with
three of its shows' covers fanned in its corner on the tint its initials would get, so the
shows colour it and no theme's palette changes. A category has a page of its own: its most
subscribed, its subcategories with how many each holds, and its grid. See all is every feed,
as the grid was. Podcasts or Blogs holds across every page. Popular, the ten most subscribed,
had a place of its own in the feed list; it is the Directory's first section instead, and its
key, g p, is gone. /api/popular stays for scripts.
The search box finds a feed in the Directory by name (#126). It said "Search items…" there and
did nothing, since loadEntries returns early for a place that lists feeds.
A feed you do not subscribe to opens a page of its own (#128): its cover, category (a link to
that category's page), how many here subscribe, a subscribe button and its latest twenty items,
each with its title, linking to the post where it has one, its first lines, date and length.
Before, a click on it did nothing; only its + did. The items come from GET /api/directory/{id},
which answers only for a feed the Directory lists, so a guessed id reaches nothing private, and
carries no file or its address. The feed's own description would belong at the top, but the
database does not keep one.
Every tile and chart row takes the keyboard, as the feed list's rows do. Feed text made plain
keeps a space at a line break or a paragraph's end, which glued "2010)Recorded" together.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Directory listed Football (6 feeds), Soccer (7) and "Soccer," (2) as categories of their
own beside Sports, and Software How-To (2) beside Technology. CATEGORIES spelt two of Apple's
Sports subcategories American Football and Football (Soccer), names no feed uses: Apple's list
says Football and Soccer, so a feed naming either fell through as a category of its own.
They are Apple's spelling now. The old spellings are aliases, since Jev picked from the list
while it had them and its answers are stored, and so is Software How-To, Apple's Technology
subcategory before its 2019 list. A comma or full stop at the end of a feed's category, as in
"Soccer,", is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API took a session cookie, a proxy's word or the shared admin token, so a script or an
agent working for one person had to sign in with their password and carry the cookie, or be
given the admin token. Settings now makes named tokens, ipx_ and 256 random bits, sent as
Authorization: Bearer. A token is its owner and no more. Only its SHA-256 is kept, in the
new api_tokens table, with when it was made and last used; it is shown once and revoked from
the same list. An unknown or revoked one gets a 401 rather than falling through to a cookie.
Cloudflare Access still stands in front of the tunnel, so from outside a token needs an
Access service token beside it; docs/sso.md says how.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Directory had one row of chips holding whatever each feed's category was, a category
(Technology) or a subcategory (Tech News, Video Games) side by side: a podcast's own
<itunes:category> is stored as its subcategory, and Jev's answers (#117) are often
subcategories too, so after the first forced scan the row held 22 chips. /api/directory now
gives each feed's Apple category and subcategory, worked out from Apple's list, and the
page shows the categories, then a picked one's subcategories on a line of their own. A
category that is not Apple's, one an admin typed, stands as a category of its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most blogs and news sites name no <itunes:category>, so they sat under no chip in the
Directory unless an admin set one by hand. When TYPESAFE_KEY is set, a feed's first read, or
a forced refresh, asks TypeSafe's Jev to pick one of Apple's categories or subcategories from
the feed's title and up to 15 item titles, for a catalogue feed with neither its own category
nor an admin's. The answer is stored as the catalogue's category, so a feed is not asked
twice, and an admin can change it. Only on a first read or a forced refresh to keep the calls
down: feeds already subscribed are categorised by one `ipx fetch --force`.
Tried on eight subscribed feeds on 2026-10-04: XDA, Daring Fireball, Ars Technica and The
Verge as Tech News, The Old New Thing as Technology, John D. Cook as Mathematics,
BoardGameGeek as Games, Pluralistic as News Commentary (0.55). About 2,000 input tokens a
feed at $0.042 a million. The likeliest answer is kept however unsure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A feed moved to its new address was only logged by follow_move. It is now an event, feed_moved
with the feed and its old and new addresses, so it goes where every other event goes: the log,
with from and to as fields, the admin page's Scans view, `ipx fetch`, and the page, which gets
the feed's new row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A feed whose address answered with a permanent redirect was read through it on every check,
and the catalogue kept the old address: 28 of 147 feeds in production, most http to https, some
to a new path or domain.
Feeds are now fetched with a client of their own that follows no redirects (Ctx::feed_client),
and feed::fetch follows them itself, up to 10 hops, so it sees each one. When every hop was
permanent (301 or 308) it says where the feed ended up, and the scan moves the feed there in the
catalogue (follow_move). A temporary hop (302, 307) anywhere moves nothing. A feed an OPML lists
is left alone, as the OPML would put the old address back, and so is a move onto an address
another feed has. A password goes only to the feed's own host, never to a redirect elsewhere;
reqwest's own following dropped it the same way. Ten hops is a loop, worded as reqwest worded
it so it still reads as redirect_loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The daemon ticked every 60 s and ran a scan pass each time: the sweep, then a check-state query
per feed (about 180) to find which were due. In the six hours before, 293 of 362 passes found
nothing due. Now, after each pass, it works out when the earliest feed is due (due_at, shared
with the scan's own check, over Db::http_states, one query) and sleeps until then: at least
30 s, so a feed that never gets a check time cannot spin it, and at most 10 minutes, so what no
command announces, ipx add or a shorter schedule, is picked up. Commands still wake it at once,
and the first pass after starting runs straight away, as the tick's did. The scan reads every
feed's state in one query too.
The prod-check skill says what to expect now: tens of scans in six hours, and pending as the
real queue.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
