Commit Graph

142 Commits

Author SHA1 Message Date
0fdd609791 Signals in a folder of their own in the feed list, as an OPML's feeds are (#255)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 17:14:45 +00:00
70990c70c7 Signals: feeds made from your subscriptions, picked by Jev (#254)
A Signal is a name and keywords. A task beside DeadAir's asks TypeSafe's Jev, on the owner's own
key, about each item of their subscribed feeds from the last 30 days and every new one: one call
an item, one noul question for each of their Signals still to hear about it, 0.5 or more a fit.
Every verdict is kept, fitting or not, in signal_verdicts, so nothing is asked twice, and new
keywords drop a Signal's verdicts to be judged afresh. A key TypeSafe refuses (401/402/403) is
passed over until its owner saves another, so a bad key is not sent the whole queue.

A Signal's items come through entries_where with an EXISTS on its hits, so filters, search, sort
and read state are an ordinary list's, and reading an item in a Signal reads it in its feed.

The page shows Signals under the places, with New signal, and Settings takes the TypeSafe key,
which is never sent back. The browser suite stubs Jev in the fixture server (IPX_TYPESAFE_URL).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 16:54:50 +00:00
1949f3e493 Compression across the board: directory answers, uploads, metrics, traces, the page (#253)
Measured 2026-10-16, uncompressed until now: the directory's answers (the feed list ipx reads,
603 KB of JSON, ~134 ms of a 159 ms Directory request), ipx's snapshot upload (~100 KB each half
hour), both /metrics (4.7 and 0.9 KB every 15 s) and every trace export to Tempo. Already
compressed: ipx-web's answers (zstd, #239), the home page and the directory's public pages
through Cloudflare (brotli, zstd).

- The directory compresses with brotli, zstd or gzip as asked, and unpacks a gzipped request.
- ipx sends its snapshot gzipped; the directory is deployed first.
- Both /metrics answer gzipped, as Prometheus asks.
- opentelemetry-otlp's gzip-http: OTEL_EXPORTER_OTLP_COMPRESSION=gzip compresses the export, set
  for all three in production's compose file; Caddy's exporter reads the same variable.
- web/build.mjs writes .br (quality 11) and .gz (level 9) beside each text file, and Caddy serves
  them in place (file_server precompressed): app.js 30 KB where zstd on the fly gave 37, app.css
  11 where 14. A .br or .gz asked for by name is refused, or /admin.js.br would be the admin
  page's script without the check that keeps it to admins; in a handle of its own, since a bare
  respond runs after the handles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 14:02:39 +00:00
cdf2c98f9f Currently Listening and the Directory open at once, or with a spinner (#248)
Both fetched on every visit and drew only when the answer came: Currently Listening took 1.3 s.
fetchPage keeps their answers by address, as the item lists are kept, sharing a request already
on its way; both draw from what is kept at once and again only if the refresh changed them, and
show the spinner with nothing kept. They are fetched ahead with the lists when the page loads,
and when the pointer or a finger reaches their row. Removing an episode from Currently Listening
forgets its kept copy, which would otherwise show it again until the refresh. The preload loop
takes a failed fetch as done instead of leaving an unhandled rejection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 15:24:25 +00:00
6640ca9612 Lists open at once from what the page keeps and fetches ahead, or with a spinner (#247)
A click on a feed, All Subscriptions or a filter fetched the list and only then drew it, so
nothing answered the click until the server did. The page now keeps the first page of each list
(the newest 40, by view, filter, sort and search), draws a kept one at once and refreshes it
behind, drawing again only if it changed. It fetches ahead: All Subscriptions and the twenty
feeds with the most unread once the page is idle, two at a time, and a feed or filter when the
pointer or a finger reaches it. A feed_row or a read forgets that feed's lists and All
Subscriptions'; the forgotten are fetched again behind. A list with nothing kept shows a
spinner if its answer takes more than 120 ms.

list_shown (#246) is reported when the list first shows, so a kept one reports its few ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 15:16:19 +00:00
62a3241139 Database connections kept open; a click's wait is measured (#241, #245, #246)
#241: on production, All Subscriptions' query runs in 31 ms, unread oldest-first in 30, the
feed list's count in 5, generic plans included (EXPLAIN ANALYZE, BUFFERS, read only), against
0.3-1.6 s in ipx. A new Postgres connection takes 80-330 ms to open, and a page load asks for
more than are idle: four of six slow loads came as the pool grew (3 to 6, 2 to 6, 3 to 5, 6 to
8), and 20 requests at once took a median 414 ms while the pool grew from 8 to 20. The daemon
and the directory's server now keep their whole pool open (min = max); a command such as `ipx
status`, and the tests, which share the server's 100 connections, open as they need.

#246: the page reports list_shown, the ms from asking for a list to drawing it, with the view
and filter, under the click's trace; background refreshes are not timed. Timings skip report()'s
once-a-minute limit, which is for errors in a loop. A dashboard panel and table, and the prod
check, read it.

#245: the load test's browse scenario subscribes by OPML and leaves the Directory, which is
ipx-directory's since #204, to it.

CLAUDE.md records the owner's leave to EXPLAIN (ANALYZE, BUFFERS) any query on production,
read only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:52:14 +00:00
a9991545f9 Tests: two people on one file, each with their own DeadAir version, place and playlist
The DeadAir tests named for each person's clock used one account. Now three play one file at
once (DeadAir at two pauses, and the original) and each keeps their own place; and two fetch
the same /media/{id}/deadair.m3u8 at once and each gets their own playlist, no-store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:04:47 +00:00
f20660a7d5 Unit tests for the page's own functions (#234)
tests/page-unit.js loads the built index.html script into the stub DOM page-smoke.js uses and
calls its helpers directly with node:test: escaping, times and sizes, file kinds, an untitled
item's name, the directory's sorts, filter words, DeadAir's playlist address. Top-level consts
are not properties of the vm context, so each check evaluates its expression in it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:47:15 +00:00
a5e64e520d Refresh buttons turn from the press to the end of their scan; quick checks keep the count (#232, #233)
The buttons followed the feeds being checked, so they stopped between one feed and the next;
7ecbe82 held them for 600 ms over each gap. Now pressing one starts it, and scan_done stops
it only when it is the end of the scan asked for: scan_done carries `only` and `forced`, so a
scheduled scan finishing first does not stop a button whose scan is queued behind it. An
error event, a job that failed without a scan_done, stops them all.

A row's spinner stood in for its unread count from the moment its feed was checked; most
checks take milliseconds, so each count blinked as the scan went by. It now waits 300 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:00:50 +00:00
724764c8df Page: catch background fetches, no HTML in errors, a trace ends after a quiet minute (#219)
The page reported NetworkError rejections from the live update's list refresh and the first
feed load, which had no catch, whenever the network was gone (a laptop asleep). A Cloudflare
502 page became a request's error message, since api() used whatever body came back. And
the trace id lived until the next tap, so background requests nine hours apart shared one
trace. The id now ends after a minute without a request; a play's position saves, every ten
seconds, keep it going.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 12:16:50 +00:00
cc155274e9 No Docker healthchecks: the daemon keeps /metrics' counts itself (#218)
Docker runs a healthcheck with exec, and containerd puts each exec's state in XDG_RUNTIME_DIR,
/run/user/0 on Tower, which elogind removes when root's last session ends. Every check then
failed and every container showed unhealthy while all of them answered. The checks are gone from
the sample compose file and production's; ipx's library counts for /metrics, which only the
healthcheck's status call refreshed, are counted every 30 s by the daemon.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:08:22 +00:00
954399ed5d Administer the directory from iPX with an rwa token (#206)
A Directory feed's page offers Check now and Remove to an admin whose token is rwa (the page
learns it from can_admin on the feed's page, asked of the directory only for admins). Admin,
Directory lists the servers sharing, from a new GET /v1/instances, and the blocked feeds by
title: a feed is often blocked because its address carries a key, so the address stays in the
directory. Unblocking or dropping a snapshot sends this server's snapshot again at once; since
directory's 403 reaches the page as 403, not 502.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:40:15 +00:00
38c1abba1d Each service its own version: ipx, ipx-directory, the page (#207)
The workspace stops holding one version. ipx keeps 0.12.1-dev and its vX.Y.Z tags; the directory
starts at 0.1.0-dev, tagged directory-vX.Y.Z, and the page at 0.1.0-dev in package.json, tagged
web-vX.Y.Z, which web/build.mjs gives the script for the logo's tooltip beside ipx's. crates/feed
is a library, not released. CLAUDE.md's release steps are for one service at a time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:35:41 +00:00
f9870938bf ipx-web: the page in its own container, behind Caddy (#203)
web/build.mjs builds a static site, web/dist, with the icons and typeface
beside the pages; Dockerfile.web serves it with Caddy, which passes /api
and /media to ipx on the same origin, so cookies, live updates and
Cloudflare Access work as before. ipx serves no page any more: build.rs
and every include_str! of the web files are gone, and its image is built
without node.

What ipx did to the page as it served it is kept another way. Caddy asks
ipx before serving a page (forward_auth): /api/page-check sends a browser
not signed in to /login and a non-admin away from /admin, and refuses them
/admin.js; /api/login-check sends someone Access vouches for past the
form; a page asked for with ?token= goes to /api/signin, which sets the
cookie. The theme and the admin link, once written into the page per
person, are set by web/boot.js in <head> from the theme cookie and the
last visit, before the first frame, and /api/me brings the version and
the account's theme from before.

Caddy strips the Cf-Access-* headers from every peer but TUNNEL_PEER, the
address cloudflared's requests come from, which is the boundary
trusted_proxies drew; ipx then trusts only Caddy's address.

The browser suite runs Caddy in front of its daemon, so it tests what
production would run. Not deployed: production's cutover is on #203.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 14:03:22 +00:00
c47aa57909 The Directory comes from the directory service (#205)
ipx asks ipx-directory on its people's behalf, with the token an admin set
in the new Admin, Directory tab, so the page never talks to it and never
sees the token or a feed's address. /api/directory, /api/popular and a
feed's Directory page answer from it; subscribing takes the address from
the directory and goes through Add feed's steps. With no directory set
up, the Directory says so. The local directory code is gone; its privacy
checks are the snapshot builder's now, and their tests moved with them.

With sharing on, ipx sends its snapshot after every scan, whenever the
config changes (set_cfg) and every half hour: each public feed and how
many here subscribe, never an OPML or Patreon creator itself or anything
private. A feed can be marked private in its settings, for a paid feed
whose address does not give it away; anyone subscribed may mark it, only
an admin unmark it. The admin's category for a feed that names none goes
with it, which the directory uses only while the feed names none, so the
category an admin gives a blog still reaches the Directory.

The directory lists only feeds it has read as a feed: the first snapshot
after a start went before the first scan, when an OPML was not yet known
to be one, and the directory listed the OPML. It now fetches what a
snapshot brings in at once instead of within the minute.

The browser suite runs its own ipx-directory on SQLite; the Directory's
tests match feeds by title, since the directory names them by address.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 04:31:32 +00:00
fca704f7ff Add a feed no longer asks for a folder (#201)
A feed's settings stopped showing its folder in 0.10.3; the dialog that adds one
still asked for it. The API keeps its folder for scripts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:52:22 +00:00
973eeb67c7 Metrics on a thread and port of their own, from memory alone (#194)
/metrics shared the web server's threads, and answered after a token
lookup and three counting queries, so a pool run dry, one of the things it
is there to show, would have stalled it, and a slow scrape held a web
thread. Now [web] metrics_bind starts a listener on its own OS thread with
a current-thread runtime, reading only what is in memory: pool counts,
channel capacity, the daemon runtime's metrics through its handle, /proc.
No sign-in, as exporters have none; publish the port to the LAN only. The
feed, pending and downloaded counts stay with the healthcheck's log lines,
which the dashboard already reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 15:04:51 +00:00
9fc1e7e248 Prometheus metrics for the daemon's state between requests (#194)
Logs and traces show requests, scans and downloads; nothing showed what
the daemon holds meanwhile. /metrics, behind sign-in, gives database
connections in use and idle against the pool's maximum, commands waiting
for the worker, live-update listeners, free torrent slots, episodes
playing through DeadAir, the library's counts, the tokio runtime's
workers, tasks and queue, and the process's memory and open files, in
Prometheus's text format, written by hand rather than with a client
library. Not logged or traced: Prometheus asks every 15 s. The iPX
dashboard gains a Daemon row of them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 14:56:54 +00:00
cd9aef36a1 A play's trace holds the playlist, audio and saved place it fetched (#192)
hls.js sends traceparent on its own requests (xhrSetup); a beacon, which
cannot set a header, and Safari's player, fetching the playlist itself,
carry it in the query, which the server reads where there is no header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 14:41:28 +00:00
6c931d0485 The page reports its errors and playback timings, and its requests carry a trace (#190)
Nothing the page did in the browser reached the logs: a DeadAir playback
failure on the phone was a toast and nothing more. POST /api/client takes
page errors, unhandled rejections, hls.js and media errors, the time from
play to sound and the length of a stall, and logs them as ev lines under
ipx::client, inside the request's span. Only those kinds are taken, the
text cut to 500 characters, and the page sends each at most once a minute.

The page's API requests send traceparent, a new trace id for each tap or
key, and the server's request span takes it as its parent, so one action's
requests are one trace in Tempo, the report included. The stub DOM the
smoke and native-bridge tests run in gains crypto and performance, which
every browser the page runs in has.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 14:15:20 +00:00
9648a79f3f Keep media out of shared caches (#189)
Played from ipodderx.sdf1.net, DeadAir's audio reached ipx as one request
with no Range header: Cloudflare caches by extension, deadair.mp3 has one,
and /media said nothing about caching, so Cloudflare fetched the whole file
and served ranges from its own copy, to anyone Access let in, without the
subscriber check (#129). /media/{id} has no extension and was never cached.
Cache-Control: private on every media response, the save included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 13:23:27 +00:00
ba9759f44f Gesture tests send a drag's touches at once, not one by one (#184)
Each touch event was awaited before the next went, so on a busy machine a
drag's ten messages took longer than the 800 ms a swipe has, and the page
took it for a slow drag. Reproduced by spacing the messages 110 ms apart:
the swipe back stayed on Second Episode, as the failing run did. The page
is right to want a swipe quick; a finger does it in well under that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 12:58:26 +00:00
e56bc9e8dd Save an episode without its dead air when DeadAir is on (#186)
The save button fetched the original while the player played it shortened.
/media/{id}/deadair sends the original's ID3 tag and then the pieces of the
person's plan one after another, read as it goes: MP3 and ADTS frames each
stand alone, so that is a file any player takes, and nothing is stored, as
with the playlist. The tag stops at the first kept frame, since what can
follow it, the Xing frame, gives the original's length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 12:38:30 +00:00
4ccf2e5511 DeadAir's pause on a slider, shown only while DeadAir is on (#183)
The four segmented buttons gave no sense that 0.2 to 0.5 s is one scale,
and stayed on screen with DeadAir off, where they did nothing. A range
input with a dot at each stop and Brisk/Relaxed at its ends says both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:14:50 +00:00
089202f72b DeadAir plays an HLS playlist of the original, with a pause of your own (#181, #182)
DeadAir made a shortened copy of each episode with ffmpeg: as much disk
again as the original, one pause length for everyone, and a minutes-long
wait after the download. Seeking past silences instead was tried on an
iPhone and left about 1.3 s of nothing at every cut (#178).

Now it finds a file's silences once (silencedetect, plus ffprobe for
each frame's byte position) and stores a plan for each pause a person
can choose, 0.2 to 0.5 s: the cuts, and the frames kept as byte ranges
of the original. /media/{id}/deadair.m3u8 hands this person's plan out
as an HLS playlist of ranges of /media/{id}/deadair.mp3, the original.
Tested on iOS Safari first (#180): no gap at the cuts, plays locked,
seeks exactly, and needs no ID3 timestamp in front of each piece.

Other browsers get hls.js (1.6.19, the light build), loaded only when
needed. Not where canPlayType says it can play HLS: Chromium answers
"maybe" and then fails on MP3 ranges with "stream parsing failed", so
only Apple's browsers play the playlist themselves.

The pause is per person and per show, on deadair_prefs.pause_ms, 0.3 s
unless chosen; 0.2 s, what the copies left, made speech sound rushed.
AAC in MP4 has no ranges a playlist can use, so only MP3 and ADTS AAC
are shortened (the library is 803 MP3s and no .m4a). The copies, their
table and #179's expiry go: the first start drops the table and deletes
the files it names, only those named name.deadair.*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:46:01 +00:00
d6c0e3e7a6 Show that DeadAir is still working on an episode (#176)
A new download in a show with DeadAir on plays at full length until its
copy is made, and nothing said so. Each file now carries
deadair_pending while DeadAir is on for its show and the copy is still
to come (not for one heard through, which it passes over), and the
episode's line and details say "Cutting dead air…". The maker sends a
deadair_done event when it finishes a file, and the page takes the list
again, so the line becomes how much shorter the episode is without a
reload.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:05:18 +00:00
827cc7489f DeadAir in the page: a switch per show, one beside the speed, and the time saved (#172)
A DeadAir switch in each show's settings, and the same switch beside
the speed for the show that is playing, drawn as a waveform with its gap
closed up, outlined when off and solid when on, with aria-pressed. A
file the server gives a copy for plays from /media/{id}/deadair. An
episode's line says how much shorter it is, and Settings says how much
DeadAir has saved you.

Turning it on or off mid-episode saves the place with a request it waits
for, not a beacon, then asks for the entry again from the new
GET /api/entries/{feed}/{guid}, whose position is in the clock of the
version it now plays, and carries on there. A copy not made yet plays
from the next time. Turning a show on wakes the copy maker, so the copy
is there in seconds rather than at its next pass.

The browser test plays a real MP3 with long silences, quiet.mp3, from a
show it adds and removes itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 18:18:41 +00:00
cb92d5e355 Say nothing in an empty list or an empty reader
Asked for: "Nothing here yet — try Scan now." and "Pick an item to read it." are gone. A search
that finds nothing still says so, as the words typed are the reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:50:22 +00:00
ecbc253894 Redesign the feed settings dialog (#167)
One column of label, box and grey paragraph, eight times at the same weight, three screens long on a
phone with Save at the bottom of the third. Approved from the mockups on #167.

- Download automatically comes first, as a switch; how many, the keywords and explicit, which only
  ever applied to automatic downloads, hang beneath it and fold away when it is off.
- How many is Default, Newest or All, said as a sentence with the number in it, over a strip of the
  feed's own newest items, each as tall as it is long and lit where it downloads; dragging along
  it sets Newest. All is 0, which now means all (#166).
- Keywords and hidden words are chips: Enter or a comma adds one, its x or Backspace removes it.
- The feed's own settings, the same for everyone and an admin's to change, sit apart in a box.
- The body scrolls and Cancel and Save stay at the foot.
- Theme colours only; iOS switches, but checkboxes in Classic. openModal takes a class for a
  dialog with a frame of its own, set whole each time so it never outlives it.

A browser test covers the strip, chips, the fold and a save that holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:25:57 +00:00
4e417389fe Catch the feed list up when the page comes back into view (#168)
All Subscriptions said 1 unread when the server had none. The badge is the page's own sum of its
feed list, loaded once and patched only by events the server sends when a scan or a download
changes a feed. Reading an item, marking everything read or unsubscribing sends none, so another
device's page kept its old counts, and an iPhone stops the event stream while the app is in the
background, missing everything meanwhile; the reconnect did not catch up.

The page now takes the feed list and the open list again when it comes back into view and when the
stream reconnects after a drop. A browser test reads an item behind the page's back and checks the
badge follows once the page is shown again; it fails without this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:13:16 +00:00
c7eef3e516 Make Glass's top bar solid on a phone, so the strip above it matches (#147)
In a home-screen app iOS paints the strip behind the status bar and the Dynamic Island itself, in
one flat colour, from theme-color. Glass's bar is see-through, so theme-color fell back to Glass's
page background, #0b0d14 (#eef1f7 light), above a bar that showed #191e34 on average, bluer at the
left and purple at the right: a near-black band, or a white one in light mode.

No flat strip can match a see-through bar, so on a phone the bar is solid, in what it showed on
average (--glass-bar, measured in Chromium off the bar's top edge), and theme-color carries the
same colour. Everywhere else Glass is as it was. The browser test checks the bar is solid and
theme-color equals it in both modes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:10:14 +00:00
a553931c76 Run the top bar's colour up behind the Dynamic Island on iOS 26 (#147)
46f9370 set theme-color, but Safari 26 ignores it. iOS 26 paints the strip behind the status bar
and the island in the background colour of a sticky or fixed element at the page's top edge, and
failing that the root's. The top bar was neither, and the root had no colour of its own (only
body did), so iOS used black or white: a band of another colour above the bar.

The bar is sticky now, though nothing scrolls under it, so iOS reads its colour; the root has the
page's background for screens with nothing at the top edge; Classic's metal bar gets a colour
under its gradient, the top of the metal, since iOS reads no gradient. theme-color stays, set to
the bar's colour, for iPhones before iOS 26 and for Android. Glass's bar is see-through, so iOS
frosts what is under it there, as the bar itself does.

Taken from developers' reports of iOS 26, not Apple's documentation, and not yet seen on a phone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:50:28 +00:00
46f9370018 Paint the bar around the Dynamic Island in the theme's background (#147)
The page named no theme-color, so the status bar iOS draws around the Dynamic Island was its own
black or white, a band across the top of every theme. The page and the admin page now carry one,
set to the page's background whenever the theme or mode changes; Android's toolbar takes it too.
Not seen on a phone yet: the browser test checks the colour matches, not what iOS does with it.

A browser test at 360px covers this and #146's row of buttons. The stub DOM gains
getComputedStyle, which every browser has.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:51:19 +00:00
c9964778cb Give SQLite several connections, so requests are answered at once (#136)
SeaORM gives a SQLite pool one connection unless told otherwise, and connect() did not say, so
every request, every scan write and every read went through it in turn. Twenty-five people
browsing in the load tests (#133) got 50 requests a second out of 12 cores, the fastest answers
in 3-30ms and the medians ten times that.

SQLite now gets eight. In WAL readers run beside the one writer. Every transaction here writes
first, so it takes the write lock or waits out the busy timeout for it; SQLite refuses at once
only a transaction holding a snapshot a write has moved past, and none here does. Db::memory
switches its file to WAL too, as Db::open does, so the tests share it as the daemon does. A test
holds a write open and reads beside it, and fails with one connection.

The same browsing: a feed's items at p95 10ms where it was 133, a search 22 where 119, the feed
list 89 where 242. The load tests' SQLite budgets are tightened to match; the Rust tests take
2.8s where they took 7.5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:55:22 +00:00
a553d050ca Have ipx add, rm and import tell a running daemon to read the catalogue again (#120)
`docker exec iPX ipx rm make` printed "removed make" and took it out of the catalogue table, but
the web UI went on listing it. add, rm and import run in a process of their own and write the
whole catalogue there; the daemon kept its own copy in memory, and its next change of its own --
an admin's edit, a feed categorised (#117), a moved feed followed (#115) -- wrote the removed
feed back and dropped any the command line had added.

A new socket command, reload, has the daemon read the catalogue and settings again from the
database. The socket answers it at once, as it answers status, with the counts after it: queued
behind a scan, the web UI could write the old copy back in the meantime. add, rm and import send
it when a daemon is live, and say so if it does not answer. User commands need nothing: the
daemon reads accounts from the database on every request.

A browser test runs the real `ipx add --list` and `ipx rm` against the test daemon, with an
admin's edit after each, and fails without this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:55:22 +00:00
9c2f3eaf1b Search Currently Listening from the search box (#127)
Currently Listening's search box said "Search items…" and did nothing: loadEntries returns early
for a place with its own list. Typing now redraws the list with the search sent along, as
/api/entries already takes one, title and text, and says so when nothing in progress matches.
The keys list says / searches it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:55:21 +00:00
bbdcbe213f Run the load tests on Postgres too, as production runs (#139)
node tests/load/run.js --postgres puts the scratch daemon in IPX_TEST_DATABASE_URL, the database
the Rust tests already use on Postgres, ipodderx_test on the databases project's server, in a
schema of its own, ipx_load, dropped and made again each run as /tmp/ipx-load is wiped. The URL
is in /src/.envrc, taken from ipx.env. It is production's server and role, so the harness refuses
a database whose name does not end in _test, and hands psql the password in its environment,
where no other process on Tower can read it. psql comes from /src/install.sh.

After seeding it runs ANALYZE: a schema filled seconds before has no planner statistics until
autovacuum gets there, which it did partway through a test, and the search's p95 swung between
40ms and 166ms from run to run. Analyzed first, three runs gave 37-38ms.

Each budget now has a value for each database, about twice what it measures on Tower: Postgres
answers the feed list at p95 54ms to SQLite's 242, having no one connection to queue for (#136).
Running it found #140.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:31:18 +00:00
b43ba89778 Load tests with k6, for what many people at once do to the server (#133)
The browser tests drive one person against a handful of feeds, one request at a time, and cannot
show what production's load does. node tests/load/run.js builds a release binary, serves 1,500
generated feeds from itself, starts a daemon of its own under /tmp/ipx-load, seeds a hundred
listeners with thirty feeds each and four downloaded files, and runs four k6 scripts, with
`ipx status` -- the healthcheck -- run every second and failing the run at its 5s timeout:

  browse     25 people at once: feed list, All Subscriptions, a feed, a search, the Directory,
             a feed's page in it
  listening  100 players saving positions every second and marking items read while scans write;
             each reads its own state back, which must be as it left it
  media      50 listeners seeking: every range checked byte for byte against the served file
  signin     a flood of wrong passwords while others browse, and the gap between refusing a
             known name and an unknown one

Each budget is about twice what it measures now. Getting here found #135 (SQLite waiting for the
disk after every write, a first scan of 1,500 feeds estimated at 50 minutes, now 27s), #136
(SQLite's single connection, left open), #137 and #138 (fixed in the commit before this). k6 is
installed from its own signed apt repository by /src/install.sh, outside this repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:38:20 +00:00
697e907c86 Keep the feed list's icons when it redraws, rather than load each again (#134)
Checking every feed made all the feeds' icons in the feed list flash while it ran. A check sends
each feed's new row as it reads the feed, and renderFeeds, run once a frame while they come in,
emptied the list and built every row anew, every <img> with it: each icon went blank and was
loaded and drawn again, many times a second through a scan of 150 feeds.

The rows are still rebuilt, but the images already drawn go into the new ones in place of the
fresh copies, matched by their markup, so only an icon that has changed is made anew. A browser
test checks the rows are new and the images in them the same elements, and fails without this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:37:35 +00:00
ef8e2526f2 Show what a feed says it is on its page in the Directory (#130)
A feed's page in the Directory (#128) had its cover, category and latest items, but not the
feed's own description, which Apple's show page leads with: iPX read every item's description
and threw the channel's away, and the feeds table had nowhere to keep it.

It is parsed now, RSS's <description>, or iTunes' summary when that is empty, or Atom's
subtitle, kept in feeds.description (kept when a later read has none, as title and image are),
and sent, sanitized, with the items from /api/directory/{id}, which is now an object, not a
list. The page shows it as plain text under the header, three lines of it, with More when there
is more. A description that only repeats the title is left out.

feeds.description is the first column added to a table that already exists. create_missing
looks for it with a SELECT and runs the ALTER only when it is missing, since it runs on every
open, the healthcheck's included, and an ALTER's lock is what made that time out before. The
same once-only step forgets every feed's ETag and Last-Modified: a feed is read whole only when
it has changed, so its description would otherwise wait for its next item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:46:22 +00:00
b86062b97a Keep a feed's items and files to the people who subscribe to it (#129)
Routes that take a feed or an enclosure id did not check who was asking. Anyone signed in could
read any feed's items through GET /api/feeds/{id}/entries, a paid feed's included, with the
addresses of its files, which can carry the subscriber's key: the Directory leaves such feeds
out for that reason, and this route handed them back to whoever guessed the id, a slug of the
title. In production it answered 25 items of a feed the asking account does not subscribe to.
/media/{id} served any downloaded file by its sequential id, POST /api/enclosures/{id}/download
and /api/feeds/{id}/download-latest queued any feed's downloads, and DELETE
/api/enclosures/{id}?force=true deleted any file.

Each now answers 404, "you do not subscribe to that feed", unless the person subscribes to it.
A feed inside an OPML has a subscription row of its own for everyone subscribed to the OPML, so
that holds for those feeds too. Found while adding the Directory's feed page (#128), which has
its own route that answers only for listed feeds and carries no files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:19:38 +00:00
95acde3046 Lay the Directory out as Apple's is, with the most subscribed in it (#124, #126, #128)
The Directory was one grid of every listed feed, 1,472 in production, under Podcasts and Blogs
tabs, a wall of about 30 category buttons and a sort menu, with a picked category's
subcategories appearing as a second row of buttons that looked like the first. It read as a
list to scroll, and the two rows were hard to tell apart.

It opens now on the ten most subscribed feeds, ranked, and the categories as tiles, each with
three of its shows' covers fanned in its corner on the tint its initials would get, so the
shows colour it and no theme's palette changes. A category has a page of its own: its most
subscribed, its subcategories with how many each holds, and its grid. See all is every feed,
as the grid was. Podcasts or Blogs holds across every page. Popular, the ten most subscribed,
had a place of its own in the feed list; it is the Directory's first section instead, and its
key, g p, is gone. /api/popular stays for scripts.

The search box finds a feed in the Directory by name (#126). It said "Search items…" there and
did nothing, since loadEntries returns early for a place that lists feeds.

A feed you do not subscribe to opens a page of its own (#128): its cover, category (a link to
that category's page), how many here subscribe, a subscribe button and its latest twenty items,
each with its title, linking to the post where it has one, its first lines, date and length.
Before, a click on it did nothing; only its + did. The items come from GET /api/directory/{id},
which answers only for a feed the Directory lists, so a guessed id reaches nothing private, and
carries no file or its address. The feed's own description would belong at the top, but the
database does not keep one.

Every tile and chart row takes the keyboard, as the feed list's rows do. Feed text made plain
keeps a space at a line break or a paragraph's end, which glued "2010)Recorded" together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:19:38 +00:00
00bd58ac9d API tokens a person makes for scripts and agents to act as them (#123)
The API took a session cookie, a proxy's word or the shared admin token, so a script or an
agent working for one person had to sign in with their password and carry the cookie, or be
given the admin token. Settings now makes named tokens, ipx_ and 256 random bits, sent as
Authorization: Bearer. A token is its owner and no more. Only its SHA-256 is kept, in the
new api_tokens table, with when it was made and last used; it is shown once and revoked from
the same list. An unknown or revoked one gets a 401 rather than falling through to a cookie.

Cloudflare Access still stands in front of the tunnel, so from outside a token needs an
Access service token beside it; docs/sso.md says how.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 00:15:17 +00:00
cc17b1ddab Sort the Directory by name either way or by subscribers (#122)
The Directory always listed A to Z, as the server sends it. A menu beside its filters now
sorts it A to Z, Z to A or by most subscribers (then by name), in the page, without asking
the server again, and the choice stays while the pane is redrawn, as the filters do. The
blurb no longer says A to Z, since it may not be.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 22:54:32 +00:00
02a46bb500 Browse the Directory by category, then subcategory, as Apple does (#118)
The Directory had one row of chips holding whatever each feed's category was, a category
(Technology) or a subcategory (Tech News, Video Games) side by side: a podcast's own
<itunes:category> is stored as its subcategory, and Jev's answers (#117) are often
subcategories too, so after the first forced scan the row held 22 chips. /api/directory now
gives each feed's Apple category and subcategory, worked out from Apple's list, and the
page shows the categories, then a picked one's subcategories on a line of their own. A
category that is not Apple's, one an admin typed, stands as a category of its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 22:33:26 +00:00
d238681ec1 Name an item without a title from its own text, not "(untitled)" (#116)
RSS 2.0 makes an item's title optional, and some blogs leave it out on purpose: Scripting News
titles almost none of its posts. Fifty rows of "(untitled)" said nothing about any of them.

entryName gives an item its title, or the opening of its text (HTML read through DOMParser, an
inert document that loads nothing; cut at a word near 120 characters), or its file's name, or
its show and date, with a flag for a name that is not a title. The list sets that one in the
regular weight, as the text it is rather than a heading; the reader leaves out the heading so
the post starts with itself; the player, the lock screen, Currently Listening, the native shell,
Share and the queued toast use the same name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:46:47 +00:00
a00516687a Keep artwork on disk and serve every image from iPX (#111)
The page loaded artwork from each publisher's server, or through /api/art, fetched every time,
for http-only hosts. Nothing was kept, every visit asked every publisher, and artwork went when
a publisher's server did.

- The page draws every image a feed or item names from /api/art. The first time, iPX fetches
  it (only an address a feed or item names, only an image, up to 5 MB, within the feed timeout)
  and keeps it in art/ beside the database, under a hash of its address with its type beside
  it (src/art.rs). Later it comes from disk, which marks it as used.
- art_cache_mb, a server setting on the admin page, 500 by default, caps what is kept: the
  sweep before each scan drops the least recently shown until it fits. 0 keeps nothing, and
  artwork is fetched through iPX each time. Settings saved before it get the default.
- Served from iPX's own address, someone else's image must stay an image: nosniff, and a CSP
  with sandbox, so an SVG opened on its own runs no script as iPX.
- The fixture server sends .jpg as image/jpeg, which /api/art requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 13:50:46 +00:00
54d827f655 Time out a hung feed, serve the precomposed touch icon, and store http artwork on https (#108, #109, #110)
#108: the HTTP client had no timeout, and scans handle feeds in order, so a hung server held
every scan. Dreamwidth answered 504 after 60-67 s for a day and each scan took 70-80 s instead of
15. A feed fetch, and a Patreon creator's show list, now gets 30 s from connecting to the last
byte (feed::FEED_TIMEOUT); the client gets a 10 s connect timeout, which bounds a download's
start but not a long download.

#109: iOS asks for /apple-touch-icon-precomposed.png first when the site is added to a home
screen; it was a 404 and the only non-feed warning in the log. It serves the same icon.

#110: the page is https and loads no http. Artwork on http came through /api/art (#90) even
when its host serves https too. A scan now tries each http artwork host on https once per feed
(feed::prefer_https) and stores the https address where the host answers with an image,
rewriting that feed's stored items from the same host (Db::secure_images). 4 of the 5 hosts in
production do; cdn.thesecretcabal.com presents another name's certificate and stays on
/api/art.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 13:31:50 +00:00
c7f13eea2f Send a changed feed's row to the page instead of it reloading the list (#105)
After a feed was checked, failed or downloaded a file, and after every item read, the page
fetched /api/feeds whole, about 60 ms for 160 rows, though one row had changed. The live event
stream now knows who is connected and, after an event that changes a feed, sends that person
its row (feed_row), built by the same code as the list (feed_rows, with Db::feed_list asked for
one feed). Marking an item read answers with the feed's row. The page puts the row in place
and redraws once a frame. A routine skip of a feed not due sends nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:41:43 +00:00
79bc006a30 Add only what is a feed or links one; refuse the rest (#102)
Adding an address looked for the feed a web page links and, finding none, added the address
as it was: every check then failed, and the sidebar called it a feed that had moved. cnn.com
is one; its page links no feed. find_feed replaces feed_behind_page: the address is added if
it is a feed or an OPML list, the feed its page links if it is a web page that links one (and
that is a feed), and otherwise the add is refused with the reason, from the web page (400, the
dialog stays open) and from `ipx add`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:06:04 +00:00