Try a feed again in minutes when its host could not be looked up for the moment (#143)

205 of the Directory's 208 feeds without a category had never been read: on 2026-10-05 at 03:24
UTC every one failed with "dns error: ... Temporary failure in name resolution" within a minute,
during an outage on Tower that also stopped iPX's log reaching Loki until Tower rebooted at 11:56.
They are working podcasts, NPR's, Megaphone's, Acast's, which resolve now. A feed nobody
subscribes to is read once a day and a failing one backs off up to a day (#99), so each sat
failing, unread and so uncategorised (#117) until the next day, and explain_failure called every
DNS error "This address no longer resolves; the site is gone.", on its page and since #142 on its
Directory page. A forced check of the 208 read 205; each has its items, its description and its
own category now.

A temporary lookup failure, EAI_AGAIN, is not recorded on the feed at all: no last_error, no
error_since to back off from or call it gone by. The daemon notes it in Ctx::retry and the feed
is due again in ten minutes, which the scan's due check and the sleep until the next scan both
honour; a restart forgets it, which only makes the retry sooner. A name that does not exist is
still the feed's error, explained as "may be gone", and the Directory page, like the feed's own,
names a cause only once a feed has been failing a day; before that its last check failed.

The page also drops "Nobody here subscribes yet", which said nothing worth a line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.claude/skills/ipx-prod-check/SKILL.md

@@ -32,12 +32,13 @@ lines had no trace id, the access log's time was `ms`, and each event was logged
 | target | fields | what |
 |---|---|---|
 | `ipx::http` | `method`, `path`, `route`, `status`, `duration_ms` | one per web request; `route` is the pattern, empty for an unrouted path |
-| `ipx::scan` | `ev` and the event's own: `feed`, `new`, `downloaded`, `failed`, `bytes`, `msg`, `url`, `feeds`, `reason`; on a failure `error.type` and, from an HTTP error, `http.response.status_code` | the daemon's events, one line each, in words; warnings are feed and download failures |
+| `ipx::scan` | `ev` and the event's own: `feed`, `new`, `downloaded`, `failed`, `bytes`, `msg`, `url`, `feeds`, `reason`, `from`, `to`; on a failure `error.type` and, from an HTTP error, `http.response.status_code` | the daemon's events, one line each, in words; warnings are feed and download failures |
 | `ipx::io` | `ev`, `feeds`, `pending`, `downloaded` on the `status` reply | commands arriving (`-> {...}`) and the healthcheck's answer |
 | `ipx` | message, sometimes fields | start-up, shutdown, account and config messages |
 
 Events (`ev`): `feed_start`, `feed_done` (new, downloaded, failed, torrents), `feed_skip` (not due,
-routine), `feed_error` (msg), `download_done` (bytes), `download_error` (msg, url),
+routine), `feed_error` (msg), `feed_moved` (from, to: a permanent redirect followed, the address
+updated), `download_done` (bytes), `download_error` (msg, url),
 `torrent_deferred`, `reaped`, `scan_done` (feeds checked), `reap_done`, `status` (feeds, pending,
 downloaded: the healthcheck's, every 30s), `error` (msg).
 
@@ -85,6 +86,12 @@ a count says something happened, the lines and traces say why.
    crash or an OOM kill: check `docker inspect iPX -f '{{.State.OOMKilled}} {{.RestartCount}}'`
    and the lines just before it. A pending count that only grows means downloads are not
    keeping up or not running.
+
+   Since 2026-10-02 the daemon sleeps until the next feed is due (at most 10 minutes) instead of
+   scanning every minute (#114), so expect tens of scans in 6 hours, not 360, nearly all with
+   `feeds` above 0; none at all for over 10 minutes means the worker is stuck. And `pending` is
+   the real queue (#113): files a scan will download on its own. Back-catalogue files are
+   `held`, listed but not counted, so it is usually 0 or a handful.
 5. **Slow and failed traces.**
    ```
    $Q traces '{resource.deployment.environment.name="production" && duration > 5s}'

.gitignore

@@ -4,4 +4,5 @@
 /playwright-report
 /web/dist
 
-.claude/settings.local.json
+.claude/settings.local.json
+__pycache__/

CHANGELOG.md

@@ -7,6 +7,80 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Changed
+
+- A feed's page in the Directory says how many people here subscribe only when someone does, without "Nobody here subscribes yet".
+
+### Fixed
+
+- A moment's DNS failure no longer marks a feed as failing for a day or calls it gone: iPX tries it again in ten minutes. One failed lookup one morning left 205 podcasts unread, uncategorised and labelled gone until the next day. A name that does not resolve is said to be maybe gone, and only once it has been failing a day, on its Directory page as on its own.
+
+## [0.10.2] - 2026-10-05
+
+### Fixed
+
+- A publisher's correction to an item already in iPX, a retitled episode, mended text, new artwork or a fixed length, arrives on the next read. Since 0.9.1 an item kept what it first said: Ain't It Cool News's items each showed the text of the one before, long after the feed was fixed.
+- A feed's page in the Directory says when the feed is failing, and why, and no longer says nobody here subscribes to a feed someone does.
+
+## [0.10.1] - 2026-10-05
+
+### Fixed
+
+- A feed whose address answers with nothing, as a lapsed domain does behind a DNS filter's block page, shows as failing, with what to do about it, instead of as a show that has not posted yet.
+- The search box finds episodes in Currently Listening. It did nothing there.
+- `ipx add`, `ipx rm` and `ipx import`, run while the daemon runs, reach it at once. It kept its old list of feeds and wrote it back at its next change, undoing them.
+- On SQLite, several requests are answered at once instead of one at a time. With 25 people browsing, a page of a feed's items comes back in 10ms rather than 130.
+- On Postgres, a write no longer waits for the database server's disk. A scan read about one feed a second and now reads 1,500 in 20 seconds.
+- Checking every feed no longer makes every icon in the feed list flash while it runs: the list keeps the icons it has already drawn.
+
+### Security
+
+- A flood of sign-in attempts no longer stalls the site for everyone else. Passwords are checked a few at a time, away from the threads that answer every other request; forty wrong passwords at a time made everything else take four seconds.
+- A wrong password is refused in the same time whether or not the name is an account here, so how long it takes no longer tells anyone which names are.
+
+## [0.10.0] - 2026-10-05
+
+### Added
+
+- API tokens: in Settings, make a named token that lets a script or an agent use iPX as you, sent as `Authorization: Bearer`, and revoke it there. Each shows when it was last used.
+- In the Directory, a feed you do not subscribe to opens a page of its own: its cover, its category, how many people here subscribe, what the feed says it is, and its latest episodes or posts, with a button to subscribe. Before, only its subscribe button did anything.
+- Feeds that don't set a category, as most blogs don't, now get one automatically, so they show up under a category in the Directory. iPX picks the closest of Apple's podcast categories from the feed's title and latest headlines. This happens once per feed: the first time a new feed is read, or for a feed you already had, the next time you force a refresh. It needs `TYPESAFE_KEY` set. An admin can change the category in the feed's settings.
+- Artwork is kept on disk once shown and loads from iPX, not from each publisher's server: fast after the first time, and still there when the publisher's server is not. The admin page sets how much is kept (500 MB by default).
+- `ipx add --list --category News <url>` puts a feed in the Directory for anyone to subscribe to,
+  and it stays there when its last subscriber leaves. Run for a feed already in the catalogue,
+  it lists it or sets its category.
+- Feeds nobody subscribes to that are dead for a month or quiet for a year are removed, so the
+  Directory lists feeds worth taking.
+
+### Changed
+
+- The Directory is laid out as Apple's is. It opens on the ten most subscribed feeds, ranked, and the categories as tiles showing some of their shows' covers. A category opens a page of its own with its most subscribed feeds, its subcategories (Tech News under News, Video Games under Leisure) and its feeds; See all lists every one. This replaces the grid of every feed under rows of category buttons.
+- The most subscribed feeds are the Directory's first section instead of a place of their own: Popular is gone from the feed list, and so is its key, g p.
+- A pinned item in a list wears the same disc as a pinned feed, in the theme's accent, instead of a plain solid pin.
+- The Directory can be sorted by name, A to Z or Z to A, or by most subscribers. It still opens A to Z.
+- An item published without a title shows its opening words, in plain text rather than bold, instead of "(untitled)"; one with no text either shows its file's name, or its show and date. Opened, it starts with its text.
+- A feed that has moved for good (a permanent redirect) is followed to its new address, which iPX then reads from, and says so in the log as `feed_moved`. A temporary redirect changes nothing.
+- The daemon sleeps until the next feed is due, at most ten minutes, instead of looking every minute; refreshing or adding a feed still wakes it at once.
+- A pinned feed's pin sits on the corner of its artwork, as a failing feed's mark does, instead of before its name.
+- The Directory lists feeds nobody subscribes to yet; its most subscribed still lists what people subscribe to.
+  A feed nobody subscribes to is checked once a day, and at once when someone subscribes. A
+  listed feed downloads nothing until someone subscribes.
+- The Directory loads faster: it asked the database three questions per feed.
+
+### Fixed
+
+- A database on SQLite, the default, no longer waits for the disk after nearly every write. On a slow disk a scan of 1,500 feeds took most of an hour and now takes half a minute. Postgres is unaffected.
+- The search box finds a feed in the Directory by name. It did nothing there.
+- Feeds that file themselves under Apple's Football or Soccer show under Sports, and those under the old Software How-To under Technology, instead of as categories of their own.
+- The download queue, in `ipx status`, `/api/status` and the dashboard, counts only what iPX will download on its own: older episodes beyond a show's limit, and files of feeds that do not download automatically, are listed but no longer counted as waiting.
+- Artwork published on http is stored on https when its host serves it there, so the page loads it directly; the rest still comes through iPX.
+- An iPhone adding the site to its home screen finds the icon at the first address it tries.
+- A feed whose server hangs no longer holds up every scan: a feed gets 30 seconds, and connecting anywhere 10.
+
+### Security
+
+- A feed's items and files reach only the people who subscribe to it. Anyone signed in could read any feed's items, a paid feed's included, with the addresses of its files, which can carry the subscriber's key, and could play, download or delete its files.
+
 ## [0.9.1] - 2026-09-29
 
 ### Added
@@ -692,7 +766,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
 - `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
 
-[unreleased]: https://git.sdf1.net/rays/ipx/compare/v0.9.0...main
+[unreleased]: https://git.sdf1.net/rays/ipx/compare/v0.10.2...main
+[0.10.2]: https://git.sdf1.net/rays/ipx/compare/v0.10.1...v0.10.2
+[0.10.1]: https://git.sdf1.net/rays/ipx/compare/v0.10.0...v0.10.1
+[0.10.0]: https://git.sdf1.net/rays/ipx/compare/v0.9.1...v0.10.0
 [0.9.1]: https://git.sdf1.net/rays/ipx/compare/v0.9.0...v0.9.1
 [0.9.0]: https://git.sdf1.net/rays/ipx/compare/v0.8.4...v0.9.0
 [0.8.4]: https://git.sdf1.net/rays/ipx/compare/v0.8.3...v0.8.4

CLAUDE.md

@@ -124,7 +124,8 @@ npx tsc -p .              # type-checks web/src
 node tests/page-smoke.js
 node tests/native-bridge.js  # the page hands playback to a native shell
 node tests/contrast.js    # every theme's palette against WCAG AA
-npx playwright test       # 40 browser tests against a real daemon on fixture feeds
+npx playwright test       # 66 browser tests against a real daemon on fixture feeds
+node tests/load/run.js    # k6: many people at once against a scratch daemon with 1,500 feeds
 ```
 
 Things about the browser suite that have cost time:
@@ -141,6 +142,31 @@ Things about the browser suite that have cost time:
 * `webServer` starts **before** `globalSetup`, which is why the fixture config is written at
   config-load time instead.
 
+The load tests (`tests/load`, issue #133) are for what one browser cannot show: twenty-five people
+browsing at once, a hundred players saving positions while a scan writes, fifty listeners
+seeking through files, a flood of wrong passwords. Things about them:
+
+* They need **k6**, which `/src/install.sh` installs from k6's own signed apt repository, and they
+  build and run a **release** binary: Argon2 in a debug build takes about a second a sign-in,
+  which would measure the build.
+* `run.js` serves the feeds itself, generated, and starts **its own daemon** under
+  `/tmp/ipx-load`, wiped each run, on ports 8793 and 8794, so it can run beside the browser suite.
+  It stops that daemon by its PID. People sign in by the `X-Load-User` header, trusted from
+  127.0.0.1, as production trusts Cloudflare Access's; only the sign-in test uses a password.
+* Each threshold is a budget well above what the request takes now: it is there to catch a query
+  that has started asking once per feed, or writers queueing on a lock, not a slow minute.
+* `ipx status`, the Docker healthcheck, runs every second throughout, and a run fails if one takes
+  the healthcheck's 5s.
+* One at a time: `node tests/load/run.js signin`. All four take about six minutes.
+* The daemon is on **SQLite** unless `--postgres`, which puts it in `IPX_TEST_DATABASE_URL`, in
+  a schema of its own (`ipx_load`) dropped and made again each run. That URL, in `/src/.envrc`, is
+  `ipodderx_test` on the `databases` project's server, production's, as `ipodderx`, taken from
+  `ipx.env`; the harness refuses a database whose name does not end in `_test`. Postgres's
+  numbers are what production would see, and a run loads the server production's database is on,
+  so not while someone is using iPX in earnest. Each budget has a value for each database.
+* The Rust tests run on Postgres with the same URL: `. /src/.envrc && cargo test`. Each test
+  makes a schema `ipxt_<pid>_<n>`, and the next run drops those an earlier one left.
+
 Non-trivial logic leaves one runnable check behind. Pure functions (`merge_policy`, `pick`,
 `matches_keywords`, `parse_interval`) are the easiest place to put it.
 
@@ -182,7 +208,8 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
   command, the healthcheck's `ipx status` included, so it must never write when nothing is
   missing: SeaORM's experimental schema sync dropped and remade an index on every open, the
   write lock that took made `ipx status` time out behind a busy daemon, and it was removed for
-  it. A new column on an existing table needs its own `ALTER`; nothing adds one for you.
+  it. A new column on an existing table needs its own `ALTER`; nothing adds one for you. Look for
+  the column first and alter only when it is missing, as `feeds.description` does.
 * **SQL written by hand in `db.rs` has to run on SQLite and Postgres both** (issue #18): `$1`
   parameters, bound only if used; `ON CONFLICT`, not `INSERT OR IGNORE`; yes/no columns tested
   as themselves (`NOT coalesce(s.read, false)`) and written as `true`/`false`, never compared to

Cargo.lock

@@ -1820,7 +1820,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
 
 [[package]]
 name = "ipx"
-version = "0.9.1"
+version = "0.10.3-dev"
 dependencies = [
  "ammonia",
  "anyhow",
@@ -1843,6 +1843,7 @@ dependencies = [
  "sea-orm",
  "serde",
  "serde_json",
+ "sha2 0.10.9",
  "tokio",
  "toml",
  "tower",

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "ipx"
-version = "0.9.1"
+version = "0.10.3-dev"
 edition = "2024"
 
 [dependencies]
@@ -25,6 +25,7 @@ rss = "2.1.1"
 sea-orm = { version = "2.0.3", default-features = false, features = ["sqlx-sqlite", "sqlx-postgres", "runtime-tokio-rustls", "macros", "with-json", "sqlite-use-returning-for-3_35"] }
 serde = { version = "1.0.229", features = ["derive"] }
 serde_json = "1.0.151"
+sha2 = "0.10.9"
 tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
 toml = "1.1.5"
 tower = { version = "0.5.3", features = ["util"] }

README.md

@@ -9,7 +9,7 @@ behind iPodderX (2004-2008, Ray Slakinski & August Trometer).
 
 ## What it does
 
-- **The web UI.** It has a toolbar, and a feed list that opens with Directory, Popular and All
+- **The web UI.** It has a toolbar, and a feed list that opens with the Directory, Currently Listening and All
   Subscriptions. Items sit in a sortable table with a Files pane, and there is a player bar. It
   comes in Dark, Light and Classic themes, and works on a phone.
 - **Several people, one copy.** Each person has their own subscriptions and their own read, pinned

docs/architecture.md

@@ -38,7 +38,9 @@ build needs node and `npm ci` run once.
    re-derived into the database each scan, never written to config.toml. A Patreon creator link
    (a token, no `show=`) with more than one show is treated the same way, before any fetch: its
    shows come from Patreon's web API and each becomes a derived feed.
-4. Record entries. A changed title or description flips the item back to unread.
+4. Record entries: the new ones, and those the feed now gives a different title, text, artwork,
+   length or number, compared with what is stored for the items it lists (#141). A corrected item
+   keeps its read state.
 5. Record enclosures. `enclosures.url` is `UNIQUE`, which is the dedupe key and subsumes the
    original's `history.dat` pickle: a reaped file keeps its row so it is never fetched twice.
 6. Apply the merged policy (see [users.md](users.md)) and mark anything rejected as `skipped` with
@@ -51,8 +53,8 @@ build needs node and `npm ci` run once.
 ## Data model
 
 ```
-feeds       id, url, title, image, etag, last_modified, last_checked, ttl_mins,
+feeds       id, url, title, image, category, description, etag, last_modified,
-            last_error, orphaned, group_id, managed
+            last_checked, ttl_mins, last_error, orphaned, group_id, managed
 entries     feed_id, guid, title, link, published, description, first_seen,
             image, duration, episode, season            PK (feed_id, guid)
 enclosures  id, feed_id, guid, url UNIQUE, mime, length, path, state,
@@ -71,7 +73,8 @@ before accounts; two bugs came from queries still reading them, and they were dr
 Schema changes: the tables are the entities in `src/entity.rs`, and `Db::open` creates whatever
 table or index a database is missing from them (`db::create_missing`), with `IF NOT EXISTS`. It
 never alters a table that exists, so a new column on one needs its own `ALTER` in
-`create_missing`, or `sea-orm-migration` once there are several. `Db::memory()` builds its
+`create_missing`, looked for first and run only when it is missing, as `feeds.description`'s is,
+or `sea-orm-migration` once there are several. `Db::memory()` builds its
 database the same way, so the tests run on the schema production gets. A database from before
 0.7 takes its last columns from the old `migrate()`, so it upgrades through a 0.7 release first.
 
@@ -89,13 +92,15 @@ printf '{"cmd":"fetch","force":true}\n' | socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/
 ```
 
 **Commands** — `fetch` (optional `feed`, `force`), `reap` (optional `dry_run`), `download`
-(`enclosure`), `status`.
+(`enclosure`), `status`, `reload` (read the catalogue and settings again from the database, which
+`ipx add`, `rm` and `import` send after changing them in a process of their own; answered with
+`status`).
 
 **Events** — `feed_start`, `feed_skip`, `feed_done`, `feed_error`, `progress`, `download_done`,
 `download_error`, `torrent_deferred`, `reaped`, `reap_done`, `scan_done`, `status`, `error`.
 `scan_done`, `reap_done` and `status` are terminal: a client that asked for work stops reading
-there. Commands run one at a time, in the order they arrive, except `status`: the socket answers it
+there. Commands run one at a time, in the order they arrive, except `status` and `reload`: the
-straight away, so the Docker healthcheck is never left waiting behind a scan or a download, and
+socket answers them straight away, so the Docker healthcheck is never left waiting behind a scan or a download, and
 answers only the client that asked, since `status` would end any other client's session.
 
 Progress carries the enclosure id, without which a UI cannot tell one download from another and
@@ -110,7 +115,8 @@ so a long download had to be able to notice the signal itself.
 ## HTTP API
 
 Everything below `/api` needs a signed-in user; the browser gets a redirect to `/login`, anything
-else a `401`.
+else a `401`. A feed's items and files (its entries, `download-latest`, `/api/enclosures/{id}` and
+`/media/{id}`) answer only someone who subscribes to the feed; anyone else gets a `404`.
 
 | Route | |
 |---|---|
@@ -126,6 +132,7 @@ else a `401`.
 | `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
 | `POST /api/fetch` | |
 | `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
+| `GET /api/directory/{id}` | a listed feed's description, why its last check failed when it did (in words, never its error, which can name its address), and latest twenty items, for its page before you subscribe: title, link, date, length and text, never a file or its address, and only for a feed the directory lists |
 | `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with an OPML's feeds in place of the OPML and everyone counted (id, title, art, count, whether it is yours, the feed's iTunes category, whether it carries audio or video; never a URL, never a private feed); subscribe by id |
 | `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
 | `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
@@ -143,6 +150,7 @@ cargo test                # parsing, filters, retention, schedules, SQL, per-use
 node tests/page-smoke.js  # the page script loads and every selector it wires at load exists
 node tests/native-bridge.js  # the page hands playback to a native shell
 npx playwright test       # a real browser against a real daemon on fixture feeds
+node tests/load/run.js    # k6: many people at once against a daemon with 1,500 generated feeds
 ```
 
 The Rust tests cannot see a wrong selector, a handler that runs and does nothing, or a page that

docs/cli.md

@@ -13,13 +13,17 @@ processes must never download the same thing. `--local` forces the work to happe
 | `ipx list` | Subscriptions and their state |
 | `ipx status` | Counts: feeds, pending, downloaded |
 | `ipx fetch [FEED] [--force]` | Scan everything, or one feed. `--force` ignores the TTL |
-| `ipx add <url> [--folder X] [--keywords a,b]` | Subscribe; the id comes from the feed title |
+| `ipx add <url> [--folder X] [--keywords a,b] [--list] [--category C]` | Subscribe; the id comes from the feed title. `--list` puts it in the Directory for anyone to subscribe to instead, and keeps it there when its last subscriber leaves; `--category` files it under one of the Directory's categories. Run for a feed already in the catalogue, these list it or set its category |
 | `ipx rm <feed>` | Unsubscribe; downloads and history are kept |
 | `ipx import <file.opml>` / `ipx export <file.opml>` | Move subscriptions in or out. Import subscribes the first admin, as the shared web token does; in the web UI it subscribes whoever is signed in |
 | `ipx reap [--dry-run]` | Run retention now |
 | `ipx user <add\|list\|passwd\|rm>` | Accounts for the web UI |
 | `ipx daemon [--web ADDR]` | Scheduler, control socket and web UI |
 
+`add`, `rm` and `import` change the catalogue in a process of their own. With a daemon running they
+then tell it to read the catalogue again; without that it kept its own copy and wrote it back at
+its next change, undoing them. If it does not answer they say so: restart it.
+
 ## Accounts
 
 Passwords are read from **stdin**, so they miss the shell history and any `ps` listing.

docs/configuration.md

@@ -4,7 +4,7 @@ Two places. **config.toml** holds what ipx needs before it reaches its database,
 who gets in: where things are (`download_dir`, `socket`, `organize`), `[torrent]` and `[web]`.
 **The database** holds the catalogue of feeds (`[feeds.<id>]` below) and the server settings the
 admin page edits (`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check`,
-`media_types`). Change those in the web UI, or with `ipx add`, `ipx rm` and `ipx import`; they
+`media_types`, `art_cache_mb`). Change those in the web UI, or with `ipx add`, `ipx rm` and `ipx import`; they
 take effect without a restart.
 
 The first time ipx meets a database that holds no catalogue, it takes the feeds and those
@@ -27,6 +27,12 @@ config.toml's default location is `$XDG_CONFIG_HOME/ipx/config.toml`
 Postgres database instead. Back SQLite up by copying `state.db` while the daemon is stopped, or
 with `sqlite3 state.db .backup`; back Postgres up with `pg_dump`.
 
+ipx does not wait for the disk to confirm each write: SQLite syncs at checkpoints
+(`synchronous=NORMAL`), and ipx's own Postgres sessions set `synchronous_commit=off`, which leaves
+other databases on the server as they are. Waiting made a scan take most of a second a feed. A
+crash of ipx loses nothing; a power cut, or a crash of the Postgres server, can lose the last moment
+of changes, a read mark or a saved position, and never corrupts the database.
+
 ## `[general]`
 
 ```toml
@@ -39,21 +45,27 @@ max_total_gb      = 50              # 0 = unlimited
 max_age_days      = 30              # 0 = keep forever
 max_new_per_check = 3               # per feed, per scan. 0 = unlimited
 media_types       = ["audio", "video"]
+art_cache_mb      = 500             # artwork kept on disk. 0 = none
 ```
 
-`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check` and `media_types` move into the
+`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check`, `media_types` and `art_cache_mb` move into the
 database as described above; `download_dir`, `socket` and `organize` stay in config.toml.
 
 * **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
   be polled *less* often, and a per-feed `schedule` overrides both. A feed that keeps failing
   waits as long as it has been failing before the next try, up to a day, and is back on schedule
-  after its first success. Admin-only from the UI.
+  after its first success. A moment's failure to look its host up does not count: the network, not
+  the feed, failed, and it is tried again in ten minutes. Admin-only from the UI.
 * **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
 * **`max_total_gb`** — the reaper deletes to get back under this, oldest first, keeping a 50 MB
   pad. Kept items are never deleted, and a file only counts as read once every subscriber has
   read it. `0` disables it entirely.
 * **`max_age_days`** — items older than this with no file on disk are pruned from the database.
   Kept ones stay. `0` disables it.
+* **`art_cache_mb`** — how much show and episode artwork iPX keeps on disk, in `art/` beside the
+  database, so the page loads it from iPX rather than from every publisher's server. Over this,
+  what has gone longest unshown goes first. `0` keeps none: artwork still comes through iPX, fetched
+  each time. 500 by default.
 * **`max_new_per_check`** — how many of a feed's newest episodes are downloaded; older ones stay
   listed to download by hand. It stops a new subscription pulling a whole back catalogue.
   `0` means every episode, for an archive; set it on the feeds you want archived, since on the
@@ -124,6 +136,7 @@ folder             = "Accidental Tech Podcast"   # default: the feed title
 schedule           = "every 6h"                  # overrides [general] for this feed
 media_types        = ["audio"]                   # overrides [general] for this feed
 category           = "Technology"                # the Directory's, if the feed names none
+listed             = true                        # in the Directory with no subscribers
 username           = "ray"                       # HTTP basic auth
 password_env       = "IPX_ATP_PASS"              # preferred over a literal `password`
 ```
@@ -133,6 +146,12 @@ With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
 config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
 and are the same for everyone. See [users.md](users.md).
 
+**The Directory** lists every feed in the catalogue, whether or not anyone subscribes to it yet.
+To put one there for others to find, `ipx add --list --category News <url>`: it subscribes no one,
+and a listed feed stays when its last subscriber leaves, where another is dropped. A feed nobody
+subscribes to is checked once a day. One that is dead (failing for 30 days) or quiet (nothing
+new in a year), with nobody subscribed and no file on disk, is removed from the catalogue.
+
 Feeds derived from a subscribed OPML are **not** in the catalogue: the OPML is the source of truth
 and they are re-derived on every scan. Editing one in the UI promotes it to a catalogue entry.
 

docs/sso.md

@@ -233,5 +233,19 @@ Set `auto_create_users = false` once everyone who should have an account has one
 proxy vouching for an unknown name is logged and refused. Make people ahead of time instead, with
 the exact name the header will carry.
 
+## API tokens for scripts and agents
+
+Anyone can make API tokens for their own account in Settings. A token is sent as
+`Authorization: Bearer ipx_...` and acts as the person who made it, admin only if they are. Only
+its SHA-256 is kept; it is shown once, when made, and revoked there too.
+
+Through the tunnel, Cloudflare Access turns a request with no Access sign-in towards Authentik
+before ipx sees it, so a token alone does not get in that way. On the LAN, `http://192.168.1.130:8099`
+takes it directly. From outside, make an Access service token, add a Service Auth policy for it to
+the `ipodderx` application, and send `CF-Access-Client-Id` and `CF-Access-Client-Secret` beside the
+`Authorization` header: Access lets the request through, vouches for no name, and ipx takes the
+API token as who is asking. Do not bypass Access for `/api/*` instead; the token would then be the
+only thing between the internet and the API.
+
 See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
 for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.

docs/users.md

@@ -70,11 +70,15 @@ list with their own read state. Unsubscribing removes it from their list alone;
 subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
 re-subscribing does not pull the back catalogue again.
 
-**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
+**Directory** sits at the top of the feed list, above your own feeds. It opens on the ten feeds
-shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
+with the most subscribers on this server, you included, ranked, then the categories, each a tile
-included. Directory shows every one of them A to Z as a grid of cover art. Above it, a filter
+showing a few of its shows' covers. A category has a page of its own: its most subscribed feeds,
-picks Podcasts (anything with audio or video) or Blogs (the rest), and chips pick the category
+then all of them as a grid of cover art, with its subcategories along the top (Tech News under
-each show gives itself in iTunes; the two combine. Your own feeds are marked Subscribed.
+News, Video Games under Leisure) and a choice of order. See all lists every feed, and the search
+box finds one by name. A feed you do not subscribe to opens a page of its own, with what it says it
+is and its latest episodes or posts, so you can see what it is first. Every page can be narrowed to Podcasts (anything with audio or video) or
+Blogs (the rest). A show's category is the one it gives itself in iTunes, or for one that names
+none, one an admin or iPX picked for it. Your own feeds are marked Subscribed.
 It shows a title, artwork and a count, never a URL or who reads it. An OPML subscription is listed
 as the feeds inside it, one by one, and never the OPML itself, so you can take just the shows you
 want. Anything that looks private is left out: a login configured for the feed, credentials in its URL,
@@ -83,6 +87,10 @@ Patreon or Supercast, which put the key in the path, and any feed inside an OPML
 itself. Those are someone's paid subscriptions, and listing them would let anyone here read what
 they pay for.
 
+**Currently Listening**, under the Directory, is every episode you started and have not finished,
+across all your feeds, with how much is left. A click picks one up where you left off. The search
+box looks through it, as it does a feed's items, and through the Directory by name.
+
 An admin can do the same from **Settings → Manage users…**: add someone (with a password, or none
 for someone the proxy signs in), tick or untick Admin, or remove an account. Removing one takes its
 subscriptions and read state with it; downloaded files stay. The only admin cannot be demoted or

package.json

@@ -7,7 +7,8 @@
     "typecheck": "tsc -p .",
     "smoke": "node tests/page-smoke.js",
     "test": "playwright test",
-    "test:headed": "playwright test --headed"
+    "test:headed": "playwright test --headed",
+    "load": "node tests/load/run.js"
   },
   "devDependencies": {
     "@playwright/test": "^1.56.0",

src/art.rs

@@ -0,0 +1,92 @@
+//! Artwork kept on disk, so the page loads it from iPX: fast after the first time, nothing asked
+//! of each publisher's server for every visit, and still there when a publisher is not.
+
+use std::path::{Path, PathBuf};
+
+pub fn dir() -> PathBuf {
+    crate::config::data_dir().join("art")
+}
+
+/// Where an image's bytes are kept; its content type is beside it, in `<name>.type`.
+// ponytail: std's hasher, 64 bits, keyed by the URL. Its algorithm may change with Rust, which
+// only makes the cache miss once and refill; a collision among a few thousand images is about
+// one in 10^12. Move to a SHA if either ever matters.
+pub fn path(url: &str) -> PathBuf {
+    use std::hash::{Hash, Hasher};
+    let mut h = std::collections::hash_map::DefaultHasher::new();
+    url.hash(&mut h);
+    dir().join(format!("{:016x}", h.finish()))
+}
+
+/// A kept image and its type, marked as just used, which is what keeps it from being trimmed.
+pub fn read(file: &Path) -> Option<(String, Vec<u8>)> {
+    let kind = std::fs::read_to_string(file.with_extension("type")).ok()?;
+    let body = std::fs::read(file).ok()?;
+    if let Ok(f) = std::fs::File::options().write(true).open(file) {
+        let _ = f.set_modified(std::time::SystemTime::now());
+    }
+    Some((kind, body))
+}
+
+/// Keeps an image, written aside and renamed into place, so a page asking for it meanwhile
+/// never reads half of one.
+pub fn write(file: &Path, kind: &str, body: &[u8]) -> std::io::Result<()> {
+    std::fs::create_dir_all(file.parent().unwrap_or(Path::new(".")))?;
+    let tmp = file.with_extension("part");
+    std::fs::write(&tmp, body)?;
+    std::fs::write(file.with_extension("type"), kind)?;
+    std::fs::rename(&tmp, file)
+}
+
+/// Removes the least recently used images until what is kept fits in `limit` bytes; 0 empties it.
+/// Returns how many went.
+pub fn trim(dir: &Path, limit: u64) -> usize {
+    let Ok(entries) = std::fs::read_dir(dir) else { return 0 };
+    let mut kept: Vec<(std::time::SystemTime, u64, PathBuf)> = entries
+        .flatten()
+        .map(|e| e.path())
+        .filter(|p| p.extension().is_none())
+        .filter_map(|p| {
+            let m = p.metadata().ok()?;
+            Some((m.modified().ok()?, m.len(), p))
+        })
+        .collect();
+    let mut total: u64 = kept.iter().map(|(_, n, _)| n).sum();
+    kept.sort();
+    let mut gone = 0;
+    for (_, n, p) in kept {
+        if total <= limit {
+            break;
+        }
+        let _ = std::fs::remove_file(p.with_extension("type"));
+        if std::fs::remove_file(&p).is_ok() {
+            total -= n;
+            gone += 1;
+        }
+    }
+    gone
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn the_least_recently_used_go_first_until_it_fits() {
+        let d = std::env::temp_dir().join(format!("ipx-art-test-{}", std::process::id()));
+        let _ = std::fs::remove_dir_all(&d);
+        for (name, age) in [("old", 300), ("mid", 200), ("new", 100)] {
+            let f = d.join(name);
+            write(&f, "image/png", &[0; 1000]).unwrap();
+            let t = std::time::SystemTime::now() - std::time::Duration::from_secs(age);
+            std::fs::File::options().write(true).open(&f).unwrap().set_modified(t).unwrap();
+        }
+        // Shown just now: no longer the oldest.
+        assert_eq!(read(&d.join("old")).unwrap().0, "image/png");
+        assert_eq!(trim(&d, 2000), 1);
+        assert!(!d.join("mid").exists() && !d.join("mid.type").exists());
+        assert!(d.join("old").exists() && d.join("new").exists());
+        assert_eq!(trim(&d, 0), 2);
+        let _ = std::fs::remove_dir_all(&d);
+    }
+}

src/auth.rs

@@ -30,6 +30,35 @@ pub fn verify_password(password: &str, stored: &str) -> bool {
         .is_ok()
 }
 
+/// How many password checks run at once: each is tens of milliseconds of CPU, and forty wrong
+/// passwords at a time, run on the async workers, made every other request wait 4s (#137).
+/// Half the cores, so a flood of sign-ins waits on itself and the rest of the server has the rest.
+static CHECKS: std::sync::LazyLock<tokio::sync::Semaphore> = std::sync::LazyLock::new(|| {
+    tokio::sync::Semaphore::new(std::thread::available_parallelism().map_or(1, |n| (n.get() / 2).max(1)))
+});
+
+/// What a name that is not an account is checked against, so that refusing it takes as long as
+/// refusing a wrong password does. Refused without a check, it came back 31ms sooner, and the
+/// time told anyone which names are accounts here (#138).
+static DECOY: std::sync::LazyLock<String> =
+    std::sync::LazyLock::new(|| hash_password("no account here has this password").expect("hashing a fixed password"));
+
+/// A sign-in's password against the account's hash, or against `DECOY` when there is no account
+/// or it has no password: false either way, in the same time. Off the async workers, and a few at
+/// a time; see `CHECKS`.
+pub async fn check_password(password: String, stored: Option<String>) -> bool {
+    let Ok(_turn) = CHECKS.acquire().await else { return false };
+    tokio::task::spawn_blocking(move || match stored {
+        Some(h) => verify_password(&password, &h),
+        None => {
+            verify_password(&password, &DECOY);
+            false
+        }
+    })
+    .await
+    .unwrap_or(false)
+}
+
 /// A session id: 256 bits of urandom, hex. Long enough that guessing is not a strategy.
 pub fn new_session_token() -> String {
     let mut bytes = [0u8; 32];
@@ -40,6 +69,19 @@ pub fn new_session_token() -> String {
     bytes.iter().map(|b| format!("{b:02x}")).collect()
 }
 
+/// A new API token: `ipx_` and a session's 256 random bits, the prefix so one found in a log
+/// or a file says what it opens (#123).
+pub fn new_api_token() -> String {
+    format!("ipx_{}", new_session_token())
+}
+
+/// What is kept of an API token. A fast hash is enough: the token is 256 random bits, not a
+/// password, so there is nothing to guess from a stolen hash.
+pub fn api_token_hash(token: &str) -> String {
+    use sha2::Digest;
+    sha2::Sha256::digest(token.as_bytes()).iter().map(|b| format!("{b:02x}")).collect()
+}
+
 fn getrandom(buf: &mut [u8]) -> std::io::Result<()> {
     use std::io::Read;
     std::fs::File::open("/dev/urandom")?.read_exact(buf)
@@ -63,6 +105,16 @@ pub fn name_from_header(raw: &str) -> Option<String> {
 mod tests {
     use super::*;
 
+    #[tokio::test]
+    async fn a_sign_in_without_an_account_is_checked_all_the_same() {
+        let h = hash_password("correct horse battery").unwrap();
+        assert!(check_password("correct horse battery".into(), Some(h.clone())).await);
+        assert!(!check_password("wrong".into(), Some(h)).await);
+        assert!(!check_password("correct horse battery".into(), None).await);
+        // A decoy that does not parse is refused before any hashing, and the time says so (#138).
+        assert!(PasswordHash::new(&DECOY).is_ok());
+    }
+
     #[test]
     fn a_password_verifies_only_against_itself() {
         let h = hash_password("correct horse battery").unwrap();
@@ -74,6 +126,21 @@ mod tests {
         assert!(hash_password("short").is_err());
     }
 
+    #[test]
+    fn api_tokens_are_prefixed_and_hash_to_hex() {
+        let t = new_api_token();
+        assert!(t.starts_with("ipx_") && t.len() == 68);
+        let h = api_token_hash(&t);
+        assert_eq!(h.len(), 64);
+        assert_eq!(h, api_token_hash(&t), "the same token, the same hash");
+        assert_ne!(h, api_token_hash(&new_api_token()));
+        assert_eq!(
+            api_token_hash("abc"),
+            "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
+            "SHA-256"
+        );
+    }
+
     #[test]
     fn session_tokens_are_long_and_distinct() {
         let a = new_session_token();

src/config.rs

@@ -39,6 +39,9 @@ pub struct General {
     /// image in an <enclosure>, so taking everything filled the disk with artwork and
     /// counted it as episodes. Empty means take anything.
     pub media_types: Vec<String>,
+    /// How much artwork iPX keeps on disk, in MB, so the page loads it from iPX rather than from
+    /// every publisher; the least recently shown goes first. 0 keeps none.
+    pub art_cache_mb: u64,
 }
 
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
@@ -163,6 +166,10 @@ pub struct Feed {
     /// Preferred over `password`: name of an env var holding the password.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub password_env: Option<String>,
+    /// Put in the Directory by an admin (`ipx add --list`): listed with no subscribers, and kept
+    /// in the catalogue when the last one leaves, where anyone else's feed is dropped.
+    #[serde(default, skip_serializing_if = "std::ops::Not::not")]
+    pub listed: bool,
 }
 
 fn yes() -> bool {
@@ -180,6 +187,7 @@ impl Default for General {
             max_age_days: 0,
             max_new_per_check: 3,
             media_types: vec!["audio".into(), "video".into()],
+            art_cache_mb: 500,
         }
     }
 }
@@ -290,10 +298,18 @@ pub struct Stored {
     pub max_age_days: u64,
     pub max_new_per_check: usize,
     pub media_types: Vec<String>,
+    /// Settings saved before it existed have none: they get the default.
+    #[serde(default = "art_cache_mb")]
+    pub art_cache_mb: u64,
+}
+
+fn art_cache_mb() -> u64 {
+    General::default().art_cache_mb
 }
 
 /// `[general]` keys that live in the database once it holds the configuration.
-const STORED_KEYS: [&str; 5] = ["schedule", "max_total_gb", "max_age_days", "max_new_per_check", "media_types"];
+const STORED_KEYS: [&str; 6] =
+    ["schedule", "max_total_gb", "max_age_days", "max_new_per_check", "media_types", "art_cache_mb"];
 
 impl Stored {
     pub fn of(cfg: &Config) -> Self {
@@ -304,6 +320,7 @@ impl Stored {
             max_age_days: g.max_age_days,
             max_new_per_check: g.max_new_per_check,
             media_types: g.media_types.clone(),
+            art_cache_mb: g.art_cache_mb,
         }
     }
 
@@ -314,6 +331,7 @@ impl Stored {
         g.max_age_days = self.max_age_days;
         g.max_new_per_check = self.max_new_per_check;
         g.media_types = self.media_types;
+        g.art_cache_mb = self.art_cache_mb;
     }
 }
 
@@ -571,6 +589,12 @@ mod tests {
         assert!(wanted_media(Some("image/jpeg"), &["image/jpeg".to_string()]));
     }
 
+    #[test]
+    fn settings_saved_before_the_art_cache_keep_the_default() {
+        let old = r#"{"schedule":"every 1h","max_total_gb":0.0,"max_age_days":0,"max_new_per_check":3,"media_types":["audio"]}"#;
+        assert_eq!(serde_json::from_str::<Stored>(old).unwrap().art_cache_mb, 500);
+    }
+
     #[test]
     fn slugs_are_readable_and_unique() {
         assert_eq!(slug("Accidental Tech Podcast"), "accidental-tech-podcast");
@@ -584,7 +608,7 @@ mod tests {
         taken.insert("the-daily".to_string(), Feed {
             url: "u".into(), folder: None, group: None, media_types: None, schedule: None, keywords: vec![], allow_explicit: false,
             auto_download: true, max_new_per_check: None, username: None,
-            password: None, password_env: None, category: None,
+            password: None, password_env: None, category: None, listed: false,
         });
         assert_eq!(unique_slug("The Daily", &taken), "the-daily-2");
     }
@@ -605,6 +629,7 @@ mod tests {
             password: Some("literal".into()),
             password_env: None,
             category: None,
+            listed: false,
         };
         assert_eq!(f.password().as_deref(), Some("literal"));
 

src/db.rs

@@ -2,7 +2,7 @@
 //! per-feed .ipxd plists, history.dat and qmcache.dat.
 
 use anyhow::{Context, Result};
-use crate::entity::{blocklists, catalogue, enclosures, entries, feeds, hidden, sessions, settings, subscriptions, users};
+use crate::entity::{api_tokens, blocklists, catalogue, enclosures, entries, feeds, hidden, sessions, settings, subscriptions, users};
 use sea_orm::sea_query::{Expr, Func};
 use sea_orm::{
     ActiveModelTrait, ColumnTrait, ConnectionTrait, EntityTrait, PaginatorTrait, QueryFilter, QueryOrder, Set,
@@ -59,6 +59,7 @@ async fn create_missing(orm: &sea_orm::DatabaseConnection) -> Result<()> {
         schema.create_table_from_entity(settings::Entity),
         schema.create_table_from_entity(blocklists::Entity),
         schema.create_table_from_entity(hidden::Entity),
+        schema.create_table_from_entity(api_tokens::Entity),
     ] {
         orm.execute(table.if_not_exists()).await.context("creating the schema")?;
     }
@@ -68,6 +69,21 @@ async fn create_missing(orm: &sea_orm::DatabaseConnection) -> Result<()> {
     ] {
         orm.execute_unprepared(sql).await.with_context(|| sql.to_owned())?;
     }
+    // A column added to a table that already exists, which CREATE TABLE IF NOT EXISTS leaves
+    // as it was. Looked for first, and added only when it is missing: this runs on every open,
+    // the healthcheck's `ipx status` included, and an ALTER takes a lock that waits behind a
+    // busy daemon, which is what made the healthcheck time out once before.
+    if orm.execute_unprepared("SELECT description FROM feeds LIMIT 0").await.is_err() {
+        // A feed is read whole only when it has changed since the last read, so each one's
+        // description would wait for its next item. Forgetting how each was last seen makes
+        // its next read, when it is due anyway, a whole one.
+        for sql in [
+            "ALTER TABLE feeds ADD COLUMN description TEXT",
+            "UPDATE feeds SET etag = NULL, last_modified = NULL",
+        ] {
+            orm.execute_unprepared(sql).await.with_context(|| sql.to_owned())?;
+        }
+    }
     Ok(())
 }
 
@@ -98,11 +114,32 @@ fn is_postgres(location: &str) -> bool {
     location.starts_with("postgres://") || location.starts_with("postgresql://")
 }
 
-/// sqlx's defaults for SQLite are what ipx wants: foreign keys on, and a five-second wait for a
+/// sqlx's defaults for SQLite are what ipx wants, foreign keys on and a five-second wait for a
-/// lock, which is what rusqlite was set to.
+/// lock, which is what rusqlite was set to, but for one.
 async fn connect(location: &str) -> Result<sea_orm::DatabaseConnection> {
     let mut opts = sea_orm::ConnectOptions::new(url_for(location));
     opts.sqlx_logging(false);
+    // NORMAL, SQLite's own advice for WAL: synced at checkpoints, not after every commit. FULL,
+    // the default, waited for the disk after nearly every statement, and a feed's items are
+    // written a row at a time: 1.3s a feed on Tower's /tmp, where one synchronous write takes
+    // 52ms, and 39s to subscribe an admin to 1,500 feeds (#135). A crash of ipx loses nothing;
+    // only a power cut can lose the last transactions, and it never corrupts the file.
+    opts.map_sqlx_sqlite_opts(|o| o.synchronous(sea_orm::sqlx::sqlite::SqliteSynchronous::Normal));
+    // The same on Postgres, for ipx's own sessions: a commit waited for the WAL to reach the
+    // disk, 12.8ms on the databases project's server, and at about fifty commits a feed a scan
+    // took most of a second a feed (#140). A crash of the Postgres server can lose the last
+    // commits, about the last 0.6s, and never corrupts anything. Added to whatever options the
+    // URL sets, so it holds for the tests' search_path too.
+    opts.map_sqlx_postgres_opts(|o| o.options([("synchronous_commit", "off")]));
+    // Several connections, where SeaORM gives SQLite one unless told: every request, every scan
+    // write and every read went through it in turn, and twenty-five people browsing waited in
+    // line for it at 50 requests a second (#136). In WAL readers run beside the one writer. Each
+    // transaction here writes first, so it takes the write lock or waits out the busy timeout for
+    // it; SQLite refuses at once only a transaction that read a snapshot a write has since moved
+    // past. Postgres keeps sqlx's ten.
+    if !is_postgres(location) {
+        opts.max_connections(8);
+    }
     sea_orm::Database::connect(opts)
         .await
         .with_context(|| format!("opening {}", redact(location)))
@@ -291,6 +328,8 @@ impl Db {
         }
         let path = std::env::temp_dir().join(format!("ipx-test-{pid}-{n}.db"));
         let orm = connect(&path.display().to_string()).await?;
+        // As Db::open has it, so the tests' connections share the file as the daemon's do.
+        orm.execute_unprepared("PRAGMA journal_mode = WAL").await?;
         create_missing(&orm).await?;
         Ok(Self { orm, tmp: Some(path) })
     }
@@ -313,6 +352,11 @@ impl Db {
         Ok(())
     }
 
+    /// What a feed says it is, for its page in the Directory.
+    pub async fn feed_description(&self, feed_id: &str) -> Result<Option<String>> {
+        Ok(feeds::Entity::find_by_id(feed_id.to_owned()).one(&self.orm).await?.and_then(|f| f.description))
+    }
+
     #[tracing::instrument(skip_all)]
     pub async fn feed_summary(&self, feed_id: &str) -> Result<FeedSummary> {
         let mut sum = feeds::Entity::find_by_id(feed_id.to_owned())
@@ -438,19 +482,27 @@ pub struct HttpState {
     pub error_since: Option<i64>,
 }
 
+impl From<feeds::Model> for HttpState {
+    fn from(f: feeds::Model) -> Self {
+        HttpState {
+            etag: f.etag,
+            last_modified: f.last_modified,
+            last_checked: f.last_checked,
+            ttl_mins: f.ttl_mins.map(|t| t.max(0) as u64),
+            error_since: f.error_since,
+        }
+    }
+}
+
 impl Db {
     pub async fn http_state(&self, feed_id: &str) -> Result<HttpState> {
-        Ok(feeds::Entity::find_by_id(feed_id.to_owned())
+        Ok(feeds::Entity::find_by_id(feed_id.to_owned()).one(&self.orm).await?.map(HttpState::from).unwrap_or_default())
-            .one(&self.orm)
+    }
-            .await?
+
-            .map(|f| HttpState {
+    /// Every feed's, in one query: a scan, and the daemon working out when the next is due, asked
-                etag: f.etag,
+    /// feed by feed, 180 queries a minute (#114).
-                last_modified: f.last_modified,
+    pub async fn http_states(&self) -> Result<std::collections::HashMap<String, HttpState>> {
-                last_checked: f.last_checked,
+        Ok(feeds::Entity::find().all(&self.orm).await?.into_iter().map(|f| (f.id.clone(), HttpState::from(f))).collect())
-                ttl_mins: f.ttl_mins.map(|t| t.max(0) as u64),
-                error_since: f.error_since,
-            })
-            .unwrap_or_default())
     }
 
     /// Upsert after a successful poll. Clears any previous error.
@@ -466,13 +518,14 @@ impl Db {
         ttl_mins: Option<u64>,
         image: Option<&str>,
         category: Option<&str>,
+        description: Option<&str>,
     ) -> Result<()> {
         // category is taken as it comes, unlike title and image: a show that leaves a category
         // should leave the Directory's chip too.
         let o = |v: Option<&str>| sea_orm::Value::from(v.map(str::to_owned));
         self.exec(
-            "INSERT INTO feeds (id, url, title, etag, last_modified, last_checked, ttl_mins, last_error, image, category)
+            "INSERT INTO feeds (id, url, title, etag, last_modified, last_checked, ttl_mins, last_error, image, category, description)
-             VALUES ($1, $2, $3, $4, $5, $6, $7, NULL, $8, $9)
+             VALUES ($1, $2, $3, $4, $5, $6, $7, NULL, $8, $9, $10)
              ON CONFLICT (id) DO UPDATE SET
                 url = excluded.url,
                 title = coalesce(excluded.title, feeds.title),
@@ -482,6 +535,7 @@ impl Db {
                 ttl_mins = excluded.ttl_mins,
                 image = coalesce(excluded.image, feeds.image),
                 category = excluded.category,
+                description = coalesce(excluded.description, feeds.description),
                 last_error = NULL,
                 error_since = NULL",
             vec![
@@ -494,6 +548,7 @@ impl Db {
                 ttl_mins.map(|t| t as i64).into(),
                 o(image),
                 o(category),
+                o(description),
             ],
         )
         .await?;
@@ -652,6 +707,35 @@ impl Db {
     /// through every show's history: 4420 files queued across 12 shows on the default of 3
     /// (#97). Unlimited, 0 in the settings, is the whole back catalogue, for an archive. An item
     /// whose files were all skipped by a filter does not hold one of the places.
+    /// Puts a feed's waiting files in the queue or out of it: 'pending' for those among its
+    /// `limit` newest items, which the scan downloads on its own, and 'held' for the rest, listed
+    /// to download by hand. With all of them 'pending', the queue counted 4420 back-catalogue
+    /// files no scan would ever take (#97, #113). 0 holds them all, for a feed nobody downloads
+    /// automatically. A held file still holds its item's place among the newest.
+    #[tracing::instrument(skip_all)]
+    pub async fn hold_back(&self, feed_id: &str, limit: usize) -> Result<()> {
+        let newest = "SELECT n.guid FROM entries n
+                       WHERE n.feed_id = $1
+                         AND EXISTS (SELECT 1 FROM enclosures y
+                                      WHERE y.feed_id = n.feed_id AND y.guid = n.guid
+                                        AND y.state <> 'skipped')
+                       ORDER BY coalesce(n.published, n.first_seen) DESC, n.guid DESC
+                       LIMIT $2";
+        let limit: sea_orm::Value = (limit.min(i64::MAX as usize) as i64).into();
+        self.exec(
+            &format!("UPDATE enclosures SET state = 'held' WHERE feed_id = $1 AND state = 'pending' AND guid NOT IN ({newest})"),
+            vec![feed_id.into(), limit.clone()],
+        )
+        .await?;
+        // And back, when a limit is raised or a feed downloads again.
+        self.exec(
+            &format!("UPDATE enclosures SET state = 'pending' WHERE feed_id = $1 AND state = 'held' AND guid IN ({newest})"),
+            vec![feed_id.into(), limit],
+        )
+        .await?;
+        Ok(())
+    }
+
     #[tracing::instrument(skip_all)]
     pub async fn pending(&self, feed_id: &str, limit: usize) -> Result<Vec<Pending>> {
         self.rows(
@@ -778,13 +862,76 @@ impl Db {
             .map(|r| Ok(r.try_get("", "id")?))
             .collect::<Result<_>>()?;
         for id in &gone {
-            for table in ["enclosures", "entry_state", "hidden", "blocklists", "entries"] {
+            self.forget_rows(id).await?;
-                self.exec(&format!("DELETE FROM {table} WHERE feed_id = $1"), vec![id.clone().into()]).await?;
-            }
         }
         Ok(gone)
     }
 
+    /// Everything stored about a feed but its row: items, file rows, read state, block list.
+    async fn forget_rows(&self, id: &str) -> Result<()> {
+        for table in ["enclosures", "entry_state", "hidden", "blocklists", "entries"] {
+            self.exec(&format!("DELETE FROM {table} WHERE feed_id = $1"), vec![id.into()]).await?;
+        }
+        Ok(())
+    }
+
+    /// A feed's stored artwork still on http, items' and its own: hosts a feed no longer names
+    /// keep their old items' artwork on http otherwise (IGN's 2009 items, #110).
+    pub async fn http_images(&self, feed_id: &str) -> Result<Vec<String>> {
+        self.rows(
+            "SELECT DISTINCT image FROM entries WHERE feed_id = $1 AND image LIKE 'http://%'
+             UNION SELECT image FROM feeds WHERE id = $1 AND image LIKE 'http://%'",
+            vec![feed_id.into()],
+        )
+        .await?
+        .iter()
+        .map(|r| Ok(r.try_get("", "image")?))
+        .collect()
+    }
+
+    /// Rewrites a feed's stored artwork on `host` from http to https, once the host is known to
+    /// serve it there (`feed::prefer_https`), for the items a scan does not write again.
+    pub async fn secure_images(&self, feed_id: &str, host: &str) -> Result<()> {
+        let like = format!("http://{host}/%");
+        for table in ["entries", "feeds"] {
+            let col = if table == "feeds" { "id" } else { "feed_id" };
+            self.exec(
+                &format!("UPDATE {table} SET image = 'https://' || substr(image, 8) WHERE {col} = $1 AND image LIKE $2"),
+                vec![feed_id.into(), like.clone().into()],
+            )
+            .await?;
+        }
+        Ok(())
+    }
+
+    /// Forgets a feed and everything stored about it.
+    pub async fn forget_feed(&self, id: &str) -> Result<()> {
+        self.forget_rows(id).await?;
+        self.exec("DELETE FROM feeds WHERE id = $1", vec![id.into()]).await?;
+        Ok(())
+    }
+
+    /// Feeds nobody subscribes to that are dead, failing since before `dead_before`, or quiet,
+    /// their newest item from before `quiet_before`: what keeps the Directory to feeds worth
+    /// taking. One with a file on disk stays, and so does one an OPML subscription lists, which
+    /// comes back with the OPML anyway. A feed with no items yet is not quiet, only new.
+    pub async fn stale_unsubscribed(&self, dead_before: i64, quiet_before: i64) -> Result<Vec<String>> {
+        self.rows(
+            "SELECT f.id FROM feeds f
+              WHERE NOT EXISTS (SELECT 1 FROM subscriptions s WHERE s.feed_id = f.id)
+                AND NOT EXISTS (SELECT 1 FROM enclosures x WHERE x.feed_id = f.id AND x.path IS NOT NULL)
+                AND f.group_id IS NULL AND NOT coalesce(f.managed, false)
+                AND ((f.error_since IS NOT NULL AND f.error_since < $1)
+                     OR (SELECT max(coalesce(e.published, e.first_seen)) FROM entries e
+                          WHERE e.feed_id = f.id) < $2)",
+            vec![dead_before.into(), quiet_before.into()],
+        )
+        .await?
+        .iter()
+        .map(|r| Ok(r.try_get("", "id")?))
+        .collect()
+    }
+
     pub async fn prune_entries(&self, older_than: i64) -> Result<usize> {
         let n = self
             .exec(
@@ -1585,6 +1732,7 @@ impl Db {
     /// The foreign key would take them anyway; this does not rely on it being switched on.
     pub async fn delete_user(&self, id: i64) -> Result<()> {
         sessions::Entity::delete_many().filter(sessions::Column::UserId.eq(id)).exec(&self.orm).await?;
+        api_tokens::Entity::delete_many().filter(api_tokens::Column::UserId.eq(id)).exec(&self.orm).await?;
         users::Entity::delete_by_id(id).exec(&self.orm).await?;
         Ok(())
     }
@@ -1637,6 +1785,56 @@ impl Db {
         Ok(found.map(User::from))
     }
 
+    /// Keeps a new API token's hash under its owner; the token itself is shown once and not kept.
+    pub async fn create_api_token(&self, user_id: i64, name: &str, hash: &str) -> Result<i64> {
+        let made = api_tokens::ActiveModel {
+            user_id: Set(user_id),
+            name: Set(name.to_owned()),
+            hash: Set(hash.to_owned()),
+            created: Set(now()),
+            last_used: Set(None),
+            ..Default::default()
+        }
+        .insert(&self.orm)
+        .await?;
+        Ok(made.id)
+    }
+
+    /// One person's tokens, newest first: names and dates, as the tokens themselves are not kept.
+    pub async fn api_tokens(&self, user_id: i64) -> Result<Vec<api_tokens::Model>> {
+        Ok(api_tokens::Entity::find()
+            .filter(api_tokens::Column::UserId.eq(user_id))
+            .order_by_desc(api_tokens::Column::Id)
+            .all(&self.orm)
+            .await?)
+    }
+
+    /// Revokes one of this person's tokens; someone else's id does nothing. Whether it was theirs.
+    pub async fn delete_api_token(&self, user_id: i64, id: i64) -> Result<bool> {
+        let gone = api_tokens::Entity::delete_many()
+            .filter(api_tokens::Column::Id.eq(id))
+            .filter(api_tokens::Column::UserId.eq(user_id))
+            .exec(&self.orm)
+            .await?;
+        Ok(gone.rows_affected > 0)
+    }
+
+    /// The person an API token acts as, noting when it was used, as a session notes `seen`.
+    pub async fn api_token_user(&self, hash: &str) -> Result<Option<User>> {
+        let found = api_tokens::Entity::find()
+            .filter(api_tokens::Column::Hash.eq(hash))
+            .find_also_related(users::Entity)
+            .one(&self.orm)
+            .await?;
+        let Some((token, Some(user))) = found else { return Ok(None) };
+        api_tokens::Entity::update_many()
+            .col_expr(api_tokens::Column::LastUsed, Expr::val(now()).into())
+            .filter(api_tokens::Column::Id.eq(token.id))
+            .exec(&self.orm)
+            .await?;
+        Ok(Some(User::from(user)))
+    }
+
     pub async fn delete_session(&self, token: &str) -> Result<()> {
         sessions::Entity::delete_by_id(token.to_owned()).exec(&self.orm).await?;
         Ok(())
@@ -1861,6 +2059,64 @@ impl Db {
         ))
     }
 
+    /// Of these items, the stored ones the feed now tells differently: a title, text, artwork,
+    /// length or number that is not what is stored, a field the feed leaves out counting as
+    /// unchanged, as record_entry writes them. A scan inserted only items it had not stored
+    /// (#96), and with that alone a publisher's correction never arrived: Ain't It Cool News's
+    /// items kept a broken copy, each with the text of the one before it, after the feed was
+    /// fixed (#141). Only what the feed lists is read back, so a scan reads as much as it was
+    /// sent, not the archive.
+    #[tracing::instrument(skip_all)]
+    pub async fn changed_items(
+        &self,
+        feed_id: &str,
+        entries: &[crate::feed::Entry],
+    ) -> Result<std::collections::HashSet<String>> {
+        if entries.is_empty() {
+            return Ok(Default::default());
+        }
+        let mut a = Args::default();
+        let feed = a.p(feed_id);
+        let guids: Vec<String> = entries.iter().map(|e| a.p(e.guid.clone())).collect();
+        let sql = format!(
+            "SELECT guid, title, description, image, duration, episode, season FROM entries
+              WHERE feed_id = {feed} AND guid IN ({})",
+            guids.join(", ")
+        );
+        type Stored = (Option<String>, Option<String>, Option<String>, Option<i64>, Option<i64>, Option<i64>);
+        let mut stored = std::collections::HashMap::<String, Stored>::new();
+        for r in self.rows(&sql, a.0).await? {
+            stored.insert(
+                r.try_get("", "guid")?,
+                (
+                    r.try_get("", "title")?,
+                    r.try_get("", "description")?,
+                    r.try_get("", "image")?,
+                    r.try_get("", "duration")?,
+                    r.try_get("", "episode")?,
+                    r.try_get("", "season")?,
+                ),
+            );
+        }
+        fn differs<T: PartialEq>(new: &Option<T>, old: &Option<T>) -> bool {
+            new.is_some() && new != old
+        }
+        Ok(entries
+            .iter()
+            .filter(|e| {
+                stored.get(&e.guid).is_some_and(|s| {
+                    differs(&e.title, &s.0)
+                        || differs(&e.description, &s.1)
+                        || differs(&e.image, &s.2)
+                        || differs(&e.duration, &s.3)
+                        || differs(&e.episode, &s.4)
+                        || differs(&e.season, &s.5)
+                })
+            })
+            .map(|e| e.guid.clone())
+            .collect())
+    }
+
     #[tracing::instrument(skip_all)]
     pub async fn skipped_by_filter(&self, feed_id: &str) -> Result<std::collections::HashMap<String, String>> {
         self.rows(
@@ -1924,6 +2180,63 @@ pub fn now() -> i64 {
 mod tests {
     use super::*;
 
+    #[tokio::test]
+    async fn a_read_does_not_wait_for_a_write_in_progress() {
+        use sea_orm::TransactionTrait;
+        let db = Db::memory().await.unwrap();
+        // A write held open, as a scan's or an import's is while it runs.
+        let tx = db.orm.begin().await.unwrap();
+        tx.execute_unprepared("INSERT INTO feeds (id, url) VALUES ('held', 'http://x/held')").await.unwrap();
+        // With one connection, which SQLite had (#136), this waited for the transaction to end.
+        let read = tokio::time::timeout(std::time::Duration::from_secs(2), db.feed_summary("other")).await;
+        assert!(read.is_ok(), "a read waited behind a write");
+        tx.commit().await.unwrap();
+        // And writers at once all get their turn, not a refusal.
+        let writes = (0..16).map(|i| {
+            let db = &db;
+            async move { db.record_feed(&format!("f{i}"), "http://x/f", Some("F"), None, None, None, None, None, None).await }
+        });
+        for r in futures_util::future::join_all(writes).await {
+            r.unwrap();
+        }
+    }
+
+    #[tokio::test]
+    async fn a_commit_does_not_wait_for_the_disk() {
+        let db = Db::memory().await.unwrap();
+        let backend = db.orm.get_database_backend();
+        if backend == sea_orm::DbBackend::Sqlite {
+            let row = db.orm.query_one_raw(Statement::from_string(backend, "PRAGMA synchronous")).await.unwrap();
+            assert_eq!(row.unwrap().try_get_by_index::<i32>(0).unwrap(), 1, "NORMAL, on every connection in the pool (#135)");
+        } else {
+            let row = db.orm.query_one_raw(Statement::from_string(backend, "SHOW synchronous_commit")).await.unwrap();
+            assert_eq!(row.unwrap().try_get_by_index::<String>(0).unwrap(), "off", "with the test's own search_path as well (#140)");
+        }
+    }
+
+    #[tokio::test]
+    async fn a_database_from_before_descriptions_gains_the_column_and_reads_each_feed_whole() {
+        let db = Db::memory().await.unwrap();
+        let record = |etag: &'static str, about: Option<&'static str>| {
+            let db = &db;
+            async move { db.record_feed("f", "https://x/f", Some("F"), Some(etag), Some("then"), None, None, None, about).await.unwrap() }
+        };
+        record("\"a\"", Some("About F")).await;
+        assert_eq!(db.feed_description("f").await.unwrap().as_deref(), Some("About F"));
+        record("\"a\"", None).await;
+        assert_eq!(db.feed_description("f").await.unwrap().as_deref(), Some("About F"), "kept when a read finds none");
+        // A database from before #130.
+        db.exec_for_test("ALTER TABLE feeds DROP COLUMN description").await.unwrap();
+        create_missing(&db.orm).await.unwrap();
+        assert_eq!(db.feed_description("f").await.unwrap(), None);
+        let st = db.http_state("f").await.unwrap();
+        assert!(st.etag.is_none() && st.last_modified.is_none(), "every feed is read whole next time, for its description");
+        // With the column there, opening the database again writes nothing.
+        record("\"b\"", Some("About F")).await;
+        create_missing(&db.orm).await.unwrap();
+        assert_eq!(db.http_state("f").await.unwrap().etag.as_deref(), Some("\"b\""));
+    }
+
     #[tokio::test]
     async fn every_sort_column_runs_and_orders_both_ways() {
         let db = Db::memory().await.unwrap();
@@ -1998,6 +2311,52 @@ mod tests {
         assert_eq!((list["f"].unread, list["g"].unread), (1, 1)); // a read, c hidden; sam's read is sam's
     }
 
+    #[tokio::test]
+    async fn a_hosts_artwork_moves_to_https_for_one_feed() {
+        let db = Db::memory().await.unwrap();
+        db.exec_for_test(
+            "INSERT INTO feeds (id, url, image) VALUES ('f','u','http://a.example/logo.png'),('g','v','http://a.example/g.png');
+             INSERT INTO entries (feed_id, guid, first_seen, image) VALUES
+                ('f','1',0,'http://a.example/1.jpg'),('f','2',0,'http://b.example/2.jpg'),('g','3',0,'http://a.example/3.jpg');",
+        ).await
+        .unwrap();
+        let mut old = db.http_images("f").await.unwrap();
+        old.sort();
+        assert_eq!(old, ["http://a.example/1.jpg", "http://a.example/logo.png", "http://b.example/2.jpg"]);
+        db.secure_images("f", "a.example").await.unwrap();
+        assert_eq!(db.feed_summary("f").await.unwrap().image.as_deref(), Some("https://a.example/logo.png"));
+        assert!(db.names_image("https://a.example/1.jpg").await.unwrap());
+        assert!(db.names_image("http://b.example/2.jpg").await.unwrap());  // another host
+        assert!(db.names_image("http://a.example/3.jpg").await.unwrap());  // another feed
+        assert_eq!(db.feed_summary("g").await.unwrap().image.as_deref(), Some("http://a.example/g.png"));
+    }
+
+    #[tokio::test]
+    async fn stale_feeds_nobody_subscribes_to_are_found_and_the_rest_left() {
+        let db = Db::memory().await.unwrap();
+        db.exec_for_test(
+            "INSERT INTO users (id, name, is_admin) VALUES (1,'ray',true);
+             INSERT INTO feeds (id, url, error_since, managed, group_id) VALUES
+                ('dead','a',100,false,null),('quiet','b',null,false,null),('news','c',null,false,null),
+                ('new','d',null,false,null),('wanted','e',100,false,null),('kept','f',100,false,null),
+                ('child','g',100,true,'opml'),('lately','h',900,false,null);
+             INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'wanted');
+             INSERT INTO entries (feed_id, guid, first_seen, published) VALUES
+                ('quiet','q',0,100),('news','n',0,950),('dead','d',0,950),('kept','k',0,100);
+             INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES (1,'kept','k','u','/tmp/k','done');",
+        ).await
+        .unwrap();
+        // Dead before 500, or nothing newer than 500: dead fails since 100, quiet's newest is 100.
+        // news published lately, new has no items yet, wanted has a subscriber, kept a file,
+        // child comes from an OPML, lately began failing after the cutoff.
+        let mut stale = db.stale_unsubscribed(500, 500).await.unwrap();
+        stale.sort();
+        assert_eq!(stale, ["dead", "quiet"]);
+        db.forget_feed("dead").await.unwrap();
+        assert_eq!(db.feed_summary("dead").await.unwrap().entries, 0);
+        assert!(!db.feed_urls().await.unwrap().iter().any(|(id, _)| id == "dead"));
+    }
+
     #[tokio::test]
     async fn a_failing_feed_nobody_subscribes_to_is_forgotten() {
         let db = Db::memory().await.unwrap();
@@ -2021,6 +2380,33 @@ mod tests {
         assert!(db.prune_abandoned_failures().await.unwrap().is_empty());
     }
 
+    #[tokio::test]
+    async fn only_the_newest_files_stay_queued_and_the_rest_are_held() {
+        let db = Db::memory().await.unwrap();
+        db.exec_for_test(
+            "INSERT INTO entries (feed_id, guid, first_seen, published) VALUES
+                ('f','e1',0,100),('f','e2',0,200),('f','e3',0,300),('f','e4',0,400);
+             INSERT INTO enclosures (id, feed_id, guid, url, state, path) VALUES
+                (1,'f','e1','u1','pending',null),(2,'f','e2','u2','pending',null),
+                (3,'f','e3','u3','done','/tmp/3'),(4,'f','e4','u4','pending',null);",
+        ).await
+        .unwrap();
+        let states = || async {
+            let mut v: Vec<(i64, String)> = db.rows("SELECT id, state FROM enclosures ORDER BY id", vec![]).await.unwrap()
+                .iter().map(|r| (r.try_get("", "id").unwrap(), r.try_get("", "state").unwrap())).collect();
+            v.sort();
+            v.into_iter().map(|(_, s)| s).collect::<Vec<_>>()
+        };
+        db.hold_back("f", 2).await.unwrap();              // newest two: e4, e3
+        assert_eq!(states().await, ["held", "held", "done", "pending"]);
+        db.hold_back("f", 3).await.unwrap();              // a limit raised brings e2 back
+        assert_eq!(states().await, ["held", "pending", "done", "pending"]);
+        db.hold_back("f", 0).await.unwrap();              // a feed nobody downloads: all held
+        assert_eq!(states().await, ["held", "held", "done", "held"]);
+        db.hold_back("f", usize::MAX).await.unwrap();     // unlimited, an archive: all queued
+        assert_eq!(states().await, ["pending", "pending", "done", "pending"]);
+    }
+
     #[tokio::test]
     async fn a_limit_takes_the_newest_episodes_not_the_back_catalogue() {
         let db = Db::memory().await.unwrap();
@@ -2068,6 +2454,31 @@ mod tests {
         assert_eq!(files, ["u1"].map(String::from).into()); // u2 is g's: left to the insert to find
     }
 
+    #[tokio::test]
+    async fn a_scan_finds_the_stored_items_the_feed_now_tells_differently() {
+        let db = Db::memory().await.unwrap();
+        let item = |guid: &str, title: &str, text: Option<&str>| crate::feed::Entry {
+            guid: guid.into(),
+            title: Some(title.into()),
+            description: text.map(str::to_owned),
+            ..Default::default()
+        };
+        db.record_entry("f", &item("a", "A", Some("the review"))).await.unwrap();
+        db.record_entry("f", &item("b", "B", Some("the next review"))).await.unwrap();
+        db.record_entry("f", &item("c", "C", Some("as it was"))).await.unwrap();
+        let changed = db
+            .changed_items("f", &[
+                item("a", "A", Some("the review, fixed")), // the text corrected
+                item("b", "B renamed", Some("the next review")), // retitled
+                item("c", "C", None), // says nothing of its text: what is stored stands
+                item("d", "D", Some("new")), // not stored: the insert's, not this
+            ])
+            .await
+            .unwrap();
+        assert_eq!(changed, ["a", "b"].map(String::from).into());
+        assert!(db.changed_items("f", &[]).await.unwrap().is_empty());
+    }
+
     #[tokio::test]
     async fn only_artwork_a_feed_names_is_fetched_for_the_page() {
         let db = Db::memory().await.unwrap();

src/download.rs

@@ -406,7 +406,7 @@ mod tests {
         let mut f = crate::config::Feed {
             url: "u".into(), folder: Some("Subscriptions/Some | Show".into()), group: None, media_types: None,
             schedule: None, keywords: vec![], allow_explicit: false, auto_download: true,
-            max_new_per_check: None, username: None, password: None, password_env: None, category: None,
+            max_new_per_check: None, username: None, password: None, password_env: None, category: None, listed: false,
         };
         assert_eq!(folder_for(&cfg, "id", &f, None), "Subscriptions/Some - Show");
 

src/entity.rs

@@ -19,6 +19,10 @@ pub mod feeds {
         /// The channel's first <itunes:category>, for the Directory.
         #[sea_orm(column_type = "Text", nullable)]
         pub category: Option<String>,
+        /// What the feed says it is, as it says it, for its page in the Directory. Added after
+        /// the table was, so `create_missing` adds it to a database without it.
+        #[sea_orm(column_type = "Text", nullable)]
+        pub description: Option<String>,
         #[sea_orm(column_type = "Text", nullable)]
         pub etag: Option<String>,
         #[sea_orm(column_type = "Text", nullable)]
@@ -102,6 +106,10 @@ pub mod enclosures {
         pub length: Option<i64>,
         #[sea_orm(column_type = "Text", nullable)]
         pub path: Option<String>,
+        /// 'pending' is queued: a scan downloads it on its own. 'held' is listed but outside the
+        /// feed's newest max_new_per_check, or its feed downloads nothing automatically; it can
+        /// be downloaded by hand (`Db::hold_back`). 'skipped' is filtered out ('last_error' says
+        /// why), then 'downloading', 'done', 'error', 'reaped'.
         #[sea_orm(column_type = "Text")]
         pub state: String,
         #[sea_orm(default_value = 0)]
@@ -283,6 +291,29 @@ pub mod sessions {
     owned_by_user!();
 }
 
+/// A token a script or agent sends as `Authorization: Bearer`, acting as the person who made
+/// it (#123). Kept as its SHA-256, so the table is no use to anyone who reads it.
+pub mod api_tokens {
+    use sea_orm::entity::prelude::*;
+
+    #[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
+    #[sea_orm(table_name = "api_tokens")]
+    pub struct Model {
+        #[sea_orm(primary_key)]
+        pub id: i64,
+        pub user_id: i64,
+        /// What its owner called it, to tell one from another when revoking.
+        #[sea_orm(column_type = "Text")]
+        pub name: String,
+        #[sea_orm(unique, column_type = "Text")]
+        pub hash: String,
+        pub created: i64,
+        pub last_used: Option<i64>,
+    }
+
+    owned_by_user!();
+}
+
 /// The catalogue: every feed configured, with its shared settings as `config::Feed` in JSON, so a
 /// new setting on a feed needs no new column. It was config.toml's `[feeds]` (issue #18).
 pub mod catalogue {

src/feed.rs

@@ -13,8 +13,11 @@ pub struct ParsedFeed {
     pub image: Option<String>,
     /// The site the feed belongs to, where a favicon can stand in for missing artwork.
     pub site: Option<String>,
-    /// The channel's first `<itunes:category>`, for the Directory's chips.
+    /// The channel's first `<itunes:category>`, for the Directory's categories.
     pub category: Option<String>,
+    /// What the feed says it is, for its page in the Directory (#130): RSS's `<description>`,
+    /// or iTunes' summary where that is empty, or Atom's subtitle. HTML as often as not.
+    pub description: Option<String>,
     pub entries: Vec<Entry>,
 }
 
@@ -53,43 +56,72 @@ pub enum Fetched {
     },
 }
 
-/// Conditional GET. reqwest handles gzip and redirects; the original's hand-rolled
+/// The longest a feed may take, connecting to the last byte: a scan handles feeds in order, and
-/// CONNECT/socket.ssl proxy path is gone -- `system-proxy` reads http_proxy/https_proxy.
+/// with no limit one hung server held every scan for as long as it did. Dreamwidth answered 504
+/// after 60-67 s for a day, and each scan took 70-80 s instead of 15 (#108). A feed is small;
+/// downloads, which are not, have no such limit.
+pub const FEED_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
+
+/// Conditional GET, following redirects itself: `client` must follow none (`feed_client`), so
+/// each hop is seen. The second value is where the feed now is when every hop said so for good
+/// (301 or 308): a publisher that moved its feed, which the catalogue should follow rather
+/// than be redirected on every read. A temporary redirect (302, 307) moves nothing.
+/// `system-proxy` reads http_proxy/https_proxy.
 #[tracing::instrument(skip_all, fields(url = %cfg.url))]
 pub async fn fetch(
     client: &reqwest::Client,
     cfg: &FeedCfg,
     etag: Option<&str>,
     last_modified: Option<&str>,
-) -> Result<Fetched> {
+) -> Result<(Fetched, Option<String>)> {
-    let mut req = client.get(&cfg.url);
+    let start = reqwest::Url::parse(&cfg.url).context("the feed's address")?;
-    if let Some(tag) = etag {
+    let mut url = start.clone();
-        req = req.header(IF_NONE_MATCH, tag);
+    let mut permanent = true;
-    }
+    for _ in 0..10 {
-    if let Some(lm) = last_modified {
+        let mut req = client.get(url.clone()).timeout(FEED_TIMEOUT);
-        req = req.header(IF_MODIFIED_SINCE, lm);
+        if let Some(tag) = etag {
-    }
+            req = req.header(IF_NONE_MATCH, tag);
-    if let Some(user) = &cfg.username {
+        }
-        req = req.basic_auth(user, cfg.password());
+        if let Some(lm) = last_modified {
-    }
+            req = req.header(IF_MODIFIED_SINCE, lm);
+        }
+        // The feed's own host only: a redirect elsewhere must not be handed the password.
+        if let Some(user) = &cfg.username
+            && url.host_str() == start.host_str()
+        {
+            req = req.basic_auth(user, cfg.password());
+        }
 
-    let resp = req.send().await.context("connecting")?;
+        let resp = req.send().await.context("connecting")?;
-    if resp.status() == StatusCode::NOT_MODIFIED {
+        let status = resp.status();
-        return Ok(Fetched::NotModified);
+        if status.is_redirection() && status != StatusCode::NOT_MODIFIED {
+            let to = resp
+                .headers()
+                .get(reqwest::header::LOCATION)
+                .and_then(|v| v.to_str().ok())
+                .ok_or_else(|| anyhow!("HTTP {status} without a Location to go to"))?;
+            url = url.join(to).with_context(|| format!("redirected to {to:?}, which is not an address"))?;
+            permanent &= matches!(status, StatusCode::MOVED_PERMANENTLY | StatusCode::PERMANENT_REDIRECT);
+            continue;
+        }
+        let moved = (permanent && url != start).then(|| url.to_string());
+        if status == StatusCode::NOT_MODIFIED {
+            return Ok((Fetched::NotModified, moved));
+        }
+        if !status.is_success() {
+            // The original surfaced 401/407 specially; the code is enough for a UI to switch on.
+            return Err(anyhow!("HTTP {status}"));
+        }
+        let header = |h: reqwest::header::HeaderName| {
+            resp.headers().get(&h).and_then(|v| v.to_str().ok()).map(str::to_owned)
+        };
+        let etag = header(ETAG);
+        let last_modified = header(LAST_MODIFIED);
+        let bytes = resp.bytes().await.context("reading body")?.to_vec();
+        return Ok((Fetched::Body { bytes, etag, last_modified }, moved));
     }
-    let status = resp.status();
+    // Worded as reqwest worded it, which failure_kind reads as a redirect loop.
-    if !status.is_success() {
+    Err(anyhow!("error following redirect for url ({url}): too many redirects"))
-        // The original surfaced 401/407 specially; the code is enough for a UI to switch on.
-        return Err(anyhow!("HTTP {status}"));
-    }
-
-    let header = |h: reqwest::header::HeaderName| {
-        resp.headers().get(&h).and_then(|v| v.to_str().ok()).map(str::to_owned)
-    };
-    let etag = header(ETAG);
-    let last_modified = header(LAST_MODIFIED);
-    let bytes = resp.bytes().await.context("reading body")?.to_vec();
-    Ok(Fetched::Body { bytes, etag, last_modified })
 }
 
 /// A stored `last_error`, translated into plain words for whoever subscribes: whose problem
@@ -158,18 +190,31 @@ pub fn explain_failure(msg: &str) -> Option<Failure> {
     if low.contains("http 401") || low.contains("http 403") {
         return Some(Failure { reason: "The site refuses ipx's requests.", new_url: None });
     }
+    if low.contains("answered with nothing") {
+        return Some(Failure { reason: "This address answers with nothing; the site may be gone.", new_url: None });
+    }
     if low.contains("http 402") {
         return Some(Failure { reason: "The feed now needs a paid plan.", new_url: None });
     }
-    if low.contains("dns error")
+    // Only a name that does not exist. A lookup that failed for the moment says nothing about the
-        || low.contains("failed to lookup address")
+    // site, and NPR's and Megaphone's feeds were labelled gone after a DNS outage (#143).
-        || low.contains("no address associated")
+    if !is_temporary_lookup_failure(msg)
+        && (low.contains("dns error") || low.contains("failed to lookup address") || low.contains("no address associated"))
     {
-        return Some(Failure { reason: "This address no longer resolves; the site is gone.", new_url: None });
+        return Some(Failure { reason: "This address no longer resolves; the site may be gone.", new_url: None });
     }
     None
 }
 
+/// A host that could not be looked up for the moment -- the resolver's EAI_AGAIN, glibc's
+/// "Temporary failure in name resolution", musl's "Try again" -- as against a name that does
+/// not exist ("Name or service not known"). It says the network was not answering, not that the
+/// feed is gone (#143).
+pub fn is_temporary_lookup_failure(msg: &str) -> bool {
+    let low = msg.to_ascii_lowercase();
+    low.contains("dns error") && (low.contains("temporary failure in name resolution") || low.contains("try again"))
+}
+
 /// True when a body is an OPML document rather than a feed.
 ///
 /// The original matched on the URL ending in ".opml" (iPXClass.py:34), which misses an
@@ -253,7 +298,7 @@ pub async fn patreon_shows(
 ) -> Result<(Option<String>, Vec<(String, String)>)> {
     // The creator feed names its campaign by number in its self link, a few hundred bytes in.
     // The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
-    let mut resp = client.get(url).send().await.context("connecting")?;
+    let mut resp = client.get(url).timeout(FEED_TIMEOUT).send().await.context("connecting")?;
     if !resp.status().is_success() {
         return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
     }
@@ -451,6 +496,30 @@ pub async fn site_icon(client: &reqwest::Client, site: &str) -> Option<String> {
     is_image(client, ico.as_str()).await.then(|| ico.into())
 }
 
+/// Artwork's address on https when its host serves it there, else as it was. The page is https
+/// and must not load http; the host is asked once per `known` (one feed's read), and an http
+/// address it does not serve on https stays, for /api/art to fetch (#90). Four of the five
+/// hosts the catalogue had on http served the same image on https; The Secret Cabal's CDN
+/// presents another name's certificate (#110).
+pub async fn prefer_https(
+    client: &reqwest::Client,
+    url: &str,
+    known: &mut std::collections::HashMap<String, bool>,
+) -> String {
+    let Some(rest) = url.strip_prefix("http://") else { return url.to_owned() };
+    let host = rest.split('/').next().unwrap_or("").to_owned();
+    let secure = format!("https://{rest}");
+    let ok = match known.get(&host) {
+        Some(ok) => *ok,
+        None => {
+            let ok = is_image(client, &secure).await;
+            known.insert(host, ok);
+            ok
+        }
+    };
+    if ok { secure } else { url.to_owned() }
+}
+
 /// Whether `url` answers with an image. A site with no favicon often answers 200 with its home
 /// page, which is not an icon.
 pub async fn is_image(client: &reqwest::Client, url: &str) -> bool {
@@ -681,6 +750,7 @@ fn from_rss(ch: rss::Channel, bytes: &[u8]) -> ParsedFeed {
         // Only the iTunes one: Apple's list is fixed, while a plain <category> is freeform and
         // would fill the Directory with one-off tags. The subcategory where there is one: Apple
         // files every tabletop and gaming show under Leisure, which says little; Games says it.
+        description: non_empty(Some(ch.description())).or_else(|| non_empty(ch.itunes_ext().and_then(|i| i.summary()))),
         category: ch
             .itunes_ext()
             .and_then(|i| i.categories().first())
@@ -750,6 +820,7 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
             .map(|l| l.href().trim().to_owned())
             .filter(|h| !h.is_empty()),
         category: None,
+        description: non_empty(feed.subtitle().map(|t| t.as_str())),
         entries,
     }
 }
@@ -955,8 +1026,151 @@ fn parse_date(s: &str) -> Option<i64> {
         .map(|d| d.timestamp())
 }
 
+/// Apple's podcast categories, each with its subcategories
+/// (https://podcasters.apple.com/support/1691-apple-podcasts-categories): the list a podcast's own
+/// `<itunes:category>` comes from, and a fixed one, so the Directory's categories stay few.
+const CATEGORIES: &[(&str, &[&str])] = &[
+    ("Arts", &["Books", "Design", "Fashion & Beauty", "Food", "Performing Arts", "Visual Arts"]),
+    ("Business", &["Careers", "Entrepreneurship", "Investing", "Management", "Marketing", "Non-Profit"]),
+    ("Comedy", &["Comedy Interviews", "Improv", "Stand-Up"]),
+    ("Education", &["Courses", "How To", "Language Learning", "Self-Improvement"]),
+    ("Fiction", &["Comedy Fiction", "Drama", "Science Fiction"]),
+    ("Government", &[]),
+    ("History", &[]),
+    ("Health & Fitness", &["Alternative Health", "Fitness", "Medicine", "Mental Health", "Nutrition", "Sexuality"]),
+    ("Kids & Family", &["Education for Kids", "Parenting", "Pets & Animals", "Stories for Kids"]),
+    ("Leisure", &["Animation & Manga", "Automotive", "Aviation", "Crafts", "Games", "Hobbies", "Home & Garden", "Video Games"]),
+    ("Music", &["Music Commentary", "Music History", "Music Interviews"]),
+    ("News", &["Business News", "Daily News", "Entertainment News", "News Commentary", "Politics", "Sports News", "Tech News"]),
+    ("Religion & Spirituality", &["Buddhism", "Christianity", "Hinduism", "Islam", "Judaism", "Religion", "Spirituality"]),
+    ("Science", &["Astronomy", "Chemistry", "Earth Sciences", "Life Sciences", "Mathematics", "Natural Sciences", "Nature", "Physics", "Social Sciences"]),
+    ("Society & Culture", &["Documentary", "Personal Journals", "Philosophy", "Places & Travel", "Relationships"]),
+    ("Sports", &["Baseball", "Basketball", "Cricket", "Fantasy Sports", "Football", "Golf", "Hockey", "Rugby", "Running", "Soccer", "Swimming", "Tennis", "Volleyball", "Wilderness", "Wrestling"]),
+    ("Technology", &[]),
+    ("True Crime", &[]),
+    ("TV & Film", &["After Shows", "Film History", "Film Interviews", "Film Reviews", "TV Reviews"]),
+];
+
+/// Names that are not in Apple's list but mean one that is (#125). Jev picked from this list when
+/// it spelt Football and Soccer its own way, and stored those answers; Software How-To is Apple's
+/// Technology subcategory from before its 2019 list, which older feeds still name.
+const ALIASES: &[(&str, &str)] =
+    &[("American Football", "Football"), ("Football (Soccer)", "Soccer"), ("Software How-To", "Technology")];
+
+/// Where a category sits in Apple's list, as (category, subcategory), so the Directory browses a
+/// category and then its subcategories as Apple's does (#118). A podcast's own is stored as its
+/// subcategory, and one an admin typed that is not Apple's stands as a category of its own.
+pub fn category_path(name: &str) -> (String, Option<String>) {
+    // A feed's own text, which can end in a stray comma: "Soccer," was a category of its own.
+    let name = name.trim().trim_end_matches([',', ';', '.']).trim_end();
+    let name = ALIASES.iter().find(|(old, _)| old.eq_ignore_ascii_case(name)).map_or(name, |(_, now)| now);
+    for (category, subs) in CATEGORIES {
+        if category.eq_ignore_ascii_case(name) {
+            return (category.to_string(), None);
+        }
+        if let Some(sub) = subs.iter().find(|s| s.eq_ignore_ascii_case(name)) {
+            return (category.to_string(), Some(sub.to_string()));
+        }
+    }
+    (name.to_string(), None)
+}
+
+/// A Directory category for a feed that names none of its own, which most blogs do not (#117):
+/// TypeSafe's Jev picks one of Apple's categories or subcategories from the feed's title and
+/// latest items, the subcategory being what a podcast's own category is stored as. The likeliest
+/// is taken however unsure, since an admin can change it and asking again would cost a call a scan.
+pub async fn classify(client: &reqwest::Client, key: &str, feed: &ParsedFeed) -> Result<String> {
+    let items: Vec<&str> = feed.entries.iter().filter_map(|e| e.title.as_deref()).take(15).collect();
+    // A subcategory says which category it is in; without that, "Games" or "Drama" is ambiguous.
+    let mut criteria = serde_json::Map::new();
+    for (category, subs) in CATEGORIES {
+        criteria.insert(category.to_string(), serde_json::Value::Null);
+        for sub in *subs {
+            criteria.insert(sub.to_string(), format!("Within {category}").into());
+        }
+    }
+    let body = serde_json::json!({
+        "model": "jev-latest",
+        "state": { "title": feed.title, "recent_items": items },
+        "questions": { "category": {
+            "type": "choice",
+            "instructions": "Which category best describes what this feed publishes?",
+            "criteria": criteria,
+        }},
+    });
+    let answer: serde_json::Value = client
+        .post("https://api.typesafe.ai/v1/systemone")
+        .bearer_auth(key)
+        .json(&body)
+        .timeout(FEED_TIMEOUT)
+        .send()
+        .await?
+        .error_for_status()?
+        .json()
+        .await?;
+    picked(&answer)
+}
+
+fn picked(answer: &serde_json::Value) -> Result<String> {
+    let choice = answer["answers"]["category"]["choice"].as_str().unwrap_or_default();
+    CATEGORIES
+        .iter()
+        .flat_map(|(c, subs)| std::iter::once(c).chain(subs.iter()))
+        .find(|c| **c == choice)
+        .map(|c| c.to_string())
+        .ok_or_else(|| anyhow!("no category in the answer: {answer}"))
+}
+
 #[cfg(test)]
 mod tests {
+    /// A server answering by path: /old moves for good to /new, /tmp for now, /chain for good
+    /// to /tmp, /new is the feed.
+    async fn redirecting_server() -> String {
+        use tokio::io::{AsyncReadExt, AsyncWriteExt};
+        let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let addr = listener.local_addr().unwrap();
+        tokio::spawn(async move {
+            loop {
+                let Ok((mut sock, _)) = listener.accept().await else { return };
+                tokio::spawn(async move {
+                    let mut buf = [0u8; 2048];
+                    let n = sock.read(&mut buf).await.unwrap_or(0);
+                    let req = String::from_utf8_lossy(&buf[..n]);
+                    let path = req.split_whitespace().nth(1).unwrap_or("/").to_owned();
+                    let body = "<?xml version=\"1.0\"?><rss version=\"2.0\"><channel><title>T</title></channel></rss>";
+                    let resp = match path.as_str() {
+                        "/old" => "HTTP/1.1 301 Moved Permanently\r\nLocation: /new\r\nContent-Length: 0\r\n\r\n".to_owned(),
+                        "/tmp" => "HTTP/1.1 302 Found\r\nLocation: /new\r\nContent-Length: 0\r\n\r\n".to_owned(),
+                        "/chain" => "HTTP/1.1 308 Permanent Redirect\r\nLocation: /tmp\r\nContent-Length: 0\r\n\r\n".to_owned(),
+                        "/loop" => "HTTP/1.1 301 Moved Permanently\r\nLocation: /loop\r\nContent-Length: 0\r\n\r\n".to_owned(),
+                        _ => format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n{body}", body.len()),
+                    };
+                    let _ = sock.write_all(resp.as_bytes()).await;
+                });
+            }
+        });
+        format!("http://{addr}")
+    }
+
+    #[tokio::test]
+    async fn a_feed_that_moved_for_good_says_where_and_one_moved_for_now_does_not() {
+        let base = redirecting_server().await;
+        let client = reqwest::Client::builder().redirect(reqwest::redirect::Policy::none()).build().unwrap();
+        let get = |path: &str| {
+            let cfg: crate::config::Feed = serde_json::from_value(serde_json::json!({ "url": format!("{base}{path}") })).unwrap();
+            let client = client.clone();
+            async move { super::fetch(&client, &cfg, None, None).await }
+        };
+        let (got, moved) = get("/old").await.unwrap();
+        assert!(matches!(got, super::Fetched::Body { .. }));
+        assert_eq!(moved, Some(format!("{base}/new")));
+        assert_eq!(get("/tmp").await.unwrap().1, None);         // 302: for now
+        assert_eq!(get("/chain").await.unwrap().1, None);       // 308 then 302: not for good
+        assert_eq!(get("/new").await.unwrap().1, None);         // never moved
+        let looped = get("/loop").await.err().unwrap().to_string();
+        assert_eq!(super::failure_kind(&looped).0, "redirect_loop");
+    }
+
     use super::*;
 
     #[test]
@@ -967,6 +1181,7 @@ mod tests {
         assert_eq!(feed.title.as_deref(), Some("Test Cast"));
         assert_eq!(feed.ttl_mins, Some(45));
         assert_eq!(feed.category.as_deref(), Some("Podcasting"), "the first, by its subcategory");
+        assert_eq!(feed.description.as_deref(), Some("A synthetic feed used by the parser tests."));
         assert_eq!(feed.entries.len(), 3);
 
         let ep = &feed.entries[0];
@@ -991,6 +1206,17 @@ mod tests {
         );
     }
 
+    #[test]
+    fn a_feed_says_what_it_is_in_itunes_summary_or_atom_subtitle() {
+        // An empty <description> gives way to iTunes' summary (#130).
+        let rss = br#"<?xml version="1.0"?><rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
+        <channel><title>R</title><description> </description><itunes:summary>Said here.</itunes:summary></channel></rss>"#;
+        assert_eq!(parse(rss).unwrap().description.as_deref(), Some("Said here."));
+        let atom = br#"<?xml version="1.0"?><feed xmlns="http://www.w3.org/2005/Atom"><title>A</title>
+        <subtitle>Or here.</subtitle><id>urn:a</id><updated>2026-01-01T00:00:00Z</updated></feed>"#;
+        assert_eq!(parse(atom).unwrap().description.as_deref(), Some("Or here."));
+    }
+
     #[test]
     fn a_file_wordpress_lists_twice_is_one_enclosure() {
         let xml = br#"<?xml version="1.0"?><rss version="2.0"><channel><title>R</title>
@@ -1097,8 +1323,17 @@ mod tests {
         assert_eq!(explain_failure("HTTP 401 Unauthorized").unwrap().reason, "The site refuses ipx's requests.");
         assert_eq!(explain_failure("HTTP 403 Forbidden").unwrap().reason, "The site refuses ipx's requests.");
         assert_eq!(explain_failure("HTTP 402 Payment Required").unwrap().reason, "The feed now needs a paid plan.");
-        let dns = explain_failure("connecting: dns error: failed to lookup address information").unwrap();
+        assert_eq!(
-        assert_eq!(dns.reason, "This address no longer resolves; the site is gone.");
+            explain_failure("the feed's address answered with nothing at all").unwrap().reason,
+            "This address answers with nothing; the site may be gone."
+        );
+        let dns = explain_failure("connecting: dns error: failed to lookup address information: Name or service not known").unwrap();
+        assert_eq!(dns.reason, "This address no longer resolves; the site may be gone.");
+        // A lookup that failed for the moment is not a site gone (#143).
+        let blip = "connecting: dns error: failed to lookup address information: Temporary failure in name resolution";
+        assert!(is_temporary_lookup_failure(blip));
+        assert!(explain_failure(blip).is_none());
+        assert!(!is_temporary_lookup_failure("connecting: dns error: failed to lookup address information: Name or service not known"));
         let moved = explain_failure("got a web page, not a feed; it links https://x/feed as its feed").unwrap();
         assert_eq!(moved.new_url.as_deref(), Some("https://x/feed"));
         assert!(explain_failure("got a web page, not a feed").unwrap().new_url.is_none());
@@ -1390,5 +1625,29 @@ mod tests {
         assert_eq!(pick_guid(None, None, Some("e"), Some("t")).as_deref(), Some("e"));
         assert_eq!(pick_guid(None, None, None, None), None);
     }
+
+    #[test]
+    fn picked_takes_only_an_apple_category() {
+        let answer = |c: &str| serde_json::json!({"answers": {"category": {"type": "choice", "choice": c}}});
+        assert_eq!(super::picked(&answer("Technology")).unwrap(), "Technology");
+        assert_eq!(super::picked(&answer("Tech News")).unwrap(), "Tech News", "a subcategory");
+        assert!(super::picked(&answer("Blogs")).is_err(), "not one of Apple's");
+        assert!(super::picked(&serde_json::json!({"error": "rate limited"})).is_err());
+    }
+
+    #[test]
+    fn category_path_finds_a_subcategory_s_category() {
+        use super::category_path;
+        assert_eq!(category_path("Tech News"), ("News".into(), Some("Tech News".into())));
+        assert_eq!(category_path("video games"), ("Leisure".into(), Some("Video Games".into())), "Apple's spelling");
+        assert_eq!(category_path("Technology"), ("Technology".into(), None));
+        assert_eq!(category_path("Homebrew"), ("Homebrew".into(), None), "not Apple's: its own category");
+        // Apple's spelling, an old answer of Jev's, a stray comma and Apple's old list (#125).
+        assert_eq!(category_path("Soccer"), ("Sports".into(), Some("Soccer".into())));
+        assert_eq!(category_path("Football (Soccer)"), ("Sports".into(), Some("Soccer".into())));
+        assert_eq!(category_path("Soccer,"), ("Sports".into(), Some("Soccer".into())));
+        assert_eq!(category_path("Football"), ("Sports".into(), Some("Football".into())));
+        assert_eq!(category_path("Software How-To"), ("Technology".into(), None));
+    }
 }
 

src/ipc.rs

@@ -15,6 +15,9 @@ pub enum Event {
     FeedSkip { feed: String, reason: String },
     FeedDone { feed: String, new: usize, downloaded: usize, failed: usize, torrents: usize },
     FeedError { feed: String, msg: String },
+    /// The feed answered from a new address after redirects that all said it moved for good,
+    /// and the catalogue now has that address (#115).
+    FeedMoved { feed: String, from: String, to: String },
     Progress {
         feed: String,
         /// Which enclosure this is about. Without it a UI cannot tell one download's
@@ -50,6 +53,7 @@ impl Event {
                 "{feed}: {new} new entries, {downloaded} downloaded, {failed} failed, {torrents} torrents deferred"
             ),
             Event::FeedError { feed, msg } => format!("{feed}: error: {msg}"),
+            Event::FeedMoved { feed, from, to } => format!("{feed}: moved for good from {from} to {to}; following it"),
             Event::Progress { file, done, total, .. } => match total {
                 Some(t) if *t > 0 => format!(
                     "    {file}: {:.1}% ({:.1}/{:.1} MB)",
@@ -69,7 +73,7 @@ impl Event {
                 *bytes as f64 / 1_048_576.0
             ),
             Event::Status { feeds, pending, downloaded } => {
-                format!("{feeds} feeds, {pending} pending, {downloaded} downloaded")
+                format!("{feeds} feeds, {pending} queued to download, {downloaded} downloaded")
             }
             Event::Error { msg } => format!("error: {msg}"),
             // Noise in a terminal; a UI still gets them on the socket.
@@ -105,6 +109,11 @@ pub enum Command {
         enclosure: i64,
     },
     Status,
+    /// Read the catalogue and server settings again from the database, which `ipx add`, `rm`
+    /// and `import` change in a process of their own. A daemon keeps them in memory, and with
+    /// its old copy wrote a removed feed back, and dropped an added one, at its next change of
+    /// its own (#120). Answered, as status is, with the counts after it.
+    Reload,
 }
 
 /// Where events go: the socket, the terminal, or both.
@@ -172,7 +181,7 @@ fn log_event(e: &Event, wire: bool) {
         ($level:ident, $target:literal, $text:expr) => {
             tracing::$level!(
                 target: $target,
-                ev = s("ev"), feed = s("feed"), msg = s("msg"), url = s("url"), reason = s("reason"),
+                ev = s("ev"), feed = s("feed"), msg = s("msg"), url = s("url"), reason = s("reason"), from = s("from"), to = s("to"),
                 new = n("new"), downloaded = n("downloaded"), failed = n("failed"),
                 torrents = n("torrents"), bytes = n("bytes"), feeds = n("feeds"),
                 pending = n("pending"), enclosure = n("enclosure"), files = n("files"),
@@ -204,13 +213,15 @@ pub async fn daemon_is_live(path: &Path) -> bool {
     UnixStream::connect(path).await.is_ok()
 }
 
-/// Answers `status` for the socket, without the worker. The worker runs one job at a time, and a
+/// Answers `status` and `reload` for the socket, without the worker. The worker runs one job at a
-/// healthcheck left waiting behind a scan or a long download timed out and called a busy daemon
+/// time, and a healthcheck left waiting behind a scan or a long download timed out and called a
-/// dead. The answer goes to the client that asked and no one else: broadcast, it ended any
+/// busy daemon dead; a reload left waiting would let the web UI write the old catalogue back in
-/// `ipx fetch` that was watching a scan, since `status` is a terminal event.
+/// the meantime. The answer goes to the client that asked and no one else: broadcast, it ended
+/// any `ipx fetch` that was watching a scan, since `status` is a terminal event.
 /// A future, since reading the counts is a database query.
-pub type StatusFn =
+pub type StatusFn = std::sync::Arc<
-    std::sync::Arc<dyn Fn() -> std::pin::Pin<Box<dyn std::future::Future<Output = Event> + Send>> + Send + Sync>;
+    dyn Fn(Command) -> std::pin::Pin<Box<dyn std::future::Future<Output = Event> + Send>> + Send + Sync,
+>;
 
 /// Accepts connections, feeding commands to `cmds` and events from `events` back out.
 pub async fn serve(
@@ -281,9 +292,9 @@ async fn handle(
         }
         match serde_json::from_str::<Command>(line) {
             // Answered here, not queued behind whatever the worker is on: see StatusFn.
-            Ok(Command::Status) => {
+            Ok(cmd @ (Command::Status | Command::Reload)) => {
                 tracing::info!(target: "ipx::io", "-> {line}");
-                let ev = status().await;
+                let ev = status(cmd).await;
                 log_event(&ev, true);
                 let _ = reply.send(ev).await;
             }
@@ -299,6 +310,26 @@ async fn handle(
     Ok(())
 }
 
+/// Tells a running daemon to read the catalogue again (see Command::Reload), and waits for it.
+pub async fn reload(path: &Path) -> Result<()> {
+    let stream = UnixStream::connect(path).await?;
+    let (read, mut write) = stream.into_split();
+    write.write_all(b"{\"cmd\":\"reload\"}\n").await?;
+    let mut lines = BufReader::new(read).lines();
+    let answer = async {
+        // Other clients' events come here too; the answer is the status, or an error.
+        while let Some(line) = lines.next_line().await? {
+            match serde_json::from_str::<Event>(&line) {
+                Ok(Event::Status { .. }) => return Ok(()),
+                Ok(Event::Error { msg }) => anyhow::bail!("{msg}"),
+                _ => continue,
+            }
+        }
+        anyhow::bail!("the daemon closed the connection without answering")
+    };
+    tokio::time::timeout(std::time::Duration::from_secs(10), answer).await.context("the daemon did not answer in 10s")?
+}
+
 /// Sends one command to a running daemon and prints the events it produces.
 pub async fn proxy(path: &Path, cmd: &Command) -> Result<()> {
     let stream = UnixStream::connect(path).await?;
@@ -328,6 +359,14 @@ pub async fn proxy(path: &Path, cmd: &Command) -> Result<()> {
 mod tests {
     use super::*;
 
+    #[test]
+    fn a_feed_that_moved_says_so_on_the_wire_and_in_words() {
+        let e = Event::FeedMoved { feed: "x".into(), from: "http://a/f".into(), to: "https://a/f".into() };
+        let wire = serde_json::to_value(&e).unwrap();
+        assert_eq!((wire["ev"].as_str(), wire["from"].as_str(), wire["to"].as_str()), (Some("feed_moved"), Some("http://a/f"), Some("https://a/f")));
+        assert_eq!(e.human().unwrap(), "x: moved for good from http://a/f to https://a/f; following it");
+    }
+
     #[test]
     fn commands_parse_from_the_wire_form() {
         let got: Command = serde_json::from_str(r#"{"cmd":"fetch"}"#).unwrap();
@@ -399,7 +438,7 @@ mod tests {
         // would end its session.
         let mut watcher = events.subscribe();
         let status: StatusFn =
-            std::sync::Arc::new(|| Box::pin(async { Event::Status { feeds: 1, pending: 2, downloaded: 3 } }));
+            std::sync::Arc::new(|_| Box::pin(async { Event::Status { feeds: 1, pending: 2, downloaded: 3 } }));
         let (client, server) = UnixStream::pair().unwrap();
         tokio::spawn(handle(server, events.subscribe(), cmds, status));
 

src/main.rs

@@ -1,4 +1,5 @@
 mod access;
+mod art;
 mod auth;
 mod config;
 mod db;
@@ -63,6 +64,12 @@ enum Command {
         /// Only take enclosures matching these keywords
         #[arg(long, value_delimiter = ',')]
         keywords: Vec<String>,
+        /// The Directory's category, for a feed that names none of its own (News, Technology)
+        #[arg(long)]
+        category: Option<String>,
+        /// Put it in the Directory for anyone to subscribe to, and keep it there when nobody does
+        #[arg(long)]
+        list: bool,
     },
     /// Unsubscribe. Downloads and history are left alone.
     Rm { feed: String },
@@ -118,6 +125,9 @@ pub struct Ctx {
     pub cfg: std::sync::RwLock<std::sync::Arc<config::Config>>,
     pub db: db::Db,
     pub client: reqwest::Client,
+    /// For feeds only: follows no redirects, so `feed::fetch` sees each hop and can tell a feed
+    /// that moved for good from one sent elsewhere for now.
+    pub feed_client: reqwest::Client,
     pub out: Emitter,
     /// Started on first use: a BitTorrent session binds ports and starts a DHT, which is
     /// rude to do for a config that has never seen a torrent.
@@ -131,9 +141,38 @@ pub struct Ctx {
     /// stops every feed scan, every HTTP download and every status command behind it.
     /// A one-shot CLI run keeps them inline, or the process would exit mid-download.
     pub detach_torrents: bool,
+    /// Feeds whose host could not be looked up for the moment, and when to try them again: a
+    /// failure that says nothing about the feed, kept off it and out of the database (#143).
+    /// Forgotten on a restart, which only makes the retry sooner.
+    pub retry: std::sync::Mutex<std::collections::HashMap<String, i64>>,
 }
 
+/// How soon a feed whose host could not be looked up is tried again (#143).
+const RETRY_LOOKUP_SECS: i64 = 600;
+
 impl Ctx {
+    /// When a feed is next due, taking a lookup retry into account: not before it.
+    fn due(&self, cfg: &config::Config, feed: &config::Feed, id: &str, state: &db::HttpState, subscribed: bool) -> Option<i64> {
+        let at = due_at(cfg, feed, state, subscribed);
+        match self.retry.lock().unwrap().get(id) {
+            Some(&retry) => Some(at.unwrap_or(0).max(retry)),
+            None => at,
+        }
+    }
+
+    /// A scan's failure, recorded on the feed, unless it was only a moment's failure to look its
+    /// host up: on 2026-10-05 a DNS outage on Tower failed 204 feeds in a minute with
+    /// "Temporary failure in name resolution", and each, read once a day and backed off as a
+    /// failing feed, sat a day failing, unread and labelled gone (#143). Tried again in minutes
+    /// instead, with nothing on the feed to say it failed.
+    async fn feed_failed(&self, id: &str, url: &str, msg: &str) -> Result<()> {
+        if feed::is_temporary_lookup_failure(msg) {
+            self.retry.lock().unwrap().insert(id.to_owned(), db::now() + RETRY_LOOKUP_SECS);
+            return Ok(());
+        }
+        self.db.set_feed_error(id, url, msg).await
+    }
+
     pub fn cfg(&self) -> std::sync::Arc<config::Config> {
         self.cfg.read().unwrap().clone()
     }
@@ -239,28 +278,40 @@ async fn main() -> Result<()> {
         return ipc::proxy(&cfg.general.socket, cmd).await;
     }
 
+    let socket = cfg.general.socket.clone();
     let cfg = assemble_config(&db, cfg, &config_path).await?;
 
     let is_daemon = matches!(cli.command, Command::Daemon { .. });
+    // These write the catalogue from a process of their own, which a running daemon, holding its
+    // own copy, would write over at its next change (#120).
+    let changes_catalogue = matches!(cli.command, Command::Add { .. } | Command::Rm { .. } | Command::Import { .. });
     let (events, _) = broadcast::channel(1024);
     let ctx = Arc::new(Ctx {
         cfg: std::sync::RwLock::new(std::sync::Arc::new(cfg)),
         db,
         client: reqwest::Client::builder()
             .user_agent(concat!("ipx/", env!("CARGO_PKG_VERSION")))
+            // Connecting only, so it bounds a download's start, not a long download (#108).
+            .connect_timeout(std::time::Duration::from_secs(10))
+            .build()?,
+        feed_client: reqwest::Client::builder()
+            .user_agent(concat!("ipx/", env!("CARGO_PKG_VERSION")))
+            .connect_timeout(std::time::Duration::from_secs(10))
+            .redirect(reqwest::redirect::Policy::none())
             .build()?,
         out: if is_daemon { Emitter::socket(events.clone(), false) } else { Emitter::terminal() },
         torrents: tokio::sync::OnceCell::new(),
         torrent_slots: Arc::new(tokio::sync::Semaphore::new(2)),
         config_path: config_path.clone(),
         detach_torrents: is_daemon,
+        retry: Default::default(),
     });
 
     let result = match cli.command {
         Command::List => list(&ctx).await,
         Command::Daemon { web } => daemon(ctx, config_path, web, events).await,
-        Command::Add { url, folder, keywords } => {
+        Command::Add { url, folder, keywords, category, list } => {
-            add(&ctx, &url, folder, keywords).await
+            add(&ctx, &url, folder, keywords, category, list).await
         }
         Command::Rm { feed } => rm(&ctx, &feed).await,
         Command::User { cmd } => user_cmd(&ctx, cmd).await,
@@ -268,6 +319,11 @@ async fn main() -> Result<()> {
         Command::Export { file } => export(&ctx, &file).await,
         _ => run(&ctx, wire_cmd.expect("only List and Daemon have no wire form")).await,
     };
+    if result.is_ok() && changes_catalogue && ipc::daemon_is_live(&socket).await {
+        if let Err(e) = ipc::reload(&socket).await {
+            eprintln!("the running daemon did not take the change ({e:#}); restart it, or it may undo it");
+        }
+    }
     // The batch exporter holds the last few seconds of spans; without this they are lost.
     if let Some(p) = otel {
         let _ = p.shutdown();
@@ -435,9 +491,19 @@ async fn run(ctx: &Arc<Ctx>, cmd: Cmd) -> Result<()> {
             ctx.out.emit(status(ctx).await);
             Ok(())
         }
+        Cmd::Reload => reload(ctx).await,
     }
 }
 
+/// The catalogue and server settings as the database now has them, in place of the copy in
+/// memory, for a daemon told another ipx changed them (#120); config.toml, read again, for where
+/// things are and who gets in.
+async fn reload(ctx: &Ctx) -> Result<()> {
+    let cfg = assemble_config(&ctx.db, config::Config::load(&ctx.config_path)?, &ctx.config_path).await?;
+    ctx.set_cfg(cfg);
+    Ok(())
+}
+
 /// The counts `ipx status` prints, and /api/status serves. A running daemon's socket answers with
 /// this directly rather than through the job queue.
 pub(crate) async fn status(ctx: &Ctx) -> Event {
@@ -494,19 +560,25 @@ async fn daemon(
     let (tx_cmd, mut rx_cmd) = mpsc::channel::<Cmd>(64);
 
     let web = start_web(&ctx, &config_path, web_addr, &tx_cmd, &events).await?;
-    // status is answered by the socket itself; everything else waits its turn in the queue.
+    // status and reload are answered by the socket itself; everything else waits its turn in the
+    // queue.
     let answer: ipc::StatusFn = {
         let ctx = ctx.clone();
-        Arc::new(move || {
+        Arc::new(move |cmd| {
             let ctx = ctx.clone();
-            Box::pin(async move { status(&ctx).await })
+            Box::pin(async move {
+                if matches!(cmd, Cmd::Reload)
+                    && let Err(e) = reload(&ctx).await
+                {
+                    return Event::Error { msg: format!("reading the catalogue again: {e:#}") };
+                }
+                status(&ctx).await
+            })
         })
     };
     let server = tokio::spawn(ipc::serve(socket.clone(), events.clone(), tx_cmd, answer));
 
     // One command at a time: the queue is what keeps two scans from overlapping.
-    let mut ticker = tokio::time::interval(std::time::Duration::from_secs(60));
-    ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
     tracing::info!(
         feeds = subscriptions(&ctx).await.map(|s| s.len()).unwrap_or(0),
         "daemon started"
@@ -543,10 +615,13 @@ async fn daemon(
     }
 
     let mut stop = rx_stop.clone();
+    // The first pass at once, as the minute's tick did: what came due while it was down.
+    let mut first = true;
     loop {
         if *stop.borrow() {
             break;
         }
+        let wait = if std::mem::take(&mut first) { std::time::Duration::ZERO } else { until_next_scan(&ctx).await };
         tokio::select! {
             biased;
             _ = stop.changed() => break,
@@ -563,7 +638,7 @@ async fn daemon(
                     break;
                 }
             }
-            _ = ticker.tick() => {
+            _ = tokio::time::sleep(wait) => {
                 // Per-feed schedule and TTL decide what actually gets polled.
                 let job = run(&ctx, Cmd::Fetch { feed: None, force: false, feeds: vec![] });
                 if !until_stopped(&ctx, &rx_stop, job).await {
@@ -673,16 +748,35 @@ async fn add(
     url: &str,
     folder: Option<String>,
     keywords: Vec<String>,
+    category: Option<String>,
+    list: bool,
 ) -> Result<()> {
     let mut cfg = (*ctx.cfg()).clone();
     let url = &feed::find_feed(&ctx.client, &feed::expand_input(url)).await?;
     // Includes feeds derived from an OPML, or the same show could be added twice.
     if let Some(existing) = subscriptions(ctx).await?.iter().find(|s| feed::same_feed(&s.cfg.url, url)) {
+        // Listing a feed the catalogue already has, or giving it a category, is the point of
+        // asking again: it changes those, and nothing else.
+        if let Some(f) = cfg.feeds.get_mut(&existing.id)
+            && (list || category.is_some())
+        {
+            f.listed |= list;
+            if category.is_some() {
+                f.category = category;
+            }
+            ctx.store_cfg(cfg).await?;
+            println!("{} is already in the catalogue; updated its Directory listing", existing.id);
+            return Ok(());
+        }
         anyhow::bail!("already subscribed as {:?}", existing.id);
     }
     let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
+    if let Some(f) = cfg.feeds.get_mut(&id) {
+        f.category = category;
+        f.listed = list;
+    }
     ctx.store_cfg(cfg).await?;
-    println!("added {id}");
+    println!("added {id}{}", if list { ", listed in the Directory" } else { "" });
     Ok(())
 }
 
@@ -709,9 +803,10 @@ pub async fn add_one(
         password: None,
         password_env: None,
         category: None,
+        listed: false,
     };
 
-    let title = match feed::fetch(&ctx.client, &probe, None, None).await {
+    let title = match feed::fetch(&ctx.feed_client, &probe, None, None).await.map(|(got, _)| got) {
         // An OPML subscription is named from its own <head><title>, not by trying to
         // parse it as a feed and falling back to the hostname.
         Ok(feed::Fetched::Body { bytes, .. }) if feed::is_opml(&bytes) => {
@@ -835,6 +930,7 @@ pub async fn subscribe_opml(
                         password: None,
                         password_env: None,
                         category: None,
+                        listed: false,
                     },
                 );
                 grew = true;
@@ -947,12 +1043,42 @@ async fn list(ctx: &Ctx) -> Result<()> {
     Ok(())
 }
 
+/// Removes feeds nobody subscribes to that are dead (failing for 30 days) or quiet (nothing new
+/// in a year), from the catalogue and the database, so the Directory lists feeds worth taking.
+/// A feed listed in it with no subscribers stays for as long as it works and publishes.
+async fn clean_directory(ctx: &Ctx) -> Result<()> {
+    const DEAD: i64 = 30 * 86_400;
+    const QUIET: i64 = 365 * 86_400;
+    let now = db::now();
+    let stale = ctx.db.stale_unsubscribed(now - DEAD, now - QUIET).await?;
+    if stale.is_empty() {
+        return Ok(());
+    }
+    let mut cfg = (*ctx.cfg()).clone();
+    if stale.iter().fold(false, |any, id| cfg.feeds.remove(id).is_some() || any) {
+        ctx.store_cfg(cfg).await?;
+    }
+    for id in &stale {
+        retire_group(ctx, id).await?;
+        ctx.db.forget_feed(id).await?;
+        tracing::info!(feed = id, "removed a feed nobody subscribes to that is dead or has published nothing in a year");
+    }
+    Ok(())
+}
+
 /// `standalone` false means this is the sweep that runs before a scan: it reports what it
 /// deleted, but must not emit the terminal ReapDone, or a client waiting on its `fetch`
 /// would stop reading before the scan had even started.
 #[tracing::instrument(name = "reap", skip_all, fields(dry_run = dry_run))]
 async fn reap(ctx: &Ctx, dry_run: bool, standalone: bool) -> Result<()> {
     let r = retention::run(&ctx.cfg(), &ctx.db, dry_run).await?;
+    if !dry_run {
+        clean_directory(ctx).await?;
+        let gone = art::trim(&art::dir(), ctx.cfg().general.art_cache_mb * 1_048_576);
+        if gone > 0 {
+            tracing::debug!(gone, "trimmed the artwork kept on disk");
+        }
+    }
     for c in r.aged_out.iter().chain(r.over_quota.iter()) {
         ctx.out.emit(Event::Reaped {
             path: c.path.clone(),
@@ -979,15 +1105,17 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
         anyhow::bail!("no feed with id {id:?}");
     }
 
+    let subscribed = ctx.db.subscriber_counts().await?;
     let mut scanned = 0;
     let mut fresh: Vec<String> = vec![];
     let in_scope = |s: &Sub| {
         scope.is_empty() || scope.contains(&s.id) || s.cfg.group.as_ref().is_some_and(|g| scope.contains(g))
     };
+    let mut states = ctx.db.http_states().await?;
     let mut due = vec![];
     for sub in subs.iter().filter(|s| only.is_none_or(|o| o == s.id) && in_scope(s)) {
         let id = &sub.id;
-        let mut state = ctx.db.http_state(id).await?;
+        let mut state = states.remove(id).unwrap_or_default();
         // A scan someone asked for reads the feed in full. With the validators it only skipped the
         // wait: a feed that had not changed answered 304 and nothing was read (#77).
         if force {
@@ -995,8 +1123,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
             state.last_modified = None;
         }
 
-        if !force && let Some(last) = state.last_checked {
+        if !force && let Some(at) = ctx.due(&cfg, &sub.cfg, id, &state, subscribed.contains_key(id)) {
-            let at = last + due_after(&cfg, &sub.cfg, state.ttl_mins, state.error_since, last) as i64;
             if at > db::now() {
                 ctx.out.emit(Event::FeedSkip {
                     feed: id.clone(),
@@ -1015,7 +1142,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
     const AHEAD: usize = 6;
     let mut bodies = futures_util::StreamExt::buffered(
         futures_util::stream::iter(due.iter().map(|(sub, state)| {
-            let (client, feed_cfg) = (ctx.client.clone(), sub.cfg.clone());
+            let (client, feed_cfg) = (ctx.feed_client.clone(), sub.cfg.clone());
             let (etag, modified) = (state.etag.clone(), state.last_modified.clone());
             let early = !feed::is_patreon_creator(&feed_cfg.url);
             tokio::spawn(tracing::Instrument::instrument(
@@ -1068,7 +1195,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
                 // One bad feed must not end the scan.
                 let msg = format!("{e:#}");
                 ctx.out.emit(Event::FeedError { feed: id.clone(), msg: msg.clone() });
-                ctx.db.set_feed_error(id, &feed_cfg.url, &msg).await?;
+                ctx.feed_failed(id, &feed_cfg.url, &msg).await?;
             }
         }
     }
@@ -1094,7 +1221,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
                 Err(e) => {
                     let msg = format!("{e:#}");
                     ctx.out.emit(Event::FeedError { feed: id.clone(), msg: msg.clone() });
-                    ctx.db.set_feed_error(id, &feed_cfg.url, &msg).await?;
+                    ctx.feed_failed(id, &feed_cfg.url, &msg).await?;
                 }
             }
         }
@@ -1156,6 +1283,7 @@ pub async fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
                 password: parent.and_then(|p| p.password.clone()),
                 password_env: parent.and_then(|p| p.password_env.clone()),
                 category: None,
+                listed: false,
             },
             managed: true,
         });
@@ -1206,6 +1334,49 @@ async fn retire_stranded(ctx: &Ctx) -> Result<usize> {
 
 /// Seconds to wait before re-checking a feed.
 ///
+/// When a feed is next due, or None for one never checked, which is due now. A feed nobody
+/// subscribes to, one listed in the Directory, is read once a day: enough to keep its entry
+/// current, without fetching it hourly for no one.
+pub fn due_at(cfg: &config::Config, feed: &config::Feed, state: &db::HttpState, subscribed: bool) -> Option<i64> {
+    let last = state.last_checked?;
+    let floor = if subscribed { 0 } else { 86_400 };
+    Some(last + due_after(cfg, feed, state.ttl_mins, state.error_since, last).max(floor) as i64)
+}
+
+/// How long the daemon may sleep before a feed is due: until the earliest one, at least 30 s and
+/// at most 10 minutes. It ticked every minute and ran a scan pass each time, 80% of them finding
+/// nothing due (#114). The floor keeps a feed that never gets a check time from spinning it; the
+/// ceiling picks up within ten minutes what no command announces, such as `ipx add` or a
+/// shorter schedule. A command, a refresh or a feed added on the page, wakes it at once anyway.
+async fn until_next_scan(ctx: &Ctx) -> std::time::Duration {
+    let next = async {
+        let cfg = ctx.cfg();
+        let subscribed = ctx.db.subscriber_counts().await?;
+        let states = ctx.db.http_states().await?;
+        anyhow::Ok(
+            subscriptions(ctx)
+                .await?
+                .iter()
+                // A feed with no row yet has never been checked: due now.
+                .map(|s| {
+                    let never = db::HttpState::default();
+                    let state = states.get(&s.id).unwrap_or(&never);
+                    ctx.due(&cfg, &s.cfg, &s.id, state, subscribed.contains_key(&s.id)).unwrap_or(0)
+                })
+                .min(),
+        )
+    };
+    let wait = match next.await {
+        Ok(Some(at)) => (at - db::now()).max(0) as u64,
+        Ok(None) => u64::MAX,
+        Err(e) => {
+            tracing::warn!(error = %format!("{e:#}"), "could not work out when the next feed is due");
+            0
+        }
+    };
+    std::time::Duration::from_secs(wait.clamp(30, 600))
+}
+
 /// A per-feed schedule is an explicit instruction and wins outright. Without one, the
 /// global schedule applies, but the feed's own <ttl> raises it when the publisher asks to
 /// be polled less often. A feed that is failing backs off (`backoff`), from `error_since`, when
@@ -1257,6 +1428,24 @@ enum Outcome {
     Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
 }
 
+/// A feed that answered from a new address after permanent redirects moves there in the
+/// catalogue, so it is read from there and no longer redirected every time. One an OPML lists
+/// is left alone, since the OPML would only put the old address back; so is a move onto an
+/// address another feed already has.
+async fn follow_move(ctx: &Ctx, id: &str, to: &str) -> Result<()> {
+    // Said in the event below, not logged here: the event is the log line.
+    if subscriptions(ctx).await?.iter().any(|s| s.id != id && feed::same_feed(&s.cfg.url, to)) {
+        tracing::info!(feed = id, to, "the feed moved to an address another feed already has; leaving it");
+        return Ok(());
+    }
+    let mut cfg = (*ctx.cfg()).clone();
+    let Some(f) = cfg.feeds.get_mut(id) else { return Ok(()) };
+    let from = std::mem::replace(&mut f.url, to.to_owned());
+    ctx.store_cfg(cfg).await?;
+    ctx.out.emit(Event::FeedMoved { feed: id.to_owned(), from, to: to.to_owned() });
+    Ok(())
+}
+
 #[tracing::instrument(name = "feed", skip_all, fields(feed = id))]
 async fn scan_one(
     ctx: &Arc<Ctx>,
@@ -1264,9 +1453,13 @@ async fn scan_one(
     feed_cfg: &config::Feed,
     state: &db::HttpState,
     force: bool,
-    // The body the scan already fetched for it, if it did (#104).
+    // The body the scan already fetched for it, if it did (#104), and where it moved, if it did.
-    prefetched: Option<Result<feed::Fetched>>,
+    prefetched: Option<Result<(feed::Fetched, Option<String>)>>,
 ) -> Result<Outcome> {
+    // The queue follows the feed's settings each time it is due, changed or not, so files a
+    // limit or auto-download no longer reaches stop counting as waiting (#113).
+    let policy = policy_for(ctx, id, feed_cfg).await?;
+    ctx.db.hold_back(id, if policy.auto_download { policy.budget } else { 0 }).await?;
     // A Patreon creator with more than one show is a list of feeds, like an OPML.
     if feed::is_patreon_creator(&feed_cfg.url) {
         match feed::patreon_shows(&ctx.client, &feed_cfg.url).await {
@@ -1292,10 +1485,13 @@ async fn scan_one(
         }
     }
 
-    let mut fetched = match prefetched {
+    let (mut fetched, moved) = match prefetched {
         Some(got) => got?,
-        None => feed::fetch(&ctx.client, feed_cfg, state.etag.as_deref(), state.last_modified.as_deref()).await?,
+        None => feed::fetch(&ctx.feed_client, feed_cfg, state.etag.as_deref(), state.last_modified.as_deref()).await?,
     };
+    if let Some(to) = moved {
+        follow_move(ctx, id, &to).await?;
+    }
 
     // A 304 while nothing is stored means the validator has outlived the data -- a restore
     // from backup, a manual edit, a cleanup that removed entries. Believe the database over
@@ -1306,7 +1502,7 @@ async fn scan_one(
     if matches!(fetched, feed::Fetched::NotModified) && (stored.entries == 0 || stored.image.is_none()) {
         tracing::info!(feed = id, "not modified, but nothing stored; refetching without the validator");
         ctx.db.clear_validators(id).await?;
-        fetched = feed::fetch(&ctx.client, feed_cfg, None, None).await?;
+        fetched = feed::fetch(&ctx.feed_client, feed_cfg, None, None).await?.0;
     }
 
     let (bytes, etag, last_modified) = match fetched {
@@ -1318,6 +1514,13 @@ async fn scan_one(
     };
 
     if bytes.iter().all(u8::is_ascii_whitespace) {
+        // From a feed that has published, nothing new, said badly (see Outcome::Empty). From one
+        // that has never stored an item, a feed that is not there: a lapsed domain behind a DNS
+        // filter's block page answers 200 and nothing, and looked like a show that had not
+        // posted yet, with no error to act on (#119). The next real read clears it.
+        if stored.entries == 0 {
+            anyhow::bail!("the feed's address answered with nothing at all");
+        }
         ctx.db.touch_feed(id, &feed_cfg.url).await?;
         return Ok(Outcome::Empty);
     }
@@ -1330,6 +1533,17 @@ async fn scan_one(
     }
 
     let mut parsed = feed::parse(&bytes)?;
+    // Artwork on http moves to https where its host serves it (#110), before it is compared
+    // with what is stored, which is then the https address too.
+    let mut secure = std::collections::HashMap::new();
+    if let Some(art) = &parsed.image {
+        parsed.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
+    }
+    for entry in parsed.entries.iter_mut() {
+        if let Some(art) = &entry.image {
+            entry.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
+        }
+    }
     // Artwork is looked at when it may have changed: the feed names different artwork from what
     // is stored, or someone asked for a refresh, so an icon the site changes or fixes still
     // follows it (#80). Looked at on every full read, a feed without validators asked its site
@@ -1354,6 +1568,20 @@ async fn scan_one(
             });
         }
     }, art_span).await;
+    // A site's icon comes back on whatever the site is on.
+    if let Some(art) = &parsed.image {
+        parsed.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
+    }
+    // Items stored before, which a scan does not write again, move with their host, and a host
+    // only they still name is asked too.
+    for art in ctx.db.http_images(id).await? {
+        feed::prefer_https(&ctx.client, &art, &mut secure).await;
+    }
+    for (host, ok) in &secure {
+        if *ok {
+            ctx.db.secure_images(id, host).await?;
+        }
+    }
     ctx.db.record_feed(
         id,
         &feed_cfg.url,
@@ -1363,9 +1591,31 @@ async fn scan_one(
         parsed.ttl_mins,
         parsed.image.as_deref(),
         parsed.category.as_deref(),
+        parsed.description.as_deref(),
     ).await?;
+    // Asked on a feed's first read, nothing of it stored yet, or when someone forces a refresh:
+    // each call costs, and a feed already here without a category can wait for a forced one.
+    // The answer becomes the catalogue's category, so a feed is not asked twice.
+    // A feed an OPML lists is not in the catalogue and has nowhere to keep one.
+    if parsed.category.is_none()
+        && (force || stored.entries == 0)
+        && !parsed.entries.is_empty()
+        && ctx.cfg().feeds.get(id).is_some_and(|f| f.category.is_none())
+        && let Ok(key) = std::env::var("TYPESAFE_KEY")
+    {
+        match feed::classify(&ctx.client, &key, &parsed).await {
+            Ok(category) => {
+                let mut cfg = (*ctx.cfg()).clone();
+                if let Some(f) = cfg.feeds.get_mut(id).filter(|f| f.category.is_none()) {
+                    tracing::info!(feed = id, category = %category, "categorised");
+                    f.category = Some(category);
+                    ctx.store_cfg(cfg).await?;
+                }
+            }
+            Err(e) => tracing::warn!(feed = id, "could not categorise: {e:#}"),
+        }
+    }
 
-    let policy = policy_for(ctx, id, feed_cfg).await?;
     if let Some(parent) = &feed_cfg.group {
         let listed: Vec<(&str, &str)> = parsed
             .entries
@@ -1380,13 +1630,17 @@ async fn scan_one(
     // changed nothing however often the feed was scanned.
     let skipped = ctx.db.skipped_by_filter(id).await?;
     let (known_items, known_files) = ctx.db.stored_items(id).await?;
+    let changed = ctx.db.changed_items(id, &parsed.entries).await?;
     let mut scan = Scan::default();
     // Its own span: the time a feed spends after its fetch was untraced (#96).
     let store = tracing::info_span!("store", items = parsed.entries.len());
     tracing::Instrument::instrument(async {
         for entry in &parsed.entries {
-            // Only what is not stored yet is inserted; the insert would find the rest and do nothing.
+            // What is not stored yet is inserted, and what the feed has changed since is written
-            if !known_items.contains(&entry.guid) && ctx.db.record_entry(id, entry).await? {
+            // again (#141); the rest is left alone. A correction keeps the item's read state.
+            if (!known_items.contains(&entry.guid) || changed.contains(&entry.guid))
+                && ctx.db.record_entry(id, entry).await?
+            {
                 scan.new_entries += 1;
             }
             for enc in &entry.enclosures {
@@ -1412,6 +1666,9 @@ async fn scan_one(
     if scan.new_entries > 0 {
         ctx.db.rehide(id).await?;   // what is new may hold someone's blocked words
     }
+    // A new item takes a place among the newest and the oldest of them leaves the queue; a file a
+    // filter lets through again may be outside them.
+    ctx.db.hold_back(id, if policy.auto_download { policy.budget } else { 0 }).await?;
 
     let budget = policy.budget;
     if policy.auto_download && budget > 0 {
@@ -1666,7 +1923,10 @@ fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Pol
 
     if subs.is_empty() {
         return Policy {
-            auto_download: feed_cfg.auto_download,
+            // A feed listed in the Directory with nobody subscribed is there to be found, not
+            // downloaded: its files would be for no one. Once someone subscribes, the files
+            // skipped for it are judged again on the next scan, by their settings (#107).
+            auto_download: feed_cfg.auto_download && !feed_cfg.listed,
             allow_explicit: feed_cfg.allow_explicit,
             wants: vec![Want { keywords: feed_cfg.keywords.clone(), blocked: vec![] }],
             budget: cap(feed_cfg.max_new_per_check),
@@ -1899,6 +2159,26 @@ fn duration(secs: u64) -> String {
 mod tests {
     use super::*;
 
+    #[test]
+    fn a_feed_is_due_after_its_wait_and_daily_with_nobody_subscribed() {
+        let cfg = config::Config::default();
+        let f = feed();
+        let at = |last: Option<i64>, subscribed| {
+            due_at(&cfg, &f, &db::HttpState { last_checked: last, ..Default::default() }, subscribed)
+        };
+        assert_eq!(at(None, true), None);                           // never checked: due now
+        assert_eq!(at(Some(1000), true), Some(1000 + 3600));        // the hourly default
+        assert_eq!(at(Some(1000), false), Some(1000 + 86_400));     // listed, nobody subscribed
+    }
+
+    #[test]
+    fn a_listed_feed_nobody_subscribes_to_downloads_nothing() {
+        let mut f = feed();
+        assert!(merge_policy(&[], &f, 3).auto_download);
+        f.listed = true;
+        assert!(!merge_policy(&[], &f, 3).auto_download);
+    }
+
     #[test]
     fn a_failing_feed_backs_off_doubling_up_to_a_day() {
         let hour = 3600;
@@ -1969,6 +2249,7 @@ mod tests {
             password: None,
             password_env: None,
             category: None,
+            listed: false,
         }
     }
 
@@ -2028,14 +2309,106 @@ mod tests {
             cfg: std::sync::RwLock::new(Arc::new(cfg)),
             db: db::Db::memory().await.unwrap(),
             client: reqwest::Client::new(),
+            feed_client: reqwest::Client::builder().redirect(reqwest::redirect::Policy::none()).build().unwrap(),
             out: Emitter::terminal(),
             torrents: tokio::sync::OnceCell::new(),
             torrent_slots: Arc::new(tokio::sync::Semaphore::new(2)),
             config_path: PathBuf::new(),
             detach_torrents: false,
+            retry: Default::default(),
         }
     }
 
+    #[tokio::test]
+    async fn a_moments_lookup_failure_is_tried_again_soon_and_not_held_against_the_feed() {
+        let ctx = test_ctx(config::Config::default()).await;
+        let blip = "connecting: dns error: failed to lookup address information: Temporary failure in name resolution";
+        ctx.feed_failed("npr", "https://feeds.npr.org/x", blip).await.unwrap();
+        // Nothing on the feed: no error, no run of failures to back off from or call it gone by.
+        let s = ctx.db.feed_summary("npr").await.unwrap();
+        assert!(s.last_error.is_none() && s.error_since.is_none(), "{:?}", s.last_error);
+        // Due again in minutes, though a feed nobody subscribes to is otherwise read once a day.
+        let state = db::HttpState { last_checked: Some(db::now() - 2 * 86_400), ..Default::default() };
+        let due = ctx.due(&config::Config::default(), &feed(), "npr", &state, false).unwrap();
+        assert!((due - db::now() - RETRY_LOOKUP_SECS).abs() <= 2, "{due}");
+        // A real failure is the feed's.
+        ctx.feed_failed("gone", "https://x/feed", "HTTP 404 Not Found").await.unwrap();
+        assert_eq!(ctx.db.feed_summary("gone").await.unwrap().last_error.as_deref(), Some("HTTP 404 Not Found"));
+    }
+
+    #[tokio::test]
+    async fn a_reload_takes_the_catalogue_another_ipx_wrote() {
+        let with = |id: &str| {
+            let mut cfg = config::Config::default();
+            cfg.feeds.insert(id.into(), config::Feed { url: format!("http://x/{id}.xml"), ..feed() });
+            cfg
+        };
+        let held = with("kept-in-memory");
+        let ctx = test_ctx(held.clone()).await;
+        ctx.db.store_config(&held).await.unwrap();
+        // `ipx add` and `ipx rm`, from a process of their own (#120).
+        ctx.db.store_config(&with("added-by-the-cli")).await.unwrap();
+        reload(&ctx).await.unwrap();
+        let urls: Vec<String> = ctx.cfg().feeds.values().map(|f| f.url.clone()).collect();
+        assert_eq!(urls, vec!["http://x/added-by-the-cli.xml".to_string()]);
+    }
+
+    /// Answers /empty with a 200 and nothing, /fixed with a feed of one item as it should read,
+    /// anything else with the same item's text broken.
+    async fn empty_or_feed_server() -> String {
+        use tokio::io::{AsyncReadExt, AsyncWriteExt};
+        let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let addr = listener.local_addr().unwrap();
+        tokio::spawn(async move {
+            loop {
+                let Ok((mut sock, _)) = listener.accept().await else { return };
+                tokio::spawn(async move {
+                    let mut buf = [0u8; 2048];
+                    let n = sock.read(&mut buf).await.unwrap_or(0);
+                    let req = String::from_utf8_lossy(&buf[..n]).into_owned();
+                    let body = if req.starts_with("GET /empty") {
+                        "\n"
+                    } else if req.starts_with("GET /fixed") {
+                        "<?xml version=\"1.0\"?><rss version=\"2.0\"><channel><title>T</title><item><title>One</title><guid>g1</guid><description>As it should read.</description></item></channel></rss>"
+                    } else {
+                        "<?xml version=\"1.0\"?><rss version=\"2.0\"><channel><title>T</title><item><title>One</title><guid>g1</guid><description>Somebody else's text.</description></item></channel></rss>"
+                    };
+                    let resp = format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n{body}", body.len());
+                    let _ = sock.write_all(resp.as_bytes()).await;
+                });
+            }
+        });
+        format!("http://{addr}")
+    }
+
+    #[tokio::test]
+    async fn a_publishers_correction_reaches_an_item_already_stored() {
+        let base = empty_or_feed_server().await;
+        let ctx = Arc::new(test_ctx(config::Config::default()).await);
+        let at = |path: &str| config::Feed { url: format!("{base}{path}"), ..feed() };
+        let none = db::HttpState::default();
+        let text = || ctx.db.strings_for_test("SELECT description FROM entries WHERE feed_id = 'aicn'");
+        scan_one(&ctx, "aicn", &at("/feed"), &none, false, None).await.unwrap();
+        assert_eq!(text().await, ["Somebody else's text."]);
+        // The feed fixed, the item already stored: it was left as it was (#141).
+        scan_one(&ctx, "aicn", &at("/fixed"), &none, false, None).await.unwrap();
+        assert_eq!(text().await, ["As it should read."]);
+    }
+
+    #[tokio::test]
+    async fn nothing_from_a_feed_that_never_posted_is_an_error_and_from_one_that_has_is_not() {
+        let base = empty_or_feed_server().await;
+        let ctx = Arc::new(test_ctx(config::Config::default()).await);
+        let at = |path: &str| config::Feed { url: format!("{base}{path}"), ..feed() };
+        let none = db::HttpState::default();
+        // A lapsed domain behind a block page: never an item, and nothing (#119).
+        let err = scan_one(&ctx, "gone", &at("/empty"), &none, false, None).await.err().expect("an error");
+        assert!(format!("{err:#}").contains("answered with nothing"), "{err:#}");
+        // A feed that has posted answering nothing has nothing new, as the Antarctic Survey does.
+        assert!(matches!(scan_one(&ctx, "quiet", &at("/feed"), &none, false, None).await, Ok(Outcome::Feed(_))));
+        assert!(matches!(scan_one(&ctx, "quiet", &at("/empty"), &none, false, None).await, Ok(Outcome::Empty)));
+    }
+
     #[tokio::test]
     async fn a_derived_feed_is_not_scanned_once_its_opml_leaves_config() {
         // davewiner: the OPML subscription left config.toml, but its 922 derived rows

src/web.rs

@@ -38,6 +38,8 @@ pub fn router(state: WebState) -> Router {
     Router::new()
         .route("/", get(index))
         .route("/api/me", get(me).patch(patch_me))
+        .route("/api/tokens", get(list_tokens).post(add_token))
+        .route("/api/tokens/{id}", axum::routing::delete(remove_token))
         .route("/api/logout", post(logout))
         .route("/api/feeds", get(feeds).post(add_feed))
         .route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
@@ -56,6 +58,7 @@ pub fn router(state: WebState) -> Router {
         .route("/api/status", get(status))
         .route("/api/popular", get(get_popular))
         .route("/api/directory", get(get_directory))
+        .route("/api/directory/{id}", get(get_listed))
         .route("/api/popular/{id}", post(subscribe_popular))
         .route("/api/users", get(list_users).post(add_user))
         .route("/api/users/{id}", patch(patch_user).delete(remove_user))
@@ -76,6 +79,8 @@ pub fn router(state: WebState) -> Router {
         .route("/favicon.png", get(favicon))
         .route("/favicon-dark.png", get(favicon_dark))
         .route("/apple-touch-icon.png", get(touch_icon))
+        // iOS asks for this one first when the site is added to a home screen (#109).
+        .route("/apple-touch-icon-precomposed.png", get(touch_icon))
         .route("/app.js", get(app_js))
         .route("/app.css", get(app_css))
         .route("/login.js", get(login_js))
@@ -98,8 +103,8 @@ pub async fn serve(state: WebState, bind: &str) -> Result<()> {
     .context("serving the web ui")
 }
 
-/// Who is asking, in order of how specific the claim is: a proxy that vouches for a name,
+/// Who is asking, in order of how specific the claim is: a proxy that vouches for a name, an
-/// a session cookie, then the shared token (which is the admin).
+/// API token, a session cookie, then the shared token (which is the admin).
 ///
 /// Any of them has to survive being put in a cookie: an `<audio src>` request is issued by
 /// the browser, and there is no way to attach a header to it.
@@ -142,6 +147,26 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
     }
     let by_proxy = user.is_some();
 
+    // An API token, made by its owner in Settings, for a script or an agent acting as them
+    // (#123). One that is not known is turned away rather than tried as anything else: a
+    // script sent with a revoked token should hear so, not fall through to a cookie.
+    if user.is_none()
+        && let Some(bearer) = req
+            .headers()
+            .get(header::AUTHORIZATION)
+            .and_then(|v| v.to_str().ok())
+            .and_then(|v| v.strip_prefix("Bearer "))
+    {
+        match state.ctx.db.api_token_user(&crate::auth::api_token_hash(bearer.trim())).await {
+            Ok(Some(u)) => user = Some(u),
+            Ok(None) => return (StatusCode::UNAUTHORIZED, "unknown or revoked API token").into_response(),
+            Err(e) => {
+                tracing::error!(error = %e, "looking up an API token");
+                return StatusCode::INTERNAL_SERVER_ERROR.into_response();
+            }
+        }
+    }
+
     // 2. A session cookie from signing in here.
     if user.is_none() {
         if let Some(sid) = cookie(&req, SESSION_COOKIE) {
@@ -290,11 +315,8 @@ async fn login(
 ) -> Result<Response, ApiError> {
     let name = body.name.trim().to_ascii_lowercase();
     let user = state.ctx.db.user_by_name(&name).await?;
-    // The same answer either way: whether a name exists is not something to leak.
+    // The same answer either way, in the same time: whether a name exists is not something to leak.
-    let ok = user
+    let ok = crate::auth::check_password(body.password, user.as_ref().and_then(|u| u.pass_hash.clone())).await;
-        .as_ref()
-        .and_then(|u| u.pass_hash.as_deref())
-        .is_some_and(|h| crate::auth::verify_password(&body.password, h));
     if !ok {
         tracing::warn!(user = %name, "failed sign-in");
         return Ok((StatusCode::UNAUTHORIZED, "wrong name or password").into_response());
@@ -390,6 +412,54 @@ fn last_admin(users: &[crate::db::User], id: i64) -> bool {
     admins == [id]
 }
 
+/// Your API tokens: names and dates, never the tokens, which are not kept.
+async fn list_tokens(
+    State(state): State<WebState>,
+    user: crate::db::User,
+) -> Result<Json<serde_json::Value>, ApiError> {
+    let tokens: Vec<_> = state
+        .ctx
+        .db
+        .api_tokens(user.id).await?
+        .iter()
+        .map(|t| serde_json::json!({ "id": t.id, "name": t.name, "created": t.created, "last_used": t.last_used }))
+        .collect();
+    Ok(Json(serde_json::json!(tokens)))
+}
+
+#[derive(Deserialize)]
+struct NewToken {
+    name: String,
+}
+
+/// Makes an API token that acts as you, and answers with it: the only time it is shown.
+async fn add_token(
+    State(state): State<WebState>,
+    user: crate::db::User,
+    Json(body): Json<NewToken>,
+) -> Result<Json<serde_json::Value>, ApiError> {
+    let name = body.name.trim();
+    if name.is_empty() {
+        return Err(ApiError::bad_request("give the token a name, to know it by when revoking it"));
+    }
+    let token = crate::auth::new_api_token();
+    let id = state.ctx.db.create_api_token(user.id, name, &crate::auth::api_token_hash(&token)).await?;
+    tracing::info!(user = %user.name, token = name, "API token made");
+    Ok(Json(serde_json::json!({ "id": id, "name": name, "token": token })))
+}
+
+async fn remove_token(
+    State(state): State<WebState>,
+    user: crate::db::User,
+    Path(id): Path<i64>,
+) -> Result<StatusCode, ApiError> {
+    if !state.ctx.db.delete_api_token(user.id, id).await? {
+        return Err(ApiError::bad_request(format!("you have no API token with id {id}")));
+    }
+    tracing::info!(user = %user.name, id, "API token revoked");
+    Ok(StatusCode::NO_CONTENT)
+}
+
 async fn list_users(
     State(state): State<WebState>,
     user: crate::db::User,
@@ -859,8 +929,10 @@ struct PopularRow {
     subscribers: i64,
     /// Yours already. Everyone counts, you included, so your own feeds are listed too.
     subscribed: bool,
-    /// The feed's own iTunes category, if it names one; most blogs do not.
+    /// Apple's category, from the feed's own iTunes category or else the catalogue's, and the
+    /// subcategory within it where there is one: the Directory browses them in that order.
     category: Option<String>,
+    subcategory: Option<String>,
     /// Any audio or video enclosure. Unlike category, every feed has an answer, so the
     /// Directory's Podcasts and Blogs between them hold everything.
     podcast: bool,
@@ -870,8 +942,13 @@ struct PopularRow {
 /// first. Popular is the top of it, the directory is all of it, and it is all that
 /// `subscribe_popular` will subscribe you to. An OPML or a Patreon creator is listed as the
 /// feeds inside it and never itself: both lists are for finding a show.
-async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
+/// The catalogue as others may see it. `everything` is the Directory: every feed, with or
+/// without subscribers, so the ones an admin listed show before anyone takes them. Without it,
+/// Popular: only what people subscribe to.
+async fn popular(state: &WebState, user_id: i64, everything: bool) -> Result<Vec<PopularRow>> {
     let db = &state.ctx.db;
+    // One pass for every feed's title, artwork and category, not three queries a feed.
+    let mut listed = db.feed_list(user_id, None).await?;
     let mine: std::collections::HashSet<String> =
         db.subscriptions_for(user_id).await?.into_iter().map(|s| s.feed_id).collect();
     let counts = db.subscriber_counts().await?;
@@ -886,23 +963,25 @@ async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
         let n = counts.get(&s.id).copied().unwrap_or(0);
         // A feed inside an OPML that looks private is as private as the OPML.
         let folder = s.cfg.group.as_deref().and_then(|g| by_id.get(g));
-        if n == 0
+        if (n == 0 && !everything)
             || is_folder.contains(s.id.as_str())
             || looks_private(&s.cfg)
             || folder.is_some_and(|f| looks_private(f))
         {
             continue;
         }
-        let sum = db.feed_summary(&s.id).await?;
+        let sum = listed.remove(&s.id).map(|l| l.summary).unwrap_or_default();
         let subscribed = mine.contains(&s.id);
+        // The feed's own wins; an admin's is for the feeds, mostly blogs, that name none.
+        let path = sum.category.or_else(|| s.cfg.category.clone()).map(|c| crate::feed::category_path(&c));
         out.push(PopularRow {
             id: s.id.clone(),
             title: sum.title,
             image: sum.image,
             subscribers: n,
             subscribed,
-            // The feed's own wins; an admin's is for the feeds, mostly blogs, that name none.
+            category: path.as_ref().map(|p| p.0.clone()),
-            category: sum.category.or_else(|| s.cfg.category.clone()),
+            subcategory: path.and_then(|p| p.1),
             podcast: media.contains(&s.id),
         });
     }
@@ -914,7 +993,7 @@ async fn get_popular(
     State(state): State<WebState>,
     user: crate::db::User,
 ) -> Result<Json<Vec<PopularRow>>, ApiError> {
-    let mut rows = popular(&state, user.id).await?;
+    let mut rows = popular(&state, user.id, false).await?;
     rows.truncate(10);
     Ok(Json(rows))
 }
@@ -924,11 +1003,74 @@ async fn get_directory(
     State(state): State<WebState>,
     user: crate::db::User,
 ) -> Result<Json<Vec<PopularRow>>, ApiError> {
-    let mut rows = popular(&state, user.id).await?;
+    let mut rows = popular(&state, user.id, true).await?;
     rows.sort_by_key(sort_name);
     Ok(Json(rows))
 }
 
+/// One of a listed feed's latest items, for its page in the Directory before anyone subscribes
+/// (#128): what it is, never its files, whose addresses are where a key would be.
+#[derive(Serialize)]
+struct ListedItem {
+    guid: String,
+    title: Option<String>,
+    link: Option<String>,
+    published: Option<i64>,
+    duration: Option<i64>,
+    description: Option<String>,
+}
+
+#[derive(Serialize)]
+struct Listed {
+    /// What the feed says it is (#130), sanitized as its items are.
+    description: Option<String>,
+    /// Why its last check failed, when it did (#142), in the words a feed's own page uses.
+    failing: Option<String>,
+    items: Vec<ListedItem>,
+}
+
+/// A feed's last error as its Directory page says it: explained once it has been failing a day, as
+/// the feed's own page does, or else that the check failed; never the error as it stands, which can
+/// name the feed's address, and the Directory names none. A day, so that a site down for an hour,
+/// or a resolver for a minute, is not called gone (#143).
+fn failure_words(err: &str, failing_for: i64) -> String {
+    Some(err)
+        .filter(|_| failing_for >= FLAG_AFTER_SECS)
+        .and_then(crate::feed::explain_failure)
+        .map_or_else(|| "Its last check failed.".to_owned(), |f| f.reason.to_owned())
+}
+
+/// A listed feed's description and latest twenty items. Only a feed the Directory lists, so a
+/// guessed id reaches nothing private, as `subscribe_popular` checks.
+async fn get_listed(
+    State(state): State<WebState>,
+    user: crate::db::User,
+    Path(id): Path<String>,
+) -> Result<Json<Listed>, ApiError> {
+    if !popular(&state, user.id, true).await?.iter().any(|p| p.id == id) {
+        return Err(ApiError::not_found(format!("{id:?} is not in the directory")));
+    }
+    let order = crate::db::order_sql("published", "desc");
+    let rows = state.ctx.db.entries_in(user.id, Some(&id), crate::db::Filter::parse("all"), None, 0, 20, &order).await?;
+    let mut sanitizer = feed_sanitizer();
+    let description = state.ctx.db.feed_description(&id).await?.map(|d| clean_description(&mut sanitizer, &d, None));
+    let summary = state.ctx.db.feed_summary(&id).await?;
+    let failing_for = summary.error_since.map_or(0, |since| crate::db::now() - since);
+    let failing = summary.last_error.as_deref().map(|e| failure_words(e, failing_for));
+    let items = rows
+        .into_iter()
+        .map(|e| ListedItem {
+            description: e.description.as_deref().map(|d| clean_description(&mut sanitizer, d, e.link.as_deref())),
+            guid: e.guid,
+            title: e.title,
+            link: e.link,
+            published: e.published,
+            duration: e.duration,
+        })
+        .collect();
+    Ok(Json(Listed { description, failing, items }))
+}
+
 fn sort_name(p: &PopularRow) -> String {
     p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase()
 }
@@ -940,10 +1082,12 @@ async fn subscribe_popular(
     user: crate::db::User,
     Path(id): Path<String>,
 ) -> Result<Json<serde_json::Value>, ApiError> {
-    if !popular(&state, user.id).await?.iter().any(|p| p.id == id) {
+    if !popular(&state, user.id, true).await?.iter().any(|p| p.id == id) {
         return Err(ApiError::bad_request(format!("{id:?} is not in the directory")));
     }
     state.ctx.db.subscribe(user.id, &id).await?;
+    // A listed feed nobody took was checked once a day at most: read it now, not in an hour.
+    scan_soon(&state, Some(id.clone())).await;
     Ok(Json(serde_json::json!({ "id": id })))
 }
 
@@ -1018,6 +1162,18 @@ impl IntoResponse for ApiError {
 mod tests {
     use super::*;
 
+    #[test]
+    fn a_failing_feed_says_why_in_the_directory_without_its_address() {
+        let day = FLAG_AFTER_SECS;
+        let nothing = "the feed's address answered with nothing at all";
+        assert_eq!(failure_words(nothing, day), "This address answers with nothing; the site may be gone.");
+        // Not before it has been failing a day, as on the feed's own page (#143).
+        assert_eq!(failure_words(nothing, 3600), "Its last check failed.");
+        // reqwest names the URL; the Directory names none.
+        let raw = "error sending request for url (http://daily-quests.com/comic/?feed=atom): connection closed";
+        assert_eq!(failure_words(raw, day), "Its last check failed.");
+    }
+
     #[test]
     fn a_relative_image_resolves_against_the_post() {
         let mut b = feed_sanitizer();
@@ -1073,6 +1229,7 @@ mod tests {
             password: None,
             password_env: None,
             category: None,
+            listed: false,
         };
         assert!(!looks_private(&f("https://feeds.twit.tv/twit.xml")));
         assert!(!looks_private(&f("https://example.com/rss?format=mp3")));
@@ -1119,7 +1276,7 @@ mod tests {
         crate::config::Feed {
             url: url.into(), folder: None, group: None, media_types: None, schedule: None, keywords: vec![], allow_explicit: false,
             auto_download: true, max_new_per_check: None, username: None,
-            password: None, password_env: None, category: None,
+            password: None, password_env: None, category: None, listed: false,
         }
     }
 
@@ -1201,9 +1358,22 @@ async fn entries(
     user: crate::db::User,
     Query(page): Query<Page>,
 ) -> Result<Json<EntryPage>, ApiError> {
+    subscribed(&state, &user, &id).await?;
     entry_page(&state, user.id, Some(&id), &page).await
 }
 
+/// A feed's items and files are for the people who subscribe to it (#129). Anyone signed in
+/// could read any feed's by its id, a paid one's included, with its enclosure addresses, which
+/// can carry the subscriber's key, and fetch, queue or delete its files by theirs. A feed in
+/// an OPML has a subscription of its own for everyone subscribed to the OPML, so this holds
+/// for those too. Not found, as for any other feed that is not yours.
+async fn subscribed(state: &WebState, user: &crate::db::User, feed_id: &str) -> Result<(), ApiError> {
+    match state.ctx.db.subscription(user.id, feed_id).await? {
+        Some(_) => Ok(()),
+        None => Err(ApiError::not_found("you do not subscribe to that feed")),
+    }
+}
+
 /// Every subscribed feed's items together, newest first: All Subscriptions.
 async fn all_entries(
     State(state): State<WebState>,
@@ -1505,8 +1675,12 @@ async fn remove_feed(
     }
 
     // Nobody is left: the feed stops being scanned. Its files and history stay, so if
-    // someone subscribes again they do not pull the back catalogue a second time.
+    // someone subscribes again they do not pull the back catalogue a second time. A feed an
+    // admin listed stays in the Directory, for the next person.
     let mut cfg = (*state.ctx.cfg()).clone();
+    if cfg.feeds.get(&id).is_some_and(|f| f.listed) {
+        return Ok(StatusCode::NO_CONTENT);
+    }
     if cfg.feeds.remove(&id).is_none() {
         // A derived feed: forget it here, though the OPML will list it again on the next
         // read unless you unsubscribe from the OPML itself.
@@ -1548,12 +1722,14 @@ async fn set_flags(
 async fn download_now(
     State(state): State<WebState>,
     Path(id): Path<i64>,
+    user: crate::db::User,
 ) -> Result<StatusCode, ApiError> {
     let enc = state
         .ctx
         .db
         .enclosure(id).await?
         .ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
+    subscribed(&state, &user, &enc.feed_id).await?;
     if enc.path.is_some() {
         return Ok(StatusCode::NO_CONTENT); // Already here.
     }
@@ -1583,6 +1759,7 @@ async fn delete_file(
         .db
         .enclosure(id).await?
         .ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
+    subscribed(&state, &user, &enc.feed_id).await?;
 
     // There is one copy of the file: deleting it deletes everyone's. Say so before doing
     // it, once, and let them decide.
@@ -1687,7 +1864,10 @@ async fn events(
 /// routine skip of a feed not due, dozens a minute that change nothing.
 fn changed_feed(ev: &Event) -> Option<&str> {
     match ev {
-        Event::FeedDone { feed, .. } | Event::FeedError { feed, .. } | Event::DownloadDone { feed, .. } => Some(feed),
+        Event::FeedDone { feed, .. }
+        | Event::FeedError { feed, .. }
+        | Event::FeedMoved { feed, .. }
+        | Event::DownloadDone { feed, .. } => Some(feed),
         Event::FeedSkip { feed, reason } if !reason.starts_with("not due") => Some(feed),
         _ => None,
     }
@@ -1697,11 +1877,15 @@ fn changed_feed(ev: &Event) -> Option<&str> {
 async fn media(
     State(state): State<WebState>,
     Path(id): Path<i64>,
+    user: crate::db::User,
     req: Request,
 ) -> Response {
     let Ok(Some(enc)) = state.ctx.db.enclosure(id).await else {
         return (StatusCode::NOT_FOUND, "no such enclosure").into_response();
     };
+    if let Err(e) = subscribed(&state, &user, &enc.feed_id).await {
+        return e.into_response();
+    }
     let Some(path) = enc.path else {
         return (StatusCode::NOT_FOUND, "not downloaded").into_response();
     };
@@ -1716,16 +1900,25 @@ struct ArtQuery {
     u: String,
 }
 
-/// Artwork a feed names on plain http, fetched here for the https page. The browser upgrades an
+/// Artwork, from iPX: the page asks here for every image a feed or item names, and the first
-/// http image on an https page to https, and a host with no https, such as The Secret Cabal's
+/// time it is fetched from the publisher and kept on disk (`art`, up to `art_cache_mb`), so
-/// CDN, then answers nothing (issue #90).
+/// later it is fast, the publisher is not asked on every visit, and it outlives the publisher's
+/// server. Only an address some feed or item names as its artwork, and only an image up to
+/// 5 MB, so the route cannot be pointed at anything else. It began as a way round http-only
+/// artwork on the https page (#90).
 async fn art(State(state): State<WebState>, Query(q): Query<ArtQuery>) -> Response {
     const MAX: usize = 5 << 20;
-    if !q.u.starts_with("http://") || !state.ctx.db.names_image(&q.u).await.unwrap_or(false) {
+    let web = q.u.starts_with("http://") || q.u.starts_with("https://");
+    if !web || !state.ctx.db.names_image(&q.u).await.unwrap_or(false) {
         return (StatusCode::NOT_FOUND, "no feed names that artwork").into_response();
     }
+    let keep = state.ctx.cfg().general.art_cache_mb > 0;
+    let file = crate::art::path(&q.u);
+    if keep && let Some((kind, body)) = crate::art::read(&file) {
+        return art_response(kind, body);
+    }
     let got = async {
-        let mut r = state.ctx.client.get(&q.u).send().await?.error_for_status()?;
+        let mut r = state.ctx.client.get(&q.u).timeout(crate::feed::FEED_TIMEOUT).send().await?.error_for_status()?;
         let kind = r.headers().get(header::CONTENT_TYPE).and_then(|v| v.to_str().ok()).unwrap_or("").to_owned();
         anyhow::ensure!(kind.starts_with("image/"), "not an image: {kind}");
         let mut body = Vec::new();
@@ -1737,12 +1930,30 @@ async fn art(State(state): State<WebState>, Query(q): Query<ArtQuery>) -> Respon
     };
     match got.await {
         Ok((kind, body)) => {
-            ([(header::CONTENT_TYPE, kind), (header::CACHE_CONTROL, "private, max-age=86400".into())], body).into_response()
+            if keep && let Err(e) = crate::art::write(&file, &kind, &body) {
+                tracing::debug!(error = %e, "could not keep the artwork");
+            }
+            art_response(kind, body)
         }
         Err(e) => (StatusCode::BAD_GATEWAY, format!("{e:#}")).into_response(),
     }
 }
 
+/// An image from someone else's server, served from iPX's own address: it must stay an image.
+/// An SVG opened on its own would otherwise run its script as iPX's page, with its cookies.
+fn art_response(kind: String, body: Vec<u8>) -> Response {
+    (
+        [
+            (header::CONTENT_TYPE, kind),
+            (header::CACHE_CONTROL, "private, max-age=2592000".into()),
+            (header::X_CONTENT_TYPE_OPTIONS, "nosniff".into()),
+            (header::CONTENT_SECURITY_POLICY, "default-src 'none'; style-src 'unsafe-inline'; sandbox".into()),
+        ],
+        body,
+    )
+        .into_response()
+}
+
 #[derive(Deserialize)]
 struct Position {
     secs: i64,
@@ -1804,8 +2015,10 @@ fn five() -> i64 {
 async fn download_latest(
     State(state): State<WebState>,
     Path(id): Path<String>,
+    user: crate::db::User,
     Json(body): Json<HowMany>,
 ) -> Result<Json<serde_json::Value>, ApiError> {
+    subscribed(&state, &user, &id).await?;
     let ids = state.ctx.db.undownloaded(&id, body.count.clamp(1, 100)).await?;
     for enc in &ids {
         state.ctx.db.requeue(*enc).await?;
@@ -1908,6 +2121,7 @@ struct Settings {
     download_dir: String,
     max_total_gb: f64,
     max_age_days: u64,
+    art_cache_mb: u64,
 }
 
 async fn get_settings(State(state): State<WebState>) -> Json<Settings> {
@@ -1920,6 +2134,7 @@ async fn get_settings(State(state): State<WebState>) -> Json<Settings> {
         download_dir: cfg.general.download_dir.display().to_string(),
         max_total_gb: cfg.general.max_total_gb,
         max_age_days: cfg.general.max_age_days,
+        art_cache_mb: cfg.general.art_cache_mb,
     })
 }
 
@@ -1930,6 +2145,7 @@ struct SettingsPatch {
     media_types: Option<Vec<String>>,
     max_total_gb: Option<f64>,
     max_age_days: Option<u64>,
+    art_cache_mb: Option<u64>,
 }
 
 async fn patch_settings(
@@ -1966,6 +2182,9 @@ async fn patch_settings(
     if let Some(v) = body.max_age_days {
         cfg.general.max_age_days = v;
     }
+    if let Some(v) = body.art_cache_mb {
+        cfg.general.art_cache_mb = v;
+    }
     state.ctx.store_cfg(cfg).await?;
     Ok(StatusCode::NO_CONTENT)
 }

tests/load/browse.js

@@ -0,0 +1,54 @@
+import http from 'k6/http';
+import { check, sleep } from 'k6';
+import { BASE, FEEDS, ITEMS, as, me, pick, feedId, p95 } from './lib.js';
+
+// An evening's browsing: twenty-five people at once opening their feed list, All Subscriptions,
+// a feed, a search, the Directory and a feed's page in it. Each answer's budget is well above what
+// it takes now, so it fails on a query that has started asking once per feed -- the Directory
+// asked the database three questions a feed until 0.10.0 -- not on a slow minute. Fifty measured
+// the queue for what was SQLite's one connection (#136) more than any query.
+export const options = {
+  scenarios: {
+    evening: {
+      executor: 'ramping-vus',
+      stages: [{ duration: '15s', target: 25 }, { duration: '45s', target: 25 }, { duration: '10s', target: 0 }],
+    },
+  },
+  thresholds: {
+    http_req_failed: ['rate==0'],
+    checks: ['rate==1'],
+    // About twice each one's p95 on 2026-10-05 on Tower, and no less than 50ms, where a few ms
+    // either way is noise. SQLite: 89, 17, 10, 22, 112, 126ms; Postgres: 54, 34, 24, 44, 71, 88ms.
+    'http_req_duration{name:feeds}': p95(200, 150),
+    'http_req_duration{name:all-entries}': p95(50, 100),
+    'http_req_duration{name:feed-entries}': p95(50, 100),
+    'http_req_duration{name:search}': p95(60, 100),
+    'http_req_duration{name:directory}': p95(250, 200),
+    'http_req_duration{name:listed}': p95(300, 200),
+  },
+};
+
+export default function () {
+  const u = me();
+  const feeds = http.get(`${BASE}/api/feeds`, as(u, 'feeds'));
+  check(feeds, { 'your thirty feeds': r => r.status === 200 && r.json().length === 30 });
+
+  const all = http.get(`${BASE}/api/entries?limit=50&filter=all&sort=published&dir=desc`, as(u, 'all-entries'));
+  check(all, { 'All Subscriptions, a page of it': r => r.status === 200 && r.json().entries.length === 50 });
+
+  const f = pick(feeds.json());
+  const one = http.get(`${BASE}/api/feeds/${f.id}/entries?limit=50&sort=title&dir=asc`, as(u, 'feed-entries'));
+  check(one, { 'a feed of yours, every item': r => r.status === 200 && r.json().entries.length === ITEMS });
+
+  // Every generated item mentions the weather, in its text, not its title.
+  const found = http.get(`${BASE}/api/entries?q=weather&limit=50`, as(u, 'search'));
+  check(found, { 'a search of everything you read': r => r.status === 200 && r.json().total === 30 * ITEMS });
+
+  const dir = http.get(`${BASE}/api/directory`, as(u, 'directory'));
+  check(dir, { 'the whole Directory': r => r.status === 200 && r.json().length === FEEDS });
+
+  const listed = http.get(`${BASE}/api/directory/${feedId(Math.floor(Math.random() * FEEDS))}`, as(u, 'listed'));
+  check(listed, { "a feed's page in the Directory": r => r.status === 200 && r.json().items.length === ITEMS });
+
+  sleep(1 + Math.random() * 2);
+}

tests/load/lib.js

@@ -0,0 +1,19 @@
+// What the load tests share. Each virtual user is a listener of its own, signed in by name with
+// the header the scratch daemon trusts from 127.0.0.1, as production trusts Cloudflare Access's.
+export const BASE = __ENV.BASE;
+export const USERS = Number(__ENV.USERS || 100);
+export const FEEDS = Number(__ENV.FEEDS || 1500);
+// Each generated feed has this many items; see run.js.
+export const ITEMS = 20;
+
+export const as = (name, tag) => ({
+  headers: { 'X-Load-User': name, 'Content-Type': 'application/json' },
+  tags: { name: tag },
+});
+/// The listener this virtual user is, one of the hundred run.js seeded, thirty feeds each.
+export const me = () => `load-${(__VU - 1) % USERS}`;
+export const pick = a => a[Math.floor(Math.random() * a.length)];
+/// A p95 budget, in ms, for whichever database the daemon is on (run.js --postgres): SQLite reads
+/// a file beside the daemon, Postgres answers over the network, and each is quicker at something.
+export const p95 = (sqlite, postgres) => [`p(95)<${__ENV.DB === 'postgres' ? postgres : sqlite}`];
+export const feedId = n => `gen-${String(n).padStart(4, '0')}`;

tests/load/listening.js

@@ -0,0 +1,57 @@
+import http from 'k6/http';
+import { check, sleep } from 'k6';
+import { BASE, as, me, pick, p95 } from './lib.js';
+
+// Players saving where people are while scans write. A hundred listeners each save a position
+// every second -- the player saves every ten, so this is a thousand people listening -- mark an
+// item read now and then, and read their position back to see it is theirs and as they left it,
+// while one of them forces a scan of all their feeds every five seconds. On SQLite every one of
+// these is a write waiting its turn for the one writer.
+export const options = {
+  scenarios: {
+    listeners: { executor: 'constant-vus', vus: 100, duration: '60s', exec: 'listen' },
+    scans: { executor: 'constant-vus', vus: 1, duration: '60s', exec: 'scan' },
+  },
+  thresholds: {
+    http_req_failed: ['rate==0'],
+    checks: ['rate==1'],
+    // About twice each one's p95 on 2026-10-05 on Tower. SQLite: 21, 277 and 134ms; Postgres: 9,
+    // 225 and 153ms. Marking read answers with the feed's row, counts and all, hence its budget.
+    'http_req_duration{name:position}': p95(100, 100),
+    'http_req_duration{name:flags}': p95(600, 500),
+    'http_req_duration{name:read-back}': p95(300, 400),
+  },
+};
+
+// Each virtual user's own episode, and how far into it it is.
+let ep = null, secs = 0, n = 0;
+
+export function listen() {
+  const u = me();
+  if (!ep) {
+    const r = http.get(`${BASE}/api/entries?limit=50`, as(u, 'pick'));
+    ep = pick(r.json().entries.filter(e => e.duration));
+  }
+  secs += 1;
+  const saved = http.post(`${BASE}/api/entries/${encodeURIComponent(ep.feed_id)}/${encodeURIComponent(ep.guid)}/position`,
+    JSON.stringify({ secs, duration: ep.duration }), as(u, 'position'));
+  check(saved, { 'position saved': r => r.status === 204 });
+  if (++n % 10 === 0) {
+    const flags = http.post(`${BASE}/api/entries/${encodeURIComponent(ep.feed_id)}/${encodeURIComponent(ep.guid)}/flags`,
+      JSON.stringify({ read: n % 20 === 0 }), as(u, 'flags'));
+    check(flags, { 'marked read or unread': r => r.status === 200 });
+    const back = http.get(`${BASE}/api/feeds/${encodeURIComponent(ep.feed_id)}/entries?limit=50`, as(u, 'read-back'));
+    const mine = back.status === 200 && back.json().entries.find(e => e.guid === ep.guid);
+    check(mine, {
+      'your position, as you left it': e => e && e.position === secs,
+      'read as you marked it': e => e && e.read === (n % 20 === 0),
+    });
+  }
+  sleep(1);
+}
+
+export function scan() {
+  const r = http.post(`${BASE}/api/fetch`, JSON.stringify({ force: true }), as('load-0', 'fetch'));
+  check(r, { 'scan queued': r => r.status === 202 || r.status === 200 });
+  sleep(5);
+}

tests/load/media.js

@@ -0,0 +1,48 @@
+import http from 'k6/http';
+import { check } from 'k6';
+import { BASE, as, pick } from './lib.js';
+
+// Listeners seeking: fifty at once asking for ranges of the same few episodes, as a player does
+// on every seek and every few seconds of playback. Every range is checked against what the feed
+// served, byte by byte at a sample of offsets, so a wrong offset fails, not only a wrong status.
+export const options = {
+  scenarios: { seeking: { executor: 'constant-vus', vus: 50, duration: '45s' } },
+  thresholds: {
+    http_req_failed: ['rate==0'],
+    checks: ['rate==1'],
+    // About four times its p95 on 2026-10-05, 48ms on SQLite and 45 on Postgres: a range is a
+    // file read, little to vary.
+    'http_req_duration{name:range}': ['p(95)<200'],
+  },
+};
+// The generated file: ID3's ten bytes, then a byte its offset gives (run.js, mediaByte).
+const SIZE = 2 * 1024 * 1024;
+const byte = k => (k * 31 + 7) & 255;
+
+export function setup() {
+  const r = http.get(`${BASE}/api/entries?filter=downloaded&limit=50`, as('listener', 'setup'));
+  const files = r.json().entries.flatMap(e => e.enclosures).filter(x => x.path).map(x => x.id);
+  if (!files.length) throw new Error('the listener has no downloaded files to seek through');
+  return { files };
+}
+
+export default function ({ files }) {
+  const start = 10 + Math.floor(Math.random() * (SIZE - 11));
+  const end = Math.min(SIZE - 1, start + Math.floor(Math.random() * 65536));
+  const r = http.get(`${BASE}/media/${pick(files)}`, {
+    headers: { 'X-Load-User': 'listener', Range: `bytes=${start}-${end}` },
+    responseType: 'binary',
+    tags: { name: 'range' },
+  });
+  const body = r.status === 206 ? new Uint8Array(r.body) : new Uint8Array(0);
+  check(r, {
+    'part of the file': r => r.status === 206,
+    'the range asked for': r => r.headers['Content-Range'] === `bytes ${start}-${end}/${SIZE}`,
+    'its bytes': () => {
+      if (body.length !== end - start + 1) return false;
+      for (const at of [0, body.length - 1, ...Array.from({ length: 30 }, () => Math.floor(Math.random() * body.length))])
+        if (body[at] !== byte(start + at)) return false;
+      return true;
+    },
+  });
+}

tests/load/run.js

@@ -0,0 +1,280 @@
+// Load tests: k6 against a scratch daemon with a catalogue the size of production's, for what
+// the browser tests cannot show -- many people at once, and what that does to latency, to the
+// database and to the healthcheck. The browser suite drives one person against a handful of
+// feeds, one request at a time.
+//
+//   node tests/load/run.js [--postgres] [browse|listening|media|signin ...]     default: all four
+//
+// It builds a release binary (debug Argon2 alone takes a second a sign-in, which would measure
+// the build, not ipx), serves generated feeds from this process, starts a daemon on its own
+// config and data under /tmp/ipx-load, wiped first, seeds listeners, then runs each k6 script.
+// While each runs, `ipx status` -- the Docker healthcheck -- is run every second, and the run
+// fails if any answer takes as long as the healthcheck's 5s timeout.
+//
+// The daemon is on SQLite unless --postgres, which puts it in IPX_TEST_DATABASE_URL, the
+// database the Rust tests use on Postgres too (ipodderx_test on the server production's is on, in
+// /src/.envrc), in a schema of its own, ipx_load, made afresh each run (#139). Postgres's numbers
+// are what production would see.
+// IPX_LOAD_FEEDS sets the catalogue's size (1500, near production's), IPX_LOAD_USERS the
+// listeners (100), IPX_LOAD_LOG=1 shows the daemon's log.
+const http = require('http');
+const fs = require('fs');
+const path = require('path');
+const { spawn, spawnSync, execFile, execFileSync } = require('child_process');
+
+const repo = path.resolve(__dirname, '../..');
+const root = '/tmp/ipx-load';
+const WEB = 8793, GEN = 8794;
+const FEEDS = Number(process.env.IPX_LOAD_FEEDS || 1500);
+const USERS = Number(process.env.IPX_LOAD_USERS || 100);
+// The first few feeds carry real files, downloaded for the media test.
+const MEDIA = 4;
+const ITEMS = 20;
+const BASE = `http://127.0.0.1:${WEB}`;
+const bin = path.join(repo, 'target/release/ipx');
+const SCRIPTS = ['browse', 'listening', 'media', 'signin'];
+const POSTGRES = process.argv.includes('--postgres');
+const PG_URL = process.env.IPX_TEST_DATABASE_URL || '';
+const PG_SCHEMA = 'ipx_load';
+const env = {
+  ...process.env,
+  IPX_CONFIG: `${root}/config/config.toml`,
+  IPX_DATA_DIR: `${root}/data`,
+  IPX_LOG: 'ipx=info',
+  IPX_LOG_FORMAT: 'json',
+  // Nothing of the test reaches production's database or traces, or a paid API.
+  // In its own schema, as Db::memory puts each Rust test, and with notices off, as url_for does.
+  IPX_DATABASE_URL: POSTGRES
+    ? `${PG_URL}${PG_URL.includes('?') ? '&' : '?'}options=-c%20search_path%3D${PG_SCHEMA}%20-c%20client_min_messages%3Dwarning`
+    : '',
+  OTEL_EXPORTER_OTLP_ENDPOINT: '',
+  TYPESAFE_KEY: '',
+};
+
+// ---- the feeds ------------------------------------------------------------------------------
+
+// Apple's categories, some with a subcategory, so the Directory has tiles and pages to draw.
+const CATS = [['Technology'], ['News', 'Tech News'], ['Comedy'], ['Leisure', 'Video Games'],
+  ['Society & Culture', 'Documentary'], ['Sports', 'Soccer'], ['Arts', 'Books'],
+  ['Science', 'Astronomy'], ['History'], ['True Crime'], ['Business', 'Investing'],
+  ['Education', 'Self-Improvement'], ['Health & Fitness', 'Mental Health'], ['Music']];
+const esc = s => s.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/"/g, '&quot;');
+const now = Date.now();
+// Every third feed is a blog: no files, as most of production's are.
+const isBlog = n => n % 3 === 2 && n >= MEDIA;
+function feedXml(n) {
+  const [cat, sub] = CATS[n % CATS.length];
+  const category = sub
+    ? `<itunes:category text="${esc(cat)}"><itunes:category text="${esc(sub)}"/></itunes:category>`
+    : `<itunes:category text="${esc(cat)}"/>`;
+  const items = Array.from({ length: ITEMS }, (_, i) => `<item>
+    <title>Episode ${ITEMS - i} of Show ${n}</title><guid>gen-${n}-${i}</guid>
+    <pubDate>${new Date(now - (i * 3 + n % 3) * 86400e3).toUTCString()}</pubDate>
+    <description>&lt;p&gt;Show ${n}, episode ${ITEMS - i}: an hour on the news, the weather and whatever came up.&lt;/p&gt;</description>
+    ${isBlog(n) ? '' : `<itunes:duration>${1800 + i * 60}</itunes:duration>
+    <enclosure url="http://127.0.0.1:${GEN}/media/${n}/${i}.mp3" length="${MEDIA_BYTES.length}" type="audio/mpeg"/>`}
+  </item>`).join('');
+  return `<?xml version="1.0"?><rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"><channel>
+    <title>Generated Show ${n}</title><link>http://127.0.0.1:${GEN}/site/${n}</link>
+    <description>Generated show number ${n}, for the load tests.</description>
+    <itunes:image href="http://127.0.0.1:${GEN}/art/${n}.jpg"/>${category}${items}</channel></rss>`;
+}
+// A file whose every byte is known from its offset, so the media test can check that a range
+// it asked for is the range it got, not merely that it got something that long. ID3 first, so
+// ipx takes it for audio.
+const mediaByte = k => (k * 31 + 7) & 255;
+const MEDIA_BYTES = Buffer.from(Uint8Array.from({ length: 2 * 1024 * 1024 }, (_, k) => mediaByte(k)));
+Buffer.from('ID3\x03\x00\x00\x00\x00\x00\x00', 'latin1').copy(MEDIA_BYTES);
+const art = fs.readFileSync(path.join(repo, 'tests/ui/fixtures/art.jpg'));
+
+function serveFeeds() {
+  return http.createServer((req, res) => {
+    const m = req.url.match(/^\/(feed|media|art)\/(\d+)/);
+    if (m?.[1] === 'feed') { res.writeHead(200, { 'content-type': 'application/rss+xml' }); return res.end(feedXml(Number(m[2]))); }
+    if (m?.[1] === 'media') { res.writeHead(200, { 'content-type': 'audio/mpeg', 'content-length': MEDIA_BYTES.length }); return res.end(MEDIA_BYTES); }
+    if (m?.[1] === 'art') { res.writeHead(200, { 'content-type': 'image/jpeg' }); return res.end(art); }
+    res.writeHead(404).end();
+  }).listen(GEN, '127.0.0.1');
+}
+const feedId = n => `gen-${String(n).padStart(4, '0')}`;
+const feedUrl = n => `http://127.0.0.1:${GEN}/feed/${n}.xml`;
+
+// ---- the daemon -----------------------------------------------------------------------------
+
+/// The load tests' schema dropped and made again, as /tmp/ipx-load is wiped. Only in a database
+/// made for tests: the URL is production's server and role, and one slip of a name would
+/// otherwise point this at production's database.
+function freshPostgres() {
+  const u = PG_URL && new URL(PG_URL);
+  if (!u || !/_test$/.test(u.pathname)) {
+    throw new Error('--postgres needs IPX_TEST_DATABASE_URL, a database whose name ends in _test (. /src/.envrc)');
+  }
+  psql(`DROP SCHEMA IF EXISTS ${PG_SCHEMA} CASCADE`, `CREATE SCHEMA ${PG_SCHEMA}`);
+}
+
+/// Statements run in the test database, given by its URL in psql's environment, not its
+/// arguments, where every process on Tower could read the password.
+function psql(...sql) {
+  const u = new URL(PG_URL);
+  const env = { ...process.env, PGHOST: u.hostname, PGPORT: u.port || '5432', PGUSER: decodeURIComponent(u.username),
+    PGPASSWORD: decodeURIComponent(u.password), PGDATABASE: u.pathname.slice(1), PGOPTIONS: `-c search_path=${PG_SCHEMA}` };
+  execFileSync('psql', ['-q', '-v', 'ON_ERROR_STOP=1', ...sql.flatMap(s => ['-c', s])], { env, stdio: ['ignore', 'ignore', 'inherit'] });
+}
+
+function writeConfig() {
+  fs.rmSync(root, { recursive: true, force: true });
+  for (const d of ['config', 'data', 'downloads']) fs.mkdirSync(path.join(root, d), { recursive: true });
+  // Read into the database's catalogue on the first start, as an existing config.toml is. Only
+  // the media feeds download: a forced scan of a listener's 30 feeds would otherwise fetch a
+  // 2 MB episode of each, every time.
+  const feeds = Array.from({ length: FEEDS }, (_, n) =>
+    `[feeds.${feedId(n)}]\nurl = "${feedUrl(n)}"\nauto_download = ${n < MEDIA}\n`).join('\n');
+  fs.writeFileSync(env.IPX_CONFIG, `
+[general]
+download_dir = "${root}/downloads"
+socket = "${root}/ipx.sock"
+schedule = "every 60m"
+max_new_per_check = 1
+
+[torrent]
+enabled = false
+
+[web]
+enabled = true
+bind = "127.0.0.1:${WEB}"
+token = "loadtokenloadtokenloadtoken12345"
+# Each k6 user signs in by name, as Cloudflare Access does in production: no password to hash
+# for every one of a hundred listeners. Only the sign-in test uses a password.
+trusted_header = "X-Load-User"
+trusted_proxies = ["127.0.0.1"]
+auto_create_users = true
+
+${feeds}`);
+}
+
+/// Resolves with the first log line matching `test`, read from the daemon's JSON log, which it
+/// writes to stderr.
+function waitForLog(daemon, test, ms) {
+  return new Promise((resolve, reject) => {
+    let buf = '';
+    const timer = setTimeout(() => { daemon.stderr.off('data', on); reject(new Error(`no log line in ${ms / 1000}s for ${test}`)); }, ms);
+    const on = chunk => {
+      buf += chunk;
+      let i;
+      while ((i = buf.indexOf('\n')) >= 0) {
+        const line = buf.slice(0, i); buf = buf.slice(i + 1);
+        let ev; try { ev = JSON.parse(line); } catch { continue; }
+        if (test(ev)) { clearTimeout(timer); daemon.stderr.off('data', on); return resolve(ev); }
+      }
+    };
+    daemon.stderr.on('data', on);
+  });
+}
+
+const as = name => ({ 'X-Load-User': name, 'Content-Type': 'application/json' });
+async function api(name, url, opts = {}) {
+  const r = await fetch(BASE + url, { ...opts, headers: { ...as(name), ...opts.headers } });
+  if (!r.ok) throw new Error(`${opts.method || 'GET'} ${url} as ${name}: ${r.status} ${await r.text()}`);
+  return r.status === 204 ? null : r.json().catch(() => null);
+}
+const opml = ns => `<?xml version="1.0"?><opml version="2.0"><head><title>load</title></head><body>${
+  ns.map(n => `<outline type="rss" text="${feedId(n)}" xmlUrl="${feedUrl(n)}"/>`).join('')}</body></opml>`;
+
+async function seed() {
+  // The first account made is the admin.
+  await api('admin', '/api/me');
+  // Every listener subscribes to 30 feeds, overlapping as people's do; one OPML each, one scan.
+  for (let u = 0; u < USERS; u++) {
+    const mine = Array.from({ length: 30 }, (_, k) => (u * 7 + k * 41) % FEEDS);
+    await api(`load-${u}`, '/api/opml', { method: 'POST', body: JSON.stringify({ xml: opml(mine) }) });
+  }
+  // The media test's listener takes the feeds with files. They may be downloaded already: the
+  // first start subscribes the admin to the whole catalogue, so the first scan fetched them.
+  for (let n = 0; n < MEDIA; n++) await api('listener', `/api/popular/${feedId(n)}`, { method: 'POST' });
+  for (let t = Date.now(); ; await new Promise(r => setTimeout(r, 500))) {
+    const got = await api('listener', '/api/entries?filter=downloaded&limit=50');
+    if (got.entries.flatMap(e => e.enclosures).filter(x => x.path).length >= MEDIA) break;
+    if (Date.now() - t > 120_000) throw new Error(`the listener's ${MEDIA} files did not download in 120s`);
+  }
+  // A password, for the sign-in test; the CLI hashes it as `ipx user add` always does.
+  execFileSync(bin, ['user', 'add', 'piper'], { input: 'piperpassword', env });
+}
+
+// ---- the run --------------------------------------------------------------------------------
+
+/// `ipx status` once a second until stopped: the slowest answer, and any that failed. Not
+/// spawnSync: blocked in it, this process stops draining the daemon's log, the pipe fills, and
+/// the daemon stalls writing its next line, which is the test measuring itself.
+function watchHealth() {
+  const seen = { slowest: 0, failed: 0 };
+  let on = true;
+  (async () => {
+    while (on) {
+      const t = Date.now();
+      const ok = await new Promise(res => execFile(bin, ['status'], { env, timeout: 5000 }, err => res(!err)));
+      seen.slowest = Math.max(seen.slowest, Date.now() - t);
+      if (!ok) seen.failed++;
+      await new Promise(res => setTimeout(res, 1000));
+    }
+  })();
+  return () => { on = false; return seen; };
+}
+
+function k6(script) {
+  return new Promise(resolve => {
+    const run = spawn('k6', ['run', '--quiet', '-e', `BASE=${BASE}`, '-e', `USERS=${USERS}`, '-e', `FEEDS=${FEEDS}`,
+      '-e', `DB=${POSTGRES ? 'postgres' : 'sqlite'}`,
+      path.join(__dirname, `${script}.js`)], { stdio: 'inherit' });
+    run.on('exit', code => resolve(code));
+  });
+}
+
+(async () => {
+  const only = process.argv.slice(2).filter(a => a !== '--postgres');
+  for (const s of only) if (!SCRIPTS.includes(s)) { console.error(`no load test called ${s}: ${SCRIPTS.join(', ')}`); process.exit(2); }
+  if (spawnSync('k6', ['version']).error) { console.error('k6 is not installed: see install.sh'); process.exit(2); }
+  console.log('building the release binary...');
+  execFileSync('cargo', ['build', '--release', '-q'], { cwd: repo, stdio: 'inherit' });
+
+  const feeds = serveFeeds();
+  writeConfig();
+  if (POSTGRES) freshPostgres();
+  console.log(`on ${POSTGRES ? 'Postgres' : 'SQLite'}`);
+  // Its log kept out of this run's output, which is k6's; IPX_LOAD_LOG=1 shows it.
+  const daemon = spawn(bin, ['daemon'], { env, stdio: ['ignore', process.env.IPX_LOAD_LOG ? 'inherit' : 'ignore', 'pipe'] });
+  daemon.stderr.setEncoding('utf8');
+  if (process.env.IPX_LOAD_LOG) daemon.stderr.on('data', d => process.stderr.write(d));
+  // Stopped by hand, the daemon goes too, by its own PID, or it holds the ports for the next run.
+  for (const sig of ['SIGINT', 'SIGTERM']) process.on(sig, () => { daemon.kill('SIGTERM'); process.exit(130); });
+  let failed = 0;
+  try {
+    const t = Date.now();
+    const first = await waitForLog(daemon, ev => ev.ev === 'scan_done' && ev.feeds > 0, 600_000);
+    console.log(`first scan: ${first.feeds} feeds in ${((Date.now() - t) / 1000).toFixed(1)}s`);
+    await seed();
+    // The planner's statistics, as production's long-standing tables have them. A schema filled
+    // seconds ago has none until autovacuum gets to it, which it did partway through a test, and
+    // the search's p95 swung from 40ms to 166ms between runs with the plan it changed.
+    if (POSTGRES) psql('ANALYZE');
+    // The log is drained from here on, or the pipe fills and the daemon blocks writing to it.
+    daemon.stderr.resume();
+    for (const script of only.length ? only : SCRIPTS) {
+      console.log(`\n=== ${script}`);
+      const stop = watchHealth();
+      const code = await k6(script);
+      const health = stop();
+      console.log(`ipx status: slowest ${health.slowest}ms, ${health.failed} failed`);
+      if (code !== 0) { failed++; console.log(`${script}: thresholds failed (k6 exit ${code})`); }
+      if (health.failed || health.slowest >= 5000) { failed++; console.log(`${script}: the healthcheck would have failed`); }
+    }
+  } catch (e) {
+    console.error(e.message);
+    failed++;
+  } finally {
+    // Its own PID, never a pkill: Tower sees production's ipx too (#38).
+    daemon.kill('SIGTERM');
+    feeds.close();
+  }
+  console.log(failed ? `\n${failed} failure(s)` : '\nall load tests passed');
+  process.exit(failed ? 1 : 0);
+})();

tests/load/signin.js

@@ -0,0 +1,44 @@
+import http from 'k6/http';
+import { check, sleep } from 'k6';
+import { Trend } from 'k6/metrics';
+import { BASE, as, me } from './lib.js';
+
+// A flood of sign-in attempts with wrong passwords while everyone else goes on using the site.
+// Two things only many requests at once show: whether checking passwords, tens of milliseconds
+// of CPU each, starves the rest of the server, and whether a wrong password for a name that is
+// an account takes longer to refuse than one for a name that is not -- the answers read the
+// same, and the time would tell anyone which names are accounts here.
+const gap = new Trend('signin_name_gap', true);
+export const options = {
+  scenarios: {
+    attempts: { executor: 'constant-vus', vus: 40, duration: '40s', exec: 'attempt' },
+    everyone: { executor: 'constant-vus', vus: 5, duration: '40s', exec: 'browse' },
+  },
+  thresholds: {
+    http_req_failed: ['rate==0'],
+    checks: ['rate==1'],
+    // 57ms on 2026-10-05 on SQLite, 68 on Postgres; 4.3s while password checks ran on the async
+    // workers (#137).
+    'http_req_duration{name:meanwhile}': ['p(95)<300'],
+    // Known name against unknown, one straight after the other: no gap but noise. 0.2ms on
+    // 2026-10-05; 31ms while an unknown name was refused without a check (#138).
+    signin_name_gap: ['med<10'],
+  },
+};
+const refused = { headers: { 'Content-Type': 'application/json' }, responseCallback: http.expectedStatuses(401) };
+
+export function attempt() {
+  const known = http.post(`${BASE}/api/login`, JSON.stringify({ name: 'piper', password: 'not-the-password' }),
+    { ...refused, tags: { name: 'signin-known' } });
+  const unknown = http.post(`${BASE}/api/login`, JSON.stringify({ name: `nobody-${__VU}-${__ITER}`, password: 'not-the-password' }),
+    { ...refused, tags: { name: 'signin-unknown' } });
+  check(known, { 'a wrong password refused': r => r.status === 401 });
+  check(unknown, { 'an unknown name refused': r => r.status === 401 });
+  gap.add(known.timings.duration - unknown.timings.duration);
+}
+
+export function browse() {
+  const r = http.get(`${BASE}/api/feeds`, as(me(), 'meanwhile'));
+  check(r, { 'the site still answers': r => r.status === 200 });
+  sleep(0.5);
+}

tests/page-smoke.js

@@ -49,7 +49,6 @@ const drive = PAGE === 'admin.html' ? [
   ['prefsModal', () => ctx.prefsModal()],
   ['opmlModal', () => ctx.opmlModal()],
   ['selectFeed (directory)', () => ctx.selectFeed(':directory')],
-  ['selectFeed (popular)', () => ctx.selectFeed(':popular')],
   ['selectFeed (currently listening)', () => ctx.selectFeed(':listening')],
   ['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
   ['keysModal', () => ctx.keysModal()],

tests/ui/app.spec.js

@@ -242,7 +242,7 @@ test('an item with several enclosures lists them all', async ({ page }) => {
   await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
 });
 
-test('Currently Listening, its own place below Popular, resumes an episode you started or forgets it', async ({ page }) => {
+test('Currently Listening, its own place below the Directory, resumes an episode you started or forgets it', async ({ page }) => {
   // Second Episode (900s) is 42 seconds in and unfinished. An earlier test may have opened it,
   // and opening marks it read; it is listed all the same, because read is not finished. This
   // test used to set it unread first, which hid exactly the bug in issue #14.
@@ -251,18 +251,27 @@ test('Currently Listening, its own place below Popular, resumes an episode you s
   await page.evaluate(() =>
     api('/api/entries/test-show/ui-2/position', { method: 'POST', body: JSON.stringify({ secs: 42 }) }));
 
-  // Popular lists feeds and nothing else; the episodes have a place of their own under it.
+  // The Directory lists feeds and nothing else; the episodes have a place of their own under it.
-  await page.locator('#feedlist .place', { hasText: 'Popular' }).click();
+  await page.locator('#feedlist .place', { hasText: 'Directory' }).click();
-  await expect(page.locator('#popular')).toBeVisible();
+  await expect(page.locator('#directory')).toBeVisible();
   await expect(page.locator('#listening')).toHaveCount(0);
   const places = await page.locator('#feedlist .place b').allTextContents();
-  expect(places.indexOf('Currently Listening')).toBe(places.indexOf('Popular') + 1);
+  expect(places.indexOf('Currently Listening')).toBe(places.indexOf('Directory') + 1);
 
   await page.locator('#feedlist .place', { hasText: 'Currently Listening' }).click();
   const row = page.locator('#listening .childrow', { hasText: 'Second Episode' });
   await expect(row).toBeVisible({ timeout: 20_000 });
   await expect(row).toContainText('14:18 left');
 
+  // The search box looks through it, as it does a feed's items (#127).
+  await page.locator('#epSearch').fill('second');
+  await expect(row).toBeVisible();
+  await page.locator('#epSearch').fill('nothing-is-called-this');
+  await expect(row).toHaveCount(0);
+  await expect(page.locator('#listening')).toContainText('Nothing you are listening to mentions');
+  await page.locator('#epSearch').fill('');
+  await expect(row).toBeVisible();
+
   // Removing it forgets where you got to, so it is still gone on the next visit.
   await row.locator('[data-a=remove]').click();
   await expect(row).toHaveCount(0);
@@ -812,7 +821,7 @@ test('ipx import subscribes the admin, and ipx export writes the feeds out', asy
   expect(xml).toContain('http://127.0.0.1:8792/two.xml');
 });
 
-test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
+test('the Directory lists what everyone here reads, the most subscribed first, but never a private feed', async ({ browser, page }) => {
   const { execFileSync } = require('child_process');
   const setup = require('./global-setup');
   const env = {
@@ -833,21 +842,23 @@ test('Popular lists what everyone here reads, but never a private feed', async (
   await piper.locator('button[type=submit]').click();
   await expect(piper.locator('#feedlist')).toContainText('No feeds.');
 
-  await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
+  // The most subscribed are the Directory's first section; Popular, their own place, is gone (#124).
-  const offered = piper.locator('#popular .childrow');
+  await expect(piper.locator('#feedlist .place', { hasText: 'Popular' })).toHaveCount(0);
-  await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
+  await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
+  const chart = piper.locator('#dchart .rank');
+  await expect(chart.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
   // An OPML is listed as the feeds inside it, and a key in a URL marks someone's paid feed.
-  await expect(offered.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
+  await expect(chart.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
-  await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
+  await expect(chart.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
 
   // No URL reaches the page at all, so neither can a key, and the server holds the same line.
-  const listed = await (await piper.request.get('/api/popular')).text();
+  const listed = await (await piper.request.get('/api/directory')).text();
   expect(listed).not.toContain('secret123');
   expect(listed).not.toContain('.xml');
   expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
 
-  // Popular is the top ten of the directory, and the directory is every listed feed A to Z,
+  // The chart is /api/popular's top ten, and the directory is every listed feed A to Z, with an
-  // with an OPML's feeds in place of the OPML in both.
+  // OPML's feeds in place of the OPML in both.
   const dir = await (await piper.request.get('/api/directory')).json();
   const top = await (await piper.request.get('/api/popular')).json();
   const names = dir.map(p => (p.title || p.id).toLowerCase());
@@ -855,48 +866,92 @@ test('Popular lists what everyone here reads, but never a private feed', async (
   const ids = dir.map(p => p.id);
   expect(ids).not.toContain('test-subscriptions');
   expect(ids).toEqual(expect.arrayContaining(['grouped-show', 'aardvark-radio']));
-  expect(top.length).toBe(Math.min(10, dir.length));
   expect(top.every(t => ids.includes(t.id))).toBe(true);
   expect(ids).not.toContain('paid-show');
-
+  await expect(chart).toHaveCount(top.length);
-  // Subscribe from the directory this time: the same feeds, as a grid of cover art.
+  await expect(chart.first().locator('.no')).toHaveText('1');
-  const tiles = piper.locator('#popular .tile');
-  const pick = (row, name) => piper.locator(`#dirbar .${row} button`, { hasText: new RegExp(`^${name}$`) });
-  await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
   await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
+
+  // Then the categories, as tiles, each a page of its own.
+  const cat = name => piper.locator('#directory .cat', { hasText: new RegExp(`^${name}\\d`) });
+  const tiles = piper.locator('#dtiles .tile');
+  expect(dir.find(p => p.id === 'test-show')).toMatchObject({ podcast: true, category: 'Technology' });
+  expect(dir.find(p => p.id === 'picture-blog')).toMatchObject({ podcast: false });
+  await cat('Technology').click();
+  await expect(piper.locator('#directory h2')).toHaveText('Technology');
+  await expect(tiles).toHaveCount(dir.filter(p => p.category === 'Technology').length);
   await expect(tiles.filter({ hasText: 'Test Show' })).toBeVisible();
+  // Its way back lands on the tile it went in by.
+  await piper.locator('#directory .dback').click();
+  await expect(cat('Technology')).toBeFocused();
+
+  // What a feed is holds on every page. No blog here names Technology, so Blogs has no such tile.
+  const kind = name => piper.locator('#directory .tabs button', { hasText: new RegExp(`^${name}$`) });
+  await kind('Blogs').click();
+  await expect(kind('Blogs')).toHaveAttribute('aria-pressed', 'true');
+  await expect(cat('Technology')).toHaveCount(0);
+  await piper.locator('#directory .dall').click();
+  await expect(tiles).toHaveCount(dir.filter(p => !p.podcast).length);
+  await expect(tiles.filter({ hasText: 'Test Show' })).toHaveCount(0);
+  await kind('All').click();
+  await expect(tiles).toHaveCount(dir.length);
   await expect(tiles.filter({ hasText: /Grouped Show|grouped-show/ })).toBeVisible();
   await expect(tiles.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
   await expect(tiles.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
+  // Sorted by name either way, A to Z as it opens, or by subscribers (#122).
+  const byName = dir.map(p => p.title || p.id).sort((a, b) => a.localeCompare(b, undefined, { sensitivity: 'base', numeric: true }));
+  await expect(tiles.first()).toContainText(byName[0]);
+  await piper.locator('#dirsort').selectOption('za');
+  await expect(tiles.first()).toContainText(byName[byName.length - 1]);
+  await expect(tiles).toHaveCount(dir.length);
+  await piper.locator('#dirsort').selectOption('subscribers');
+  await expect(tiles).toHaveCount(dir.length);
+  await piper.locator('#dirsort').selectOption('az');
+
+  // The search box looks through the Directory by name (#126), and clearing it goes back.
+  await piper.locator('#epSearch').fill('test show');
+  await expect(piper.locator('#directory h2')).toHaveText('Results for “test show”');
+  await expect(tiles).toHaveCount(dir.filter(p => (p.title || p.id).toLowerCase().includes('test show')).length);
+  await piper.locator('#epSearch').fill('');
   await expect(tiles).toHaveCount(dir.length);
 
-  // Two filters that combine: what a feed is, and what it is about.
+  // A feed you do not subscribe to opens its page here: what it is and its latest items (#128).
-  expect(dir.find(p => p.id === 'test-show')).toMatchObject({ podcast: true, category: 'Technology' });
+  await tiles.filter({ hasText: 'Test Show' }).locator('.txt').click();
-  expect(dir.find(p => p.id === 'picture-blog')).toMatchObject({ podcast: false });
+  await expect(piper.locator('.show h2')).toHaveText('Test Show');
-  await pick('tabs', 'Blogs').click();
+  await expect(piper.locator('#dshow .lep', { hasText: 'First Episode' })).toBeVisible();
-  await expect(tiles).toHaveCount(dir.filter(p => !p.podcast).length);
+  // What the feed says it is, from its own <description> (#130).
-  await expect(tiles.filter({ hasText: 'Test Show' })).toHaveCount(0);
+  await expect(piper.locator('#dabout p')).toHaveText('A fixture feed.');
-  // No empty chips: no blog here names Technology, so Blogs does not offer it.
+  await expect(piper.locator('#dabout button')).toBeHidden();
-  await expect(pick('chips', 'Technology')).toHaveCount(0);
+  // Back, onto the tile it was opened from, the page under it as it was.
-  await pick('tabs', 'Podcasts').click();
+  await piper.locator('#directory .dback').click();
-  await pick('chips', 'Technology').click();
+  await expect(tiles.filter({ hasText: 'Test Show' })).toBeFocused();
-  await expect(pick('chips', 'Technology')).toHaveAttribute('aria-pressed', 'true');
-  await expect(tiles).toHaveCount(dir.filter(p => p.podcast && p.category === 'Technology').length);
-  // A second press lifts the chip and leaves the kind as it was.
-  await pick('chips', 'Technology').click();
-  await expect(tiles).toHaveCount(dir.filter(p => p.podcast).length);
-  await pick('tabs', 'All').click();
   await expect(tiles).toHaveCount(dir.length);
+  // Only a listed feed has one, and it carries no file or its address.
+  const preview = await (await piper.request.get('/api/directory/test-show')).text();
+  expect(preview).toContain('First Episode');
+  expect(JSON.parse(preview).description).toBe('A fixture feed.');
+  expect(preview).not.toContain('.mp3');
+  expect((await piper.request.get('/api/directory/paid-show')).status()).toBe(404);
+  // Nor does anything else give a feed's items or files to someone who does not subscribe (#129).
+  expect((await piper.request.get('/api/feeds/test-show/entries')).status()).toBe(404);
+  expect((await piper.request.get('/api/feeds/paid-show/entries')).status()).toBe(404);
+  const pics = await page.evaluate(() => api('/api/feeds/picture-blog/entries'));
+  const file = pics.entries.flatMap(e => e.enclosures).find(x => x.path);
+  expect(file).toBeTruthy();
+  expect((await page.request.get(`/media/${file.id}`)).status()).toBe(200);
+  expect((await piper.request.get(`/media/${file.id}`)).status()).toBe(404);
+  expect((await piper.request.post(`/api/enclosures/${file.id}/download`)).status()).toBe(404);
+  expect((await piper.request.delete(`/api/enclosures/${file.id}?force=true`)).status()).toBe(404);
+  expect((await piper.request.post('/api/feeds/picture-blog/download-latest', { data: { count: 1 } })).status()).toBe(404);
 
-  // Add a feed is for an address; Popular and Directory are where you browse (issue #30).
+  // Add a feed is for an address; the Directory is where you browse (issue #30).
   await piper.locator('#addFeed').click();
   await expect(piper.locator('#nurl')).toBeVisible();
-  await expect(piper.locator('#modalCard .childrow')).toHaveCount(0);
+  await expect(piper.locator('#modalCard .tile, #modalCard .rank')).toHaveCount(0);
-  await expect(tiles).toHaveCount(dir.length);
   await piper.locator('#modalCard button[title="Cancel"]').click();
 
   const row = async () =>
-    (await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
+    (await (await piper.request.get('/api/directory')).json()).find(p => p.id === 'test-show');
   const before = await row();
   expect(before.subscribed).toBe(false);
   await tiles.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
@@ -904,9 +959,10 @@ test('Popular lists what everyone here reads, but never a private feed', async (
 
   // Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
   expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
-  await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
+  expect((await piper.request.get('/api/feeds/test-show/entries')).status()).toBe(200);
-  await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
+  await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
-  await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
+  await expect(chart.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
+  await expect(chart.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
 
   // All Subscriptions is every item from piper's feeds and only those: the admin's Picture
   // Blog is not among them.
@@ -919,6 +975,46 @@ test('Popular lists what everyone here reads, but never a private feed', async (
   await ctx.close();
 });
 
+test('the feed list keeps the icons it has drawn when a feed\'s new row comes in (#134)', async ({ page }) => {
+  await expect(page.locator('#feedlist .feed').first()).toBeVisible({ timeout: 20_000 });
+  const kept = await page.evaluate(async () => {
+    const frames = () => new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)));
+    // Artwork that loads, on a feed of its own row: the fixtures' fails on purpose, for initials.
+    const row = { ...S.feeds.find(f => !f.group && !S.feeds.some(c => c.group === f.id)), image: '/favicon.png' };
+    patchFeed(row); await frames();
+    const rows = [...document.querySelectorAll('#feedlist .feed')];
+    const before = [...document.querySelectorAll('#feedlist .feed img')];
+    // The same feed's new row again, as a check of every feed sends one for each feed it reads.
+    patchFeed({ ...row }); await frames();
+    const after = [...document.querySelectorAll('#feedlist .feed img')];
+    return { drawn: before.length > 0, redrawn: !rows[0].isConnected, same: after.length === before.length && after.every((img, i) => img === before[i]) };
+  });
+  // The rows are new, the images in them the ones already drawn.
+  expect(kept).toEqual({ drawn: true, redrawn: true, same: true });
+});
+
+test('ipx add and rm while the daemon runs reach it, and its next change keeps them (#120)', async ({ page }) => {
+  const { execFileSync } = require('child_process');
+  const setup = require('./global-setup');
+  const env = { ...process.env, IPX_CONFIG: `${setup.root}/config/config.toml`, IPX_DATA_DIR: `${setup.root}/data` };
+  const ipx = (...args) => execFileSync('./target/debug/ipx', args, { env, encoding: 'utf8' });
+  const listed = async () => (await page.evaluate(() => api('/api/directory'))).map(p => p.id);
+  // A change of the daemon's own: it writes the whole catalogue from the copy it holds, which
+  // wrote a removed feed back and dropped an added one when that copy was old.
+  const daemonWrites = category => page.evaluate(c =>
+    api('/api/feeds/picture-blog', { method: 'PATCH', body: JSON.stringify({ category: c }) }), category);
+
+  const id = ipx('add', '--list', 'http://127.0.0.1:8792/cli.xml').match(/^added (\S+?),? /m)[1];
+  await expect.poll(listed).toContain(id);
+  await daemonWrites('Comedy');
+  expect(await listed()).toContain(id);
+
+  ipx('rm', id);
+  await expect.poll(listed).not.toContain(id);
+  await daemonWrites(null);
+  expect(await listed()).not.toContain(id);
+});
+
 test('adding a feed scans it straight away', async ({ page }) => {
   await page.locator('#addFeed').click();
   await page.locator('#nurl').fill('http://127.0.0.1:8792/fresh.xml');
@@ -956,6 +1052,37 @@ test('reading an item updates its feed\'s count without reloading the list', asy
   expect(lists, 'the row came back with the read, not by reloading the list').toEqual([]);
 });
 
+test('artwork comes from iPX, kept, and only as an image', async ({ page }) => {
+  // Every image a feed names is drawn from iPX's address.
+  expect(await page.evaluate(() => artHTML('https://example.com/a.jpg', 'A'))).toContain('src="/api/art?u=https%3A%2F%2Fexample.com%2Fa.jpg"');
+  // Multi Show's items name art.jpg.
+  await expect(page.locator('.feed', { hasText: 'Multi Show' })).toBeVisible({ timeout: 20_000 });
+  const src = '/api/art?u=' + encodeURIComponent('http://127.0.0.1:8792/art.jpg');
+  for (const _ of [1, 2]) {                                         // fetched, then kept
+    const r = await page.request.get(src);
+    expect(r.status()).toBe(200);
+    expect(r.headers()['content-type']).toMatch(/^image\//);
+    expect(r.headers()['content-security-policy']).toContain('sandbox');
+  }
+  // An address no feed names is not fetched.
+  expect((await page.request.get('/api/art?u=' + encodeURIComponent('http://127.0.0.1:8792/show.xml'))).status()).toBe(404);
+});
+
+test('an item without a title is named from its text, then its file, then its show and date', async ({ page }) => {
+  const names = await page.evaluate(() => [
+    entryName({ title: 'A Title' }),
+    entryName({ title: ' ', description: '<p>I like the way <b>AI</b> is   evolving.</p><img src="x.png">' }),
+    entryName({ description: '<p>' + 'word '.repeat(40) + '</p>' }).text.length,
+    entryName({ enclosures: [{ url: 'https://k.example/media/KARTAS691.mp3?x=1' }] }),
+    entryName({ feed_id: 'nobody', published: 0 }),
+  ]);
+  expect(names[0]).toEqual({ text: 'A Title', derived: false });
+  expect(names[1]).toEqual({ text: 'I like the way AI is evolving.', derived: true });
+  expect(names[2]).toBeLessThanOrEqual(121);                         // cut at a word, with …
+  expect(names[3]).toEqual({ text: 'KARTAS691', derived: true });
+  expect(names[4]).toEqual({ text: 'nobody', derived: true });
+});
+
 test('a deleted file looks as if it was never downloaded', async ({ page }) => {
   // Other people subscribe to Picture Blog by now, so both prompts come; take them.
   page.on('dialog', d => d.accept());
@@ -1140,13 +1267,49 @@ test('keys move through items and places, after Feedly', async ({ page }) => {
   await expect(page.locator('#count')).toContainText('All Subscriptions');
 });
 
+test('an API token acts as its owner until it is revoked', async ({ page, request }) => {
+  // Made in Settings, shown once.
+  await page.locator('#prefs').click();
+  await page.locator('#tokname').fill('test agent');
+  await page.locator('#tokadd').click();
+  const token = await page.locator('#tokval').inputValue();
+  expect(token).toMatch(/^ipx_[0-9a-f]{64}$/);
+  await expect(page.locator('.tokrow', { hasText: 'test agent' })).toContainText('last used never');
+
+  // Sent as a Bearer header, with no cookie, it is the person who made it.
+  const as = t => ({ headers: { Authorization: `Bearer ${t}` } });
+  const me = await (await request.get('/api/me', as(token))).json();
+  expect(me.name).toBe(await page.evaluate(() => S.me.name));
+  expect((await request.get('/api/feeds', as(token))).status()).toBe(200);
+  // A wrong one is turned away, not let through as anyone.
+  expect((await request.get('/api/me', as('ipx_' + '0'.repeat(64)))).status()).toBe(401);
+  // Only the hash is kept: the list never carries the token.
+  const listed = await page.evaluate(() => api('/api/tokens'));
+  expect(JSON.stringify(listed)).not.toContain(token);
+  expect(listed.find(t => t.name === 'test agent').last_used).toBeTruthy();
+
+  // Revoked, it opens nothing.
+  await page.locator('.tokrow', { hasText: 'test agent' }).locator('[data-tok]').click();
+  await expect(page.locator('.tokrow', { hasText: 'test agent' })).toHaveCount(0);
+  expect((await request.get('/api/me', as(token))).status()).toBe(401);
+});
+
 test('an admin can give a blog its Directory category', async ({ page }) => {
   const patch = (id, category) => page.evaluate(([id, category]) =>
     api(`/api/feeds/${id}`, { method: 'PATCH', body: JSON.stringify({ category }) }), [id, category]);
   const listed = async id => (await page.evaluate(() => api('/api/directory'))).find(p => p.id === id);
 
   await patch('picture-blog', 'Visual Arts');
-  expect(await listed('picture-blog')).toMatchObject({ podcast: false, category: 'Visual Arts' });
+  // Apple's subcategory, under its category, as the Directory browses them (#118).
+  expect(await listed('picture-blog')).toMatchObject({ podcast: false, category: 'Arts', subcategory: 'Visual Arts' });
+  // A category's subcategories are on its own page, as Apple's directory has them.
+  const pill = name => page.locator('#directory .pills button', { hasText: new RegExp(`^${name}\\d`) });
+  await page.locator('#feedlist .place', { hasText: 'Directory' }).click();
+  await expect(page.locator('#directory .pills')).toHaveCount(0);
+  await page.locator('#directory .cat', { hasText: /^Arts\d/ }).click();
+  await pill('Visual Arts').click();
+  await expect(pill('Visual Arts')).toHaveAttribute('aria-pressed', 'true');
+  await expect(page.locator('#dtiles .tile')).toHaveCount(1);
   // A feed's own iTunes category wins over one given here.
   await patch('test-show', 'Comedy');
   expect((await listed('test-show')).category).toBe('Technology');
@@ -1233,7 +1396,7 @@ test('the favicon is the logo, square, from both pages', async ({ page }) => {
   await page.evaluate(() => setTheme('modern', 'dark'));
   await expect(page.locator('#favicon')).toHaveAttribute('href', /^\/favicon-dark\.png\?v=[0-9a-f]{12}$/);
   // A browser asks for /favicon.ico on its own, signed in or not.
-  for (const path of ['/favicon.ico', '/favicon.png', '/favicon-dark.png', '/apple-touch-icon.png']) {
+  for (const path of ['/favicon.ico', '/favicon.png', '/favicon-dark.png', '/apple-touch-icon.png', '/apple-touch-icon-precomposed.png']) {
     const r = await page.request.get(path, { headers: { cookie: '' } });
     expect(r.status(), path).toBe(200);
     expect(r.headers()['content-type'], path).toBe('image/png');

tests/ui/fixtures/cli.xml

@@ -0,0 +1,5 @@
+<?xml version="1.0"?>
+<rss version="2.0"><channel><title>Command Line Show</title><link>http://127.0.0.1:8792/</link>
+<description>Added with ipx add while the daemon runs.</description>
+<item><title>From the Command Line</title><guid>cli-1</guid><description>x</description></item>
+</channel></rss>

tests/ui/fixtures/serve.js

@@ -12,7 +12,10 @@ http.createServer((req, res) => {
   fs.readFile(file, (err, body) => {
     if (err) { res.writeHead(404).end('no'); return; }
     const type = file.endsWith('.mp3') ? 'audio/mpeg'
-      : file.endsWith('.opml') ? 'text/x-opml' : 'application/xml';
+      : file.endsWith('.opml') ? 'text/x-opml'
+      // Artwork as an image, or /api/art refuses it as not one.
+      : file.endsWith('.jpg') ? 'image/jpeg'
+      : file.endsWith('.html') ? 'text/html' : 'application/xml';
     res.writeHead(200, { 'content-type': type, 'content-length': body.length });
     res.end(body);
   });

web/app.css

@@ -566,7 +566,14 @@ input:focus,select:focus{outline:0;border-color:var(--accent)}
 /* A show sits under its folder's title, a size down, so an open folder reads as one. */
 .feed.child{margin-left:11px}
 /* Pinned feeds, at the top of the list: a small pin before the name, and a rule under the last. */
-.feed .fpin .i{width:10px;height:10px;margin-right:5px;vertical-align:-1px;color:var(--accent)}
+/* Pinned: a disc on the artwork's corner, as a failing feed's mark is, in the accent and the ink
+   the theme pairs with it. A feed both pinned and failing keeps the error below, the pin above. */
+.fpin{
+  position:absolute;right:-5px;bottom:-5px;width:17px;height:17px;border-radius:50%;
+  display:grid;place-items:center;background:var(--accent);color:var(--ink);border:2px solid var(--bg);
+}
+.fpin .i{width:9px;height:9px}
+.feed.err .fpin{bottom:auto;top:-5px}
 .feed.lastpin{margin-bottom:9px}
 .feed.lastpin::after{content:"";position:absolute;left:8px;right:8px;bottom:-5px;border-bottom:1px solid var(--line)}
 .feed.child .art{width:28px;height:28px;font-size:11px}
@@ -630,24 +637,102 @@ body.scan-this .fhead [data-a=scan] .i,body.scan-any .fhead [data-a=scanall] .i,
 /* Quiet buttons: a filled one on every row outshouted the row that is playing. */
 #listening [data-a=play]{color:var(--accent)}
 #listening [data-a=remove]{color:var(--faint)}
-/* Directory's filters: .tabs for what a feed is, as the item filters are, and chips for what it
+/* The Directory, laid out as Apple's is: a large title, then sections. Every page carries what
-   is about. A picked chip is underlined in --accent2, as the download bar and the now-playing EQ
+   kind of feed (.tabs, as the item filters are) in its title row. */
-   are; a .badge's fill already means unread in the sidebar. */
+.dhead{margin-bottom:6px}
-.dirbar{display:flex;flex-wrap:wrap;align-items:center;gap:8px 14px;margin-top:4px}
+.dback{display:inline-flex;align-items:center;gap:5px;margin:0 0 8px -5px;padding:3px 5px;border-radius:6px;font-size:13px;color:var(--accent)}
-.dirbar .tabs{flex:none}
+.dback .i{width:11px;height:11px}
-.chips{display:flex;flex-wrap:wrap;gap:2px 6px;flex:1 1 0;min-width:0}
+.dback:hover{background:var(--raise)}
-.chips button{flex:none;padding:4px 6px;font-size:13px;color:var(--dim);white-space:nowrap;border-bottom:2px solid transparent}
+.dtitle{display:flex;flex-wrap:wrap;align-items:center;justify-content:space-between;gap:10px 18px}
-.chips button:hover{color:var(--fg)}
+.dtitle h2{margin:0;font-size:30px;line-height:1.15;font-weight:700;letter-spacing:-.02em;overflow-wrap:anywhere}
-.chips button[aria-pressed="true"]{color:var(--fg);border-bottom-color:var(--accent2)}
+.dhead .sub{margin:5px 0 0;color:var(--dim);font-size:13px;font-variant-numeric:tabular-nums}
+.dsec{margin-top:28px}
+.dsech{display:flex;align-items:baseline;justify-content:space-between;gap:12px;margin-bottom:10px}
+.dsech h3{margin:0;font-size:19px;line-height:1.25;font-weight:650;letter-spacing:-.01em}
+.dall{font-size:13px;color:var(--accent);flex:none}
+.dall:hover{text-decoration:underline}
+/* The most subscribed, numbered because they are ranked: down one column, then the next. */
+.chart{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));grid-template-rows:repeat(var(--rows,5),auto);grid-auto-flow:column;column-gap:32px}
+.rank{display:grid;grid-template-columns:22px 52px minmax(0,1fr) auto;align-items:center;gap:12px;padding:8px 0;border-bottom:1px solid var(--line)}
+.rank .no{font-size:17px;font-weight:650;color:var(--faint);text-align:right;font-variant-numeric:tabular-nums}
+.rank .art{width:52px;height:52px;font-size:16px}
+.rank .txt b{display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;overflow-wrap:anywhere;font-weight:500;font-size:14px;line-height:1.3}
+.rank .txt small{display:block;margin-top:2px;color:var(--faint);font-size:12px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
+/* A category: its name, and its three most subscribed shows' covers fanned in the corner, so the
+   shows are what colour it. Not tinted: a tint by the name's hash meant nothing, and the five that
+   drew --good read as yours, which green means everywhere else (#131). The fan opens a little
+   under the pointer, the only thing on the page that moves. */
+.cats{display:grid;grid-template-columns:repeat(auto-fill,minmax(210px,1fr));gap:12px}
+.cat{
+  position:relative;overflow:hidden;height:116px;padding:13px 15px;border-radius:12px;text-align:left;
+  display:flex;flex-direction:column;align-items:flex-start;
+  background:var(--panel);border:1px solid var(--line);
+}
+.cat:hover{border-color:var(--accent)}
+/* The whole width: the fan sits below two lines of name, and a narrower name broke Government
+   in two in the wider faces, Classic's and Glass's. */
+.cat b{font-size:16px;line-height:1.2;font-weight:650;color:var(--fg);overflow-wrap:break-word}
+.cat small{margin-top:3px;font-size:12px;color:var(--dim);font-variant-numeric:tabular-nums}
+.cat .fan{position:absolute;right:14px;bottom:-12px;width:112px;height:80px;pointer-events:none}
+.cat .fan .art{
+  position:absolute;bottom:0;width:62px;height:62px;font-size:16px;border-radius:8px;box-shadow:0 3px 10px rgba(0,0,0,.3);
+  /* Each cover on a layer of its own before the pointer arrives. Given one only as the fan began
+     to open, the browser drew the covers again on the first frames, and again on the way back,
+     and the fan opened in a jerk (#132). All three move, for as long as the eye can follow, as
+     one motion that settles rather than stops. */
+  will-change:transform;
+  transition:rotate .45s cubic-bezier(.2,.7,.2,1),translate .45s cubic-bezier(.2,.7,.2,1);
+}
+.cat .fan .art:nth-last-child(1){right:0;rotate:8deg;translate:0 -12px}
+.cat .fan .art:nth-last-child(2){right:25px;rotate:-3deg;translate:0 -4px}
+.cat .fan .art:nth-last-child(3){right:50px;rotate:-13deg;translate:0 4px}
+.cat:hover .fan .art:nth-last-child(1){rotate:13deg;translate:5px -17px}
+.cat:hover .fan .art:nth-last-child(2){rotate:-2deg;translate:0 -8px}
+.cat:hover .fan .art:nth-last-child(3){rotate:-19deg;translate:-7px 3px}
+/* A category's subcategories, each with how many it holds, then the order. A picked one is
+   drawn as a picked tab is. */
+.dtools{display:flex;align-items:center;gap:10px 14px;margin-bottom:14px}
+.pills{display:flex;flex-wrap:wrap;gap:6px;flex:1 1 0;min-width:0}
+.pills button{flex:none;padding:4px 11px;border:1px solid var(--line);border-radius:999px;font-size:13px;color:var(--dim);white-space:nowrap}
+.pills button small{margin-left:6px;color:var(--faint);font-size:11.5px;font-variant-numeric:tabular-nums}
+.pills button:hover{border-color:var(--accent);color:var(--fg)}
+.pills button.on{background:var(--raise);border-color:var(--raise);color:var(--fg);font-weight:500}
+.dtools .grow{flex:1}
+#dirsort{flex:none;width:auto;padding:4px 6px;font-size:13px}
 /* Square cover art, title underneath: podcast art is made to be known at a glance. The subscribe
    button stays visible, since a button that only shows on hover cannot be reached on a phone. */
-.tiles{display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:18px 14px;margin-top:14px}
+.tiles{display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:18px 14px}
 .tiles>.hint{grid-column:1/-1}
-.tile{display:grid;grid-template-columns:minmax(0,1fr) auto;gap:6px 2px;align-items:start;cursor:pointer}
+.tile{display:grid;grid-template-columns:minmax(0,1fr) auto;gap:6px 2px;align-items:start}
 .tile .art{grid-column:1/-1;width:100%;height:auto;aspect-ratio:1;font-size:34px}
 .tile .txt b{display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;overflow-wrap:anywhere;font-weight:500;font-size:13px;line-height:1.3}
 .tile .txt small{display:block;color:var(--faint);font-size:11.5px}
-.tile:hover .txt b{color:var(--accent)}
+/* Every one opens: yours its feed, the rest their page here. */
+.tile,.rank{cursor:pointer}
+.tile:hover .txt b,.rank:hover .txt b{color:var(--accent)}
+/* A feed's page before you subscribe, as Apple's show page: the cover large, what it is and who
+   here reads it beside it, then what it has published lately. */
+.show{display:flex;gap:22px;align-items:flex-end;margin-top:4px}
+.show>.art{width:180px;height:180px;font-size:48px;border-radius:12px;box-shadow:var(--shadow)}
+.show .meta{min-width:0;flex:1;display:flex;flex-direction:column;align-items:flex-start;gap:5px}
+.show h2{margin:0 0 2px;font-size:30px;line-height:1.15;font-weight:700;letter-spacing:-.02em;overflow-wrap:anywhere}
+.show .sub{margin:0;color:var(--dim);font-size:13px;font-variant-numeric:tabular-nums}
+.show .dfail{color:var(--bad)}
+.show .acts{margin-top:8px}
+.dlink{padding:0;font-size:13px;color:var(--accent);text-align:left}
+.dlink:hover{text-decoration:underline}
+/* What the feed says it is, three lines of it until More. */
+.about{max-width:760px;margin-top:18px}
+.about p{margin:0;color:var(--dim);font-size:14px;line-height:1.55;display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:3;overflow:hidden;overflow-wrap:anywhere}
+.about.open p{display:block}
+.about .dlink{margin-top:4px}
+.latest{max-width:760px}
+.lep{padding:12px 0;border-bottom:1px solid var(--line)}
+.lep b{display:block;font-size:15px;font-weight:600;line-height:1.35;overflow-wrap:anywhere}
+.lep b a{color:var(--fg);text-decoration:none}
+.lep b a:hover{color:var(--accent)}
+.lep p{margin:4px 0 0;color:var(--dim);font-size:13px;line-height:1.5;display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:3;overflow:hidden}
+.lep small{display:flex;align-items:center;gap:7px;margin-top:6px;color:var(--faint);font-size:12px;font-variant-numeric:tabular-nums}
 /* Outlined, not filled: on --raise the warning and error colours fell short of AA in most light
    themes, where on the row's own ground they clear it. */
 .tag{
@@ -676,7 +761,7 @@ body.scan-this .fhead [data-a=scan] .i,body.scan-any .fhead [data-a=scanall] .i,
 /* A selected row is --raise too, and the zero count's pill vanished into it. */
 .sel .badge.zero{background:var(--bg)}
 /* Counts, sizes and times that change in place should not jiggle the text around them. */
-.badge,.feed small,.childrow small,.tile small,.ep,.fhead .sub,.dmeta,#status,#seekrow{font-variant-numeric:tabular-nums}
+.badge,.feed small,.childrow small,.tile small,.rank small,.ep,.fhead .sub,.dmeta,#status,#seekrow{font-variant-numeric:tabular-nums}
 
 /* ---------- main ---------- */
 #main{overflow:hidden;min-height:0;min-width:0;display:flex;flex-direction:column}
@@ -824,7 +909,10 @@ body.adminpage #log{max-width:none}
    icon, so it spilled 3px right of centre and off line with its heading. */
 .ep .st,.ep .fl{width:22px;height:22px;padding:0;border-radius:5px;display:grid;place-items:center;font-size:11px;color:var(--accent2)}
 .ep .fl{color:var(--faint);font-size:13px}
-.ep .fl.on{color:var(--fg)}
+/* Pinned: the pinned feed's disc (.fpin), the accent and its ink, drawn inside the 22px button so
+   the row does not move when it changes. */
+.ep .fl.on,.ep .fl.on:hover{color:var(--ink);background:radial-gradient(circle,var(--accent) 8.5px,transparent 9px)}
+.ep .fl.on .i{width:9px;height:9px}
 /* The icon's EQ bars mark what is playing: standing still, and moving only while it plays. A
    paused animation was tried first; the frames it held were a pixel apart and read as dots. */
 .eq{display:inline-flex;align-items:flex-end;gap:2px;width:13px;height:12px;flex:none}
@@ -837,6 +925,8 @@ body.playing .eq i:nth-child(3){animation-delay:-.6s}
 @keyframes eq{from{transform:scaleY(.35)}}
 .ep .st:hover,.ep .fl:hover{background:var(--raise)}
 .ep .t{font-weight:600;font-size:13.5px;display:block;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
+/* An item with no title shows its opening words: set as text, not as a heading. */
+.ep .t.notitle{font-weight:400}
 .ep.read .t{color:var(--dim);font-weight:500}
 .ep .line{display:flex;gap:9px;align-items:center;flex-wrap:wrap;color:var(--faint);font-size:11.5px}
 .ep .line:empty{display:none}
@@ -937,6 +1027,9 @@ input[type=range]::-moz-range-thumb{width:12px;height:12px;border:0;border-radiu
 .inline{display:flex;gap:6px;align-items:stretch}
 .inline input{flex:1;min-width:0}
 .inline .btn{white-space:nowrap;flex:none}
+/* An API token in Settings: its name and dates, and the button that revokes it. */
+.tokrow{display:flex;align-items:center;justify-content:space-between;gap:8px;margin-bottom:6px}
+.tokrow .hint{display:block}
 .inline select{flex:none;width:auto}
 .inline input[type=number]{flex:none;width:90px}
 .check input{width:16px;height:16px;accent-color:var(--accent)}
@@ -1043,9 +1136,22 @@ body:has(#player.on) #status{padding-bottom:3px}
   #pnow .txt small,#pright #vol{display:none}
   .wrap.plain{padding:14px}
   .card{padding:16px}
-  /* The chips get a line of their own and scroll sideways rather than wrap, so they never push
+  /* The subcategories get a line of their own and scroll sideways rather than wrap, so they
-     the grid down. */
+     never push the grid down. */
-  .chips{flex:1 1 100%;flex-wrap:nowrap;overflow-x:auto}
+  .dtitle h2{font-size:25px}
+  .dsec{margin-top:22px}
+  .dsech h3{font-size:17px}
+  .chart{grid-template-columns:minmax(0,1fr);grid-template-rows:none;grid-auto-flow:row}
+  .cats{grid-template-columns:repeat(auto-fill,minmax(150px,1fr));gap:10px}
+  .cat{height:100px;padding:11px 12px}
+  .cat b{font-size:15px}
+  .cat .fan{right:6px;bottom:-16px;transform:scale(.7);transform-origin:bottom right}
+  .pills{flex:1 1 100%;flex-wrap:nowrap;overflow-x:auto}
+  .dtools{flex-wrap:wrap}
+  .show{flex-direction:column;align-items:flex-start;gap:14px}
+  .show>.art{width:150px;height:150px;font-size:40px}
+  .show h2{font-size:24px}
+  #dirsort{margin-left:auto}
   .tiles{grid-template-columns:repeat(auto-fill,minmax(104px,1fr));gap:14px 10px}
 
   /* A log line has no room for four columns: keep time and message, wrap as prose. */

web/src/admin.ts

@@ -49,6 +49,10 @@ async function drawServer(){
       items are never touched.</span></div>
     <div class="field"><label>Delete items older than (days, 0 = keep)</label>
       <input type="number" id="gage" min="0" value="${g.max_age_days}"></div>
+    <div class="field"><label>Artwork kept (MB, 0 = none)</label>
+      <input type="number" id="gart" min="0" value="${g.art_cache_mb}">
+      <span class="hint">Show and episode artwork is kept here once shown, so it loads from iPX
+      instead of each publisher. Over this, what has gone longest unshown is dropped first.</span></div>
     <div class="field"><label>Download folder</label>
       <span class="hint" style="overflow-wrap:anywhere">${esc(g.download_dir)}</span></div>
     <div class="cardacts"><button class="btn primary" id="gsave">${ICON.check} Save</button></div>`;
@@ -59,7 +63,8 @@ async function drawServer(){
         max_new_per_check: Math.max(0, Number($('#gmax').value) || 0),
         media_types: $('#gtypes').value.split(',').map(t => t.trim()).filter(Boolean),
         max_total_gb: Number($('#gquota').value) || 0,
-        max_age_days: Number($('#gage').value) || 0})});
+        max_age_days: Number($('#gage').value) || 0,
+        art_cache_mb: Math.max(0, Number($('#gart').value) || 0)})});
       toast('Settings saved');
     }catch(e){ toast(e.message, true); }
   };

web/src/dialogs.ts

@@ -34,97 +34,238 @@ $('#addFeed').onclick=()=>{
   };
 };
 
-// What everyone here reads, you included, as a place to start. The rows carry an id, never a
+// The rows carry an id, never a URL, so a key in someone's feed address never reaches this page.
-// URL, so a key in someone's feed address never reaches this page.
 const NONE_LISTED='<p class="hint">Nothing yet. Feeds people here subscribe to show up here.</p>';
-async function listFeeds(url,box){
-  let rows=[];
-  try{ rows=await api(url)||[]; }catch{}
-  box.innerHTML=rows.length?'':NONE_LISTED;
-  for(const p of rows) box.appendChild(listedFeed(p,'childrow'));
-  return rows.length;
-}
 
-/// One listed feed: a row in Popular and the Add a feed dialog, a tile in Directory's grid. The
+/// One listed feed: a tile in the Directory's grid or, given its place, a row in its chart. The
-/// parts are the same either way; the class lays them out.
+/// parts are the same either way; the class lays them out. Yours opens its feed; any other, its
-function listedFeed(p,cls){
+/// page in the Directory, to see what it is before you take it (#128).
+function listedFeed(p,cls,rank?: number){
   const el=document.createElement('div');
-  el.className=cls;
+  el.className=cls+(p.subscribed?' mine':'');
-  el.innerHTML=artHTML(p.image,p.title||p.id)+
+  el.tabIndex=0; el.dataset.id=p.id;
-    `<div class="txt"><b>${esc(p.title||p.id)}</b>`+
+  // As the feed list's rows: Enter or Space opens it, and stops there, or Space would also
-    `<small class="meta">${p.subscribers} subscriber${p.subscribers===1?'':'s'}</small></div>`+
+  // play or pause.
+  el.onkeydown=ev=>{
+    if((ev.key==='Enter'||ev.key===' ')&&ev.target===el){ ev.preventDefault(); ev.stopPropagation(); el.click(); }
+  };
+  // A chart says what a show is about, as Apple's does; the grid says who here reads it, when
+  // anyone does. "0 subscribers" under most of the grid said nothing.
+  const meta=rank?(p.subcategory||p.category):p.subscribers?plural(p.subscribers,'subscriber'):'';
+  el.innerHTML=(rank?`<span class="no">${rank}</span>`:'')+artHTML(p.image,NAME(p))+
+    `<div class="txt"><b>${esc(NAME(p))}</b>${meta?`<small class="meta">${esc(meta)}</small>`:''}</div>`+
     // Green, as a downloaded file is: it is already yours. Plus, beside it, is the way to get one.
     (p.subscribed?`<span class="subbed" title="Subscribed: click to open it" aria-label="Subscribed">${ICON.subbed}</span>`
       :`<button class="btn ico" data-a="sub" title="Subscribe" aria-label="Subscribe">${ICON.plus}</button>`);
-  // Yours already: the row opens it instead.
+  if(p.subscribed){ el.onclick=()=>selectFeed(p.id); return el; }
-  if(p.subscribed){ el.onclick=()=>{ closeModal(); selectFeed(p.id); }; return el; }
+  el.onclick=()=>{ dirShow=p.id; $('#content').scrollTop=0; drawDirectory(undefined,'.dback'); };
-  $('[data-a="sub"]',el).onclick=async()=>{
+  // Its own click, not the tile's as well.
-    try{
+  $('[data-a="sub"]',el).onclick=ev=>{ ev.stopPropagation(); subscribeListed(p); };
-      await api(`/api/popular/${encodeURIComponent(p.id)}`,{method:'POST'});
-      closeModal(); toast(`Subscribed to ${p.title||p.id}`);
-      await loadFeeds(true); selectFeed(p.id);
-    }catch(e){ toast(e.message,true); }
-  };
   return el;
 }
 
-// Directory's filters. Kept out here because a finished scan redraws the pane, which would
+async function subscribeListed(p){
-// otherwise clear them.
+  try{
-let dirKind='All', dirCat=null;
+    await api(`/api/popular/${encodeURIComponent(p.id)}`,{method:'POST'});
-const KINDS={All:()=>true,Podcasts:p=>p.podcast,Blogs:p=>!p.podcast};
+    toast(`Subscribed to ${NAME(p)}`);
-/// Directory: every listed feed as its cover art, under two filters that combine: what a feed is
+    await loadFeeds(true); selectFeed(p.id);
-/// (Podcasts, anything with audio or video, or Blogs, the rest) and what it is about (its iTunes
+  }catch(e){ toast(e.message,true); }
-/// category, as chips). Both filter in place, without asking the server again.
-async function renderDirectory(url,box){
-  let rows=[];
-  try{ rows=await api(url)||[]; }catch{}
-  if(!rows.length){ box.innerHTML=NONE_LISTED; return 0; }
-  const bar=$('#dirbar');
-  // Only a filter when the server has both kinds.
-  const both=rows.some(KINDS.Podcasts)&&rows.some(KINDS.Blogs);
-  const btn=(k,v,on)=>`<button type="button" data-${k}="${esc(v)}" class="${on?'on':''}" aria-pressed="${on}">${esc(v)}</button>`;
-  const draw=()=>{
-    if(!both) dirKind='All';
-    const ofKind=rows.filter(KINDS[dirKind]);
-    // No empty chips: only the categories among the feeds the kind lets through.
-    const cats=[...new Set(ofKind.map(p=>p.category).filter(Boolean))].sort();
-    if(!cats.includes(dirCat)) dirCat=null;
-    bar.innerHTML=
-      (both?`<div class="tabs" role="group" aria-label="Kind">${Object.keys(KINDS).map(k=>btn('kind',k,k===dirKind)).join('')}</div>`:'')+
-      (cats.length?`<div class="chips" role="group" aria-label="Category">${cats.map(c=>btn('cat',c,c===dirCat)).join('')}</div>`:'');
-    // A picked chip lifts on a second press. Everything is redrawn, so the keyboard goes back to
-    // the button just pressed.
-    for(const b of $$('button',bar)) b.onclick=()=>{
-      const k=b.dataset.kind!=null?'kind':'cat', v=b.dataset[k];
-      if(k==='kind') dirKind=v; else dirCat=dirCat===v?null:v;
-      draw(); $(`[data-${k}="${CSS.escape(v)}"]`,bar)?.focus();
-    };
-    box.innerHTML='';
-    for(const p of ofKind.filter(p=>!dirCat||p.category===dirCat)) box.appendChild(listedFeed(p,'tile'));
-  };
-  draw();
-  return rows.length;
 }
 
-/// Directory and Popular open in the main pane, as the original's Directory did.
+// Where you are in the Directory: what kind of feed, which category and subcategory, in what
+// order. Out here because a search redraws the page, and the kind and order hold across visits.
+let dirKind='All', dirCat=null, dirSub=null, dirSort='az', dirRows=[];
+// The feed whose page is open over all that, by id; going back leaves the rest as it was.
+let dirShow=null;
+// The two pages that are not a category: every feed, and the feeds that name none. A category is
+// Apple's or one an admin typed, neither of which starts with a colon.
+const EVERY=':every', OTHER=':other';
+const NAME=p=>(p.title||p.id);
+const byName=(a,b)=>NAME(a).localeCompare(NAME(b),undefined,{sensitivity:'base',numeric:true});
+const bySubs=(a,b)=>b.subscribers-a.subscribers||byName(a,b);
+/// The grid's orders. It opens A to Z, as the server sends it.
+const DIR_SORTS={
+  subscribers:['Most subscribers',bySubs],
+  az:['Name, A to Z',byName],
+  za:['Name, Z to A',(a,b)=>byName(b,a)],
+};
+const KINDS={All:()=>true,Podcasts:p=>p.podcast,Blogs:p=>!p.podcast};
+const KIND_WORD={All:'feed',Podcasts:'podcast',Blogs:'blog'};
+
+/// The Directory, as Apple's is laid out: it opens on the most subscribed feeds and the
+/// categories, and a category has a page of its own, its subcategories along the top of its
+/// grid. What a feed is (Podcasts, anything with audio or video, or Blogs, the rest) holds across
+/// every page, and the search box looks through the lot. One fetch draws all of it.
+async function renderDirectory(url,box){
+  // Each visit starts at the front, as going back to a tab does.
+  dirCat=null; dirSub=null; dirShow=null;
+  try{ dirRows=await api(url)||[]; }catch{ dirRows=[]; }
+  drawDirectory(box);
+  return dirRows.length;
+}
+
+/// `focus` is the control to put the keyboard back on, since the whole page is redrawn.
+function drawDirectory(box=$('#directory'), focus?: string){
+  if(!box) return;
+  if(!dirRows.length){ box.innerHTML=NONE_LISTED; return; }
+  // Only a choice when the server has both kinds.
+  const both=dirRows.some(KINDS.Podcasts)&&dirRows.some(KINDS.Blogs);
+  if(!both) dirKind='All';
+  const rows=dirRows.filter(KINDS[dirKind]), word=KIND_WORD[dirKind];
+  const count=n=>`${n.toLocaleString()} ${word}${n===1?'':'s'}`;
+  const of=c=>p=>c===EVERY||(c===OTHER?!p.category:p.category===c);
+  // A kind can leave a category with nothing in it; that goes back to the front.
+  if(dirCat&&!rows.some(of(dirCat))) dirCat=null;
+  const catName=c=>c===EVERY?`All ${word}s`:c===OTHER?'Other':c;
+  const top=list=>list.filter(p=>p.subscribers).sort(bySubs).slice(0,10);
+  const btn=(k,v,label,on)=>`<button type="button" data-${k}="${esc(v)}" aria-pressed="${on}"${on?' class="on"':''}>${label}</button>`;
+  // back: where its button goes, '' for the front; none on the front itself.
+  const head=(title,sub,back?)=>`<div class="dhead">${back==null?'':
+      `<button type="button" class="dback" data-go="${esc(back)}">${ICON.left}${esc(back?catName(back):'Directory')}</button>`}
+    <div class="dtitle"><h2>${esc(title)}</h2>${both?`<div class="tabs" role="group" aria-label="Kind">${
+      Object.keys(KINDS).map(k=>btn('kind',k,k,k===dirKind)).join('')}</div>`:''}</div>
+    <p class="sub">${esc(sub)}</p></div>`;
+  const sec=(title,body,extra='')=>`<section class="dsec">${title?`<div class="dsech"><h3>${esc(title)}</h3>${extra}</div>`:''}${body}</section>`;
+  const tools=(pills='')=>`<div class="dtools">${pills||'<span class="grow"></span>'}<select id="dirsort" aria-label="Sort">${
+    Object.entries(DIR_SORTS).map(([k,[label]])=>`<option value="${k}"${k===dirSort?' selected':''}>${label}</option>`).join('')}</select></div>`;
+  const q=S.q.trim(), grid='<div class="tiles" id="dtiles"></div>', board='<div class="chart" id="dchart"></div>';
+  let html, chart=[], tiles=null;
+  const show=dirShow&&dirRows.find(p=>p.id===dirShow);
+  if(show){
+    const n=show.subscribers;
+    html=`<div class="dhead"><button type="button" class="dback" data-back>${ICON.left}${esc(q?'Results':dirCat?catName(dirCat):'Directory')}</button></div>
+      <div class="show">${artHTML(show.image,NAME(show))}<div class="meta"><h2>${esc(NAME(show))}</h2>${
+        // Its category's page is one click away from here.
+        show.category?`<button type="button" class="dlink" data-go="${esc(show.category)}">${esc(show.subcategory||show.category)}</button>`:''}
+        ${n?`<p class="sub">${n} ${n===1?'person here subscribes':'people here subscribe'}</p>`:''}
+        <p class="sub dfail" id="dfail" hidden></p>
+        <div class="acts"><button class="btn ico primary" data-a="sub" title="Subscribe" aria-label="Subscribe">${ICON.plus}</button></div></div></div>
+      <div class="about" id="dabout" hidden><p></p><button type="button" class="dlink" aria-expanded="false" hidden>More</button></div>`
+      +sec(show.podcast?'Latest episodes':'Latest posts','<div class="latest" id="dshow"><p class="hint">Loading…</p></div>');
+  }else if(q){
+    // Names only: a category is a tap away, and "news" matching every News feed buried the
+    // one called that.
+    tiles=rows.filter(p=>NAME(p).toLocaleLowerCase().includes(q.toLocaleLowerCase()));
+    html=head(`Results for “${q}”`,tiles.length?count(tiles.length):`No ${word} here has that in its name.`,dirCat||'')+tools()+grid;
+  }else if(!dirCat){
+    chart=top(rows);
+    const cats=new Map<string,any[]>();
+    for(const p of rows){ const c=p.category||OTHER; if(!cats.has(c)) cats.set(c,[]); cats.get(c).push(p); }
+    // Other, the feeds with no category, goes last.
+    const order=[...cats.keys()].sort((a,b)=>Number(a===OTHER)-Number(b===OTHER)||a.localeCompare(b));
+    html=head('Directory',`${count(rows.length)} on this server. Private feeds are never listed.`)
+      +(chart.length?sec('Most subscribed',board):'')
+      +sec('Categories',`<div class="cats">${order.map(c=>catTile(c,catName(c),count(cats.get(c).length),cats.get(c))).join('')}</div>`,
+        `<button type="button" class="dall" data-go="${EVERY}">See all ${rows.length.toLocaleString()}</button>`);
+  }else{
+    const inCat=rows.filter(of(dirCat));
+    const subs=dirCat===EVERY?[]:[...new Set(inCat.map(p=>p.subcategory).filter(Boolean))].sort();
+    if(!subs.includes(dirSub)) dirSub=null;
+    if(dirCat!==EVERY) chart=top(inCat);
+    tiles=inCat.filter(p=>!dirSub||p.subcategory===dirSub);
+    const pill=(v,label,n)=>btn('sub',v,`${esc(label)}<small>${n}</small>`,(dirSub||'')===v);
+    html=head(catName(dirCat),dirCat===OTHER?`${count(inCat.length)} that name no category`:count(inCat.length),'')
+      +(chart.length?sec('Most subscribed',board):'')
+      // Its own heading only when the chart is above it, to say where one list ends.
+      +sec(chart.length?`${word[0].toUpperCase()+word.slice(1)}s`:'',tools(subs.length?`<div class="pills" role="group" aria-label="Subcategory">${
+        pill('','All',inCat.length)+subs.map(s=>pill(s,s,inCat.filter(p=>p.subcategory===s).length)).join('')}</div>`:'')+grid);
+  }
+  box.innerHTML=html;
+  if(show){ $('.show [data-a="sub"]',box).onclick=()=>subscribeListed(show); fillShow(box,show); }
+  const c=$('#dchart',box);
+  if(c){ c.style.setProperty('--rows',String(Math.ceil(chart.length/2))); chart.forEach((p,i)=>c.appendChild(listedFeed(p,'rank',i+1))); }
+  const t=$('#dtiles',box);
+  if(t) for(const p of tiles.sort(DIR_SORTS[dirSort][1])) t.appendChild(listedFeed(p,'tile'));
+  box.onclick=ev=>{
+    const b=(ev.target as Element).closest<HTMLElement>('[data-go],[data-kind],[data-sub],[data-back]');
+    if(!b) return;
+    // Back from a feed's page, onto the tile or row it was opened from.
+    if(b.dataset.back!=null){ const id=dirShow; dirShow=null; return drawDirectory(box,`[data-id="${CSS.escape(id)}"]`); }
+    if(b.dataset.kind!=null){ dirKind=b.dataset.kind; return drawDirectory(box,`[data-kind="${CSS.escape(dirKind)}"]`); }
+    if(b.dataset.sub!=null){ dirSub=b.dataset.sub||null; return drawDirectory(box,`[data-sub="${CSS.escape(b.dataset.sub)}"]`); }
+    // Going anywhere ends a search.
+    if(S.q){ S.q=''; $('#epSearch').value=''; }
+    const from=dirCat;
+    dirCat=b.dataset.go||null; dirSub=null; dirShow=null;
+    $('#content').scrollTop=0;
+    // Into a page, onto its way back; back out, onto the tile you went in by.
+    drawDirectory(box,dirCat?'.dback':from?`[data-go="${CSS.escape(from)}"]:not(.dback)`:undefined);
+  };
+  const s=$('#dirsort',box);
+  if(s) s.onchange=()=>{ dirSort=s.value; drawDirectory(box,'#dirsort'); };
+  if(focus) $(focus,box)?.focus();
+}
+
+/// A listed feed's page, filled in: what it says it is, its first lines unless you ask for the
+/// rest, and its latest items, each one's title, opening its post on the publisher's site where
+/// it has one, its first lines, and when and how long.
+async function fillShow(box,p){
+  const list=$('#dshow',box), about=$('#dabout',box);
+  let got;
+  try{ got=await api(`/api/directory/${encodeURIComponent(p.id)}`); }
+  catch(e){ list.innerHTML=`<p class="hint">${esc(e.message)}</p>`; return; }
+  // Gone back, or on to another, while it loaded.
+  if(dirShow!==p.id||!list.isConnected) return;
+  // Some feeds give their title again as their description, which says nothing new.
+  const said=plainText(got.description);
+  if(said&&said!==NAME(p)){
+    const text=$('p',about), more=$('button',about);
+    text.textContent=said; about.hidden=false;
+    // Only when there is more than the three lines shown.
+    more.hidden=text.scrollHeight<=text.clientHeight+1;
+    more.onclick=()=>{
+      const open=about.classList.toggle('open');
+      more.textContent=open?'Less':'More'; more.setAttribute('aria-expanded',String(open));
+    };
+  }
+  // A broken feed says so, or it reads as a quiet one (#142).
+  if(got.failing){ const f=$('#dfail',box); f.textContent=got.failing; f.hidden=false; }
+  const items=got.items;
+  list.innerHTML=items.length?items.map(e=>{
+    const name=entryName(e), text=name.derived?'':plainText(e.description).slice(0,500);
+    const when=[dateOf(e.published),e.duration&&clock(e.duration)].filter(Boolean);
+    return `<article class="lep"><b>${e.link?`<a href="${esc(e.link)}" target="_blank" rel="noopener">${esc(name.text)}</a>`:esc(name.text)}</b>`
+      +(text?`<p>${esc(text)}</p>`:'')
+      +(when.length?`<small>${when.map(s=>`<span>${s}</span>`).join('<span class="dot"></span>')}</small>`:'')+'</article>';
+  }).join(''):`<p class="hint">Nothing read from it yet.${p.subscribers||got.failing?''
+    :' A feed nobody here subscribes to is checked once a day.'}</p>`;
+}
+
+/// A category as a tile: its name, how many feeds, and the covers of its three most subscribed
+/// fanned in the corner, so each one is coloured by what is in it, as Apple's are.
+function catTile(c,name,count,rows){
+  // Covers before initials, which are all Other had; the likeliest first, and first is in front:
+  // the fan is drawn back to front.
+  const art=[...rows].sort((a,b)=>Number(!a.image)-Number(!b.image)||bySubs(a,b)).slice(0,3).reverse();
+  return `<button type="button" class="cat" data-go="${esc(c)}"><b>${esc(name)}</b><small>${count}</small>`+
+    `<span class="fan" aria-hidden="true">${art.map(p=>artHTML(p.image,NAME(p))).join('')}</span></button>`;
+}
+
+/// The Directory and Currently Listening open in the main pane, as the original's Directory did.
 async function renderListed(v){
   const box=$('#content');
   box.classList.add('plain');
   $('#tbRemove').disabled=true;
   syncTools(null);
+  if(v===VIEWS[':directory']){
+    $('#epSearch').placeholder='Find a feed…';
+    box.innerHTML='<div id="directory"><p class="hint">Loading…</p></div>';
+    $('#count').textContent=v.title;
+    const n=await renderDirectory(v.url,$('#directory'));
+    if(VIEWS[S.feed]===v) $('#count').textContent=`${v.title}: ${plural(n,'feed')}`;
+    return;
+  }
   $('#epSearch').placeholder='Search items…';
-  const listening=v===VIEWS[':listening'], grid=v===VIEWS[':directory'];
   box.innerHTML=`
     <div class="fhead slim">
       <div class="art">${v.icon}</div>
-      <div class="meta"><h2>${v.title}</h2>
+      <div class="meta"><h2>${v.title}</h2><div class="sub">${v.blurb}</div></div>
-        <div class="sub">${v.blurb}${listening?'':' Everyone counts, you included. Private feeds are never listed.'}</div></div>
     </div>
-    ${grid?'<div class="dirbar" id="dirbar"></div>':''}
+    <div class="childlist" id="listening"><p class="hint">Loading…</p></div>`;
-    <div class="${grid?'tiles':'childlist'}" id="${listening?'listening':'popular'}"><p class="hint">Loading…</p></div>`;
   $('#count').textContent=v.title;
-  const n=await (listening?renderListening:grid?renderDirectory:listFeeds)(v.url,$(listening?'#listening':'#popular',box));
+  // The search box looks through these too, title and text, as it does a feed's items (#127).
-  if(VIEWS[S.feed]===v) $('#count').textContent=`${v.title}: ${plural(n,listening?'episode':'feed')}`;
+  const n=await renderListening(S.q?`${v.url}&q=${encodeURIComponent(S.q)}`:v.url,$('#listening',box));
+  if(VIEWS[S.feed]===v) $('#count').textContent=`${v.title}: ${plural(n,'episode')}`;
 }
 
 /// Currently Listening: episodes you started and have not finished, across every feed you
@@ -133,14 +274,15 @@ async function renderListed(v){
 async function renderListening(url,box){
   let rows=[];
   try{ rows=(await api(url)).entries||[]; }catch{}
-  box.innerHTML=rows.length?'':'<p class="hint">Nothing in progress. Episodes you start and do not finish show up here.</p>';
+  box.innerHTML=rows.length?'':S.q?`<p class="hint">Nothing you are listening to mentions “${esc(S.q)}”.</p>`
+    :'<p class="hint">Nothing in progress. Episodes you start and do not finish show up here.</p>';
   for(const e of rows){
     // Carries its episode, for paintListenRow to repaint as the player moves.
     const el: HTMLDivElement & {entry?: any}=document.createElement('div');
     el.className='childrow';
     el.entry=e;
     el.innerHTML=artHTML(e.image||feedArt(e.feed_id),e.title||'')+
-      `<div class="txt"><b>${EQ}<span>${esc(e.title||'(untitled)')}</span></b>`+
+      `<div class="txt"><b>${EQ}<span>${esc(entryName(e).text)}</span></b>`+
       `<small><span class="fd">${esc(feedName(e.feed_id))}</span><span class="left"></span></small></div>`+
       `<button class="iconbtn" data-a="play"></button>`+
       `<button class="iconbtn" data-a="remove" title="Remove from Currently Listening" aria-label="Remove from Currently Listening">${ICON.close}</button>`+
@@ -193,7 +335,12 @@ $('#tbRead').onclick=()=>{ const e=cur(); if(e) epAction('read',e,null); };
 $('#tbFlag').onclick=()=>{ const e=cur(); if(e) epAction('flag',e,null); };
 let searchT;
 $('#epSearch').oninput=ev=>{ clearTimeout(searchT);
-  searchT=setTimeout(()=>{ S.q=ev.target.value; S.offset=0; loadEntries(); },250); };
+  // The Directory looks through what it already has; Currently Listening and the rest ask the
+  // server (#126, #127).
+  searchT=setTimeout(()=>{ S.q=ev.target.value; S.offset=0;
+    if(S.feed===':directory'){ dirShow=null; drawDirectory(); }
+    else if(S.feed===':listening') renderListed(VIEWS[':listening']);
+    else loadEntries(); },250); };
 // Crossing the phone breakpoint moves the files between their pane and the text.
 window.matchMedia?.('(max-width:820px)')?.addEventListener?.('change',()=>{ const e=cur(); if(e) showDetail(e); });
 
@@ -239,12 +386,35 @@ async function prefsModal(){
       <span class="hint">Comma separated words or phrases, in every feed you read. An item with
       one in its title or text is hidden from you and not downloaded for you. Each feed's
       settings can add more.</span></div>
+    <div class="field"><label for="tokname">API tokens</label>
+      <div id="tokens"></div>
+      <div class="inline"><input type="text" id="tokname" placeholder="What it is for">
+        <button class="btn" id="tokadd" title="Make an API token" aria-label="Make an API token">${ICON.plus} Make</button></div>
+      <span class="hint">A token lets a script or an agent use iPX as you, adding and removing
+      your feeds and reading your items: it sends <code>Authorization: Bearer</code> and the token.
+      Anyone holding one is you here, so revoke one you no longer use.</span></div>
     <div class="field"><label>Feeds are checked every</label>
       <span class="hint">${everyText(g.every_mins)}, for every feed that does not set its own.
       ${admin?'This and the rest of the server\'s settings are on the <a href="/admin">admin page</a>.':'Only an admin changes this.'}</span></div>`);
   $('#stheme').onchange=e=>setTheme(e.target.value,undefined,true);
   $('#smode').onchange=e=>setTheme(undefined,e.target.value,true);
   $('#gopml').onclick=opmlModal;
+  drawTokens();
+  $('#tokadd').onclick=async()=>{
+    const name=$('#tokname').value.trim(); if(!name){ $('#tokname').focus(); return; }
+    try{
+      const t=await api('/api/tokens',{method:'POST',body:JSON.stringify({name})});
+      $('#tokname').value='';
+      await drawTokens();
+      // Shown this once: only its hash is kept, so a lost token is revoked and made again.
+      $('#tokens').insertAdjacentHTML('afterbegin',`<div class="field" id="toknew"><span class="hint">Your new token,
+        ${esc(t.name)}. Copy it now: it is not shown again.</span>
+        <div class="inline"><input type="text" id="tokval" readonly value="${esc(t.token)}">
+        <button class="btn ico" id="tokcopy" title="Copy" aria-label="Copy">${ICON.copy}</button></div></div>`);
+      $('#tokcopy').onclick=()=>copyText(t.token,$('#tokcopy'));
+      $('#tokval').select();
+    }catch(err){ toast(err.message,true); }
+  };
   $('#sblock').onchange=async e=>{
     const blocked=splitWords(e.target.value);
     try{
@@ -255,6 +425,19 @@ async function prefsModal(){
   };
 }
 
+/// Your API tokens, each with when it was made and last used, and a button to revoke it.
+async function drawTokens(){
+  const box=$('#tokens'); if(!box) return;
+  const list=await api('/api/tokens').catch(()=>[]);
+  box.innerHTML=list.map(t=>`<div class="tokrow"><span><b>${esc(t.name)}</b>
+    <span class="hint">made ${dateOf(t.created)}, last used ${ago(t.last_used)}</span></span>
+    <button class="btn ico" data-tok="${t.id}" title="Revoke ${esc(t.name)}" aria-label="Revoke ${esc(t.name)}">${ICON.trash}</button></div>`).join('');
+  for(const b of $$('[data-tok]',box)) b.onclick=async()=>{
+    try{ await api(`/api/tokens/${b.dataset.tok}`,{method:'DELETE'}); toast('Revoked'); drawTokens(); }
+    catch(err){ toast(err.message,true); }
+  };
+}
+
 const splitWords=(s: string)=>s.split(',').map(w=>w.trim()).filter(Boolean);
 
 function settingsModal(f, newUrl?: string){
@@ -307,7 +490,7 @@ function settingsModal(f, newUrl?: string){
   // Offer the categories the Directory already shows, so a blog about games joins Games rather
   // than starting a second chip beside it.
   if($('#scats')) api('/api/directory').then(rows=>{ $('#scats').innerHTML=[...new Set((rows||[])
-    .map(p=>p.category).filter(Boolean))].sort().map(c=>`<option value="${esc(c)}">`).join(''); }).catch(()=>{});
+    .flatMap(p=>[p.category,p.subcategory]).filter(Boolean))].sort().map(c=>`<option value="${esc(c)}">`).join(''); }).catch(()=>{});
   $('#ssave').onclick=async()=>{
     const max=$('#smax').value;
     try{

web/src/feeds.ts

@@ -28,10 +28,8 @@ const unreadFirst=(a,b)=>Number(b.unread>0)-Number(a.unread>0);
 // The original's source list opened with these, above the feeds. They are places, not feeds:
 // an id starting with ':' can never be a feed's, since feed ids are slugs.
 const VIEWS={
-  ':directory':{title:'Directory',icon:ICON.directory,url:'/api/directory',
+  // The Directory holds the most subscribed feeds too, which had a place of their own, Popular (#124).
-    blurb:'Every feed anyone on this server subscribes to, A to Z. The feeds inside an OPML are listed one by one, not the OPML.'},
+  ':directory':{title:'Directory',icon:ICON.directory,url:'/api/directory'},
-  ':popular':{title:'Popular',icon:ICON.popular,url:'/api/popular',
-    blurb:'The ten feeds with the most subscribers here. The feeds inside an OPML count one by one, not the OPML.'},
   ':listening':{title:'Currently Listening',icon:ICON.audio,url:'/api/entries?filter=in_progress&limit=50',
     blurb:'Episodes you started and have not finished, across every feed you subscribe to. Pick one up where you left off.'},
   ':all':{title:'All Subscriptions',icon:ICON.all},
@@ -68,6 +66,11 @@ function renderFeeds(){
     const row=back&&list.querySelector(`[data-id="${CSS.escape(back[0])}"]`);
     if(row) (back[1]&&$('.chev',row)||row).focus();
   };
+  // The images already drawn, to go into the rows that replace theirs. Made anew, every icon in
+  // the list went blank and loaded again, once a frame through a check of every feed, as each
+  // feed's new row came in (#134). Matched by their markup, so a changed one is made anew.
+  const drawn=new Map<string,HTMLImageElement[]>();
+  for(const img of $$('img',list)){ const k=img.outerHTML; if(!drawn.has(k)) drawn.set(k,[]); drawn.get(k).push(img); }
   list.innerHTML='';
   const unreadAll=S.feeds.reduce((n,f)=>n+(f.unread||0),0);
   const places=document.createElement('div');
@@ -130,8 +133,9 @@ function renderFeeds(){
       // The mark sits on the artwork, the thing the eye scans the list by, and the line under
       // the name says what is wrong in place of the counts.
       `<span class="fart">${mine.length?folderArt(f,mine):artHTML(f.image,f.title||f.id)}`+
-      (err?`<span class="ferr" role="img" title="${esc(err)}" aria-label="Error: ${esc(err)}">${ICON.alert}</span>`:'')+`</span>`+
+      (err?`<span class="ferr" role="img" title="${esc(err)}" aria-label="Error: ${esc(err)}">${ICON.alert}</span>`:'')+
-      `<div class="txt"><b>${f.pinned?`<span class="fpin" title="Pinned">${ICON.pinOn}</span>`:''}${esc(f.title||f.id)}</b><small>`+
+      (f.pinned?`<span class="fpin" role="img" title="Pinned" aria-label="Pinned">${ICON.pinOn}</span>`:'')+`</span>`+
+      `<div class="txt"><b>${esc(f.title||f.id)}</b><small>`+
       (nbad?`${plural(nbad,'feed')} not updating`
         :err?esc(f.failing?.reason||'The last check failed')
         :`${mine.length?plural(mine.length,'feed'):plural(eps,'item')} · ${saved} downloaded`)+
@@ -140,6 +144,7 @@ function renderFeeds(){
       `<span class="badge${unread?'':' zero'}" title="${unread} unread">${unread>999?'999+':unread}</span>`;
     el.onclick=()=>{ selectFeed(f.id); nav(false); };
     if(kids) $('.chev',el).onclick=ev=>{ ev.stopPropagation(); toggleGroup(f.id); };
+    for(const img of $$('img',el)){ const was=drawn.get(img.outerHTML)?.shift(); if(was) img.replaceWith(was); }
     list.appendChild(el);
   }
   paintScanning();

web/src/items.ts

@@ -1,7 +1,7 @@
 /* ---------------- items ---------------- */
 let loadSeq=0;
 async function loadEntries(append?: boolean){
-  // Directory and Popular list feeds, not items.
+  // The Directory lists feeds, and Currently Listening draws its own episodes.
   if(!S.feed || VIEWS[S.feed]?.url) return;
   const p=new URLSearchParams({offset:String(S.offset),limit:String(LIMIT),filter:S.filter,sort:S.sort.col,dir:S.sort.dir});
   if(S.q) p.set('q',S.q);
@@ -65,13 +65,14 @@ function epEl(e){
   el.dataset.guid=e.guid;
   const num=[e.season?`S${e.season}`:'',e.episode?`E${e.episode}`:''].filter(Boolean).join('');
   const left = e.position>10 && e.duration ? `${clock(e.duration-e.position)} left` : (e.duration?clock(e.duration):'');
+  const name=entryName(e);
   el.innerHTML=`
     <button class="st" data-a="read" title="Mark ${e.read?'unread':'read'}">${
       player.guid===e.guid?EQ:(e.read?'':'●')}</button>
     <button class="fl${e.flagged?' on':''}" data-a="flag" aria-pressed="${!!e.flagged}" title="${
       e.flagged?'Unpin':'Pin, so it is never deleted'}">${e.flagged?ICON.pinOn:ICON.pin}</button>
     <div class="body">
-      <span class="t">${esc(e.title||'(untitled)')}</span>
+      <span class="t${name.derived?' notitle':''}">${esc(name.text)}</span>
       <div class="line">${[
         num&&`<span>${num}</span>`,
         left&&`<span>${left}</span>`,
@@ -136,6 +137,28 @@ function kindIcon(enc){
 function feedArt(id=S.feed){ const f=S.feeds.find(x=>x.id===id); return f&&f.image; }
 const feedName=id=>{ const f=S.feeds.find(x=>x.id===id); return f?(f.title||f.id):id; };
 
+/// What an item is called. Its title, or for one published without (RSS 2.0 makes it optional,
+/// and Scripting News titles almost none of its posts) the opening of its text, then its file's
+/// name, then its feed and date: "(untitled)" fifty times down a list said nothing about any of
+/// them. `derived` marks a name that is not a title, which the list sets as text, not heading.
+/// Feed HTML as plain words. An inert document: nothing in it loads or runs, where a detached
+/// element fetches its images. A line break or paragraph's end is a space, or "2010)<br>Recorded"
+/// read "2010)Recorded".
+const plainText=(html?: string)=>html
+  ? (new DOMParser().parseFromString(html.replace(/<(br|\/p|\/div|\/li|\/h\d)\b[^>]*>/gi,' $&'),'text/html').body.textContent||'')
+    .replace(/\s+/g,' ').trim() : '';
+function entryName(e): {text: string, derived: boolean}{
+  if(e.title&&e.title.trim()) return {text:e.title, derived:false};
+  const words=plainText(e.description);
+  if(words){
+    const cut=words.length>120 ? words.slice(0,120).replace(/\s+\S*$/,'')+'…' : words;
+    return {text:cut, derived:true};
+  }
+  const file=((e.enclosures||[])[0]?.url||'').split(/[?#]/)[0].split('/').pop().replace(/\.[a-z0-9]{1,5}$/i,'');
+  if(file){ try{ return {text:decodeURIComponent(file), derived:true}; }catch{ return {text:file, derived:true}; } }
+  return {text:[feedName(e.feed_id), dateOf(e.published)].filter(Boolean).join(', '), derived:true};
+}
+
 /// Selecting an item shows it in the pane below, rather than expanding the row.
 /// Replaces one row with a fresh one, leaving the rest of the list and its scroll alone.
 function swapRow(e){
@@ -271,7 +294,8 @@ function detailHtml(e){
   // description was sanitized server-side with ammonia before it ever reached here
   return `
     <button class="btn ico" id="dback" title="Back to the items" aria-label="Back to the items">${ICON.left}</button>
-    <h3 class="dt">${esc(e.title||'(untitled)')}</h3>
+    ${/* A post without a title starts with its text: its own first words as a heading above
+         themselves would read as a mistake. */ e.title&&e.title.trim() ? `<h3 class="dt">${esc(e.title)}</h3>` : ''}
     <div class="dmeta">
       ${/* Joined, so a missing date or number leaves no stray dot behind. */
         [f&&esc(f.title||f.id), num, dateOf(e.published), e.duration&&clock(e.duration)]
@@ -328,7 +352,7 @@ function encBox(x){
   // Nothing on disk. For an image or a PDF you usually just want to look at it, so link
   // straight to the publisher's copy in a new tab -- no download, and nothing proxied
   // through here, which would make ipx a fetch-anything relay.
-  const viewable = !isPlayable(x) && x.state !== 'pending';
+  const viewable = !isPlayable(x) && x.state !== 'pending' && x.state !== 'held';
   return `<div class="encbox">
     ${kindIcon(x)}
     <span class="meta" style="flex:1">${size}</span>
@@ -346,13 +370,13 @@ async function epAction(a: string, e, el, encId?: number){
   const path=`/api/entries/${encodeURIComponent(e.feed_id)}/${encodeURIComponent(e.guid)}`;
   try{
     if(a==='play') play(e, encId!=null ? enc : undefined);
-    if(a==='share') await share(e.title||'', encId!=null ? enc.url : e.link, el);
+    if(a==='share') await share(entryName(e).text, encId!=null ? enc.url : e.link, el);
     if(a==='flag'){ e.flagged=!e.flagged; await api(path+'/flags',{method:'POST',body:JSON.stringify({flagged:e.flagged})}); redraw(); }
     if(a==='read'){ await setRead(e,!e.read); redraw(); }
     if(a==='get'){
       if(!enc) return;
       await api(`/api/enclosures/${enc.id}/download`,{method:'POST'});
-      toast('Queued: '+(e.title||'item'));
+      toast('Queued: '+entryName(e).text);
     }
     if(a==='del'){
       const f=S.feeds.find(x=>x.id===e.feed_id);

web/src/native.ts

@@ -77,7 +77,7 @@ function installNativePlayback(){
       realLoad.call(audio);
       const e = player.entry, f = player.feed;
       post({t:'load', url:v, enc:player.enc, feedId:f, guid:player.guid,
-        title:(e && e.title) || '', feedTitle:feedName(f),
+        title:e ? entryName(e).text : '', feedTitle:feedName(f),
         artwork:(e && e.image) || feedArt(f) || null,
         // Where the host starts is not this: the seek to where you left off is player.ts's, on
         // loadedmetadata, so one piece of code decides it. This is for the host's now-playing

web/src/player.ts

@@ -32,7 +32,7 @@ function play(e,enc=e.enclosures.find(isPlayable)){
     if(e.position>5) audio.addEventListener('loadedmetadata',()=>{audio.currentTime=e.position},{once:true});
     // Initials, if it comes to that, are the feed's: the episode's read as "SE" beside the feed's art.
     $('#partwrap').innerHTML=artHTML(e.image||feedArt(e.feed_id),feedName(e.feed_id));
-    $('#ptitle').textContent=e.title||'(untitled)';
+    $('#ptitle').textContent=entryName(e).text;
     const f=S.feeds.find(x=>x.id===e.feed_id);
     $('#pfeed').textContent=f?(f.title||f.id):'';
     $('#player').classList.add('on');
@@ -45,8 +45,8 @@ function play(e,enc=e.enclosures.find(isPlayable)){
 function mediaSession(e,f){
   if(!('mediaSession' in navigator)) return;
   navigator.mediaSession.metadata=new MediaMetadata({
-    title:e.title||'', artist:f?(f.title||f.id):'', album:f?(f.title||''):'',
+    title:entryName(e).text, artist:f?(f.title||f.id):'', album:f?(f.title||''):'',
-    artwork:(e.image||(f&&f.image))?[{src:e.image||f.image,sizes:'512x512'}]:[],
+    artwork:(e.image||(f&&f.image))?[{src:artSrc(e.image||f.image),sizes:'512x512'}]:[],
   });
   const h={play:()=>audio.play(),pause:()=>audio.pause(),
     seekbackward:()=>audio.currentTime-=15,seekforward:()=>audio.currentTime+=30};
@@ -135,7 +135,7 @@ document.addEventListener('keydown',ev=>{
 // Feedly's keys, vim's j and k among them: a letter to move through items or feeds, g and a
 // letter to go somewhere, ? to list them. None fire with Ctrl, Alt or Cmd held, so the browser's
 // own shortcuts still work, or while a dialog is open.
-const GO={a:':all',d:':directory',p:':popular',l:':listening'};
+const GO={a:':all',d:':directory',l:':listening'};
 let gAt=0;
 function stepEntry(by){
   if(VIEWS[S.feed]?.url||!S.entries.length) return;
@@ -176,10 +176,10 @@ function keysModal(){
   const k=s=>`<kbd>${esc(s)}</kbd>`, g=c=>k('g')+' '+k(c);
   const rows=[
     ['Go to'],
-    [g('a'),'All Subscriptions'],[g('d'),'Directory'],[g('p'),'Popular'],
+    [g('a'),'All Subscriptions'],[g('d'),'Directory'],
     [g('l'),'Currently Listening'],[g('s'),'Settings'],
     [k('Shift')+' '+k('J'),'Next feed'],[k('Shift')+' '+k('K'),'Previous feed'],
-    [k('/'),'Search items'],[k('r'),'Refresh'],[k('['),'Show or hide the feed list'],
+    [k('/'),'Search items, Currently Listening or the Directory'],[k('r'),'Refresh'],[k('['),'Show or hide the feed list'],
     ['Items'],
     [k('j')+' or '+k('n'),'Next item'],[k('k')+' or '+k('p'),'Previous item'],
     [k('Shift')+' '+k('A'),'Mark all read'],

web/src/util.ts

@@ -30,7 +30,6 @@ const ICON={
   close:fa('0 0 384 512','<path fill="currentColor" d="M55.1 73.4c-12.5-12.5-32.8-12.5-45.3 0s-12.5 32.8 0 45.3L147.2 256 9.9 393.4c-12.5 12.5-12.5 32.8 0 45.3s32.8 12.5 45.3 0L192.5 301.3 329.9 438.6c12.5 12.5 32.8 12.5 45.3 0s12.5-32.8 0-45.3L237.8 256 375.1 118.6c12.5-12.5 12.5-32.8 0-45.3s-32.8-12.5-45.3 0L192.5 210.7 55.1 73.4z"/>'), // solid/xmark
   menu:fa('0 0 448 512','<path fill="currentColor" d="M0 96C0 78.3 14.3 64 32 64l384 0c17.7 0 32 14.3 32 32s-14.3 32-32 32L32 128C14.3 128 0 113.7 0 96zM0 256c0-17.7 14.3-32 32-32l384 0c17.7 0 32 14.3 32 32s-14.3 32-32 32L32 288c-17.7 0-32-14.3-32-32zM448 416c0 17.7-14.3 32-32 32L32 448c-17.7 0-32-14.3-32-32s14.3-32 32-32l384 0c17.7 0 32 14.3 32 32z"/>'), // solid/bars
   directory:fa('0 0 448 512','<path fill="currentColor" d="M0 96C0 60.7 28.7 32 64 32l320 0c35.3 0 64 28.7 64 64l0 320c0 35.3-28.7 64-64 64L64 480c-35.3 0-64-28.7-64-64L0 96zm64 0l0 64 64 0 0-64-64 0zm320 0l-192 0 0 64 192 0 0-64zM64 224l0 64 64 0 0-64-64 0zm320 0l-192 0 0 64 192 0 0-64zM64 352l0 64 64 0 0-64-64 0zm320 0l-192 0 0 64 192 0 0-64z"/>'), // solid/table-list
-  popular:fa('0 0 576 512','<path fill="currentColor" d="M309.5-18.9c-4.1-8-12.4-13.1-21.4-13.1s-17.3 5.1-21.4 13.1L193.1 125.3 33.2 150.7c-8.9 1.4-16.3 7.7-19.1 16.3s-.5 18 5.8 24.4l114.4 114.5-25.2 159.9c-1.4 8.9 2.3 17.9 9.6 23.2s16.9 6.1 25 2L288.1 417.6 432.4 491c8 4.1 17.7 3.3 25-2s11-14.2 9.6-23.2L441.7 305.9 556.1 191.4c6.4-6.4 8.6-15.8 5.8-24.4s-10.1-14.9-19.1-16.3L383 125.3 309.5-18.9z"/>'), // solid/star
   all:fa('0 0 512 512','<path fill="currentColor" d="M232.5 5.2c14.9-6.9 32.1-6.9 47 0l218.6 101c8.5 3.9 13.9 12.4 13.9 21.8s-5.4 17.9-13.9 21.8l-218.6 101c-14.9 6.9-32.1 6.9-47 0L13.9 149.8C5.4 145.8 0 137.3 0 128s5.4-17.9 13.9-21.8L232.5 5.2zM48.1 218.4l164.3 75.9c27.7 12.8 59.6 12.8 87.3 0l164.3-75.9 34.1 15.8c8.5 3.9 13.9 12.4 13.9 21.8s-5.4 17.9-13.9 21.8l-218.6 101c-14.9 6.9-32.1 6.9-47 0L13.9 277.8C5.4 273.8 0 265.3 0 256s5.4-17.9 13.9-21.8l34.1-15.8zM13.9 362.2l34.1-15.8 164.3 75.9c27.7 12.8 59.6 12.8 87.3 0l164.3-75.9 34.1 15.8c8.5 3.9 13.9 12.4 13.9 21.8s-5.4 17.9-13.9 21.8l-218.6 101c-14.9 6.9-32.1 6.9-47 0L13.9 405.8C5.4 401.8 0 393.3 0 384s5.4-17.9 13.9-21.8z"/>'), // solid/layer-group
   unread:fa('0 0 512 512','<path fill="currentColor" d="M48 64c-26.5 0-48 21.5-48 48 0 15.1 7.1 29.3 19.2 38.4l208 156c17.1 12.8 40.5 12.8 57.6 0l208-156c12.1-9.1 19.2-23.3 19.2-38.4 0-26.5-21.5-48-48-48L48 64zM0 196L0 384c0 35.3 28.7 64 64 64l384 0c35.3 0 64-28.7 64-64l0-188-198.4 148.8c-34.1 25.6-81.1 25.6-115.2 0L0 196z"/>'), // solid/envelope: a closed letter, not a record button
   audio:fa('0 0 448 512','<path fill="currentColor" d="M64 224c0-88.4 71.6-160 160-160s160 71.6 160 160l0 37.5c-10-3.5-20.8-5.5-32-5.5l-16 0c-26.5 0-48 21.5-48 48l0 128c0 26.5 21.5 48 48 48l16 0c53 0 96-43 96-96l0-160C448 100.3 347.7 0 224 0S0 100.3 0 224L0 384c0 53 43 96 96 96l16 0c26.5 0 48-21.5 48-48l0-128c0-26.5-21.5-48-48-48l-16 0c-11.2 0-22 1.9-32 5.5L64 224z"/>'), // solid/headphones
@@ -145,10 +144,11 @@ const tint=name=>{ let h=0; for(const c of name||'?') h=(h*31+c.charCodeAt(0))>>
 function tileHTML(name,cls){
   return `<div class="art ini ${cls||''}" style="--tint:${tint(name)}">${esc(initials(name))}</div>`;
 }
-/// On the https page, the browser upgrades an http:// image to https, and a host that has no
+/// Every image a feed names comes from iPX, which keeps a copy: fast after the first time, and
-/// https shows nothing (issue #90); ipx fetches those itself.
+/// no publisher's server asked on every visit. It began with http-only artwork, which the https
+/// page could not load (#90).
 function artSrc(url: string){
-  return location.protocol==='https:'&&/^http:\/\//i.test(url) ? '/api/art?u='+encodeURIComponent(url) : url;
+  return /^https?:\/\//i.test(url) ? '/api/art?u='+encodeURIComponent(url) : url;
 }
 function artHTML(url: string | null, name: string, cls?: string){
   return url