Say plainly when a feed is given a category, in the changelog

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.claude/skills/ipx-prod-check/SKILL.md

@@ -32,12 +32,13 @@ lines had no trace id, the access log's time was `ms`, and each event was logged
 | target | fields | what |
 |---|---|---|
 | `ipx::http` | `method`, `path`, `route`, `status`, `duration_ms` | one per web request; `route` is the pattern, empty for an unrouted path |
-| `ipx::scan` | `ev` and the event's own: `feed`, `new`, `downloaded`, `failed`, `bytes`, `msg`, `url`, `feeds`, `reason`; on a failure `error.type` and, from an HTTP error, `http.response.status_code` | the daemon's events, one line each, in words; warnings are feed and download failures |
+| `ipx::scan` | `ev` and the event's own: `feed`, `new`, `downloaded`, `failed`, `bytes`, `msg`, `url`, `feeds`, `reason`, `from`, `to`; on a failure `error.type` and, from an HTTP error, `http.response.status_code` | the daemon's events, one line each, in words; warnings are feed and download failures |
 | `ipx::io` | `ev`, `feeds`, `pending`, `downloaded` on the `status` reply | commands arriving (`-> {...}`) and the healthcheck's answer |
 | `ipx` | message, sometimes fields | start-up, shutdown, account and config messages |
 
 Events (`ev`): `feed_start`, `feed_done` (new, downloaded, failed, torrents), `feed_skip` (not due,
-routine), `feed_error` (msg), `download_done` (bytes), `download_error` (msg, url),
+routine), `feed_error` (msg), `feed_moved` (from, to: a permanent redirect followed, the address
+updated), `download_done` (bytes), `download_error` (msg, url),
 `torrent_deferred`, `reaped`, `scan_done` (feeds checked), `reap_done`, `status` (feeds, pending,
 downloaded: the healthcheck's, every 30s), `error` (msg).
 
@@ -85,6 +86,12 @@ a count says something happened, the lines and traces say why.
    crash or an OOM kill: check `docker inspect iPX -f '{{.State.OOMKilled}} {{.RestartCount}}'`
    and the lines just before it. A pending count that only grows means downloads are not
    keeping up or not running.
+
+   Since 2026-10-02 the daemon sleeps until the next feed is due (at most 10 minutes) instead of
+   scanning every minute (#114), so expect tens of scans in 6 hours, not 360, nearly all with
+   `feeds` above 0; none at all for over 10 minutes means the worker is stuck. And `pending` is
+   the real queue (#113): files a scan will download on its own. Back-catalogue files are
+   `held`, listed but not counted, so it is usually 0 or a handful.
 5. **Slow and failed traces.**
    ```
    $Q traces '{resource.deployment.environment.name="production" && duration > 5s}'

.gitignore

@@ -5,3 +5,4 @@
 /web/dist
 
 .claude/settings.local.json
+__pycache__/

CHANGELOG.md

@@ -7,6 +7,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Feeds that don't set a category, as most blogs don't, now get one automatically, so they show up under a category in the Directory. iPX picks the closest of Apple's podcast categories from the feed's title and latest headlines. This happens once per feed: the first time a new feed is read, or for a feed you already had, the next time you force a refresh. It needs `TYPESAFE_KEY` set. An admin can change the category in the feed's settings.
+- Artwork is kept on disk once shown and loads from iPX, not from each publisher's server: fast after the first time, and still there when the publisher's server is not. The admin page sets how much is kept (500 MB by default).
+- `ipx add --list --category News <url>` puts a feed in the Directory for anyone to subscribe to,
+  and it stays there when its last subscriber leaves. Run for a feed already in the catalogue,
+  it lists it or sets its category.
+- Feeds nobody subscribes to that are dead for a month or quiet for a year are removed, so the
+  Directory lists feeds worth taking.
+
+### Changed
+
+- The Directory browses as Apple's does: a row of categories, and once one is picked, a row of its subcategories (Tech News under News, Video Games under Leisure), instead of one row mixing both.
+- A pinned item in a list wears the same disc as a pinned feed, in the theme's accent, instead of a plain solid pin.
+- The Directory can be sorted by name, A to Z or Z to A, or by most subscribers. It still opens A to Z.
+- API tokens: in Settings, make a named token that lets a script or an agent use iPX as you, sent as `Authorization: Bearer`, and revoke it there. Each shows when it was last used.
+- An item published without a title shows its opening words, in plain text rather than bold, instead of "(untitled)"; one with no text either shows its file's name, or its show and date. Opened, it starts with its text.
+- A feed that has moved for good (a permanent redirect) is followed to its new address, which iPX then reads from, and says so in the log as `feed_moved`. A temporary redirect changes nothing.
+- The daemon sleeps until the next feed is due, at most ten minutes, instead of looking every minute; refreshing or adding a feed still wakes it at once.
+- A pinned feed's pin sits on the corner of its artwork, as a failing feed's mark does, instead of before its name.
+- The Directory lists feeds nobody subscribes to yet; Popular still lists what people subscribe to.
+  A feed nobody subscribes to is checked once a day, and at once when someone subscribes. A
+  listed feed downloads nothing until someone subscribes.
+- The Directory loads faster: it asked the database three questions per feed.
+
+### Fixed
+
+- The download queue, in `ipx status`, `/api/status` and the dashboard, counts only what iPX will download on its own: older episodes beyond a show's limit, and files of feeds that do not download automatically, are listed but no longer counted as waiting.
+- Artwork published on http is stored on https when its host serves it there, so the page loads it directly; the rest still comes through iPX.
+- An iPhone adding the site to its home screen finds the icon at the first address it tries.
+- A feed whose server hangs no longer holds up every scan: a feed gets 30 seconds, and connecting anywhere 10.
+
 ## [0.9.1] - 2026-09-29
 
 ### Added

Cargo.lock

@@ -1820,7 +1820,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
 
 [[package]]
 name = "ipx"
-version = "0.9.1"
+version = "0.9.2-dev"
 dependencies = [
  "ammonia",
  "anyhow",
@@ -1843,6 +1843,7 @@ dependencies = [
  "sea-orm",
  "serde",
  "serde_json",
+ "sha2 0.10.9",
  "tokio",
  "toml",
  "tower",

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "ipx"
-version = "0.9.1"
+version = "0.9.2-dev"
 edition = "2024"
 
 [dependencies]
@@ -25,6 +25,7 @@ rss = "2.1.1"
 sea-orm = { version = "2.0.3", default-features = false, features = ["sqlx-sqlite", "sqlx-postgres", "runtime-tokio-rustls", "macros", "with-json", "sqlite-use-returning-for-3_35"] }
 serde = { version = "1.0.229", features = ["derive"] }
 serde_json = "1.0.151"
+sha2 = "0.10.9"
 tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
 toml = "1.1.5"
 tower = { version = "0.5.3", features = ["util"] }

docs/configuration.md

@@ -4,7 +4,7 @@ Two places. **config.toml** holds what ipx needs before it reaches its database,
 who gets in: where things are (`download_dir`, `socket`, `organize`), `[torrent]` and `[web]`.
 **The database** holds the catalogue of feeds (`[feeds.<id>]` below) and the server settings the
 admin page edits (`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check`,
-`media_types`). Change those in the web UI, or with `ipx add`, `ipx rm` and `ipx import`; they
+`media_types`, `art_cache_mb`). Change those in the web UI, or with `ipx add`, `ipx rm` and `ipx import`; they
 take effect without a restart.
 
 The first time ipx meets a database that holds no catalogue, it takes the feeds and those
@@ -39,9 +39,10 @@ max_total_gb      = 50              # 0 = unlimited
 max_age_days      = 30              # 0 = keep forever
 max_new_per_check = 3               # per feed, per scan. 0 = unlimited
 media_types       = ["audio", "video"]
+art_cache_mb      = 500             # artwork kept on disk. 0 = none
 ```
 
-`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check` and `media_types` move into the
+`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check`, `media_types` and `art_cache_mb` move into the
 database as described above; `download_dir`, `socket` and `organize` stay in config.toml.
 
 * **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
@@ -54,6 +55,10 @@ database as described above; `download_dir`, `socket` and `organize` stay in con
   read it. `0` disables it entirely.
 * **`max_age_days`** — items older than this with no file on disk are pruned from the database.
   Kept ones stay. `0` disables it.
+* **`art_cache_mb`** — how much show and episode artwork iPX keeps on disk, in `art/` beside the
+  database, so the page loads it from iPX rather than from every publisher's server. Over this,
+  what has gone longest unshown goes first. `0` keeps none: artwork still comes through iPX, fetched
+  each time. 500 by default.
 * **`max_new_per_check`** — how many of a feed's newest episodes are downloaded; older ones stay
   listed to download by hand. It stops a new subscription pulling a whole back catalogue.
   `0` means every episode, for an archive; set it on the feeds you want archived, since on the
@@ -124,6 +129,7 @@ folder             = "Accidental Tech Podcast"   # default: the feed title
 schedule           = "every 6h"                  # overrides [general] for this feed
 media_types        = ["audio"]                   # overrides [general] for this feed
 category           = "Technology"                # the Directory's, if the feed names none
+listed             = true                        # in the Directory with no subscribers
 username           = "ray"                       # HTTP basic auth
 password_env       = "IPX_ATP_PASS"              # preferred over a literal `password`
 ```
@@ -133,6 +139,12 @@ With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
 config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
 and are the same for everyone. See [users.md](users.md).
 
+**The Directory** lists every feed in the catalogue, whether or not anyone subscribes to it yet.
+To put one there for others to find, `ipx add --list --category News <url>`: it subscribes no one,
+and a listed feed stays when its last subscriber leaves, where another is dropped. A feed nobody
+subscribes to is checked once a day. One that is dead (failing for 30 days) or quiet (nothing
+new in a year), with nobody subscribed and no file on disk, is removed from the catalogue.
+
 Feeds derived from a subscribed OPML are **not** in the catalogue: the OPML is the source of truth
 and they are re-derived on every scan. Editing one in the UI promotes it to a catalogue entry.
 

docs/sso.md

@@ -233,5 +233,19 @@ Set `auto_create_users = false` once everyone who should have an account has one
 proxy vouching for an unknown name is logged and refused. Make people ahead of time instead, with
 the exact name the header will carry.
 
+## API tokens for scripts and agents
+
+Anyone can make API tokens for their own account in Settings. A token is sent as
+`Authorization: Bearer ipx_...` and acts as the person who made it, admin only if they are. Only
+its SHA-256 is kept; it is shown once, when made, and revoked there too.
+
+Through the tunnel, Cloudflare Access turns a request with no Access sign-in towards Authentik
+before ipx sees it, so a token alone does not get in that way. On the LAN, `http://192.168.1.130:8099`
+takes it directly. From outside, make an Access service token, add a Service Auth policy for it to
+the `ipodderx` application, and send `CF-Access-Client-Id` and `CF-Access-Client-Secret` beside the
+`Authorization` header: Access lets the request through, vouches for no name, and ipx takes the
+API token as who is asking. Do not bypass Access for `/api/*` instead; the token would then be the
+only thing between the internet and the API.
+
 See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
 for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.

src/art.rs

@@ -0,0 +1,92 @@
+//! Artwork kept on disk, so the page loads it from iPX: fast after the first time, nothing asked
+//! of each publisher's server for every visit, and still there when a publisher is not.
+
+use std::path::{Path, PathBuf};
+
+pub fn dir() -> PathBuf {
+    crate::config::data_dir().join("art")
+}
+
+/// Where an image's bytes are kept; its content type is beside it, in `<name>.type`.
+// ponytail: std's hasher, 64 bits, keyed by the URL. Its algorithm may change with Rust, which
+// only makes the cache miss once and refill; a collision among a few thousand images is about
+// one in 10^12. Move to a SHA if either ever matters.
+pub fn path(url: &str) -> PathBuf {
+    use std::hash::{Hash, Hasher};
+    let mut h = std::collections::hash_map::DefaultHasher::new();
+    url.hash(&mut h);
+    dir().join(format!("{:016x}", h.finish()))
+}
+
+/// A kept image and its type, marked as just used, which is what keeps it from being trimmed.
+pub fn read(file: &Path) -> Option<(String, Vec<u8>)> {
+    let kind = std::fs::read_to_string(file.with_extension("type")).ok()?;
+    let body = std::fs::read(file).ok()?;
+    if let Ok(f) = std::fs::File::options().write(true).open(file) {
+        let _ = f.set_modified(std::time::SystemTime::now());
+    }
+    Some((kind, body))
+}
+
+/// Keeps an image, written aside and renamed into place, so a page asking for it meanwhile
+/// never reads half of one.
+pub fn write(file: &Path, kind: &str, body: &[u8]) -> std::io::Result<()> {
+    std::fs::create_dir_all(file.parent().unwrap_or(Path::new(".")))?;
+    let tmp = file.with_extension("part");
+    std::fs::write(&tmp, body)?;
+    std::fs::write(file.with_extension("type"), kind)?;
+    std::fs::rename(&tmp, file)
+}
+
+/// Removes the least recently used images until what is kept fits in `limit` bytes; 0 empties it.
+/// Returns how many went.
+pub fn trim(dir: &Path, limit: u64) -> usize {
+    let Ok(entries) = std::fs::read_dir(dir) else { return 0 };
+    let mut kept: Vec<(std::time::SystemTime, u64, PathBuf)> = entries
+        .flatten()
+        .map(|e| e.path())
+        .filter(|p| p.extension().is_none())
+        .filter_map(|p| {
+            let m = p.metadata().ok()?;
+            Some((m.modified().ok()?, m.len(), p))
+        })
+        .collect();
+    let mut total: u64 = kept.iter().map(|(_, n, _)| n).sum();
+    kept.sort();
+    let mut gone = 0;
+    for (_, n, p) in kept {
+        if total <= limit {
+            break;
+        }
+        let _ = std::fs::remove_file(p.with_extension("type"));
+        if std::fs::remove_file(&p).is_ok() {
+            total -= n;
+            gone += 1;
+        }
+    }
+    gone
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn the_least_recently_used_go_first_until_it_fits() {
+        let d = std::env::temp_dir().join(format!("ipx-art-test-{}", std::process::id()));
+        let _ = std::fs::remove_dir_all(&d);
+        for (name, age) in [("old", 300), ("mid", 200), ("new", 100)] {
+            let f = d.join(name);
+            write(&f, "image/png", &[0; 1000]).unwrap();
+            let t = std::time::SystemTime::now() - std::time::Duration::from_secs(age);
+            std::fs::File::options().write(true).open(&f).unwrap().set_modified(t).unwrap();
+        }
+        // Shown just now: no longer the oldest.
+        assert_eq!(read(&d.join("old")).unwrap().0, "image/png");
+        assert_eq!(trim(&d, 2000), 1);
+        assert!(!d.join("mid").exists() && !d.join("mid.type").exists());
+        assert!(d.join("old").exists() && d.join("new").exists());
+        assert_eq!(trim(&d, 0), 2);
+        let _ = std::fs::remove_dir_all(&d);
+    }
+}

src/auth.rs

@@ -40,6 +40,19 @@ pub fn new_session_token() -> String {
     bytes.iter().map(|b| format!("{b:02x}")).collect()
 }
 
+/// A new API token: `ipx_` and a session's 256 random bits, the prefix so one found in a log
+/// or a file says what it opens (#123).
+pub fn new_api_token() -> String {
+    format!("ipx_{}", new_session_token())
+}
+
+/// What is kept of an API token. A fast hash is enough: the token is 256 random bits, not a
+/// password, so there is nothing to guess from a stolen hash.
+pub fn api_token_hash(token: &str) -> String {
+    use sha2::Digest;
+    sha2::Sha256::digest(token.as_bytes()).iter().map(|b| format!("{b:02x}")).collect()
+}
+
 fn getrandom(buf: &mut [u8]) -> std::io::Result<()> {
     use std::io::Read;
     std::fs::File::open("/dev/urandom")?.read_exact(buf)
@@ -74,6 +87,21 @@ mod tests {
         assert!(hash_password("short").is_err());
     }
 
+    #[test]
+    fn api_tokens_are_prefixed_and_hash_to_hex() {
+        let t = new_api_token();
+        assert!(t.starts_with("ipx_") && t.len() == 68);
+        let h = api_token_hash(&t);
+        assert_eq!(h.len(), 64);
+        assert_eq!(h, api_token_hash(&t), "the same token, the same hash");
+        assert_ne!(h, api_token_hash(&new_api_token()));
+        assert_eq!(
+            api_token_hash("abc"),
+            "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
+            "SHA-256"
+        );
+    }
+
     #[test]
     fn session_tokens_are_long_and_distinct() {
         let a = new_session_token();

src/config.rs

@@ -39,6 +39,9 @@ pub struct General {
     /// image in an <enclosure>, so taking everything filled the disk with artwork and
     /// counted it as episodes. Empty means take anything.
     pub media_types: Vec<String>,
+    /// How much artwork iPX keeps on disk, in MB, so the page loads it from iPX rather than from
+    /// every publisher; the least recently shown goes first. 0 keeps none.
+    pub art_cache_mb: u64,
 }
 
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
@@ -163,6 +166,10 @@ pub struct Feed {
     /// Preferred over `password`: name of an env var holding the password.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub password_env: Option<String>,
+    /// Put in the Directory by an admin (`ipx add --list`): listed with no subscribers, and kept
+    /// in the catalogue when the last one leaves, where anyone else's feed is dropped.
+    #[serde(default, skip_serializing_if = "std::ops::Not::not")]
+    pub listed: bool,
 }
 
 fn yes() -> bool {
@@ -180,6 +187,7 @@ impl Default for General {
             max_age_days: 0,
             max_new_per_check: 3,
             media_types: vec!["audio".into(), "video".into()],
+            art_cache_mb: 500,
         }
     }
 }
@@ -290,10 +298,18 @@ pub struct Stored {
     pub max_age_days: u64,
     pub max_new_per_check: usize,
     pub media_types: Vec<String>,
+    /// Settings saved before it existed have none: they get the default.
+    #[serde(default = "art_cache_mb")]
+    pub art_cache_mb: u64,
+}
+
+fn art_cache_mb() -> u64 {
+    General::default().art_cache_mb
 }
 
 /// `[general]` keys that live in the database once it holds the configuration.
-const STORED_KEYS: [&str; 5] = ["schedule", "max_total_gb", "max_age_days", "max_new_per_check", "media_types"];
+const STORED_KEYS: [&str; 6] =
+    ["schedule", "max_total_gb", "max_age_days", "max_new_per_check", "media_types", "art_cache_mb"];
 
 impl Stored {
     pub fn of(cfg: &Config) -> Self {
@@ -304,6 +320,7 @@ impl Stored {
             max_age_days: g.max_age_days,
             max_new_per_check: g.max_new_per_check,
             media_types: g.media_types.clone(),
+            art_cache_mb: g.art_cache_mb,
         }
     }
 
@@ -314,6 +331,7 @@ impl Stored {
         g.max_age_days = self.max_age_days;
         g.max_new_per_check = self.max_new_per_check;
         g.media_types = self.media_types;
+        g.art_cache_mb = self.art_cache_mb;
     }
 }
 
@@ -571,6 +589,12 @@ mod tests {
         assert!(wanted_media(Some("image/jpeg"), &["image/jpeg".to_string()]));
     }
 
+    #[test]
+    fn settings_saved_before_the_art_cache_keep_the_default() {
+        let old = r#"{"schedule":"every 1h","max_total_gb":0.0,"max_age_days":0,"max_new_per_check":3,"media_types":["audio"]}"#;
+        assert_eq!(serde_json::from_str::<Stored>(old).unwrap().art_cache_mb, 500);
+    }
+
     #[test]
     fn slugs_are_readable_and_unique() {
         assert_eq!(slug("Accidental Tech Podcast"), "accidental-tech-podcast");
@@ -584,7 +608,7 @@ mod tests {
         taken.insert("the-daily".to_string(), Feed {
             url: "u".into(), folder: None, group: None, media_types: None, schedule: None, keywords: vec![], allow_explicit: false,
             auto_download: true, max_new_per_check: None, username: None,
-            password: None, password_env: None, category: None,
+            password: None, password_env: None, category: None, listed: false,
         });
         assert_eq!(unique_slug("The Daily", &taken), "the-daily-2");
     }
@@ -605,6 +629,7 @@ mod tests {
             password: Some("literal".into()),
             password_env: None,
             category: None,
+            listed: false,
         };
         assert_eq!(f.password().as_deref(), Some("literal"));
 

src/db.rs

@@ -2,7 +2,7 @@
 //! per-feed .ipxd plists, history.dat and qmcache.dat.
 
 use anyhow::{Context, Result};
-use crate::entity::{blocklists, catalogue, enclosures, entries, feeds, hidden, sessions, settings, subscriptions, users};
+use crate::entity::{api_tokens, blocklists, catalogue, enclosures, entries, feeds, hidden, sessions, settings, subscriptions, users};
 use sea_orm::sea_query::{Expr, Func};
 use sea_orm::{
     ActiveModelTrait, ColumnTrait, ConnectionTrait, EntityTrait, PaginatorTrait, QueryFilter, QueryOrder, Set,
@@ -59,6 +59,7 @@ async fn create_missing(orm: &sea_orm::DatabaseConnection) -> Result<()> {
         schema.create_table_from_entity(settings::Entity),
         schema.create_table_from_entity(blocklists::Entity),
         schema.create_table_from_entity(hidden::Entity),
+        schema.create_table_from_entity(api_tokens::Entity),
     ] {
         orm.execute(table.if_not_exists()).await.context("creating the schema")?;
     }
@@ -438,19 +439,27 @@ pub struct HttpState {
     pub error_since: Option<i64>,
 }
 
+impl From<feeds::Model> for HttpState {
+    fn from(f: feeds::Model) -> Self {
+        HttpState {
+            etag: f.etag,
+            last_modified: f.last_modified,
+            last_checked: f.last_checked,
+            ttl_mins: f.ttl_mins.map(|t| t.max(0) as u64),
+            error_since: f.error_since,
+        }
+    }
+}
+
 impl Db {
     pub async fn http_state(&self, feed_id: &str) -> Result<HttpState> {
-        Ok(feeds::Entity::find_by_id(feed_id.to_owned())
-            .one(&self.orm)
-            .await?
-            .map(|f| HttpState {
-                etag: f.etag,
-                last_modified: f.last_modified,
-                last_checked: f.last_checked,
-                ttl_mins: f.ttl_mins.map(|t| t.max(0) as u64),
-                error_since: f.error_since,
-            })
-            .unwrap_or_default())
+        Ok(feeds::Entity::find_by_id(feed_id.to_owned()).one(&self.orm).await?.map(HttpState::from).unwrap_or_default())
+    }
+
+    /// Every feed's, in one query: a scan, and the daemon working out when the next is due, asked
+    /// feed by feed, 180 queries a minute (#114).
+    pub async fn http_states(&self) -> Result<std::collections::HashMap<String, HttpState>> {
+        Ok(feeds::Entity::find().all(&self.orm).await?.into_iter().map(|f| (f.id.clone(), HttpState::from(f))).collect())
     }
 
     /// Upsert after a successful poll. Clears any previous error.
@@ -652,6 +661,35 @@ impl Db {
     /// through every show's history: 4420 files queued across 12 shows on the default of 3
     /// (#97). Unlimited, 0 in the settings, is the whole back catalogue, for an archive. An item
     /// whose files were all skipped by a filter does not hold one of the places.
+    /// Puts a feed's waiting files in the queue or out of it: 'pending' for those among its
+    /// `limit` newest items, which the scan downloads on its own, and 'held' for the rest, listed
+    /// to download by hand. With all of them 'pending', the queue counted 4420 back-catalogue
+    /// files no scan would ever take (#97, #113). 0 holds them all, for a feed nobody downloads
+    /// automatically. A held file still holds its item's place among the newest.
+    #[tracing::instrument(skip_all)]
+    pub async fn hold_back(&self, feed_id: &str, limit: usize) -> Result<()> {
+        let newest = "SELECT n.guid FROM entries n
+                       WHERE n.feed_id = $1
+                         AND EXISTS (SELECT 1 FROM enclosures y
+                                      WHERE y.feed_id = n.feed_id AND y.guid = n.guid
+                                        AND y.state <> 'skipped')
+                       ORDER BY coalesce(n.published, n.first_seen) DESC, n.guid DESC
+                       LIMIT $2";
+        let limit: sea_orm::Value = (limit.min(i64::MAX as usize) as i64).into();
+        self.exec(
+            &format!("UPDATE enclosures SET state = 'held' WHERE feed_id = $1 AND state = 'pending' AND guid NOT IN ({newest})"),
+            vec![feed_id.into(), limit.clone()],
+        )
+        .await?;
+        // And back, when a limit is raised or a feed downloads again.
+        self.exec(
+            &format!("UPDATE enclosures SET state = 'pending' WHERE feed_id = $1 AND state = 'held' AND guid IN ({newest})"),
+            vec![feed_id.into(), limit],
+        )
+        .await?;
+        Ok(())
+    }
+
     #[tracing::instrument(skip_all)]
     pub async fn pending(&self, feed_id: &str, limit: usize) -> Result<Vec<Pending>> {
         self.rows(
@@ -778,13 +816,76 @@ impl Db {
             .map(|r| Ok(r.try_get("", "id")?))
             .collect::<Result<_>>()?;
         for id in &gone {
-            for table in ["enclosures", "entry_state", "hidden", "blocklists", "entries"] {
-                self.exec(&format!("DELETE FROM {table} WHERE feed_id = $1"), vec![id.clone().into()]).await?;
-            }
+            self.forget_rows(id).await?;
         }
         Ok(gone)
     }
 
+    /// Everything stored about a feed but its row: items, file rows, read state, block list.
+    async fn forget_rows(&self, id: &str) -> Result<()> {
+        for table in ["enclosures", "entry_state", "hidden", "blocklists", "entries"] {
+            self.exec(&format!("DELETE FROM {table} WHERE feed_id = $1"), vec![id.into()]).await?;
+        }
+        Ok(())
+    }
+
+    /// A feed's stored artwork still on http, items' and its own: hosts a feed no longer names
+    /// keep their old items' artwork on http otherwise (IGN's 2009 items, #110).
+    pub async fn http_images(&self, feed_id: &str) -> Result<Vec<String>> {
+        self.rows(
+            "SELECT DISTINCT image FROM entries WHERE feed_id = $1 AND image LIKE 'http://%'
+             UNION SELECT image FROM feeds WHERE id = $1 AND image LIKE 'http://%'",
+            vec![feed_id.into()],
+        )
+        .await?
+        .iter()
+        .map(|r| Ok(r.try_get("", "image")?))
+        .collect()
+    }
+
+    /// Rewrites a feed's stored artwork on `host` from http to https, once the host is known to
+    /// serve it there (`feed::prefer_https`), for the items a scan does not write again.
+    pub async fn secure_images(&self, feed_id: &str, host: &str) -> Result<()> {
+        let like = format!("http://{host}/%");
+        for table in ["entries", "feeds"] {
+            let col = if table == "feeds" { "id" } else { "feed_id" };
+            self.exec(
+                &format!("UPDATE {table} SET image = 'https://' || substr(image, 8) WHERE {col} = $1 AND image LIKE $2"),
+                vec![feed_id.into(), like.clone().into()],
+            )
+            .await?;
+        }
+        Ok(())
+    }
+
+    /// Forgets a feed and everything stored about it.
+    pub async fn forget_feed(&self, id: &str) -> Result<()> {
+        self.forget_rows(id).await?;
+        self.exec("DELETE FROM feeds WHERE id = $1", vec![id.into()]).await?;
+        Ok(())
+    }
+
+    /// Feeds nobody subscribes to that are dead, failing since before `dead_before`, or quiet,
+    /// their newest item from before `quiet_before`: what keeps the Directory to feeds worth
+    /// taking. One with a file on disk stays, and so does one an OPML subscription lists, which
+    /// comes back with the OPML anyway. A feed with no items yet is not quiet, only new.
+    pub async fn stale_unsubscribed(&self, dead_before: i64, quiet_before: i64) -> Result<Vec<String>> {
+        self.rows(
+            "SELECT f.id FROM feeds f
+              WHERE NOT EXISTS (SELECT 1 FROM subscriptions s WHERE s.feed_id = f.id)
+                AND NOT EXISTS (SELECT 1 FROM enclosures x WHERE x.feed_id = f.id AND x.path IS NOT NULL)
+                AND f.group_id IS NULL AND NOT coalesce(f.managed, false)
+                AND ((f.error_since IS NOT NULL AND f.error_since < $1)
+                     OR (SELECT max(coalesce(e.published, e.first_seen)) FROM entries e
+                          WHERE e.feed_id = f.id) < $2)",
+            vec![dead_before.into(), quiet_before.into()],
+        )
+        .await?
+        .iter()
+        .map(|r| Ok(r.try_get("", "id")?))
+        .collect()
+    }
+
     pub async fn prune_entries(&self, older_than: i64) -> Result<usize> {
         let n = self
             .exec(
@@ -1585,6 +1686,7 @@ impl Db {
     /// The foreign key would take them anyway; this does not rely on it being switched on.
     pub async fn delete_user(&self, id: i64) -> Result<()> {
         sessions::Entity::delete_many().filter(sessions::Column::UserId.eq(id)).exec(&self.orm).await?;
+        api_tokens::Entity::delete_many().filter(api_tokens::Column::UserId.eq(id)).exec(&self.orm).await?;
         users::Entity::delete_by_id(id).exec(&self.orm).await?;
         Ok(())
     }
@@ -1637,6 +1739,56 @@ impl Db {
         Ok(found.map(User::from))
     }
 
+    /// Keeps a new API token's hash under its owner; the token itself is shown once and not kept.
+    pub async fn create_api_token(&self, user_id: i64, name: &str, hash: &str) -> Result<i64> {
+        let made = api_tokens::ActiveModel {
+            user_id: Set(user_id),
+            name: Set(name.to_owned()),
+            hash: Set(hash.to_owned()),
+            created: Set(now()),
+            last_used: Set(None),
+            ..Default::default()
+        }
+        .insert(&self.orm)
+        .await?;
+        Ok(made.id)
+    }
+
+    /// One person's tokens, newest first: names and dates, as the tokens themselves are not kept.
+    pub async fn api_tokens(&self, user_id: i64) -> Result<Vec<api_tokens::Model>> {
+        Ok(api_tokens::Entity::find()
+            .filter(api_tokens::Column::UserId.eq(user_id))
+            .order_by_desc(api_tokens::Column::Id)
+            .all(&self.orm)
+            .await?)
+    }
+
+    /// Revokes one of this person's tokens; someone else's id does nothing. Whether it was theirs.
+    pub async fn delete_api_token(&self, user_id: i64, id: i64) -> Result<bool> {
+        let gone = api_tokens::Entity::delete_many()
+            .filter(api_tokens::Column::Id.eq(id))
+            .filter(api_tokens::Column::UserId.eq(user_id))
+            .exec(&self.orm)
+            .await?;
+        Ok(gone.rows_affected > 0)
+    }
+
+    /// The person an API token acts as, noting when it was used, as a session notes `seen`.
+    pub async fn api_token_user(&self, hash: &str) -> Result<Option<User>> {
+        let found = api_tokens::Entity::find()
+            .filter(api_tokens::Column::Hash.eq(hash))
+            .find_also_related(users::Entity)
+            .one(&self.orm)
+            .await?;
+        let Some((token, Some(user))) = found else { return Ok(None) };
+        api_tokens::Entity::update_many()
+            .col_expr(api_tokens::Column::LastUsed, Expr::val(now()).into())
+            .filter(api_tokens::Column::Id.eq(token.id))
+            .exec(&self.orm)
+            .await?;
+        Ok(Some(User::from(user)))
+    }
+
     pub async fn delete_session(&self, token: &str) -> Result<()> {
         sessions::Entity::delete_by_id(token.to_owned()).exec(&self.orm).await?;
         Ok(())
@@ -1998,6 +2150,52 @@ mod tests {
         assert_eq!((list["f"].unread, list["g"].unread), (1, 1)); // a read, c hidden; sam's read is sam's
     }
 
+    #[tokio::test]
+    async fn a_hosts_artwork_moves_to_https_for_one_feed() {
+        let db = Db::memory().await.unwrap();
+        db.exec_for_test(
+            "INSERT INTO feeds (id, url, image) VALUES ('f','u','http://a.example/logo.png'),('g','v','http://a.example/g.png');
+             INSERT INTO entries (feed_id, guid, first_seen, image) VALUES
+                ('f','1',0,'http://a.example/1.jpg'),('f','2',0,'http://b.example/2.jpg'),('g','3',0,'http://a.example/3.jpg');",
+        ).await
+        .unwrap();
+        let mut old = db.http_images("f").await.unwrap();
+        old.sort();
+        assert_eq!(old, ["http://a.example/1.jpg", "http://a.example/logo.png", "http://b.example/2.jpg"]);
+        db.secure_images("f", "a.example").await.unwrap();
+        assert_eq!(db.feed_summary("f").await.unwrap().image.as_deref(), Some("https://a.example/logo.png"));
+        assert!(db.names_image("https://a.example/1.jpg").await.unwrap());
+        assert!(db.names_image("http://b.example/2.jpg").await.unwrap());  // another host
+        assert!(db.names_image("http://a.example/3.jpg").await.unwrap());  // another feed
+        assert_eq!(db.feed_summary("g").await.unwrap().image.as_deref(), Some("http://a.example/g.png"));
+    }
+
+    #[tokio::test]
+    async fn stale_feeds_nobody_subscribes_to_are_found_and_the_rest_left() {
+        let db = Db::memory().await.unwrap();
+        db.exec_for_test(
+            "INSERT INTO users (id, name, is_admin) VALUES (1,'ray',true);
+             INSERT INTO feeds (id, url, error_since, managed, group_id) VALUES
+                ('dead','a',100,false,null),('quiet','b',null,false,null),('news','c',null,false,null),
+                ('new','d',null,false,null),('wanted','e',100,false,null),('kept','f',100,false,null),
+                ('child','g',100,true,'opml'),('lately','h',900,false,null);
+             INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'wanted');
+             INSERT INTO entries (feed_id, guid, first_seen, published) VALUES
+                ('quiet','q',0,100),('news','n',0,950),('dead','d',0,950),('kept','k',0,100);
+             INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES (1,'kept','k','u','/tmp/k','done');",
+        ).await
+        .unwrap();
+        // Dead before 500, or nothing newer than 500: dead fails since 100, quiet's newest is 100.
+        // news published lately, new has no items yet, wanted has a subscriber, kept a file,
+        // child comes from an OPML, lately began failing after the cutoff.
+        let mut stale = db.stale_unsubscribed(500, 500).await.unwrap();
+        stale.sort();
+        assert_eq!(stale, ["dead", "quiet"]);
+        db.forget_feed("dead").await.unwrap();
+        assert_eq!(db.feed_summary("dead").await.unwrap().entries, 0);
+        assert!(!db.feed_urls().await.unwrap().iter().any(|(id, _)| id == "dead"));
+    }
+
     #[tokio::test]
     async fn a_failing_feed_nobody_subscribes_to_is_forgotten() {
         let db = Db::memory().await.unwrap();
@@ -2021,6 +2219,33 @@ mod tests {
         assert!(db.prune_abandoned_failures().await.unwrap().is_empty());
     }
 
+    #[tokio::test]
+    async fn only_the_newest_files_stay_queued_and_the_rest_are_held() {
+        let db = Db::memory().await.unwrap();
+        db.exec_for_test(
+            "INSERT INTO entries (feed_id, guid, first_seen, published) VALUES
+                ('f','e1',0,100),('f','e2',0,200),('f','e3',0,300),('f','e4',0,400);
+             INSERT INTO enclosures (id, feed_id, guid, url, state, path) VALUES
+                (1,'f','e1','u1','pending',null),(2,'f','e2','u2','pending',null),
+                (3,'f','e3','u3','done','/tmp/3'),(4,'f','e4','u4','pending',null);",
+        ).await
+        .unwrap();
+        let states = || async {
+            let mut v: Vec<(i64, String)> = db.rows("SELECT id, state FROM enclosures ORDER BY id", vec![]).await.unwrap()
+                .iter().map(|r| (r.try_get("", "id").unwrap(), r.try_get("", "state").unwrap())).collect();
+            v.sort();
+            v.into_iter().map(|(_, s)| s).collect::<Vec<_>>()
+        };
+        db.hold_back("f", 2).await.unwrap();              // newest two: e4, e3
+        assert_eq!(states().await, ["held", "held", "done", "pending"]);
+        db.hold_back("f", 3).await.unwrap();              // a limit raised brings e2 back
+        assert_eq!(states().await, ["held", "pending", "done", "pending"]);
+        db.hold_back("f", 0).await.unwrap();              // a feed nobody downloads: all held
+        assert_eq!(states().await, ["held", "held", "done", "held"]);
+        db.hold_back("f", usize::MAX).await.unwrap();     // unlimited, an archive: all queued
+        assert_eq!(states().await, ["pending", "pending", "done", "pending"]);
+    }
+
     #[tokio::test]
     async fn a_limit_takes_the_newest_episodes_not_the_back_catalogue() {
         let db = Db::memory().await.unwrap();

src/download.rs

@@ -406,7 +406,7 @@ mod tests {
         let mut f = crate::config::Feed {
             url: "u".into(), folder: Some("Subscriptions/Some | Show".into()), group: None, media_types: None,
             schedule: None, keywords: vec![], allow_explicit: false, auto_download: true,
-            max_new_per_check: None, username: None, password: None, password_env: None, category: None,
+            max_new_per_check: None, username: None, password: None, password_env: None, category: None, listed: false,
         };
         assert_eq!(folder_for(&cfg, "id", &f, None), "Subscriptions/Some - Show");
 

src/entity.rs

@@ -102,6 +102,10 @@ pub mod enclosures {
         pub length: Option<i64>,
         #[sea_orm(column_type = "Text", nullable)]
         pub path: Option<String>,
+        /// 'pending' is queued: a scan downloads it on its own. 'held' is listed but outside the
+        /// feed's newest max_new_per_check, or its feed downloads nothing automatically; it can
+        /// be downloaded by hand (`Db::hold_back`). 'skipped' is filtered out ('last_error' says
+        /// why), then 'downloading', 'done', 'error', 'reaped'.
         #[sea_orm(column_type = "Text")]
         pub state: String,
         #[sea_orm(default_value = 0)]
@@ -283,6 +287,29 @@ pub mod sessions {
     owned_by_user!();
 }
 
+/// A token a script or agent sends as `Authorization: Bearer`, acting as the person who made
+/// it (#123). Kept as its SHA-256, so the table is no use to anyone who reads it.
+pub mod api_tokens {
+    use sea_orm::entity::prelude::*;
+
+    #[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
+    #[sea_orm(table_name = "api_tokens")]
+    pub struct Model {
+        #[sea_orm(primary_key)]
+        pub id: i64,
+        pub user_id: i64,
+        /// What its owner called it, to tell one from another when revoking.
+        #[sea_orm(column_type = "Text")]
+        pub name: String,
+        #[sea_orm(unique, column_type = "Text")]
+        pub hash: String,
+        pub created: i64,
+        pub last_used: Option<i64>,
+    }
+
+    owned_by_user!();
+}
+
 /// The catalogue: every feed configured, with its shared settings as `config::Feed` in JSON, so a
 /// new setting on a feed needs no new column. It was config.toml's `[feeds]` (issue #18).
 pub mod catalogue {

src/feed.rs

@@ -53,43 +53,72 @@ pub enum Fetched {
     },
 }
 
-/// Conditional GET. reqwest handles gzip and redirects; the original's hand-rolled
-/// CONNECT/socket.ssl proxy path is gone -- `system-proxy` reads http_proxy/https_proxy.
+/// The longest a feed may take, connecting to the last byte: a scan handles feeds in order, and
+/// with no limit one hung server held every scan for as long as it did. Dreamwidth answered 504
+/// after 60-67 s for a day, and each scan took 70-80 s instead of 15 (#108). A feed is small;
+/// downloads, which are not, have no such limit.
+pub const FEED_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
+
+/// Conditional GET, following redirects itself: `client` must follow none (`feed_client`), so
+/// each hop is seen. The second value is where the feed now is when every hop said so for good
+/// (301 or 308): a publisher that moved its feed, which the catalogue should follow rather
+/// than be redirected on every read. A temporary redirect (302, 307) moves nothing.
+/// `system-proxy` reads http_proxy/https_proxy.
 #[tracing::instrument(skip_all, fields(url = %cfg.url))]
 pub async fn fetch(
     client: &reqwest::Client,
     cfg: &FeedCfg,
     etag: Option<&str>,
     last_modified: Option<&str>,
-) -> Result<Fetched> {
-    let mut req = client.get(&cfg.url);
-    if let Some(tag) = etag {
-        req = req.header(IF_NONE_MATCH, tag);
-    }
-    if let Some(lm) = last_modified {
-        req = req.header(IF_MODIFIED_SINCE, lm);
-    }
-    if let Some(user) = &cfg.username {
-        req = req.basic_auth(user, cfg.password());
-    }
+) -> Result<(Fetched, Option<String>)> {
+    let start = reqwest::Url::parse(&cfg.url).context("the feed's address")?;
+    let mut url = start.clone();
+    let mut permanent = true;
+    for _ in 0..10 {
+        let mut req = client.get(url.clone()).timeout(FEED_TIMEOUT);
+        if let Some(tag) = etag {
+            req = req.header(IF_NONE_MATCH, tag);
+        }
+        if let Some(lm) = last_modified {
+            req = req.header(IF_MODIFIED_SINCE, lm);
+        }
+        // The feed's own host only: a redirect elsewhere must not be handed the password.
+        if let Some(user) = &cfg.username
+            && url.host_str() == start.host_str()
+        {
+            req = req.basic_auth(user, cfg.password());
+        }
 
-    let resp = req.send().await.context("connecting")?;
-    if resp.status() == StatusCode::NOT_MODIFIED {
-        return Ok(Fetched::NotModified);
+        let resp = req.send().await.context("connecting")?;
+        let status = resp.status();
+        if status.is_redirection() && status != StatusCode::NOT_MODIFIED {
+            let to = resp
+                .headers()
+                .get(reqwest::header::LOCATION)
+                .and_then(|v| v.to_str().ok())
+                .ok_or_else(|| anyhow!("HTTP {status} without a Location to go to"))?;
+            url = url.join(to).with_context(|| format!("redirected to {to:?}, which is not an address"))?;
+            permanent &= matches!(status, StatusCode::MOVED_PERMANENTLY | StatusCode::PERMANENT_REDIRECT);
+            continue;
+        }
+        let moved = (permanent && url != start).then(|| url.to_string());
+        if status == StatusCode::NOT_MODIFIED {
+            return Ok((Fetched::NotModified, moved));
+        }
+        if !status.is_success() {
+            // The original surfaced 401/407 specially; the code is enough for a UI to switch on.
+            return Err(anyhow!("HTTP {status}"));
+        }
+        let header = |h: reqwest::header::HeaderName| {
+            resp.headers().get(&h).and_then(|v| v.to_str().ok()).map(str::to_owned)
+        };
+        let etag = header(ETAG);
+        let last_modified = header(LAST_MODIFIED);
+        let bytes = resp.bytes().await.context("reading body")?.to_vec();
+        return Ok((Fetched::Body { bytes, etag, last_modified }, moved));
     }
-    let status = resp.status();
-    if !status.is_success() {
-        // The original surfaced 401/407 specially; the code is enough for a UI to switch on.
-        return Err(anyhow!("HTTP {status}"));
-    }
-
-    let header = |h: reqwest::header::HeaderName| {
-        resp.headers().get(&h).and_then(|v| v.to_str().ok()).map(str::to_owned)
-    };
-    let etag = header(ETAG);
-    let last_modified = header(LAST_MODIFIED);
-    let bytes = resp.bytes().await.context("reading body")?.to_vec();
-    Ok(Fetched::Body { bytes, etag, last_modified })
+    // Worded as reqwest worded it, which failure_kind reads as a redirect loop.
+    Err(anyhow!("error following redirect for url ({url}): too many redirects"))
 }
 
 /// A stored `last_error`, translated into plain words for whoever subscribes: whose problem
@@ -253,7 +282,7 @@ pub async fn patreon_shows(
 ) -> Result<(Option<String>, Vec<(String, String)>)> {
     // The creator feed names its campaign by number in its self link, a few hundred bytes in.
     // The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
-    let mut resp = client.get(url).send().await.context("connecting")?;
+    let mut resp = client.get(url).timeout(FEED_TIMEOUT).send().await.context("connecting")?;
     if !resp.status().is_success() {
         return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
     }
@@ -451,6 +480,30 @@ pub async fn site_icon(client: &reqwest::Client, site: &str) -> Option<String> {
     is_image(client, ico.as_str()).await.then(|| ico.into())
 }
 
+/// Artwork's address on https when its host serves it there, else as it was. The page is https
+/// and must not load http; the host is asked once per `known` (one feed's read), and an http
+/// address it does not serve on https stays, for /api/art to fetch (#90). Four of the five
+/// hosts the catalogue had on http served the same image on https; The Secret Cabal's CDN
+/// presents another name's certificate (#110).
+pub async fn prefer_https(
+    client: &reqwest::Client,
+    url: &str,
+    known: &mut std::collections::HashMap<String, bool>,
+) -> String {
+    let Some(rest) = url.strip_prefix("http://") else { return url.to_owned() };
+    let host = rest.split('/').next().unwrap_or("").to_owned();
+    let secure = format!("https://{rest}");
+    let ok = match known.get(&host) {
+        Some(ok) => *ok,
+        None => {
+            let ok = is_image(client, &secure).await;
+            known.insert(host, ok);
+            ok
+        }
+    };
+    if ok { secure } else { url.to_owned() }
+}
+
 /// Whether `url` answers with an image. A site with no favicon often answers 200 with its home
 /// page, which is not an icon.
 pub async fn is_image(client: &reqwest::Client, url: &str) -> bool {
@@ -955,8 +1008,142 @@ fn parse_date(s: &str) -> Option<i64> {
         .map(|d| d.timestamp())
 }
 
+/// Apple's podcast categories, each with its subcategories
+/// (https://podcasters.apple.com/support/1691-apple-podcasts-categories): the list a podcast's own
+/// `<itunes:category>` comes from, and a fixed one, so the Directory's chips stay few.
+const CATEGORIES: &[(&str, &[&str])] = &[
+    ("Arts", &["Books", "Design", "Fashion & Beauty", "Food", "Performing Arts", "Visual Arts"]),
+    ("Business", &["Careers", "Entrepreneurship", "Investing", "Management", "Marketing", "Non-Profit"]),
+    ("Comedy", &["Comedy Interviews", "Improv", "Stand-Up"]),
+    ("Education", &["Courses", "How To", "Language Learning", "Self-Improvement"]),
+    ("Fiction", &["Comedy Fiction", "Drama", "Science Fiction"]),
+    ("Government", &[]),
+    ("History", &[]),
+    ("Health & Fitness", &["Alternative Health", "Fitness", "Medicine", "Mental Health", "Nutrition", "Sexuality"]),
+    ("Kids & Family", &["Education for Kids", "Parenting", "Pets & Animals", "Stories for Kids"]),
+    ("Leisure", &["Animation & Manga", "Automotive", "Aviation", "Crafts", "Games", "Hobbies", "Home & Garden", "Video Games"]),
+    ("Music", &["Music Commentary", "Music History", "Music Interviews"]),
+    ("News", &["Business News", "Daily News", "Entertainment News", "News Commentary", "Politics", "Sports News", "Tech News"]),
+    ("Religion & Spirituality", &["Buddhism", "Christianity", "Hinduism", "Islam", "Judaism", "Religion", "Spirituality"]),
+    ("Science", &["Astronomy", "Chemistry", "Earth Sciences", "Life Sciences", "Mathematics", "Natural Sciences", "Nature", "Physics", "Social Sciences"]),
+    ("Society & Culture", &["Documentary", "Personal Journals", "Philosophy", "Places & Travel", "Relationships"]),
+    ("Sports", &["Baseball", "Basketball", "Cricket", "Fantasy Sports", "American Football", "Golf", "Hockey", "Rugby", "Running", "Football (Soccer)", "Swimming", "Tennis", "Volleyball", "Wilderness", "Wrestling"]),
+    ("Technology", &[]),
+    ("True Crime", &[]),
+    ("TV & Film", &["After Shows", "Film History", "Film Interviews", "Film Reviews", "TV Reviews"]),
+];
+
+/// Where a category sits in Apple's list, as (category, subcategory), so the Directory browses a
+/// category and then its subcategories as Apple's does (#118). A podcast's own is stored as its
+/// subcategory, and one an admin typed that is not Apple's stands as a category of its own.
+pub fn category_path(name: &str) -> (String, Option<String>) {
+    for (category, subs) in CATEGORIES {
+        if category.eq_ignore_ascii_case(name) {
+            return (category.to_string(), None);
+        }
+        if let Some(sub) = subs.iter().find(|s| s.eq_ignore_ascii_case(name)) {
+            return (category.to_string(), Some(sub.to_string()));
+        }
+    }
+    (name.to_string(), None)
+}
+
+/// A Directory category for a feed that names none of its own, which most blogs do not (#117):
+/// TypeSafe's Jev picks one of Apple's categories or subcategories from the feed's title and
+/// latest items, the subcategory being what a podcast's own category is stored as. The likeliest
+/// is taken however unsure, since an admin can change it and asking again would cost a call a scan.
+pub async fn classify(client: &reqwest::Client, key: &str, feed: &ParsedFeed) -> Result<String> {
+    let items: Vec<&str> = feed.entries.iter().filter_map(|e| e.title.as_deref()).take(15).collect();
+    // A subcategory says which category it is in; without that, "Games" or "Drama" is ambiguous.
+    let mut criteria = serde_json::Map::new();
+    for (category, subs) in CATEGORIES {
+        criteria.insert(category.to_string(), serde_json::Value::Null);
+        for sub in *subs {
+            criteria.insert(sub.to_string(), format!("Within {category}").into());
+        }
+    }
+    let body = serde_json::json!({
+        "model": "jev-latest",
+        "state": { "title": feed.title, "recent_items": items },
+        "questions": { "category": {
+            "type": "choice",
+            "instructions": "Which category best describes what this feed publishes?",
+            "criteria": criteria,
+        }},
+    });
+    let answer: serde_json::Value = client
+        .post("https://api.typesafe.ai/v1/systemone")
+        .bearer_auth(key)
+        .json(&body)
+        .timeout(FEED_TIMEOUT)
+        .send()
+        .await?
+        .error_for_status()?
+        .json()
+        .await?;
+    picked(&answer)
+}
+
+fn picked(answer: &serde_json::Value) -> Result<String> {
+    let choice = answer["answers"]["category"]["choice"].as_str().unwrap_or_default();
+    CATEGORIES
+        .iter()
+        .flat_map(|(c, subs)| std::iter::once(c).chain(subs.iter()))
+        .find(|c| **c == choice)
+        .map(|c| c.to_string())
+        .ok_or_else(|| anyhow!("no category in the answer: {answer}"))
+}
+
 #[cfg(test)]
 mod tests {
+    /// A server answering by path: /old moves for good to /new, /tmp for now, /chain for good
+    /// to /tmp, /new is the feed.
+    async fn redirecting_server() -> String {
+        use tokio::io::{AsyncReadExt, AsyncWriteExt};
+        let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let addr = listener.local_addr().unwrap();
+        tokio::spawn(async move {
+            loop {
+                let Ok((mut sock, _)) = listener.accept().await else { return };
+                tokio::spawn(async move {
+                    let mut buf = [0u8; 2048];
+                    let n = sock.read(&mut buf).await.unwrap_or(0);
+                    let req = String::from_utf8_lossy(&buf[..n]);
+                    let path = req.split_whitespace().nth(1).unwrap_or("/").to_owned();
+                    let body = "<?xml version=\"1.0\"?><rss version=\"2.0\"><channel><title>T</title></channel></rss>";
+                    let resp = match path.as_str() {
+                        "/old" => "HTTP/1.1 301 Moved Permanently\r\nLocation: /new\r\nContent-Length: 0\r\n\r\n".to_owned(),
+                        "/tmp" => "HTTP/1.1 302 Found\r\nLocation: /new\r\nContent-Length: 0\r\n\r\n".to_owned(),
+                        "/chain" => "HTTP/1.1 308 Permanent Redirect\r\nLocation: /tmp\r\nContent-Length: 0\r\n\r\n".to_owned(),
+                        "/loop" => "HTTP/1.1 301 Moved Permanently\r\nLocation: /loop\r\nContent-Length: 0\r\n\r\n".to_owned(),
+                        _ => format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n{body}", body.len()),
+                    };
+                    let _ = sock.write_all(resp.as_bytes()).await;
+                });
+            }
+        });
+        format!("http://{addr}")
+    }
+
+    #[tokio::test]
+    async fn a_feed_that_moved_for_good_says_where_and_one_moved_for_now_does_not() {
+        let base = redirecting_server().await;
+        let client = reqwest::Client::builder().redirect(reqwest::redirect::Policy::none()).build().unwrap();
+        let get = |path: &str| {
+            let cfg: crate::config::Feed = serde_json::from_value(serde_json::json!({ "url": format!("{base}{path}") })).unwrap();
+            let client = client.clone();
+            async move { super::fetch(&client, &cfg, None, None).await }
+        };
+        let (got, moved) = get("/old").await.unwrap();
+        assert!(matches!(got, super::Fetched::Body { .. }));
+        assert_eq!(moved, Some(format!("{base}/new")));
+        assert_eq!(get("/tmp").await.unwrap().1, None);         // 302: for now
+        assert_eq!(get("/chain").await.unwrap().1, None);       // 308 then 302: not for good
+        assert_eq!(get("/new").await.unwrap().1, None);         // never moved
+        let looped = get("/loop").await.err().unwrap().to_string();
+        assert_eq!(super::failure_kind(&looped).0, "redirect_loop");
+    }
+
     use super::*;
 
     #[test]
@@ -1390,5 +1577,23 @@ mod tests {
         assert_eq!(pick_guid(None, None, Some("e"), Some("t")).as_deref(), Some("e"));
         assert_eq!(pick_guid(None, None, None, None), None);
     }
+
+    #[test]
+    fn picked_takes_only_an_apple_category() {
+        let answer = |c: &str| serde_json::json!({"answers": {"category": {"type": "choice", "choice": c}}});
+        assert_eq!(super::picked(&answer("Technology")).unwrap(), "Technology");
+        assert_eq!(super::picked(&answer("Tech News")).unwrap(), "Tech News", "a subcategory");
+        assert!(super::picked(&answer("Blogs")).is_err(), "not one of Apple's");
+        assert!(super::picked(&serde_json::json!({"error": "rate limited"})).is_err());
+    }
+
+    #[test]
+    fn category_path_finds_a_subcategory_s_category() {
+        use super::category_path;
+        assert_eq!(category_path("Tech News"), ("News".into(), Some("Tech News".into())));
+        assert_eq!(category_path("video games"), ("Leisure".into(), Some("Video Games".into())), "Apple's spelling");
+        assert_eq!(category_path("Technology"), ("Technology".into(), None));
+        assert_eq!(category_path("Homebrew"), ("Homebrew".into(), None), "not Apple's: its own category");
+    }
 }
 

src/ipc.rs

@@ -15,6 +15,9 @@ pub enum Event {
     FeedSkip { feed: String, reason: String },
     FeedDone { feed: String, new: usize, downloaded: usize, failed: usize, torrents: usize },
     FeedError { feed: String, msg: String },
+    /// The feed answered from a new address after redirects that all said it moved for good,
+    /// and the catalogue now has that address (#115).
+    FeedMoved { feed: String, from: String, to: String },
     Progress {
         feed: String,
         /// Which enclosure this is about. Without it a UI cannot tell one download's
@@ -50,6 +53,7 @@ impl Event {
                 "{feed}: {new} new entries, {downloaded} downloaded, {failed} failed, {torrents} torrents deferred"
             ),
             Event::FeedError { feed, msg } => format!("{feed}: error: {msg}"),
+            Event::FeedMoved { feed, from, to } => format!("{feed}: moved for good from {from} to {to}; following it"),
             Event::Progress { file, done, total, .. } => match total {
                 Some(t) if *t > 0 => format!(
                     "    {file}: {:.1}% ({:.1}/{:.1} MB)",
@@ -69,7 +73,7 @@ impl Event {
                 *bytes as f64 / 1_048_576.0
             ),
             Event::Status { feeds, pending, downloaded } => {
-                format!("{feeds} feeds, {pending} pending, {downloaded} downloaded")
+                format!("{feeds} feeds, {pending} queued to download, {downloaded} downloaded")
             }
             Event::Error { msg } => format!("error: {msg}"),
             // Noise in a terminal; a UI still gets them on the socket.
@@ -172,7 +176,7 @@ fn log_event(e: &Event, wire: bool) {
         ($level:ident, $target:literal, $text:expr) => {
             tracing::$level!(
                 target: $target,
-                ev = s("ev"), feed = s("feed"), msg = s("msg"), url = s("url"), reason = s("reason"),
+                ev = s("ev"), feed = s("feed"), msg = s("msg"), url = s("url"), reason = s("reason"), from = s("from"), to = s("to"),
                 new = n("new"), downloaded = n("downloaded"), failed = n("failed"),
                 torrents = n("torrents"), bytes = n("bytes"), feeds = n("feeds"),
                 pending = n("pending"), enclosure = n("enclosure"), files = n("files"),
@@ -328,6 +332,14 @@ pub async fn proxy(path: &Path, cmd: &Command) -> Result<()> {
 mod tests {
     use super::*;
 
+    #[test]
+    fn a_feed_that_moved_says_so_on_the_wire_and_in_words() {
+        let e = Event::FeedMoved { feed: "x".into(), from: "http://a/f".into(), to: "https://a/f".into() };
+        let wire = serde_json::to_value(&e).unwrap();
+        assert_eq!((wire["ev"].as_str(), wire["from"].as_str(), wire["to"].as_str()), (Some("feed_moved"), Some("http://a/f"), Some("https://a/f")));
+        assert_eq!(e.human().unwrap(), "x: moved for good from http://a/f to https://a/f; following it");
+    }
+
     #[test]
     fn commands_parse_from_the_wire_form() {
         let got: Command = serde_json::from_str(r#"{"cmd":"fetch"}"#).unwrap();

src/main.rs

@@ -1,4 +1,5 @@
 mod access;
+mod art;
 mod auth;
 mod config;
 mod db;
@@ -63,6 +64,12 @@ enum Command {
         /// Only take enclosures matching these keywords
         #[arg(long, value_delimiter = ',')]
         keywords: Vec<String>,
+        /// The Directory's category, for a feed that names none of its own (News, Technology)
+        #[arg(long)]
+        category: Option<String>,
+        /// Put it in the Directory for anyone to subscribe to, and keep it there when nobody does
+        #[arg(long)]
+        list: bool,
     },
     /// Unsubscribe. Downloads and history are left alone.
     Rm { feed: String },
@@ -118,6 +125,9 @@ pub struct Ctx {
     pub cfg: std::sync::RwLock<std::sync::Arc<config::Config>>,
     pub db: db::Db,
     pub client: reqwest::Client,
+    /// For feeds only: follows no redirects, so `feed::fetch` sees each hop and can tell a feed
+    /// that moved for good from one sent elsewhere for now.
+    pub feed_client: reqwest::Client,
     pub out: Emitter,
     /// Started on first use: a BitTorrent session binds ports and starts a DHT, which is
     /// rude to do for a config that has never seen a torrent.
@@ -248,6 +258,13 @@ async fn main() -> Result<()> {
         db,
         client: reqwest::Client::builder()
             .user_agent(concat!("ipx/", env!("CARGO_PKG_VERSION")))
+            // Connecting only, so it bounds a download's start, not a long download (#108).
+            .connect_timeout(std::time::Duration::from_secs(10))
+            .build()?,
+        feed_client: reqwest::Client::builder()
+            .user_agent(concat!("ipx/", env!("CARGO_PKG_VERSION")))
+            .connect_timeout(std::time::Duration::from_secs(10))
+            .redirect(reqwest::redirect::Policy::none())
             .build()?,
         out: if is_daemon { Emitter::socket(events.clone(), false) } else { Emitter::terminal() },
         torrents: tokio::sync::OnceCell::new(),
@@ -259,8 +276,8 @@ async fn main() -> Result<()> {
     let result = match cli.command {
         Command::List => list(&ctx).await,
         Command::Daemon { web } => daemon(ctx, config_path, web, events).await,
-        Command::Add { url, folder, keywords } => {
-            add(&ctx, &url, folder, keywords).await
+        Command::Add { url, folder, keywords, category, list } => {
+            add(&ctx, &url, folder, keywords, category, list).await
         }
         Command::Rm { feed } => rm(&ctx, &feed).await,
         Command::User { cmd } => user_cmd(&ctx, cmd).await,
@@ -505,8 +522,6 @@ async fn daemon(
     let server = tokio::spawn(ipc::serve(socket.clone(), events.clone(), tx_cmd, answer));
 
     // One command at a time: the queue is what keeps two scans from overlapping.
-    let mut ticker = tokio::time::interval(std::time::Duration::from_secs(60));
-    ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
     tracing::info!(
         feeds = subscriptions(&ctx).await.map(|s| s.len()).unwrap_or(0),
         "daemon started"
@@ -543,10 +558,13 @@ async fn daemon(
     }
 
     let mut stop = rx_stop.clone();
+    // The first pass at once, as the minute's tick did: what came due while it was down.
+    let mut first = true;
     loop {
         if *stop.borrow() {
             break;
         }
+        let wait = if std::mem::take(&mut first) { std::time::Duration::ZERO } else { until_next_scan(&ctx).await };
         tokio::select! {
             biased;
             _ = stop.changed() => break,
@@ -563,7 +581,7 @@ async fn daemon(
                     break;
                 }
             }
-            _ = ticker.tick() => {
+            _ = tokio::time::sleep(wait) => {
                 // Per-feed schedule and TTL decide what actually gets polled.
                 let job = run(&ctx, Cmd::Fetch { feed: None, force: false, feeds: vec![] });
                 if !until_stopped(&ctx, &rx_stop, job).await {
@@ -673,16 +691,35 @@ async fn add(
     url: &str,
     folder: Option<String>,
     keywords: Vec<String>,
+    category: Option<String>,
+    list: bool,
 ) -> Result<()> {
     let mut cfg = (*ctx.cfg()).clone();
     let url = &feed::find_feed(&ctx.client, &feed::expand_input(url)).await?;
     // Includes feeds derived from an OPML, or the same show could be added twice.
     if let Some(existing) = subscriptions(ctx).await?.iter().find(|s| feed::same_feed(&s.cfg.url, url)) {
+        // Listing a feed the catalogue already has, or giving it a category, is the point of
+        // asking again: it changes those, and nothing else.
+        if let Some(f) = cfg.feeds.get_mut(&existing.id)
+            && (list || category.is_some())
+        {
+            f.listed |= list;
+            if category.is_some() {
+                f.category = category;
+            }
+            ctx.store_cfg(cfg).await?;
+            println!("{} is already in the catalogue; updated its Directory listing", existing.id);
+            return Ok(());
+        }
         anyhow::bail!("already subscribed as {:?}", existing.id);
     }
     let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
+    if let Some(f) = cfg.feeds.get_mut(&id) {
+        f.category = category;
+        f.listed = list;
+    }
     ctx.store_cfg(cfg).await?;
-    println!("added {id}");
+    println!("added {id}{}", if list { ", listed in the Directory" } else { "" });
     Ok(())
 }
 
@@ -709,9 +746,10 @@ pub async fn add_one(
         password: None,
         password_env: None,
         category: None,
+        listed: false,
     };
 
-    let title = match feed::fetch(&ctx.client, &probe, None, None).await {
+    let title = match feed::fetch(&ctx.feed_client, &probe, None, None).await.map(|(got, _)| got) {
         // An OPML subscription is named from its own <head><title>, not by trying to
         // parse it as a feed and falling back to the hostname.
         Ok(feed::Fetched::Body { bytes, .. }) if feed::is_opml(&bytes) => {
@@ -835,6 +873,7 @@ pub async fn subscribe_opml(
                         password: None,
                         password_env: None,
                         category: None,
+                        listed: false,
                     },
                 );
                 grew = true;
@@ -947,12 +986,42 @@ async fn list(ctx: &Ctx) -> Result<()> {
     Ok(())
 }
 
+/// Removes feeds nobody subscribes to that are dead (failing for 30 days) or quiet (nothing new
+/// in a year), from the catalogue and the database, so the Directory lists feeds worth taking.
+/// A feed listed in it with no subscribers stays for as long as it works and publishes.
+async fn clean_directory(ctx: &Ctx) -> Result<()> {
+    const DEAD: i64 = 30 * 86_400;
+    const QUIET: i64 = 365 * 86_400;
+    let now = db::now();
+    let stale = ctx.db.stale_unsubscribed(now - DEAD, now - QUIET).await?;
+    if stale.is_empty() {
+        return Ok(());
+    }
+    let mut cfg = (*ctx.cfg()).clone();
+    if stale.iter().fold(false, |any, id| cfg.feeds.remove(id).is_some() || any) {
+        ctx.store_cfg(cfg).await?;
+    }
+    for id in &stale {
+        retire_group(ctx, id).await?;
+        ctx.db.forget_feed(id).await?;
+        tracing::info!(feed = id, "removed a feed nobody subscribes to that is dead or has published nothing in a year");
+    }
+    Ok(())
+}
+
 /// `standalone` false means this is the sweep that runs before a scan: it reports what it
 /// deleted, but must not emit the terminal ReapDone, or a client waiting on its `fetch`
 /// would stop reading before the scan had even started.
 #[tracing::instrument(name = "reap", skip_all, fields(dry_run = dry_run))]
 async fn reap(ctx: &Ctx, dry_run: bool, standalone: bool) -> Result<()> {
     let r = retention::run(&ctx.cfg(), &ctx.db, dry_run).await?;
+    if !dry_run {
+        clean_directory(ctx).await?;
+        let gone = art::trim(&art::dir(), ctx.cfg().general.art_cache_mb * 1_048_576);
+        if gone > 0 {
+            tracing::debug!(gone, "trimmed the artwork kept on disk");
+        }
+    }
     for c in r.aged_out.iter().chain(r.over_quota.iter()) {
         ctx.out.emit(Event::Reaped {
             path: c.path.clone(),
@@ -979,15 +1048,17 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
         anyhow::bail!("no feed with id {id:?}");
     }
 
+    let subscribed = ctx.db.subscriber_counts().await?;
     let mut scanned = 0;
     let mut fresh: Vec<String> = vec![];
     let in_scope = |s: &Sub| {
         scope.is_empty() || scope.contains(&s.id) || s.cfg.group.as_ref().is_some_and(|g| scope.contains(g))
     };
+    let mut states = ctx.db.http_states().await?;
     let mut due = vec![];
     for sub in subs.iter().filter(|s| only.is_none_or(|o| o == s.id) && in_scope(s)) {
         let id = &sub.id;
-        let mut state = ctx.db.http_state(id).await?;
+        let mut state = states.remove(id).unwrap_or_default();
         // A scan someone asked for reads the feed in full. With the validators it only skipped the
         // wait: a feed that had not changed answered 304 and nothing was read (#77).
         if force {
@@ -995,8 +1066,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
             state.last_modified = None;
         }
 
-        if !force && let Some(last) = state.last_checked {
-            let at = last + due_after(&cfg, &sub.cfg, state.ttl_mins, state.error_since, last) as i64;
+        if !force && let Some(at) = due_at(&cfg, &sub.cfg, &state, subscribed.contains_key(id)) {
             if at > db::now() {
                 ctx.out.emit(Event::FeedSkip {
                     feed: id.clone(),
@@ -1015,7 +1085,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
     const AHEAD: usize = 6;
     let mut bodies = futures_util::StreamExt::buffered(
         futures_util::stream::iter(due.iter().map(|(sub, state)| {
-            let (client, feed_cfg) = (ctx.client.clone(), sub.cfg.clone());
+            let (client, feed_cfg) = (ctx.feed_client.clone(), sub.cfg.clone());
             let (etag, modified) = (state.etag.clone(), state.last_modified.clone());
             let early = !feed::is_patreon_creator(&feed_cfg.url);
             tokio::spawn(tracing::Instrument::instrument(
@@ -1156,6 +1226,7 @@ pub async fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
                 password: parent.and_then(|p| p.password.clone()),
                 password_env: parent.and_then(|p| p.password_env.clone()),
                 category: None,
+                listed: false,
             },
             managed: true,
         });
@@ -1206,6 +1277,47 @@ async fn retire_stranded(ctx: &Ctx) -> Result<usize> {
 
 /// Seconds to wait before re-checking a feed.
 ///
+/// When a feed is next due, or None for one never checked, which is due now. A feed nobody
+/// subscribes to, one listed in the Directory, is read once a day: enough to keep its entry
+/// current, without fetching it hourly for no one.
+pub fn due_at(cfg: &config::Config, feed: &config::Feed, state: &db::HttpState, subscribed: bool) -> Option<i64> {
+    let last = state.last_checked?;
+    let floor = if subscribed { 0 } else { 86_400 };
+    Some(last + due_after(cfg, feed, state.ttl_mins, state.error_since, last).max(floor) as i64)
+}
+
+/// How long the daemon may sleep before a feed is due: until the earliest one, at least 30 s and
+/// at most 10 minutes. It ticked every minute and ran a scan pass each time, 80% of them finding
+/// nothing due (#114). The floor keeps a feed that never gets a check time from spinning it; the
+/// ceiling picks up within ten minutes what no command announces, such as `ipx add` or a
+/// shorter schedule. A command, a refresh or a feed added on the page, wakes it at once anyway.
+async fn until_next_scan(ctx: &Ctx) -> std::time::Duration {
+    let next = async {
+        let cfg = ctx.cfg();
+        let subscribed = ctx.db.subscriber_counts().await?;
+        let states = ctx.db.http_states().await?;
+        anyhow::Ok(
+            subscriptions(ctx)
+                .await?
+                .iter()
+                // A feed with no row yet has never been checked: due now.
+                .map(|s| {
+                    states.get(&s.id).and_then(|st| due_at(&cfg, &s.cfg, st, subscribed.contains_key(&s.id))).unwrap_or(0)
+                })
+                .min(),
+        )
+    };
+    let wait = match next.await {
+        Ok(Some(at)) => (at - db::now()).max(0) as u64,
+        Ok(None) => u64::MAX,
+        Err(e) => {
+            tracing::warn!(error = %format!("{e:#}"), "could not work out when the next feed is due");
+            0
+        }
+    };
+    std::time::Duration::from_secs(wait.clamp(30, 600))
+}
+
 /// A per-feed schedule is an explicit instruction and wins outright. Without one, the
 /// global schedule applies, but the feed's own <ttl> raises it when the publisher asks to
 /// be polled less often. A feed that is failing backs off (`backoff`), from `error_since`, when
@@ -1257,6 +1369,24 @@ enum Outcome {
     Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
 }
 
+/// A feed that answered from a new address after permanent redirects moves there in the
+/// catalogue, so it is read from there and no longer redirected every time. One an OPML lists
+/// is left alone, since the OPML would only put the old address back; so is a move onto an
+/// address another feed already has.
+async fn follow_move(ctx: &Ctx, id: &str, to: &str) -> Result<()> {
+    // Said in the event below, not logged here: the event is the log line.
+    if subscriptions(ctx).await?.iter().any(|s| s.id != id && feed::same_feed(&s.cfg.url, to)) {
+        tracing::info!(feed = id, to, "the feed moved to an address another feed already has; leaving it");
+        return Ok(());
+    }
+    let mut cfg = (*ctx.cfg()).clone();
+    let Some(f) = cfg.feeds.get_mut(id) else { return Ok(()) };
+    let from = std::mem::replace(&mut f.url, to.to_owned());
+    ctx.store_cfg(cfg).await?;
+    ctx.out.emit(Event::FeedMoved { feed: id.to_owned(), from, to: to.to_owned() });
+    Ok(())
+}
+
 #[tracing::instrument(name = "feed", skip_all, fields(feed = id))]
 async fn scan_one(
     ctx: &Arc<Ctx>,
@@ -1264,9 +1394,13 @@ async fn scan_one(
     feed_cfg: &config::Feed,
     state: &db::HttpState,
     force: bool,
-    // The body the scan already fetched for it, if it did (#104).
-    prefetched: Option<Result<feed::Fetched>>,
+    // The body the scan already fetched for it, if it did (#104), and where it moved, if it did.
+    prefetched: Option<Result<(feed::Fetched, Option<String>)>>,
 ) -> Result<Outcome> {
+    // The queue follows the feed's settings each time it is due, changed or not, so files a
+    // limit or auto-download no longer reaches stop counting as waiting (#113).
+    let policy = policy_for(ctx, id, feed_cfg).await?;
+    ctx.db.hold_back(id, if policy.auto_download { policy.budget } else { 0 }).await?;
     // A Patreon creator with more than one show is a list of feeds, like an OPML.
     if feed::is_patreon_creator(&feed_cfg.url) {
         match feed::patreon_shows(&ctx.client, &feed_cfg.url).await {
@@ -1292,10 +1426,13 @@ async fn scan_one(
         }
     }
 
-    let mut fetched = match prefetched {
+    let (mut fetched, moved) = match prefetched {
         Some(got) => got?,
-        None => feed::fetch(&ctx.client, feed_cfg, state.etag.as_deref(), state.last_modified.as_deref()).await?,
+        None => feed::fetch(&ctx.feed_client, feed_cfg, state.etag.as_deref(), state.last_modified.as_deref()).await?,
     };
+    if let Some(to) = moved {
+        follow_move(ctx, id, &to).await?;
+    }
 
     // A 304 while nothing is stored means the validator has outlived the data -- a restore
     // from backup, a manual edit, a cleanup that removed entries. Believe the database over
@@ -1306,7 +1443,7 @@ async fn scan_one(
     if matches!(fetched, feed::Fetched::NotModified) && (stored.entries == 0 || stored.image.is_none()) {
         tracing::info!(feed = id, "not modified, but nothing stored; refetching without the validator");
         ctx.db.clear_validators(id).await?;
-        fetched = feed::fetch(&ctx.client, feed_cfg, None, None).await?;
+        fetched = feed::fetch(&ctx.feed_client, feed_cfg, None, None).await?.0;
     }
 
     let (bytes, etag, last_modified) = match fetched {
@@ -1330,6 +1467,17 @@ async fn scan_one(
     }
 
     let mut parsed = feed::parse(&bytes)?;
+    // Artwork on http moves to https where its host serves it (#110), before it is compared
+    // with what is stored, which is then the https address too.
+    let mut secure = std::collections::HashMap::new();
+    if let Some(art) = &parsed.image {
+        parsed.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
+    }
+    for entry in parsed.entries.iter_mut() {
+        if let Some(art) = &entry.image {
+            entry.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
+        }
+    }
     // Artwork is looked at when it may have changed: the feed names different artwork from what
     // is stored, or someone asked for a refresh, so an icon the site changes or fixes still
     // follows it (#80). Looked at on every full read, a feed without validators asked its site
@@ -1354,6 +1502,20 @@ async fn scan_one(
             });
         }
     }, art_span).await;
+    // A site's icon comes back on whatever the site is on.
+    if let Some(art) = &parsed.image {
+        parsed.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
+    }
+    // Items stored before, which a scan does not write again, move with their host, and a host
+    // only they still name is asked too.
+    for art in ctx.db.http_images(id).await? {
+        feed::prefer_https(&ctx.client, &art, &mut secure).await;
+    }
+    for (host, ok) in &secure {
+        if *ok {
+            ctx.db.secure_images(id, host).await?;
+        }
+    }
     ctx.db.record_feed(
         id,
         &feed_cfg.url,
@@ -1364,8 +1526,29 @@ async fn scan_one(
         parsed.image.as_deref(),
         parsed.category.as_deref(),
     ).await?;
+    // Asked on a feed's first read, nothing of it stored yet, or when someone forces a refresh:
+    // each call costs, and a feed already here without a category can wait for a forced one.
+    // The answer becomes the catalogue's category, so a feed is not asked twice.
+    // A feed an OPML lists is not in the catalogue and has nowhere to keep one.
+    if parsed.category.is_none()
+        && (force || stored.entries == 0)
+        && !parsed.entries.is_empty()
+        && ctx.cfg().feeds.get(id).is_some_and(|f| f.category.is_none())
+        && let Ok(key) = std::env::var("TYPESAFE_KEY")
+    {
+        match feed::classify(&ctx.client, &key, &parsed).await {
+            Ok(category) => {
+                let mut cfg = (*ctx.cfg()).clone();
+                if let Some(f) = cfg.feeds.get_mut(id).filter(|f| f.category.is_none()) {
+                    tracing::info!(feed = id, category = %category, "categorised");
+                    f.category = Some(category);
+                    ctx.store_cfg(cfg).await?;
+                }
+            }
+            Err(e) => tracing::warn!(feed = id, "could not categorise: {e:#}"),
+        }
+    }
 
-    let policy = policy_for(ctx, id, feed_cfg).await?;
     if let Some(parent) = &feed_cfg.group {
         let listed: Vec<(&str, &str)> = parsed
             .entries
@@ -1412,6 +1595,9 @@ async fn scan_one(
     if scan.new_entries > 0 {
         ctx.db.rehide(id).await?;   // what is new may hold someone's blocked words
     }
+    // A new item takes a place among the newest and the oldest of them leaves the queue; a file a
+    // filter lets through again may be outside them.
+    ctx.db.hold_back(id, if policy.auto_download { policy.budget } else { 0 }).await?;
 
     let budget = policy.budget;
     if policy.auto_download && budget > 0 {
@@ -1666,7 +1852,10 @@ fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Pol
 
     if subs.is_empty() {
         return Policy {
-            auto_download: feed_cfg.auto_download,
+            // A feed listed in the Directory with nobody subscribed is there to be found, not
+            // downloaded: its files would be for no one. Once someone subscribes, the files
+            // skipped for it are judged again on the next scan, by their settings (#107).
+            auto_download: feed_cfg.auto_download && !feed_cfg.listed,
             allow_explicit: feed_cfg.allow_explicit,
             wants: vec![Want { keywords: feed_cfg.keywords.clone(), blocked: vec![] }],
             budget: cap(feed_cfg.max_new_per_check),
@@ -1899,6 +2088,26 @@ fn duration(secs: u64) -> String {
 mod tests {
     use super::*;
 
+    #[test]
+    fn a_feed_is_due_after_its_wait_and_daily_with_nobody_subscribed() {
+        let cfg = config::Config::default();
+        let f = feed();
+        let at = |last: Option<i64>, subscribed| {
+            due_at(&cfg, &f, &db::HttpState { last_checked: last, ..Default::default() }, subscribed)
+        };
+        assert_eq!(at(None, true), None);                           // never checked: due now
+        assert_eq!(at(Some(1000), true), Some(1000 + 3600));        // the hourly default
+        assert_eq!(at(Some(1000), false), Some(1000 + 86_400));     // listed, nobody subscribed
+    }
+
+    #[test]
+    fn a_listed_feed_nobody_subscribes_to_downloads_nothing() {
+        let mut f = feed();
+        assert!(merge_policy(&[], &f, 3).auto_download);
+        f.listed = true;
+        assert!(!merge_policy(&[], &f, 3).auto_download);
+    }
+
     #[test]
     fn a_failing_feed_backs_off_doubling_up_to_a_day() {
         let hour = 3600;
@@ -1969,6 +2178,7 @@ mod tests {
             password: None,
             password_env: None,
             category: None,
+            listed: false,
         }
     }
 
@@ -2028,6 +2238,7 @@ mod tests {
             cfg: std::sync::RwLock::new(Arc::new(cfg)),
             db: db::Db::memory().await.unwrap(),
             client: reqwest::Client::new(),
+            feed_client: reqwest::Client::builder().redirect(reqwest::redirect::Policy::none()).build().unwrap(),
             out: Emitter::terminal(),
             torrents: tokio::sync::OnceCell::new(),
             torrent_slots: Arc::new(tokio::sync::Semaphore::new(2)),

src/web.rs

@@ -38,6 +38,8 @@ pub fn router(state: WebState) -> Router {
     Router::new()
         .route("/", get(index))
         .route("/api/me", get(me).patch(patch_me))
+        .route("/api/tokens", get(list_tokens).post(add_token))
+        .route("/api/tokens/{id}", axum::routing::delete(remove_token))
         .route("/api/logout", post(logout))
         .route("/api/feeds", get(feeds).post(add_feed))
         .route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
@@ -76,6 +78,8 @@ pub fn router(state: WebState) -> Router {
         .route("/favicon.png", get(favicon))
         .route("/favicon-dark.png", get(favicon_dark))
         .route("/apple-touch-icon.png", get(touch_icon))
+        // iOS asks for this one first when the site is added to a home screen (#109).
+        .route("/apple-touch-icon-precomposed.png", get(touch_icon))
         .route("/app.js", get(app_js))
         .route("/app.css", get(app_css))
         .route("/login.js", get(login_js))
@@ -98,8 +102,8 @@ pub async fn serve(state: WebState, bind: &str) -> Result<()> {
     .context("serving the web ui")
 }
 
-/// Who is asking, in order of how specific the claim is: a proxy that vouches for a name,
-/// a session cookie, then the shared token (which is the admin).
+/// Who is asking, in order of how specific the claim is: a proxy that vouches for a name, an
+/// API token, a session cookie, then the shared token (which is the admin).
 ///
 /// Any of them has to survive being put in a cookie: an `<audio src>` request is issued by
 /// the browser, and there is no way to attach a header to it.
@@ -142,6 +146,26 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
     }
     let by_proxy = user.is_some();
 
+    // An API token, made by its owner in Settings, for a script or an agent acting as them
+    // (#123). One that is not known is turned away rather than tried as anything else: a
+    // script sent with a revoked token should hear so, not fall through to a cookie.
+    if user.is_none()
+        && let Some(bearer) = req
+            .headers()
+            .get(header::AUTHORIZATION)
+            .and_then(|v| v.to_str().ok())
+            .and_then(|v| v.strip_prefix("Bearer "))
+    {
+        match state.ctx.db.api_token_user(&crate::auth::api_token_hash(bearer.trim())).await {
+            Ok(Some(u)) => user = Some(u),
+            Ok(None) => return (StatusCode::UNAUTHORIZED, "unknown or revoked API token").into_response(),
+            Err(e) => {
+                tracing::error!(error = %e, "looking up an API token");
+                return StatusCode::INTERNAL_SERVER_ERROR.into_response();
+            }
+        }
+    }
+
     // 2. A session cookie from signing in here.
     if user.is_none() {
         if let Some(sid) = cookie(&req, SESSION_COOKIE) {
@@ -390,6 +414,54 @@ fn last_admin(users: &[crate::db::User], id: i64) -> bool {
     admins == [id]
 }
 
+/// Your API tokens: names and dates, never the tokens, which are not kept.
+async fn list_tokens(
+    State(state): State<WebState>,
+    user: crate::db::User,
+) -> Result<Json<serde_json::Value>, ApiError> {
+    let tokens: Vec<_> = state
+        .ctx
+        .db
+        .api_tokens(user.id).await?
+        .iter()
+        .map(|t| serde_json::json!({ "id": t.id, "name": t.name, "created": t.created, "last_used": t.last_used }))
+        .collect();
+    Ok(Json(serde_json::json!(tokens)))
+}
+
+#[derive(Deserialize)]
+struct NewToken {
+    name: String,
+}
+
+/// Makes an API token that acts as you, and answers with it: the only time it is shown.
+async fn add_token(
+    State(state): State<WebState>,
+    user: crate::db::User,
+    Json(body): Json<NewToken>,
+) -> Result<Json<serde_json::Value>, ApiError> {
+    let name = body.name.trim();
+    if name.is_empty() {
+        return Err(ApiError::bad_request("give the token a name, to know it by when revoking it"));
+    }
+    let token = crate::auth::new_api_token();
+    let id = state.ctx.db.create_api_token(user.id, name, &crate::auth::api_token_hash(&token)).await?;
+    tracing::info!(user = %user.name, token = name, "API token made");
+    Ok(Json(serde_json::json!({ "id": id, "name": name, "token": token })))
+}
+
+async fn remove_token(
+    State(state): State<WebState>,
+    user: crate::db::User,
+    Path(id): Path<i64>,
+) -> Result<StatusCode, ApiError> {
+    if !state.ctx.db.delete_api_token(user.id, id).await? {
+        return Err(ApiError::bad_request(format!("you have no API token with id {id}")));
+    }
+    tracing::info!(user = %user.name, id, "API token revoked");
+    Ok(StatusCode::NO_CONTENT)
+}
+
 async fn list_users(
     State(state): State<WebState>,
     user: crate::db::User,
@@ -859,8 +931,10 @@ struct PopularRow {
     subscribers: i64,
     /// Yours already. Everyone counts, you included, so your own feeds are listed too.
     subscribed: bool,
-    /// The feed's own iTunes category, if it names one; most blogs do not.
+    /// Apple's category, from the feed's own iTunes category or else the catalogue's, and the
+    /// subcategory within it where there is one: the Directory browses them in that order.
     category: Option<String>,
+    subcategory: Option<String>,
     /// Any audio or video enclosure. Unlike category, every feed has an answer, so the
     /// Directory's Podcasts and Blogs between them hold everything.
     podcast: bool,
@@ -870,8 +944,13 @@ struct PopularRow {
 /// first. Popular is the top of it, the directory is all of it, and it is all that
 /// `subscribe_popular` will subscribe you to. An OPML or a Patreon creator is listed as the
 /// feeds inside it and never itself: both lists are for finding a show.
-async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
+/// The catalogue as others may see it. `everything` is the Directory: every feed, with or
+/// without subscribers, so the ones an admin listed show before anyone takes them. Without it,
+/// Popular: only what people subscribe to.
+async fn popular(state: &WebState, user_id: i64, everything: bool) -> Result<Vec<PopularRow>> {
     let db = &state.ctx.db;
+    // One pass for every feed's title, artwork and category, not three queries a feed.
+    let mut listed = db.feed_list(user_id, None).await?;
     let mine: std::collections::HashSet<String> =
         db.subscriptions_for(user_id).await?.into_iter().map(|s| s.feed_id).collect();
     let counts = db.subscriber_counts().await?;
@@ -886,23 +965,25 @@ async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
         let n = counts.get(&s.id).copied().unwrap_or(0);
         // A feed inside an OPML that looks private is as private as the OPML.
         let folder = s.cfg.group.as_deref().and_then(|g| by_id.get(g));
-        if n == 0
+        if (n == 0 && !everything)
             || is_folder.contains(s.id.as_str())
             || looks_private(&s.cfg)
             || folder.is_some_and(|f| looks_private(f))
         {
             continue;
         }
-        let sum = db.feed_summary(&s.id).await?;
+        let sum = listed.remove(&s.id).map(|l| l.summary).unwrap_or_default();
         let subscribed = mine.contains(&s.id);
+        // The feed's own wins; an admin's is for the feeds, mostly blogs, that name none.
+        let path = sum.category.or_else(|| s.cfg.category.clone()).map(|c| crate::feed::category_path(&c));
         out.push(PopularRow {
             id: s.id.clone(),
             title: sum.title,
             image: sum.image,
             subscribers: n,
             subscribed,
-            // The feed's own wins; an admin's is for the feeds, mostly blogs, that name none.
-            category: sum.category.or_else(|| s.cfg.category.clone()),
+            category: path.as_ref().map(|p| p.0.clone()),
+            subcategory: path.and_then(|p| p.1),
             podcast: media.contains(&s.id),
         });
     }
@@ -914,7 +995,7 @@ async fn get_popular(
     State(state): State<WebState>,
     user: crate::db::User,
 ) -> Result<Json<Vec<PopularRow>>, ApiError> {
-    let mut rows = popular(&state, user.id).await?;
+    let mut rows = popular(&state, user.id, false).await?;
     rows.truncate(10);
     Ok(Json(rows))
 }
@@ -924,7 +1005,7 @@ async fn get_directory(
     State(state): State<WebState>,
     user: crate::db::User,
 ) -> Result<Json<Vec<PopularRow>>, ApiError> {
-    let mut rows = popular(&state, user.id).await?;
+    let mut rows = popular(&state, user.id, true).await?;
     rows.sort_by_key(sort_name);
     Ok(Json(rows))
 }
@@ -940,10 +1021,12 @@ async fn subscribe_popular(
     user: crate::db::User,
     Path(id): Path<String>,
 ) -> Result<Json<serde_json::Value>, ApiError> {
-    if !popular(&state, user.id).await?.iter().any(|p| p.id == id) {
+    if !popular(&state, user.id, true).await?.iter().any(|p| p.id == id) {
         return Err(ApiError::bad_request(format!("{id:?} is not in the directory")));
     }
     state.ctx.db.subscribe(user.id, &id).await?;
+    // A listed feed nobody took was checked once a day at most: read it now, not in an hour.
+    scan_soon(&state, Some(id.clone())).await;
     Ok(Json(serde_json::json!({ "id": id })))
 }
 
@@ -1073,6 +1156,7 @@ mod tests {
             password: None,
             password_env: None,
             category: None,
+            listed: false,
         };
         assert!(!looks_private(&f("https://feeds.twit.tv/twit.xml")));
         assert!(!looks_private(&f("https://example.com/rss?format=mp3")));
@@ -1119,7 +1203,7 @@ mod tests {
         crate::config::Feed {
             url: url.into(), folder: None, group: None, media_types: None, schedule: None, keywords: vec![], allow_explicit: false,
             auto_download: true, max_new_per_check: None, username: None,
-            password: None, password_env: None, category: None,
+            password: None, password_env: None, category: None, listed: false,
         }
     }
 
@@ -1505,8 +1589,12 @@ async fn remove_feed(
     }
 
     // Nobody is left: the feed stops being scanned. Its files and history stay, so if
-    // someone subscribes again they do not pull the back catalogue a second time.
+    // someone subscribes again they do not pull the back catalogue a second time. A feed an
+    // admin listed stays in the Directory, for the next person.
     let mut cfg = (*state.ctx.cfg()).clone();
+    if cfg.feeds.get(&id).is_some_and(|f| f.listed) {
+        return Ok(StatusCode::NO_CONTENT);
+    }
     if cfg.feeds.remove(&id).is_none() {
         // A derived feed: forget it here, though the OPML will list it again on the next
         // read unless you unsubscribe from the OPML itself.
@@ -1687,7 +1775,10 @@ async fn events(
 /// routine skip of a feed not due, dozens a minute that change nothing.
 fn changed_feed(ev: &Event) -> Option<&str> {
     match ev {
-        Event::FeedDone { feed, .. } | Event::FeedError { feed, .. } | Event::DownloadDone { feed, .. } => Some(feed),
+        Event::FeedDone { feed, .. }
+        | Event::FeedError { feed, .. }
+        | Event::FeedMoved { feed, .. }
+        | Event::DownloadDone { feed, .. } => Some(feed),
         Event::FeedSkip { feed, reason } if !reason.starts_with("not due") => Some(feed),
         _ => None,
     }
@@ -1716,16 +1807,25 @@ struct ArtQuery {
     u: String,
 }
 
-/// Artwork a feed names on plain http, fetched here for the https page. The browser upgrades an
-/// http image on an https page to https, and a host with no https, such as The Secret Cabal's
-/// CDN, then answers nothing (issue #90).
+/// Artwork, from iPX: the page asks here for every image a feed or item names, and the first
+/// time it is fetched from the publisher and kept on disk (`art`, up to `art_cache_mb`), so
+/// later it is fast, the publisher is not asked on every visit, and it outlives the publisher's
+/// server. Only an address some feed or item names as its artwork, and only an image up to
+/// 5 MB, so the route cannot be pointed at anything else. It began as a way round http-only
+/// artwork on the https page (#90).
 async fn art(State(state): State<WebState>, Query(q): Query<ArtQuery>) -> Response {
     const MAX: usize = 5 << 20;
-    if !q.u.starts_with("http://") || !state.ctx.db.names_image(&q.u).await.unwrap_or(false) {
+    let web = q.u.starts_with("http://") || q.u.starts_with("https://");
+    if !web || !state.ctx.db.names_image(&q.u).await.unwrap_or(false) {
         return (StatusCode::NOT_FOUND, "no feed names that artwork").into_response();
     }
+    let keep = state.ctx.cfg().general.art_cache_mb > 0;
+    let file = crate::art::path(&q.u);
+    if keep && let Some((kind, body)) = crate::art::read(&file) {
+        return art_response(kind, body);
+    }
     let got = async {
-        let mut r = state.ctx.client.get(&q.u).send().await?.error_for_status()?;
+        let mut r = state.ctx.client.get(&q.u).timeout(crate::feed::FEED_TIMEOUT).send().await?.error_for_status()?;
         let kind = r.headers().get(header::CONTENT_TYPE).and_then(|v| v.to_str().ok()).unwrap_or("").to_owned();
         anyhow::ensure!(kind.starts_with("image/"), "not an image: {kind}");
         let mut body = Vec::new();
@@ -1737,12 +1837,30 @@ async fn art(State(state): State<WebState>, Query(q): Query<ArtQuery>) -> Respon
     };
     match got.await {
         Ok((kind, body)) => {
-            ([(header::CONTENT_TYPE, kind), (header::CACHE_CONTROL, "private, max-age=86400".into())], body).into_response()
+            if keep && let Err(e) = crate::art::write(&file, &kind, &body) {
+                tracing::debug!(error = %e, "could not keep the artwork");
+            }
+            art_response(kind, body)
         }
         Err(e) => (StatusCode::BAD_GATEWAY, format!("{e:#}")).into_response(),
     }
 }
 
+/// An image from someone else's server, served from iPX's own address: it must stay an image.
+/// An SVG opened on its own would otherwise run its script as iPX's page, with its cookies.
+fn art_response(kind: String, body: Vec<u8>) -> Response {
+    (
+        [
+            (header::CONTENT_TYPE, kind),
+            (header::CACHE_CONTROL, "private, max-age=2592000".into()),
+            (header::X_CONTENT_TYPE_OPTIONS, "nosniff".into()),
+            (header::CONTENT_SECURITY_POLICY, "default-src 'none'; style-src 'unsafe-inline'; sandbox".into()),
+        ],
+        body,
+    )
+        .into_response()
+}
+
 #[derive(Deserialize)]
 struct Position {
     secs: i64,
@@ -1908,6 +2026,7 @@ struct Settings {
     download_dir: String,
     max_total_gb: f64,
     max_age_days: u64,
+    art_cache_mb: u64,
 }
 
 async fn get_settings(State(state): State<WebState>) -> Json<Settings> {
@@ -1920,6 +2039,7 @@ async fn get_settings(State(state): State<WebState>) -> Json<Settings> {
         download_dir: cfg.general.download_dir.display().to_string(),
         max_total_gb: cfg.general.max_total_gb,
         max_age_days: cfg.general.max_age_days,
+        art_cache_mb: cfg.general.art_cache_mb,
     })
 }
 
@@ -1930,6 +2050,7 @@ struct SettingsPatch {
     media_types: Option<Vec<String>>,
     max_total_gb: Option<f64>,
     max_age_days: Option<u64>,
+    art_cache_mb: Option<u64>,
 }
 
 async fn patch_settings(
@@ -1966,6 +2087,9 @@ async fn patch_settings(
     if let Some(v) = body.max_age_days {
         cfg.general.max_age_days = v;
     }
+    if let Some(v) = body.art_cache_mb {
+        cfg.general.art_cache_mb = v;
+    }
     state.ctx.store_cfg(cfg).await?;
     Ok(StatusCode::NO_CONTENT)
 }

tests/ui/app.spec.js

@@ -887,6 +887,16 @@ test('Popular lists what everyone here reads, but never a private feed', async (
   await expect(tiles).toHaveCount(dir.filter(p => p.podcast).length);
   await pick('tabs', 'All').click();
   await expect(tiles).toHaveCount(dir.length);
+  // Sorted by name either way, A to Z as it opens, or by subscribers (#122).
+  const byName = dir.map(p => p.title || p.id).sort((a, b) => a.localeCompare(b, undefined, { sensitivity: 'base', numeric: true }));
+  await piper.locator('#dirsort').selectOption('az');
+  await expect(tiles.first()).toContainText(byName[0]);
+  await piper.locator('#dirsort').selectOption('za');
+  await expect(tiles.first()).toContainText(byName[byName.length - 1]);
+  await expect(tiles).toHaveCount(dir.length);
+  await piper.locator('#dirsort').selectOption('subscribers');
+  await expect(tiles).toHaveCount(dir.length);
+  await piper.locator('#dirsort').selectOption('az');
 
   // Add a feed is for an address; Popular and Directory are where you browse (issue #30).
   await piper.locator('#addFeed').click();
@@ -956,6 +966,37 @@ test('reading an item updates its feed\'s count without reloading the list', asy
   expect(lists, 'the row came back with the read, not by reloading the list').toEqual([]);
 });
 
+test('artwork comes from iPX, kept, and only as an image', async ({ page }) => {
+  // Every image a feed names is drawn from iPX's address.
+  expect(await page.evaluate(() => artHTML('https://example.com/a.jpg', 'A'))).toContain('src="/api/art?u=https%3A%2F%2Fexample.com%2Fa.jpg"');
+  // Multi Show's items name art.jpg.
+  await expect(page.locator('.feed', { hasText: 'Multi Show' })).toBeVisible({ timeout: 20_000 });
+  const src = '/api/art?u=' + encodeURIComponent('http://127.0.0.1:8792/art.jpg');
+  for (const _ of [1, 2]) {                                         // fetched, then kept
+    const r = await page.request.get(src);
+    expect(r.status()).toBe(200);
+    expect(r.headers()['content-type']).toMatch(/^image\//);
+    expect(r.headers()['content-security-policy']).toContain('sandbox');
+  }
+  // An address no feed names is not fetched.
+  expect((await page.request.get('/api/art?u=' + encodeURIComponent('http://127.0.0.1:8792/show.xml'))).status()).toBe(404);
+});
+
+test('an item without a title is named from its text, then its file, then its show and date', async ({ page }) => {
+  const names = await page.evaluate(() => [
+    entryName({ title: 'A Title' }),
+    entryName({ title: ' ', description: '<p>I like the way <b>AI</b> is   evolving.</p><img src="x.png">' }),
+    entryName({ description: '<p>' + 'word '.repeat(40) + '</p>' }).text.length,
+    entryName({ enclosures: [{ url: 'https://k.example/media/KARTAS691.mp3?x=1' }] }),
+    entryName({ feed_id: 'nobody', published: 0 }),
+  ]);
+  expect(names[0]).toEqual({ text: 'A Title', derived: false });
+  expect(names[1]).toEqual({ text: 'I like the way AI is evolving.', derived: true });
+  expect(names[2]).toBeLessThanOrEqual(121);                         // cut at a word, with …
+  expect(names[3]).toEqual({ text: 'KARTAS691', derived: true });
+  expect(names[4]).toEqual({ text: 'nobody', derived: true });
+});
+
 test('a deleted file looks as if it was never downloaded', async ({ page }) => {
   // Other people subscribe to Picture Blog by now, so both prompts come; take them.
   page.on('dialog', d => d.accept());
@@ -1140,13 +1181,49 @@ test('keys move through items and places, after Feedly', async ({ page }) => {
   await expect(page.locator('#count')).toContainText('All Subscriptions');
 });
 
+test('an API token acts as its owner until it is revoked', async ({ page, request }) => {
+  // Made in Settings, shown once.
+  await page.locator('#prefs').click();
+  await page.locator('#tokname').fill('test agent');
+  await page.locator('#tokadd').click();
+  const token = await page.locator('#tokval').inputValue();
+  expect(token).toMatch(/^ipx_[0-9a-f]{64}$/);
+  await expect(page.locator('.tokrow', { hasText: 'test agent' })).toContainText('last used never');
+
+  // Sent as a Bearer header, with no cookie, it is the person who made it.
+  const as = t => ({ headers: { Authorization: `Bearer ${t}` } });
+  const me = await (await request.get('/api/me', as(token))).json();
+  expect(me.name).toBe(await page.evaluate(() => S.me.name));
+  expect((await request.get('/api/feeds', as(token))).status()).toBe(200);
+  // A wrong one is turned away, not let through as anyone.
+  expect((await request.get('/api/me', as('ipx_' + '0'.repeat(64)))).status()).toBe(401);
+  // Only the hash is kept: the list never carries the token.
+  const listed = await page.evaluate(() => api('/api/tokens'));
+  expect(JSON.stringify(listed)).not.toContain(token);
+  expect(listed.find(t => t.name === 'test agent').last_used).toBeTruthy();
+
+  // Revoked, it opens nothing.
+  await page.locator('.tokrow', { hasText: 'test agent' }).locator('[data-tok]').click();
+  await expect(page.locator('.tokrow', { hasText: 'test agent' })).toHaveCount(0);
+  expect((await request.get('/api/me', as(token))).status()).toBe(401);
+});
+
 test('an admin can give a blog its Directory category', async ({ page }) => {
   const patch = (id, category) => page.evaluate(([id, category]) =>
     api(`/api/feeds/${id}`, { method: 'PATCH', body: JSON.stringify({ category }) }), [id, category]);
   const listed = async id => (await page.evaluate(() => api('/api/directory'))).find(p => p.id === id);
 
   await patch('picture-blog', 'Visual Arts');
-  expect(await listed('picture-blog')).toMatchObject({ podcast: false, category: 'Visual Arts' });
+  // Apple's subcategory, under its category, as the Directory browses them (#118).
+  expect(await listed('picture-blog')).toMatchObject({ podcast: false, category: 'Arts', subcategory: 'Visual Arts' });
+  // A subcategory's chips show once its category is picked, as Apple's directory does.
+  const chip = (row, name) => page.locator(`#dirbar .${row} button`, { hasText: new RegExp(`^${name}$`) });
+  await page.locator('#feedlist .place', { hasText: 'Directory' }).click();
+  await expect(chip('subs', 'Visual Arts')).toHaveCount(0);
+  await chip('chips', 'Arts').click();
+  await chip('subs', 'Visual Arts').click();
+  await expect(chip('subs', 'Visual Arts')).toHaveAttribute('aria-pressed', 'true');
+  await expect(page.locator('#popular .tile')).toHaveCount(1);
   // A feed's own iTunes category wins over one given here.
   await patch('test-show', 'Comedy');
   expect((await listed('test-show')).category).toBe('Technology');
@@ -1233,7 +1310,7 @@ test('the favicon is the logo, square, from both pages', async ({ page }) => {
   await page.evaluate(() => setTheme('modern', 'dark'));
   await expect(page.locator('#favicon')).toHaveAttribute('href', /^\/favicon-dark\.png\?v=[0-9a-f]{12}$/);
   // A browser asks for /favicon.ico on its own, signed in or not.
-  for (const path of ['/favicon.ico', '/favicon.png', '/favicon-dark.png', '/apple-touch-icon.png']) {
+  for (const path of ['/favicon.ico', '/favicon.png', '/favicon-dark.png', '/apple-touch-icon.png', '/apple-touch-icon-precomposed.png']) {
     const r = await page.request.get(path, { headers: { cookie: '' } });
     expect(r.status(), path).toBe(200);
     expect(r.headers()['content-type'], path).toBe('image/png');

tests/ui/fixtures/serve.js

@@ -12,7 +12,10 @@ http.createServer((req, res) => {
   fs.readFile(file, (err, body) => {
     if (err) { res.writeHead(404).end('no'); return; }
     const type = file.endsWith('.mp3') ? 'audio/mpeg'
-      : file.endsWith('.opml') ? 'text/x-opml' : 'application/xml';
+      : file.endsWith('.opml') ? 'text/x-opml'
+      // Artwork as an image, or /api/art refuses it as not one.
+      : file.endsWith('.jpg') ? 'image/jpeg'
+      : file.endsWith('.html') ? 'text/html' : 'application/xml';
     res.writeHead(200, { 'content-type': type, 'content-length': body.length });
     res.end(body);
   });

web/app.css

@@ -566,7 +566,14 @@ input:focus,select:focus{outline:0;border-color:var(--accent)}
 /* A show sits under its folder's title, a size down, so an open folder reads as one. */
 .feed.child{margin-left:11px}
 /* Pinned feeds, at the top of the list: a small pin before the name, and a rule under the last. */
-.feed .fpin .i{width:10px;height:10px;margin-right:5px;vertical-align:-1px;color:var(--accent)}
+/* Pinned: a disc on the artwork's corner, as a failing feed's mark is, in the accent and the ink
+   the theme pairs with it. A feed both pinned and failing keeps the error below, the pin above. */
+.fpin{
+  position:absolute;right:-5px;bottom:-5px;width:17px;height:17px;border-radius:50%;
+  display:grid;place-items:center;background:var(--accent);color:var(--ink);border:2px solid var(--bg);
+}
+.fpin .i{width:9px;height:9px}
+.feed.err .fpin{bottom:auto;top:-5px}
 .feed.lastpin{margin-bottom:9px}
 .feed.lastpin::after{content:"";position:absolute;left:8px;right:8px;bottom:-5px;border-bottom:1px solid var(--line)}
 .feed.child .art{width:28px;height:28px;font-size:11px}
@@ -635,7 +642,10 @@ body.scan-this .fhead [data-a=scan] .i,body.scan-any .fhead [data-a=scanall] .i,
    are; a .badge's fill already means unread in the sidebar. */
 .dirbar{display:flex;flex-wrap:wrap;align-items:center;gap:8px 14px;margin-top:4px}
 .dirbar .tabs{flex:none}
+#dirsort{flex:none;width:auto;padding:4px 6px;font-size:13px}
 .chips{display:flex;flex-wrap:wrap;gap:2px 6px;flex:1 1 0;min-width:0}
+/* A category's subcategories go on a line of their own, under the category picked. */
+.chips.subs{flex-basis:100%}
 .chips button{flex:none;padding:4px 6px;font-size:13px;color:var(--dim);white-space:nowrap;border-bottom:2px solid transparent}
 .chips button:hover{color:var(--fg)}
 .chips button[aria-pressed="true"]{color:var(--fg);border-bottom-color:var(--accent2)}
@@ -824,7 +834,10 @@ body.adminpage #log{max-width:none}
    icon, so it spilled 3px right of centre and off line with its heading. */
 .ep .st,.ep .fl{width:22px;height:22px;padding:0;border-radius:5px;display:grid;place-items:center;font-size:11px;color:var(--accent2)}
 .ep .fl{color:var(--faint);font-size:13px}
-.ep .fl.on{color:var(--fg)}
+/* Pinned: the pinned feed's disc (.fpin), the accent and its ink, drawn inside the 22px button so
+   the row does not move when it changes. */
+.ep .fl.on,.ep .fl.on:hover{color:var(--ink);background:radial-gradient(circle,var(--accent) 8.5px,transparent 9px)}
+.ep .fl.on .i{width:9px;height:9px}
 /* The icon's EQ bars mark what is playing: standing still, and moving only while it plays. A
    paused animation was tried first; the frames it held were a pixel apart and read as dots. */
 .eq{display:inline-flex;align-items:flex-end;gap:2px;width:13px;height:12px;flex:none}
@@ -837,6 +850,8 @@ body.playing .eq i:nth-child(3){animation-delay:-.6s}
 @keyframes eq{from{transform:scaleY(.35)}}
 .ep .st:hover,.ep .fl:hover{background:var(--raise)}
 .ep .t{font-weight:600;font-size:13.5px;display:block;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
+/* An item with no title shows its opening words: set as text, not as a heading. */
+.ep .t.notitle{font-weight:400}
 .ep.read .t{color:var(--dim);font-weight:500}
 .ep .line{display:flex;gap:9px;align-items:center;flex-wrap:wrap;color:var(--faint);font-size:11.5px}
 .ep .line:empty{display:none}
@@ -937,6 +952,9 @@ input[type=range]::-moz-range-thumb{width:12px;height:12px;border:0;border-radiu
 .inline{display:flex;gap:6px;align-items:stretch}
 .inline input{flex:1;min-width:0}
 .inline .btn{white-space:nowrap;flex:none}
+/* An API token in Settings: its name and dates, and the button that revokes it. */
+.tokrow{display:flex;align-items:center;justify-content:space-between;gap:8px;margin-bottom:6px}
+.tokrow .hint{display:block}
 .inline select{flex:none;width:auto}
 .inline input[type=number]{flex:none;width:90px}
 .check input{width:16px;height:16px;accent-color:var(--accent)}

web/src/admin.ts

@@ -49,6 +49,10 @@ async function drawServer(){
       items are never touched.</span></div>
     <div class="field"><label>Delete items older than (days, 0 = keep)</label>
       <input type="number" id="gage" min="0" value="${g.max_age_days}"></div>
+    <div class="field"><label>Artwork kept (MB, 0 = none)</label>
+      <input type="number" id="gart" min="0" value="${g.art_cache_mb}">
+      <span class="hint">Show and episode artwork is kept here once shown, so it loads from iPX
+      instead of each publisher. Over this, what has gone longest unshown is dropped first.</span></div>
     <div class="field"><label>Download folder</label>
       <span class="hint" style="overflow-wrap:anywhere">${esc(g.download_dir)}</span></div>
     <div class="cardacts"><button class="btn primary" id="gsave">${ICON.check} Save</button></div>`;
@@ -59,7 +63,8 @@ async function drawServer(){
         max_new_per_check: Math.max(0, Number($('#gmax').value) || 0),
         media_types: $('#gtypes').value.split(',').map(t => t.trim()).filter(Boolean),
         max_total_gb: Number($('#gquota').value) || 0,
-        max_age_days: Number($('#gage').value) || 0})});
+        max_age_days: Number($('#gage').value) || 0,
+        art_cache_mb: Math.max(0, Number($('#gart').value) || 0)})});
       toast('Settings saved');
     }catch(e){ toast(e.message, true); }
   };

web/src/dialogs.ts

@@ -70,11 +70,19 @@ function listedFeed(p,cls){
 
 // Directory's filters. Kept out here because a finished scan redraws the pane, which would
 // otherwise clear them.
-let dirKind='All', dirCat=null;
+let dirKind='All', dirCat=null, dirSub=null, dirSort='az';
+/// The Directory's orders. It opens A to Z, as the server sends it.
+const NAME=p=>(p.title||p.id);
+const DIR_SORTS={
+  subscribers:['Most subscribers',(a,b)=>b.subscribers-a.subscribers||NAME(a).localeCompare(NAME(b),undefined,{sensitivity:'base',numeric:true})],
+  az:['Name, A to Z',(a,b)=>NAME(a).localeCompare(NAME(b),undefined,{sensitivity:'base',numeric:true})],
+  za:['Name, Z to A',(a,b)=>NAME(b).localeCompare(NAME(a),undefined,{sensitivity:'base',numeric:true})],
+};
 const KINDS={All:()=>true,Podcasts:p=>p.podcast,Blogs:p=>!p.podcast};
 /// Directory: every listed feed as its cover art, under two filters that combine: what a feed is
 /// (Podcasts, anything with audio or video, or Blogs, the rest) and what it is about (its iTunes
-/// category, as chips). Both filter in place, without asking the server again.
+/// category, as chips, then once one is picked its subcategories, as Apple's directory does).
+/// They filter in place, without asking the server again.
 async function renderDirectory(url,box){
   let rows=[];
   try{ rows=await api(url)||[]; }catch{}
@@ -89,18 +97,27 @@ async function renderDirectory(url,box){
     // No empty chips: only the categories among the feeds the kind lets through.
     const cats=[...new Set(ofKind.map(p=>p.category).filter(Boolean))].sort();
     if(!cats.includes(dirCat)) dirCat=null;
+    const ofCat=ofKind.filter(p=>!dirCat||p.category===dirCat);
+    const subs=dirCat?[...new Set(ofCat.map(p=>p.subcategory).filter(Boolean))].sort():[];
+    if(!subs.includes(dirSub)) dirSub=null;
     bar.innerHTML=
       (both?`<div class="tabs" role="group" aria-label="Kind">${Object.keys(KINDS).map(k=>btn('kind',k,k===dirKind)).join('')}</div>`:'')+
-      (cats.length?`<div class="chips" role="group" aria-label="Category">${cats.map(c=>btn('cat',c,c===dirCat)).join('')}</div>`:'');
+      (cats.length?`<div class="chips" role="group" aria-label="Category">${cats.map(c=>btn('cat',c,c===dirCat)).join('')}</div>`:'')+
+      `<select id="dirsort" aria-label="Sort">${Object.entries(DIR_SORTS).map(([k,[label]])=>
+        `<option value="${k}"${k===dirSort?' selected':''}>${label}</option>`).join('')}</select>`+
+      (subs.length?`<div class="chips subs" role="group" aria-label="Subcategory">${subs.map(c=>btn('sub',c,c===dirSub)).join('')}</div>`:'');
     // A picked chip lifts on a second press. Everything is redrawn, so the keyboard goes back to
     // the button just pressed.
     for(const b of $$('button',bar)) b.onclick=()=>{
-      const k=b.dataset.kind!=null?'kind':'cat', v=b.dataset[k];
-      if(k==='kind') dirKind=v; else dirCat=dirCat===v?null:v;
+      const k=b.dataset.kind!=null?'kind':b.dataset.cat!=null?'cat':'sub', v=b.dataset[k];
+      if(k==='kind') dirKind=v;
+      else if(k==='cat'){ dirCat=dirCat===v?null:v; dirSub=null; }
+      else dirSub=dirSub===v?null:v;
       draw(); $(`[data-${k}="${CSS.escape(v)}"]`,bar)?.focus();
     };
+    $('#dirsort',bar).onchange=e=>{ dirSort=e.target.value; draw(); $('#dirsort',bar).focus(); };
     box.innerHTML='';
-    for(const p of ofKind.filter(p=>!dirCat||p.category===dirCat)) box.appendChild(listedFeed(p,'tile'));
+    for(const p of ofCat.filter(p=>!dirSub||p.subcategory===dirSub).sort(DIR_SORTS[dirSort][1])) box.appendChild(listedFeed(p,'tile'));
   };
   draw();
   return rows.length;
@@ -140,7 +157,7 @@ async function renderListening(url,box){
     el.className='childrow';
     el.entry=e;
     el.innerHTML=artHTML(e.image||feedArt(e.feed_id),e.title||'')+
-      `<div class="txt"><b>${EQ}<span>${esc(e.title||'(untitled)')}</span></b>`+
+      `<div class="txt"><b>${EQ}<span>${esc(entryName(e).text)}</span></b>`+
       `<small><span class="fd">${esc(feedName(e.feed_id))}</span><span class="left"></span></small></div>`+
       `<button class="iconbtn" data-a="play"></button>`+
       `<button class="iconbtn" data-a="remove" title="Remove from Currently Listening" aria-label="Remove from Currently Listening">${ICON.close}</button>`+
@@ -239,12 +256,35 @@ async function prefsModal(){
       <span class="hint">Comma separated words or phrases, in every feed you read. An item with
       one in its title or text is hidden from you and not downloaded for you. Each feed's
       settings can add more.</span></div>
+    <div class="field"><label for="tokname">API tokens</label>
+      <div id="tokens"></div>
+      <div class="inline"><input type="text" id="tokname" placeholder="What it is for">
+        <button class="btn" id="tokadd" title="Make an API token" aria-label="Make an API token">${ICON.plus} Make</button></div>
+      <span class="hint">A token lets a script or an agent use iPX as you, adding and removing
+      your feeds and reading your items: it sends <code>Authorization: Bearer</code> and the token.
+      Anyone holding one is you here, so revoke one you no longer use.</span></div>
     <div class="field"><label>Feeds are checked every</label>
       <span class="hint">${everyText(g.every_mins)}, for every feed that does not set its own.
       ${admin?'This and the rest of the server\'s settings are on the <a href="/admin">admin page</a>.':'Only an admin changes this.'}</span></div>`);
   $('#stheme').onchange=e=>setTheme(e.target.value,undefined,true);
   $('#smode').onchange=e=>setTheme(undefined,e.target.value,true);
   $('#gopml').onclick=opmlModal;
+  drawTokens();
+  $('#tokadd').onclick=async()=>{
+    const name=$('#tokname').value.trim(); if(!name){ $('#tokname').focus(); return; }
+    try{
+      const t=await api('/api/tokens',{method:'POST',body:JSON.stringify({name})});
+      $('#tokname').value='';
+      await drawTokens();
+      // Shown this once: only its hash is kept, so a lost token is revoked and made again.
+      $('#tokens').insertAdjacentHTML('afterbegin',`<div class="field" id="toknew"><span class="hint">Your new token,
+        ${esc(t.name)}. Copy it now: it is not shown again.</span>
+        <div class="inline"><input type="text" id="tokval" readonly value="${esc(t.token)}">
+        <button class="btn ico" id="tokcopy" title="Copy" aria-label="Copy">${ICON.copy}</button></div></div>`);
+      $('#tokcopy').onclick=()=>copyText(t.token,$('#tokcopy'));
+      $('#tokval').select();
+    }catch(err){ toast(err.message,true); }
+  };
   $('#sblock').onchange=async e=>{
     const blocked=splitWords(e.target.value);
     try{
@@ -255,6 +295,19 @@ async function prefsModal(){
   };
 }
 
+/// Your API tokens, each with when it was made and last used, and a button to revoke it.
+async function drawTokens(){
+  const box=$('#tokens'); if(!box) return;
+  const list=await api('/api/tokens').catch(()=>[]);
+  box.innerHTML=list.map(t=>`<div class="tokrow"><span><b>${esc(t.name)}</b>
+    <span class="hint">made ${dateOf(t.created)}, last used ${ago(t.last_used)}</span></span>
+    <button class="btn ico" data-tok="${t.id}" title="Revoke ${esc(t.name)}" aria-label="Revoke ${esc(t.name)}">${ICON.trash}</button></div>`).join('');
+  for(const b of $$('[data-tok]',box)) b.onclick=async()=>{
+    try{ await api(`/api/tokens/${b.dataset.tok}`,{method:'DELETE'}); toast('Revoked'); drawTokens(); }
+    catch(err){ toast(err.message,true); }
+  };
+}
+
 const splitWords=(s: string)=>s.split(',').map(w=>w.trim()).filter(Boolean);
 
 function settingsModal(f, newUrl?: string){
@@ -307,7 +360,7 @@ function settingsModal(f, newUrl?: string){
   // Offer the categories the Directory already shows, so a blog about games joins Games rather
   // than starting a second chip beside it.
   if($('#scats')) api('/api/directory').then(rows=>{ $('#scats').innerHTML=[...new Set((rows||[])
-    .map(p=>p.category).filter(Boolean))].sort().map(c=>`<option value="${esc(c)}">`).join(''); }).catch(()=>{});
+    .flatMap(p=>[p.category,p.subcategory]).filter(Boolean))].sort().map(c=>`<option value="${esc(c)}">`).join(''); }).catch(()=>{});
   $('#ssave').onclick=async()=>{
     const max=$('#smax').value;
     try{

web/src/feeds.ts

@@ -29,7 +29,7 @@ const unreadFirst=(a,b)=>Number(b.unread>0)-Number(a.unread>0);
 // an id starting with ':' can never be a feed's, since feed ids are slugs.
 const VIEWS={
   ':directory':{title:'Directory',icon:ICON.directory,url:'/api/directory',
-    blurb:'Every feed anyone on this server subscribes to, A to Z. The feeds inside an OPML are listed one by one, not the OPML.'},
+    blurb:'Every feed anyone on this server subscribes to. The feeds inside an OPML are listed one by one, not the OPML.'},
   ':popular':{title:'Popular',icon:ICON.popular,url:'/api/popular',
     blurb:'The ten feeds with the most subscribers here. The feeds inside an OPML count one by one, not the OPML.'},
   ':listening':{title:'Currently Listening',icon:ICON.audio,url:'/api/entries?filter=in_progress&limit=50',
@@ -130,8 +130,9 @@ function renderFeeds(){
       // The mark sits on the artwork, the thing the eye scans the list by, and the line under
       // the name says what is wrong in place of the counts.
       `<span class="fart">${mine.length?folderArt(f,mine):artHTML(f.image,f.title||f.id)}`+
-      (err?`<span class="ferr" role="img" title="${esc(err)}" aria-label="Error: ${esc(err)}">${ICON.alert}</span>`:'')+`</span>`+
-      `<div class="txt"><b>${f.pinned?`<span class="fpin" title="Pinned">${ICON.pinOn}</span>`:''}${esc(f.title||f.id)}</b><small>`+
+      (err?`<span class="ferr" role="img" title="${esc(err)}" aria-label="Error: ${esc(err)}">${ICON.alert}</span>`:'')+
+      (f.pinned?`<span class="fpin" role="img" title="Pinned" aria-label="Pinned">${ICON.pinOn}</span>`:'')+`</span>`+
+      `<div class="txt"><b>${esc(f.title||f.id)}</b><small>`+
       (nbad?`${plural(nbad,'feed')} not updating`
         :err?esc(f.failing?.reason||'The last check failed')
         :`${mine.length?plural(mine.length,'feed'):plural(eps,'item')} · ${saved} downloaded`)+

web/src/items.ts

@@ -65,13 +65,14 @@ function epEl(e){
   el.dataset.guid=e.guid;
   const num=[e.season?`S${e.season}`:'',e.episode?`E${e.episode}`:''].filter(Boolean).join('');
   const left = e.position>10 && e.duration ? `${clock(e.duration-e.position)} left` : (e.duration?clock(e.duration):'');
+  const name=entryName(e);
   el.innerHTML=`
     <button class="st" data-a="read" title="Mark ${e.read?'unread':'read'}">${
       player.guid===e.guid?EQ:(e.read?'':'●')}</button>
     <button class="fl${e.flagged?' on':''}" data-a="flag" aria-pressed="${!!e.flagged}" title="${
       e.flagged?'Unpin':'Pin, so it is never deleted'}">${e.flagged?ICON.pinOn:ICON.pin}</button>
     <div class="body">
-      <span class="t">${esc(e.title||'(untitled)')}</span>
+      <span class="t${name.derived?' notitle':''}">${esc(name.text)}</span>
       <div class="line">${[
         num&&`<span>${num}</span>`,
         left&&`<span>${left}</span>`,
@@ -136,6 +137,25 @@ function kindIcon(enc){
 function feedArt(id=S.feed){ const f=S.feeds.find(x=>x.id===id); return f&&f.image; }
 const feedName=id=>{ const f=S.feeds.find(x=>x.id===id); return f?(f.title||f.id):id; };
 
+/// What an item is called. Its title, or for one published without (RSS 2.0 makes it optional,
+/// and Scripting News titles almost none of its posts) the opening of its text, then its file's
+/// name, then its feed and date: "(untitled)" fifty times down a list said nothing about any of
+/// them. `derived` marks a name that is not a title, which the list sets as text, not heading.
+function entryName(e): {text: string, derived: boolean}{
+  if(e.title&&e.title.trim()) return {text:e.title, derived:false};
+  // An inert document: nothing in it loads or runs, where a detached element fetches its images.
+  const words=e.description
+    ? (new DOMParser().parseFromString(e.description,'text/html').body.textContent||'').replace(/\s+/g,' ').trim()
+    : '';
+  if(words){
+    const cut=words.length>120 ? words.slice(0,120).replace(/\s+\S*$/,'')+'…' : words;
+    return {text:cut, derived:true};
+  }
+  const file=((e.enclosures||[])[0]?.url||'').split(/[?#]/)[0].split('/').pop().replace(/\.[a-z0-9]{1,5}$/i,'');
+  if(file){ try{ return {text:decodeURIComponent(file), derived:true}; }catch{ return {text:file, derived:true}; } }
+  return {text:[feedName(e.feed_id), dateOf(e.published)].filter(Boolean).join(', '), derived:true};
+}
+
 /// Selecting an item shows it in the pane below, rather than expanding the row.
 /// Replaces one row with a fresh one, leaving the rest of the list and its scroll alone.
 function swapRow(e){
@@ -271,7 +291,8 @@ function detailHtml(e){
   // description was sanitized server-side with ammonia before it ever reached here
   return `
     <button class="btn ico" id="dback" title="Back to the items" aria-label="Back to the items">${ICON.left}</button>
-    <h3 class="dt">${esc(e.title||'(untitled)')}</h3>
+    ${/* A post without a title starts with its text: its own first words as a heading above
+         themselves would read as a mistake. */ e.title&&e.title.trim() ? `<h3 class="dt">${esc(e.title)}</h3>` : ''}
     <div class="dmeta">
       ${/* Joined, so a missing date or number leaves no stray dot behind. */
         [f&&esc(f.title||f.id), num, dateOf(e.published), e.duration&&clock(e.duration)]
@@ -328,7 +349,7 @@ function encBox(x){
   // Nothing on disk. For an image or a PDF you usually just want to look at it, so link
   // straight to the publisher's copy in a new tab -- no download, and nothing proxied
   // through here, which would make ipx a fetch-anything relay.
-  const viewable = !isPlayable(x) && x.state !== 'pending';
+  const viewable = !isPlayable(x) && x.state !== 'pending' && x.state !== 'held';
   return `<div class="encbox">
     ${kindIcon(x)}
     <span class="meta" style="flex:1">${size}</span>
@@ -346,13 +367,13 @@ async function epAction(a: string, e, el, encId?: number){
   const path=`/api/entries/${encodeURIComponent(e.feed_id)}/${encodeURIComponent(e.guid)}`;
   try{
     if(a==='play') play(e, encId!=null ? enc : undefined);
-    if(a==='share') await share(e.title||'', encId!=null ? enc.url : e.link, el);
+    if(a==='share') await share(entryName(e).text, encId!=null ? enc.url : e.link, el);
     if(a==='flag'){ e.flagged=!e.flagged; await api(path+'/flags',{method:'POST',body:JSON.stringify({flagged:e.flagged})}); redraw(); }
     if(a==='read'){ await setRead(e,!e.read); redraw(); }
     if(a==='get'){
       if(!enc) return;
       await api(`/api/enclosures/${enc.id}/download`,{method:'POST'});
-      toast('Queued: '+(e.title||'item'));
+      toast('Queued: '+entryName(e).text);
     }
     if(a==='del'){
       const f=S.feeds.find(x=>x.id===e.feed_id);

web/src/native.ts

@@ -77,7 +77,7 @@ function installNativePlayback(){
       realLoad.call(audio);
       const e = player.entry, f = player.feed;
       post({t:'load', url:v, enc:player.enc, feedId:f, guid:player.guid,
-        title:(e && e.title) || '', feedTitle:feedName(f),
+        title:e ? entryName(e).text : '', feedTitle:feedName(f),
         artwork:(e && e.image) || feedArt(f) || null,
         // Where the host starts is not this: the seek to where you left off is player.ts's, on
         // loadedmetadata, so one piece of code decides it. This is for the host's now-playing

web/src/player.ts

@@ -32,7 +32,7 @@ function play(e,enc=e.enclosures.find(isPlayable)){
     if(e.position>5) audio.addEventListener('loadedmetadata',()=>{audio.currentTime=e.position},{once:true});
     // Initials, if it comes to that, are the feed's: the episode's read as "SE" beside the feed's art.
     $('#partwrap').innerHTML=artHTML(e.image||feedArt(e.feed_id),feedName(e.feed_id));
-    $('#ptitle').textContent=e.title||'(untitled)';
+    $('#ptitle').textContent=entryName(e).text;
     const f=S.feeds.find(x=>x.id===e.feed_id);
     $('#pfeed').textContent=f?(f.title||f.id):'';
     $('#player').classList.add('on');
@@ -45,8 +45,8 @@ function play(e,enc=e.enclosures.find(isPlayable)){
 function mediaSession(e,f){
   if(!('mediaSession' in navigator)) return;
   navigator.mediaSession.metadata=new MediaMetadata({
-    title:e.title||'', artist:f?(f.title||f.id):'', album:f?(f.title||''):'',
-    artwork:(e.image||(f&&f.image))?[{src:e.image||f.image,sizes:'512x512'}]:[],
+    title:entryName(e).text, artist:f?(f.title||f.id):'', album:f?(f.title||''):'',
+    artwork:(e.image||(f&&f.image))?[{src:artSrc(e.image||f.image),sizes:'512x512'}]:[],
   });
   const h={play:()=>audio.play(),pause:()=>audio.pause(),
     seekbackward:()=>audio.currentTime-=15,seekforward:()=>audio.currentTime+=30};

web/src/util.ts

@@ -145,10 +145,11 @@ const tint=name=>{ let h=0; for(const c of name||'?') h=(h*31+c.charCodeAt(0))>>
 function tileHTML(name,cls){
   return `<div class="art ini ${cls||''}" style="--tint:${tint(name)}">${esc(initials(name))}</div>`;
 }
-/// On the https page, the browser upgrades an http:// image to https, and a host that has no
-/// https shows nothing (issue #90); ipx fetches those itself.
+/// Every image a feed names comes from iPX, which keeps a copy: fast after the first time, and
+/// no publisher's server asked on every visit. It began with http-only artwork, which the https
+/// page could not load (#90).
 function artSrc(url: string){
-  return location.protocol==='https:'&&/^http:\/\//i.test(url) ? '/api/art?u='+encodeURIComponent(url) : url;
+  return /^https?:\/\//i.test(url) ? '/api/art?u='+encodeURIComponent(url) : url;
 }
 function artHTML(url: string | null, name: string, cls?: string){
   return url