Show what a feed says it is on its page in the Directory (#130)

A feed's page in the Directory (#128) had its cover, category and latest items, but not the
feed's own description, which Apple's show page leads with: iPX read every item's description
and threw the channel's away, and the feeds table had nowhere to keep it.

It is parsed now, RSS's <description>, or iTunes' summary when that is empty, or Atom's
subtitle, kept in feeds.description (kept when a later read has none, as title and image are),
and sent, sanitized, with the items from /api/directory/{id}, which is now an object, not a
list. The page shows it as plain text under the header, three lines of it, with More when there
is more. A description that only repeats the title is left out.

feeds.description is the first column added to a table that already exists. create_missing
looks for it with a SELECT and runs the ALTER only when it is missing, since it runs on every
open, the healthcheck's included, and an ALTER's lock is what made that time out before. The
same once-only step forgets every feed's ETag and Last-Modified: a feed is read whole only when
it has changed, so its description would otherwise wait for its next item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
-- In the Directory, a feed you do not subscribe to opens a page of its own: its cover, its category, how many people here subscribe, and its latest episodes or posts, with a button to subscribe. Before, only its subscribe button did anything.
+- In the Directory, a feed you do not subscribe to opens a page of its own: its cover, its category, how many people here subscribe, what the feed says it is, and its latest episodes or posts, with a button to subscribe. Before, only its subscribe button did anything.
 - Feeds that don't set a category, as most blogs don't, now get one automatically, so they show up under a category in the Directory. iPX picks the closest of Apple's podcast categories from the feed's title and latest headlines. This happens once per feed: the first time a new feed is read, or for a feed you already had, the next time you force a refresh. It needs `TYPESAFE_KEY` set. An admin can change the category in the feed's settings.
 - Artwork is kept on disk once shown and loads from iPX, not from each publisher's server: fast after the first time, and still there when the publisher's server is not. The admin page sets how much is kept (500 MB by default).
 - `ipx add --list --category News <url>` puts a feed in the Directory for anyone to subscribe to,

CLAUDE.md

@@ -182,7 +182,8 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
   command, the healthcheck's `ipx status` included, so it must never write when nothing is
   missing: SeaORM's experimental schema sync dropped and remade an index on every open, the
   write lock that took made `ipx status` time out behind a busy daemon, and it was removed for
-  it. A new column on an existing table needs its own `ALTER`; nothing adds one for you.
+  it. A new column on an existing table needs its own `ALTER`; nothing adds one for you. Look for
+  the column first and alter only when it is missing, as `feeds.description` does.
 * **SQL written by hand in `db.rs` has to run on SQLite and Postgres both** (issue #18): `$1`
   parameters, bound only if used; `ON CONFLICT`, not `INSERT OR IGNORE`; yes/no columns tested
   as themselves (`NOT coalesce(s.read, false)`) and written as `true`/`false`, never compared to

docs/architecture.md

@@ -51,8 +51,8 @@ build needs node and `npm ci` run once.
 ## Data model
 
 ```
-feeds       id, url, title, image, etag, last_modified, last_checked, ttl_mins,
+feeds       id, url, title, image, category, description, etag, last_modified,
-            last_error, orphaned, group_id, managed
+            last_checked, ttl_mins, last_error, orphaned, group_id, managed
 entries     feed_id, guid, title, link, published, description, first_seen,
             image, duration, episode, season            PK (feed_id, guid)
 enclosures  id, feed_id, guid, url UNIQUE, mime, length, path, state,
@@ -71,7 +71,8 @@ before accounts; two bugs came from queries still reading them, and they were dr
 Schema changes: the tables are the entities in `src/entity.rs`, and `Db::open` creates whatever
 table or index a database is missing from them (`db::create_missing`), with `IF NOT EXISTS`. It
 never alters a table that exists, so a new column on one needs its own `ALTER` in
-`create_missing`, or `sea-orm-migration` once there are several. `Db::memory()` builds its
+`create_missing`, looked for first and run only when it is missing, as `feeds.description`'s is,
+or `sea-orm-migration` once there are several. `Db::memory()` builds its
 database the same way, so the tests run on the schema production gets. A database from before
 0.7 takes its last columns from the old `migrate()`, so it upgrades through a 0.7 release first.
 
@@ -127,7 +128,7 @@ else a `401`. A feed's items and files (its entries, `download-latest`, `/api/en
 | `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
 | `POST /api/fetch` | |
 | `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
-| `GET /api/directory/{id}` | a listed feed's latest twenty items, for its page before you subscribe: title, link, date, length and text, never a file or its address, and only for a feed the directory lists |
+| `GET /api/directory/{id}` | a listed feed's description and latest twenty items, for its page before you subscribe: title, link, date, length and text, never a file or its address, and only for a feed the directory lists |
 | `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with an OPML's feeds in place of the OPML and everyone counted (id, title, art, count, whether it is yours, the feed's iTunes category, whether it carries audio or video; never a URL, never a private feed); subscribe by id |
 | `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
 | `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |

docs/users.md

@@ -75,8 +75,8 @@ with the most subscribers on this server, you included, ranked, then the categor
 showing a few of its shows' covers. A category has a page of its own: its most subscribed feeds,
 then all of them as a grid of cover art, with its subcategories along the top (Tech News under
 News, Video Games under Leisure) and a choice of order. See all lists every feed, and the search
-box finds one by name. A feed you do not subscribe to opens a page of its own, with its latest
+box finds one by name. A feed you do not subscribe to opens a page of its own, with what it says it
-episodes or posts, so you can see what it is first. Every page can be narrowed to Podcasts (anything with audio or video) or
+is and its latest episodes or posts, so you can see what it is first. Every page can be narrowed to Podcasts (anything with audio or video) or
 Blogs (the rest). A show's category is the one it gives itself in iTunes, or for one that names
 none, one an admin or iPX picked for it. Your own feeds are marked Subscribed.
 It shows a title, artwork and a count, never a URL or who reads it. An OPML subscription is listed

src/db.rs

@@ -69,6 +69,21 @@ async fn create_missing(orm: &sea_orm::DatabaseConnection) -> Result<()> {
     ] {
         orm.execute_unprepared(sql).await.with_context(|| sql.to_owned())?;
     }
+    // A column added to a table that already exists, which CREATE TABLE IF NOT EXISTS leaves
+    // as it was. Looked for first, and added only when it is missing: this runs on every open,
+    // the healthcheck's `ipx status` included, and an ALTER takes a lock that waits behind a
+    // busy daemon, which is what made the healthcheck time out once before.
+    if orm.execute_unprepared("SELECT description FROM feeds LIMIT 0").await.is_err() {
+        // A feed is read whole only when it has changed since the last read, so each one's
+        // description would wait for its next item. Forgetting how each was last seen makes
+        // its next read, when it is due anyway, a whole one.
+        for sql in [
+            "ALTER TABLE feeds ADD COLUMN description TEXT",
+            "UPDATE feeds SET etag = NULL, last_modified = NULL",
+        ] {
+            orm.execute_unprepared(sql).await.with_context(|| sql.to_owned())?;
+        }
+    }
     Ok(())
 }
 
@@ -314,6 +329,11 @@ impl Db {
         Ok(())
     }
 
+    /// What a feed says it is, for its page in the Directory.
+    pub async fn feed_description(&self, feed_id: &str) -> Result<Option<String>> {
+        Ok(feeds::Entity::find_by_id(feed_id.to_owned()).one(&self.orm).await?.and_then(|f| f.description))
+    }
+
     #[tracing::instrument(skip_all)]
     pub async fn feed_summary(&self, feed_id: &str) -> Result<FeedSummary> {
         let mut sum = feeds::Entity::find_by_id(feed_id.to_owned())
@@ -475,13 +495,14 @@ impl Db {
         ttl_mins: Option<u64>,
         image: Option<&str>,
         category: Option<&str>,
+        description: Option<&str>,
     ) -> Result<()> {
         // category is taken as it comes, unlike title and image: a show that leaves a category
         // should leave the Directory's chip too.
         let o = |v: Option<&str>| sea_orm::Value::from(v.map(str::to_owned));
         self.exec(
-            "INSERT INTO feeds (id, url, title, etag, last_modified, last_checked, ttl_mins, last_error, image, category)
+            "INSERT INTO feeds (id, url, title, etag, last_modified, last_checked, ttl_mins, last_error, image, category, description)
-             VALUES ($1, $2, $3, $4, $5, $6, $7, NULL, $8, $9)
+             VALUES ($1, $2, $3, $4, $5, $6, $7, NULL, $8, $9, $10)
              ON CONFLICT (id) DO UPDATE SET
                 url = excluded.url,
                 title = coalesce(excluded.title, feeds.title),
@@ -491,6 +512,7 @@ impl Db {
                 ttl_mins = excluded.ttl_mins,
                 image = coalesce(excluded.image, feeds.image),
                 category = excluded.category,
+                description = coalesce(excluded.description, feeds.description),
                 last_error = NULL,
                 error_since = NULL",
             vec![
@@ -503,6 +525,7 @@ impl Db {
                 ttl_mins.map(|t| t as i64).into(),
                 o(image),
                 o(category),
+                o(description),
             ],
         )
         .await?;
@@ -2076,6 +2099,29 @@ pub fn now() -> i64 {
 mod tests {
     use super::*;
 
+    #[tokio::test]
+    async fn a_database_from_before_descriptions_gains_the_column_and_reads_each_feed_whole() {
+        let db = Db::memory().await.unwrap();
+        let record = |etag: &'static str, about: Option<&'static str>| {
+            let db = &db;
+            async move { db.record_feed("f", "https://x/f", Some("F"), Some(etag), Some("then"), None, None, None, about).await.unwrap() }
+        };
+        record("\"a\"", Some("About F")).await;
+        assert_eq!(db.feed_description("f").await.unwrap().as_deref(), Some("About F"));
+        record("\"a\"", None).await;
+        assert_eq!(db.feed_description("f").await.unwrap().as_deref(), Some("About F"), "kept when a read finds none");
+        // A database from before #130.
+        db.exec_for_test("ALTER TABLE feeds DROP COLUMN description").await.unwrap();
+        create_missing(&db.orm).await.unwrap();
+        assert_eq!(db.feed_description("f").await.unwrap(), None);
+        let st = db.http_state("f").await.unwrap();
+        assert!(st.etag.is_none() && st.last_modified.is_none(), "every feed is read whole next time, for its description");
+        // With the column there, opening the database again writes nothing.
+        record("\"b\"", Some("About F")).await;
+        create_missing(&db.orm).await.unwrap();
+        assert_eq!(db.http_state("f").await.unwrap().etag.as_deref(), Some("\"b\""));
+    }
+
     #[tokio::test]
     async fn every_sort_column_runs_and_orders_both_ways() {
         let db = Db::memory().await.unwrap();

src/entity.rs

@@ -19,6 +19,10 @@ pub mod feeds {
         /// The channel's first <itunes:category>, for the Directory.
         #[sea_orm(column_type = "Text", nullable)]
         pub category: Option<String>,
+        /// What the feed says it is, as it says it, for its page in the Directory. Added after
+        /// the table was, so `create_missing` adds it to a database without it.
+        #[sea_orm(column_type = "Text", nullable)]
+        pub description: Option<String>,
         #[sea_orm(column_type = "Text", nullable)]
         pub etag: Option<String>,
         #[sea_orm(column_type = "Text", nullable)]

src/feed.rs

@@ -15,6 +15,9 @@ pub struct ParsedFeed {
     pub site: Option<String>,
     /// The channel's first `<itunes:category>`, for the Directory's categories.
     pub category: Option<String>,
+    /// What the feed says it is, for its page in the Directory (#130): RSS's `<description>`,
+    /// or iTunes' summary where that is empty, or Atom's subtitle. HTML as often as not.
+    pub description: Option<String>,
     pub entries: Vec<Entry>,
 }
 
@@ -734,6 +737,7 @@ fn from_rss(ch: rss::Channel, bytes: &[u8]) -> ParsedFeed {
         // Only the iTunes one: Apple's list is fixed, while a plain <category> is freeform and
         // would fill the Directory with one-off tags. The subcategory where there is one: Apple
         // files every tabletop and gaming show under Leisure, which says little; Games says it.
+        description: non_empty(Some(ch.description())).or_else(|| non_empty(ch.itunes_ext().and_then(|i| i.summary()))),
         category: ch
             .itunes_ext()
             .and_then(|i| i.categories().first())
@@ -803,6 +807,7 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
             .map(|l| l.href().trim().to_owned())
             .filter(|h| !h.is_empty()),
         category: None,
+        description: non_empty(feed.subtitle().map(|t| t.as_str())),
         entries,
     }
 }
@@ -1163,6 +1168,7 @@ mod tests {
         assert_eq!(feed.title.as_deref(), Some("Test Cast"));
         assert_eq!(feed.ttl_mins, Some(45));
         assert_eq!(feed.category.as_deref(), Some("Podcasting"), "the first, by its subcategory");
+        assert_eq!(feed.description.as_deref(), Some("A synthetic feed used by the parser tests."));
         assert_eq!(feed.entries.len(), 3);
 
         let ep = &feed.entries[0];
@@ -1187,6 +1193,17 @@ mod tests {
         );
     }
 
+    #[test]
+    fn a_feed_says_what_it_is_in_itunes_summary_or_atom_subtitle() {
+        // An empty <description> gives way to iTunes' summary (#130).
+        let rss = br#"<?xml version="1.0"?><rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
+        <channel><title>R</title><description> </description><itunes:summary>Said here.</itunes:summary></channel></rss>"#;
+        assert_eq!(parse(rss).unwrap().description.as_deref(), Some("Said here."));
+        let atom = br#"<?xml version="1.0"?><feed xmlns="http://www.w3.org/2005/Atom"><title>A</title>
+        <subtitle>Or here.</subtitle><id>urn:a</id><updated>2026-01-01T00:00:00Z</updated></feed>"#;
+        assert_eq!(parse(atom).unwrap().description.as_deref(), Some("Or here."));
+    }
+
     #[test]
     fn a_file_wordpress_lists_twice_is_one_enclosure() {
         let xml = br#"<?xml version="1.0"?><rss version="2.0"><channel><title>R</title>

src/main.rs

@@ -1525,6 +1525,7 @@ async fn scan_one(
         parsed.ttl_mins,
         parsed.image.as_deref(),
         parsed.category.as_deref(),
+        parsed.description.as_deref(),
     ).await?;
     // Asked on a feed's first read, nothing of it stored yet, or when someone forces a refresh:
     // each call costs, and a feed already here without a category can wait for a forced one.

src/web.rs

@@ -1023,21 +1023,29 @@ struct ListedItem {
     description: Option<String>,
 }
 
-/// A listed feed's latest twenty items. Only a feed the Directory lists, so a guessed id reaches
+#[derive(Serialize)]
-/// nothing private, as `subscribe_popular` checks.
+struct Listed {
+    /// What the feed says it is (#130), sanitized as its items are.
+    description: Option<String>,
+    items: Vec<ListedItem>,
+}
+
+/// A listed feed's description and latest twenty items. Only a feed the Directory lists, so a
+/// guessed id reaches nothing private, as `subscribe_popular` checks.
 async fn get_listed(
     State(state): State<WebState>,
     user: crate::db::User,
     Path(id): Path<String>,
-) -> Result<Json<Vec<ListedItem>>, ApiError> {
+) -> Result<Json<Listed>, ApiError> {
     if !popular(&state, user.id, true).await?.iter().any(|p| p.id == id) {
         return Err(ApiError::not_found(format!("{id:?} is not in the directory")));
     }
     let order = crate::db::order_sql("published", "desc");
     let rows = state.ctx.db.entries_in(user.id, Some(&id), crate::db::Filter::parse("all"), None, 0, 20, &order).await?;
     let mut sanitizer = feed_sanitizer();
-    Ok(Json(
+    let description = state.ctx.db.feed_description(&id).await?.map(|d| clean_description(&mut sanitizer, &d, None));
-        rows.into_iter()
+    let items = rows
+        .into_iter()
         .map(|e| ListedItem {
             description: e.description.as_deref().map(|d| clean_description(&mut sanitizer, d, e.link.as_deref())),
             guid: e.guid,
@@ -1046,8 +1054,8 @@ async fn get_listed(
             published: e.published,
             duration: e.duration,
         })
-            .collect(),
+        .collect();
-    ))
+    Ok(Json(Listed { description, items }))
 }
 
 fn sort_name(p: &PopularRow) -> String {

tests/ui/app.spec.js

@@ -910,6 +910,9 @@ test('the Directory lists what everyone here reads, the most subscribed first, b
   await tiles.filter({ hasText: 'Test Show' }).locator('.txt').click();
   await expect(piper.locator('.show h2')).toHaveText('Test Show');
   await expect(piper.locator('#dshow .lep', { hasText: 'First Episode' })).toBeVisible();
+  // What the feed says it is, from its own <description> (#130).
+  await expect(piper.locator('#dabout p')).toHaveText('A fixture feed.');
+  await expect(piper.locator('#dabout button')).toBeHidden();
   // Back, onto the tile it was opened from, the page under it as it was.
   await piper.locator('#directory .dback').click();
   await expect(tiles.filter({ hasText: 'Test Show' })).toBeFocused();
@@ -917,6 +920,7 @@ test('the Directory lists what everyone here reads, the most subscribed first, b
   // Only a listed feed has one, and it carries no file or its address.
   const preview = await (await piper.request.get('/api/directory/test-show')).text();
   expect(preview).toContain('First Episode');
+  expect(JSON.parse(preview).description).toBe('A fixture feed.');
   expect(preview).not.toContain('.mp3');
   expect((await piper.request.get('/api/directory/paid-show')).status()).toBe(404);
   // Nor does anything else give a feed's items or files to someone who does not subscribe (#129).

web/app.css

@@ -720,6 +720,11 @@ body.scan-this .fhead [data-a=scan] .i,body.scan-any .fhead [data-a=scanall] .i,
 .show .acts{margin-top:8px}
 .dlink{padding:0;font-size:13px;color:var(--accent);text-align:left}
 .dlink:hover{text-decoration:underline}
+/* What the feed says it is, three lines of it until More. */
+.about{max-width:760px;margin-top:18px}
+.about p{margin:0;color:var(--dim);font-size:14px;line-height:1.55;display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:3;overflow:hidden;overflow-wrap:anywhere}
+.about.open p{display:block}
+.about .dlink{margin-top:4px}
 .latest{max-width:760px}
 .lep{padding:12px 0;border-bottom:1px solid var(--line)}
 .lep b{display:block;font-size:15px;font-weight:600;line-height:1.35;overflow-wrap:anywhere}

web/src/dialogs.ts

@@ -138,7 +138,8 @@ function drawDirectory(box=$('#directory'), focus?: string){
         // Its category's page is one click away from here.
         show.category?`<button type="button" class="dlink" data-go="${esc(show.category)}">${esc(show.subcategory||show.category)}</button>`:''}
         <p class="sub">${n?`${n} ${n===1?'person here subscribes':'people here subscribe'}`:'Nobody here subscribes yet'}</p>
-        <div class="acts"><button class="btn ico primary" data-a="sub" title="Subscribe" aria-label="Subscribe">${ICON.plus}</button></div></div></div>`
+        <div class="acts"><button class="btn ico primary" data-a="sub" title="Subscribe" aria-label="Subscribe">${ICON.plus}</button></div></div></div>
+      <div class="about" id="dabout" hidden><p></p><button type="button" class="dlink" aria-expanded="false" hidden>More</button></div>`
       +sec(show.podcast?'Latest episodes':'Latest posts','<div class="latest" id="dshow"><p class="hint">Loading…</p></div>');
   }else if(q){
     // Names only: a category is a tap away, and "news" matching every News feed buried the
@@ -169,7 +170,7 @@ function drawDirectory(box=$('#directory'), focus?: string){
         pill('','All',inCat.length)+subs.map(s=>pill(s,s,inCat.filter(p=>p.subcategory===s).length)).join('')}</div>`:'')+grid);
   }
   box.innerHTML=html;
-  if(show){ $('.show [data-a="sub"]',box).onclick=()=>subscribeListed(show); fillShow($('#dshow',box),show); }
+  if(show){ $('.show [data-a="sub"]',box).onclick=()=>subscribeListed(show); fillShow(box,show); }
   const c=$('#dchart',box);
   if(c){ c.style.setProperty('--rows',String(Math.ceil(chart.length/2))); chart.forEach((p,i)=>c.appendChild(listedFeed(p,'rank',i+1))); }
   const t=$('#dtiles',box);
@@ -194,14 +195,29 @@ function drawDirectory(box=$('#directory'), focus?: string){
   if(focus) $(focus,box)?.focus();
 }
 
-/// A listed feed's latest items, on its page: each one's title, opening its post on the
+/// A listed feed's page, filled in: what it says it is, its first lines unless you ask for the
-/// publisher's site where it has one, its first lines, and when and how long.
+/// rest, and its latest items, each one's title, opening its post on the publisher's site where
-async function fillShow(list,p){
+/// it has one, its first lines, and when and how long.
-  let items;
+async function fillShow(box,p){
-  try{ items=await api(`/api/directory/${encodeURIComponent(p.id)}`); }
+  const list=$('#dshow',box), about=$('#dabout',box);
+  let got;
+  try{ got=await api(`/api/directory/${encodeURIComponent(p.id)}`); }
   catch(e){ list.innerHTML=`<p class="hint">${esc(e.message)}</p>`; return; }
   // Gone back, or on to another, while it loaded.
   if(dirShow!==p.id||!list.isConnected) return;
+  // Some feeds give their title again as their description, which says nothing new.
+  const said=plainText(got.description);
+  if(said&&said!==NAME(p)){
+    const text=$('p',about), more=$('button',about);
+    text.textContent=said; about.hidden=false;
+    // Only when there is more than the three lines shown.
+    more.hidden=text.scrollHeight<=text.clientHeight+1;
+    more.onclick=()=>{
+      const open=about.classList.toggle('open');
+      more.textContent=open?'Less':'More'; more.setAttribute('aria-expanded',String(open));
+    };
+  }
+  const items=got.items;
   list.innerHTML=items.length?items.map(e=>{
     const name=entryName(e), text=name.derived?'':plainText(e.description).slice(0,500);
     const when=[dateOf(e.published),e.duration&&clock(e.duration)].filter(Boolean);