A feed's page in the Directory (#128) had its cover, category and latest items, but not the
feed's own description, which Apple's show page leads with: iPX read every item's description
and threw the channel's away, and the feeds table had nowhere to keep it.
It is parsed now, RSS's <description>, or iTunes' summary when that is empty, or Atom's
subtitle, kept in feeds.description (kept when a later read has none, as title and image are),
and sent, sanitized, with the items from /api/directory/{id}, which is now an object, not a
list. The page shows it as plain text under the header, three lines of it, with More when there
is more. A description that only repeats the title is left out.
feeds.description is the first column added to a table that already exists. create_missing
looks for it with a SELECT and runs the ALTER only when it is missing, since it runs on every
open, the healthcheck's included, and an ALTER's lock is what made that time out before. The
same once-only step forgets every feed's ETag and Last-Modified: a feed is read whole only when
it has changed, so its description would otherwise wait for its next item.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Directory listed Football (6 feeds), Soccer (7) and "Soccer," (2) as categories of their
own beside Sports, and Software How-To (2) beside Technology. CATEGORIES spelt two of Apple's
Sports subcategories American Football and Football (Soccer), names no feed uses: Apple's list
says Football and Soccer, so a feed naming either fell through as a category of its own.
They are Apple's spelling now. The old spellings are aliases, since Jev picked from the list
while it had them and its answers are stored, and so is Software How-To, Apple's Technology
subcategory before its 2019 list. A comma or full stop at the end of a feed's category, as in
"Soccer,", is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Directory had one row of chips holding whatever each feed's category was, a category
(Technology) or a subcategory (Tech News, Video Games) side by side: a podcast's own
<itunes:category> is stored as its subcategory, and Jev's answers (#117) are often
subcategories too, so after the first forced scan the row held 22 chips. /api/directory now
gives each feed's Apple category and subcategory, worked out from Apple's list, and the
page shows the categories, then a picked one's subcategories on a line of their own. A
category that is not Apple's, one an admin typed, stands as a category of its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most blogs and news sites name no <itunes:category>, so they sat under no chip in the
Directory unless an admin set one by hand. When TYPESAFE_KEY is set, a feed's first read, or
a forced refresh, asks TypeSafe's Jev to pick one of Apple's categories or subcategories from
the feed's title and up to 15 item titles, for a catalogue feed with neither its own category
nor an admin's. The answer is stored as the catalogue's category, so a feed is not asked
twice, and an admin can change it. Only on a first read or a forced refresh to keep the calls
down: feeds already subscribed are categorised by one `ipx fetch --force`.
Tried on eight subscribed feeds on 2026-10-04: XDA, Daring Fireball, Ars Technica and The
Verge as Tech News, The Old New Thing as Technology, John D. Cook as Mathematics,
BoardGameGeek as Games, Pluralistic as News Commentary (0.55). About 2,000 input tokens a
feed at $0.042 a million. The likeliest answer is kept however unsure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A feed whose address answered with a permanent redirect was read through it on every check,
and the catalogue kept the old address: 28 of 147 feeds in production, most http to https, some
to a new path or domain.
Feeds are now fetched with a client of their own that follows no redirects (Ctx::feed_client),
and feed::fetch follows them itself, up to 10 hops, so it sees each one. When every hop was
permanent (301 or 308) it says where the feed ended up, and the scan moves the feed there in the
catalogue (follow_move). A temporary hop (302, 307) anywhere moves nothing. A feed an OPML lists
is left alone, as the OPML would put the old address back, and so is a move onto an address
another feed has. A password goes only to the feed's own host, never to a redirect elsewhere;
reqwest's own following dropped it the same way. Ten hops is a loop, worded as reqwest worded
it so it still reads as redirect_loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#108: the HTTP client had no timeout, and scans handle feeds in order, so a hung server held
every scan. Dreamwidth answered 504 after 60-67 s for a day and each scan took 70-80 s instead of
15. A feed fetch, and a Patreon creator's show list, now gets 30 s from connecting to the last
byte (feed::FEED_TIMEOUT); the client gets a 10 s connect timeout, which bounds a download's
start but not a long download.
#109: iOS asks for /apple-touch-icon-precomposed.png first when the site is added to a home
screen; it was a 404 and the only non-feed warning in the log. It serves the same icon.
#110: the page is https and loads no http. Artwork on http came through /api/art (#90) even
when its host serves https too. A scan now tries each http artwork host on https once per feed
(feed::prefer_https) and stores the https address where the host answers with an image,
rewriting that feed's stored items from the same host (Db::secure_images). 4 of the 5 hosts in
production do; cdn.thesecretcabal.com presents another name's certificate and stays on
/api/art.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adding an address looked for the feed a web page links and, finding none, added the address
as it was: every check then failed, and the sidebar called it a feed that had moved. cnn.com
is one; its page links no feed. find_feed replaces feed_behind_page: the address is added if
it is a feed or an OPML list, the feed its page links if it is a web page that links one (and
that is a feed), and otherwise the add is refused with the reason, from the web page (400, the
dialog stays open) and from `ipx add`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cnn-com was added as 'cnn.com', stored as typed, and every check failed with "relative URL
without a base" before it reached the site to look for its feed. expand_input, which both the
web page and `ipx add` pass the address through, now makes one without a scheme https, and a
protocol-relative //host/path https too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From Dash0's structured logging guide, what applies here:
- Each JSON line inside a traced span ends with its trace_id and span_id, so a line in Loki leads
to its trace in Tempo; the access log is written inside its request's span so it has one too.
The JSON formatter takes no extra fields, so WithTrace appends them to the object it writes.
- A feed or download failure carries error.type (the HTTP status, or dns, redirect_loop,
timeout, ...) and http.response.status_code, from failure_kind beside explain_failure, so
failures group by kind without a regex over msg.
- Each event was logged twice: words under ipx::scan and fields under ipx::io. It is now one
line under ipx::scan with both; the wire copy is at debug, for the admin page's Daemon I/O tab,
and out of production's log. The healthcheck's status reply stays under ipx::io.
- The access log's ms is duration_ms. The dashboard and the prod-check skill follow.
- error fields are Display with the anyhow chain everywhere, not a mix of Debug and Display.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A feed's itunes:image or <image><url> was stored without being asked
for, so a dead one stood in the way of the site's icon. Ken and Robin
Talk About Stuff names http://kenandrobin.wpengine.com/.../kartas_podcast.png,
a 404, while its site's apple-touch-icon works. The feed's artwork now has
to answer as an image, as the site icon already did, when the feed is
read in full; otherwise the site's icon is looked for.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ipx had no spans, only log lines, so there was no way to see where a
slow scan, download or request spent its time. With
OTEL_EXPORTER_OTLP_ENDPOINT set, the daemon now exports traces over
OTLP/HTTP (Tempo on Tower): a scan, each feed in it, the feed fetch and
site icon lookup, downloads, torrents, reaps, and web requests. Log lines
inside a span ride along as its events.
Only the daemon exports: the healthcheck runs ipx status every 30s and
would bury everything else. The web event stream and the log view's
polling get no span, for the same reason. The exporter shares ipx's
reqwest 0.13, so no second HTTP stack comes in.
The stderr log now prefixes lines inside a span with it, as
tracing-subscriber's fmt layer does (scan{only=None force=false}: ...).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
site_icon took the icon a site's page names in its <link> tags without
asking for it, so a dead one was stored and /favicon.ico never tried.
antirez.com names /images/favicon.png, which is a 404, while its
/favicon.ico is there; the feed showed no artwork, and since the lookup
happens once, never would. The named icon now has to answer with an
image, as /favicon.ico already did.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both add paths, the CLI's and the web's, look behind the URL first: a web
page that names its feed with <link rel="alternate"> is swapped for that
feed, before the duplicate check so it finds a feed someone already has.
Before, the page itself was added and every scan failed on it.
alternate_feed_link found tags in a to_lowercase() copy and sliced the
original at those offsets; Unicode lowercasing changes some characters'
length, so a page with one before its <link> tags lost the href or
panicked off a char boundary. ASCII lowercasing keeps offsets aligned.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a feed names no image and none is stored, the scan fetches the
channel's site link (RSS <link>, Atom rel=alternate) and uses the
apple-touch-icon or icon it names, falling back to /favicon.ico when that
answers with an image.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Substack puts a post's subtitle in <description> and leaves it out of
content:encoded, and body() took content:encoded alone, so the subtitle
was lost (a known gap in CLAUDE.md). A description is now shown above the
body, as <p><em>, when it is short plain text the body does not already
contain. Podcast feeds that repeat their notes in both, whole or cut short
with an ellipsis, are unchanged; the comparison is by words, since a tag
taken out of the body leaves stray spaces around punctuation.
Checked against Experimental History's feed (subtitles appear) and The
Daily's (notes in both fields, shown once). Entries are inserted with ON
CONFLICT DO NOTHING, so only posts first seen from now on get it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Inter (#11): the pages are set in Inter's variable font, served from the
binary at /inter.woff2 as the icon is, with its OFL licence beside it in
web/. Classic keeps Lucida Grande, the 2004 app's face.
Double audio (#12): WordPress numbers each audio player on a page by
adding ?_=N to its file's URL, so a post that embeds the file it encloses
listed it twice, and it was downloaded twice. The parser keeps the first
of an item's enclosures that differ only by that number. At startup the
repeats already stored fold into the first; where only the repeat had
been downloaded its file moves to the first rather than being deleted.
Pin heading (#13): the rows' icon buttons kept the browser's side
padding, which pushed their 16px icon 3px right of centre, and the
heading's icon sat at the left of its column. Both are centred now, and
the heading row takes the pixel of border the rows have, so every
heading sits over its column.
Closes#11, closes#12, closes#13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An Atom title of type="html", and an RSS title in CDATA, reach the parser
with their entities intact, so The Verge's "Meta’s" showed as typed:
55 stored titles across 17 feeds. Titles are decoded one entity at a time
with quick-xml's HTML5 table, leaving an & that starts none ("Q&A") alone
rather than failing the whole title.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Apple puts every tabletop and gaming show under Leisure, so the top level
alone put most of this server's podcasts behind one chip. Games says what
they are; a show with no subcategory keeps its top-level one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Feeds take their channel's first <itunes:category> into a new feeds.category
column; the migration drops ETag and Last-Modified once so every feed re-reads
on its normal schedule and picks one up. /api/popular and /api/directory carry
category and podcast (any audio or video enclosure). Directory becomes a grid of
cover-art tiles under a chip rail: All, Podcasts, Blogs, and a podcast's
categories once Podcasts is picked. Popular and Add a feed keep their rows.
Closes#4, closes#5, closes#6, closes#7.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
doghouse answers 200 with "Unable to establish a DB connection", and parse()
reported two parser errors about end of input that buried it. A body that does
not start with < now reports its first line, and explain_failure flags it as
the publisher's problem. Malformed XML keeps the parsers' errors.
Fixes#2
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SZbKERNSt4vQfyGV8rvkqp
- Stop scanning an OPML/Patreon feed's derived rows once nobody subscribes
to it; retire them (drop or orphan) the way sync_group already does when
the list itself drops one. This is what let 922 defunct davewiner feeds
keep scanning hourly after the OPML left config.
- Repair feed XML with a bare `&`, and give a plain reason (moved web page
with its new address when linked, or nothing yet for an empty body)
instead of a raw parser error.
- Show a failing feed's plain-English reason and next step (Unsubscribe /
Use the new address) in the sidebar and on its own page, once it has
been down a day.
- Fix four small UI bugs: show-note links open in a new tab, video files
play as video, an opened item no longer disappears from the Unread tab,
and Subscribe/Unsubscribe get their own icons.
- Fix Settings disappearing for non-admin accounts: it was hiding the
whole modal instead of just the admin-only parts (Users, the editable
schedule/quota, Save), which are the only parts the server actually
refuses them.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DmQfE1eFPApnXWyPHBWqUA
libsyn served Daily Meditation Podcast's content:encoded cut at the '>'
inside a Tailwind class pasted from a web app, so 57 items began halfway
through a tag and the page showed the rest of it as text. Their
description was whole. A body that closes an attribute list before any
tag opens now falls back to the description, for RSS and Atom alike.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
A Patreon token pasted into Add feed, or a creator link without
&show=, becomes a folder of that creator's shows, found through
Patreon's web API and kept in step like a subscribed OPML (sync_group,
split out of sync_opml). A creator already read as one feed is split
too: each show takes over the files and read state it held
(Db::adopt). A creator with one show stays a plain feed.
Filter verdicts are judged again every scan, so turning on Allow
explicit brings skipped items back. Add feed has an explicit box.
Feeds in a group follow your settings on the group, as its dialog
said. A new feed no longer takes the id of a removed one at a
different URL and shows its old items. See CHANGELOG.md [Unreleased]
and docs/history.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wi22VSVrkAvqNj61eqHsm9
Resolves in order of deliberateness: itunes:image, media:thumbnail, a
media:content that says it is an image, then an image enclosure -- which
is where a blog's article picture actually lives, so those entries had
artwork available all along and showed none. Audio enclosures are never
taken for pictures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
The rss crate keeps at most one enclosure per item and, when a feed ships
several, silently keeps the last -- so a two-file item lost its first
file. enclosures_by_item reads them from the XML in document order,
unescaping attributes so a URL's & survives. The row summarises the
one you would act on and counts the rest; the pane below lists them all.
Non-media enclosures gain a View link opening in a new tab: the
publisher's URL, or the local copy once downloaded. A direct link, not a
proxy, so the daemon does not become a fetch-anything relay.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
A feed whose body sniffs as OPML is treated as a subscription list and
re-read on every scan, as iPodderX did. Listed feeds become real config
entries grouped under it, inherit its settings, land in one nested folder,
and are scanned in the same run.
When a feed leaves the OPML: removed if nothing was downloaded, kept and
flagged otherwise, so a downloaded file is never orphaned.
folder_for sanitized the whole folder string and would have flattened the
nesting; each segment is sanitized separately now, and a traversal still
cannot escape the download directory. Db::memory() also runs migrate(),
which it did not, so a migration-only column passed tests while missing in
production.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdXho5tTkjFLeUXKbEjKBh
Confirmed byte-for-byte against a live feed, and tested against a fixture
whose itunes:title differs: the RSS title wins, and season/episode stay
metadata rather than being folded into the displayed name. An item with a
season but no episode number keeps a null episode instead of inventing one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
Rewrites the page around a persistent player (speed, seek, resume,
MediaSession, keyboard shortcuts), artwork, filter tabs, episode search,
pagination and live progress, with modals and toasts replacing prompt()
and a status line.
Backend gains the metadata that makes that possible: feed and episode
artwork, durations, season/episode numbers and playback position, plus
filters, search, totals, mark-all-read, download-latest and OPML over
HTTP. Schema changes arrive through a real migration, since CREATE TABLE
IF NOT EXISTS does nothing to an installed database.
Fixes filtering, which returned 500 whenever no search term was given:
the search clause was dropped while its parameter was still bound.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
librqbit replaces the vendored BitTorrent 4.2.1 tree. Torrents download
in place because seeding serves the files it downloaded, so the planned
stage-then-move would have broken it. The stall budget now also covers
magnet metadata resolution, which otherwise never returns against a dead
swarm and wedged the scan.
Adds add/rm/import/export, tracing setup, systemd units and README.
A successful swarm download is unverified: no reachable peers here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
Conditional GET plus an RSS-first, Atom-fallback parser normalising both
into one entry model, with iTunes explicit and ttl handling carried over
from the Python. Enclosures are recorded as pending; nothing downloads yet.
GUID falls back guid -> link -> enclosure url -> title rather than hashing
the description as the original did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
