A feed's page in the Directory (#128) had its cover, category and latest items, but not the
feed's own description, which Apple's show page leads with: iPX read every item's description
and threw the channel's away, and the feeds table had nowhere to keep it.
It is parsed now, RSS's <description>, or iTunes' summary when that is empty, or Atom's
subtitle, kept in feeds.description (kept when a later read has none, as title and image are),
and sent, sanitized, with the items from /api/directory/{id}, which is now an object, not a
list. The page shows it as plain text under the header, three lines of it, with More when there
is more. A description that only repeats the title is left out.
feeds.description is the first column added to a table that already exists. create_missing
looks for it with a SELECT and runs the ALTER only when it is missing, since it runs on every
open, the healthcheck's included, and an ALTER's lock is what made that time out before. The
same once-only step forgets every feed's ETag and Last-Modified: a feed is read whole only when
it has changed, so its description would otherwise wait for its next item.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API took a session cookie, a proxy's word or the shared admin token, so a script or an
agent working for one person had to sign in with their password and carry the cookie, or be
given the admin token. Settings now makes named tokens, ipx_ and 256 random bits, sent as
Authorization: Bearer. A token is its owner and no more. Only its SHA-256 is kept, in the
new api_tokens table, with when it was made and last used; it is shown once and revoked from
the same list. An unknown or revoked one gets a 401 rather than falling through to a cookie.
Cloudflare Access still stands in front of the tunnel, so from outside a token needs an
Access service token beside it; docs/sso.md says how.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every file in state 'pending' counted as waiting to download: 4420 in production, across 12
shows. Since #97 a scan only downloads among a feed's newest max_new_per_check items, so those
were back-catalogue episodes no scan would take; the real queue was 0.
A new state, 'held': listed and downloadable by hand, but outside the feed's newest items, or of
a feed nothing downloads automatically. Db::hold_back moves a feed's waiting files between
'pending' and 'held' each time the feed is due, changed or not, and again after its items are
stored, so a new episode, a raised limit or auto-download turned on or off moves them. A held
file keeps its item's place among the newest, as a downloaded one does. 'pending' now means
queued, so ipx status, /api/status and the dashboard read true without changing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each person has a list for every feed they read and one per feed. An item whose title or text
holds one of the words, matched as whole words so "ai" does not hide everything that "said"
anything, is hidden from them and, since the scanner now keeps each subscriber's filters
separate, is fetched only if someone else still wants it.
Whole-word matching is not something LIKE can do on both SQLite and Postgres, so the matches are
worked out in Rust into a `hidden` table whenever a list changes, someone subscribes, or a scan
brings in new items, and the queries only look that table up. Both new tables are tables rather
than columns because create_missing adds tables but never columns. Hidden counts as read for
the reaper and for "others still want this file", since whoever it is hidden from is as done
with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 3 of #18. Two tables: catalogue (each feed's config::Feed as JSON, so a
new feed setting needs no column) and settings (general: the five server
settings the admin page edits). config.toml keeps what is needed before the
database is reached, or decides who gets in: paths, [torrent], [web].
ipx still runs from one in-memory Config, assembled at start from both
(assemble_config). The eight places that saved config.toml and re-read it now
call Ctx::store_cfg, which writes the database and swaps the copy in memory; the
first-run web token, which is config.toml's, is written there.
The first start on a database with no catalogue imports config.toml's feeds and
settings in one transaction whose first insert is the settings row, so two ipx
starting at once cannot both import; it then trims config.toml, keeping the
original as config.toml.pre-database. After that, feeds written into the file are
ignored with a warning. copy-db skips it, and copies both tables.
Rehearsed on a clone of production's database with production's config: all 130
feeds imported, the file trimmed, and the feed list, settings and directory
identical to the live server's.
Postgres connections now ask for no notices. Every CREATE ... IF NOT EXISTS on an
existing table sends one, eleven per open; sqlx logs them, and
tracing-subscriber 0.3.23's per-layer filters then dropped the next line ipx
logged -- the import's own message went missing that way. Proved by toggling it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last nineteen functions move to SeaORM: recording feeds, items and
enclosures, managed OPML feeds, folding WordPress's repeated files, and handing a
Patreon creator's files to its shows. Two SQLite-only forms go: GLOB becomes a
LIKE with the underscore escaped (broader, harmlessly: the fold still keys on
`_=` and digits), and UPDATE OR IGNORE becomes an UPDATE ... WHERE NOT EXISTS.
The two transactions are SeaORM transactions.
With nothing left on it, rusqlite goes, with the SQL schema and migrate(). The
entities are the schema: create_missing makes whatever tables and indexes a
database lacks, from them, with CREATE ... IF NOT EXISTS. Production's schema
already has every column migrate() added and none it dropped.
Not SeaORM's schema sync, used until now: despite its docs it drops a unique
index the entities do not describe, so it dropped users_name_lower on every open.
Every `ipx` command then took a write lock, and against a daemon busy writing,
`ipx status` -- the healthcheck -- failed 7 times in 15 where the old code
failed none. Now 15 in 15, as before. On Postgres it would not have started.
WAL is set only when a file is not already in it: setting it takes a lock that
cannot wait out a busy daemon.
Checked on copies of production: a forced scan of all 162 feeds against the real
feeds with no database errors; the feed list, filters, sorts, search and the
reaper's candidates against the old code on the same data, earlier in the
branch. The column comments from the SQL schema move to the entities.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first step of moving to SeaORM (#18, phase 1). Nothing a user sees changes.
- src/entity.rs: the seven tables as SeaORM entities, matching the SQLite schema.
Strings are Text, as the columns are; yes/no columns are bool, which is BOOLEAN
on Postgres and stays INTEGER in the existing SQLite file (sync notes the
difference and leaves it alone).
- Db holds a SeaORM connection to the same SQLite file beside the rusqlite one;
functions move to it one at a time, and rusqlite goes with the last of them.
- db::sync creates what a database is missing from the entities (SeaORM's
schema-sync, experimental, so sea-orm is pinned to ~2.0), plus the two indexes
an entity cannot express. Checked against a copy of production: it added the
lower(name) index and changed nothing else.
- Test databases are now built from the entities alone, in a temporary file
(two connections to one ":memory:" are two databases), so every test also
checks that the entities describe what the queries need. That caught the one
difference: finding a user by name relied on COLLATE NOCASE, which Postgres
lacks; it now compares lower() on both sides.
- rusqlite steps back to 0.39: 0.40's libsqlite3-sys is newer than sqlx accepts,
and only one may link SQLite. It goes away at the end of this phase.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
