From e312f11bb10c08ea76de72dc22056b6587221cd4 Mon Sep 17 00:00:00 2001 From: rays Date: Fri, 18 Sep 2026 16:47:21 +0000 Subject: [PATCH] Remove docs/history.md It had grown past 1,700 lines, too large to be read or kept up. What it held -- what was wrong before a change and what it cost to find -- goes in commit message bodies now, beside the change. CLAUDE.md says so; the README and the changelog no longer point at it. It remains in git history. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 3 - CLAUDE.md | 6 +- README.md | 1 - docs/history.md | 1732 ----------------------------------------------- 4 files changed, 3 insertions(+), 1739 deletions(-) delete mode 100644 docs/history.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 23b31aa..482d32f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,9 +5,6 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -The long form, with what was wrong before and how it was found, is in -[docs/history.md](docs/history.md). - ## [Unreleased] ## [0.7.0] - 2026-09-18 diff --git a/CLAUDE.md b/CLAUDE.md index cc48253..f163bf1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -154,9 +154,9 @@ addressed to the person using it. Every change gets one line under `## [Unreleased]` in [CHANGELOG.md](CHANGELOG.md), in its [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) group: Added, Changed, Deprecated, Removed, Fixed or Security. Say it the way someone using ipx would notice it. When there is more to -say, such as what was wrong before or what it cost to find out, write it up at the top of -[docs/history.md](docs/history.md), dated. That record has been more useful than the git log more -than once. +say, such as what was wrong before or what it cost to find out, it goes in the commit message's +body, where `git log` and `git blame` find it beside the change. (There was a long-form +`docs/history.md` until 0.7.0; it grew too large to be useful and was removed. It is in git.) Cutting a release: rename `[Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD` and open a new empty `[Unreleased]` above it, bump `version` in `Cargo.toml`, tag the commit `vX.Y.Z`, and update the diff --git a/README.md b/README.md index b143bfd..a35630a 100644 --- a/README.md +++ b/README.md @@ -63,7 +63,6 @@ The UI is plain HTTP, so put TLS in front of it if it is reachable from outside | [docs/sso.md](docs/sso.md) | Signing in through Cloudflare Zero Trust or Authentik | | [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, control socket, HTTP API | | [CHANGELOG.md](CHANGELOG.md) | What changed, by release | -| [docs/history.md](docs/history.md) | How it was built, with what was wrong and why | | [CLAUDE.md](CLAUDE.md) | Notes for working on the code, including how production is deployed | ## Tests diff --git a/docs/history.md b/docs/history.md deleted file mode 100644 index 5ffe572..0000000 --- a/docs/history.md +++ /dev/null @@ -1,1732 +0,0 @@ -# How ipx was built - -The long-form record, newest first: what landed each day, what was wrong before, and what it cost -to find out. [CHANGELOG.md](../CHANGELOG.md) has the short version, by release; this is where the -reasoning lives. New write-ups go at the top. - -See [README.md](../README.md) for what the thing is. - -## 2026-09-15 — Kristen Bell again: a length minutes out, and an idle player - -After 0.6.0 the Kristen Bell episode dropped out of Currently Listening again, with eleven -minutes left in the player. Its saved position was 41:15 of a file 43:48 long, 94% and finished -by any measure, while the listener was at about 32:48. The first guess was a wrong length from -the feed, and it was wrong: the feed does say 41:23, but no length makes 41:15 unfinished. - -The log had the answer. Saves every ten seconds while it played, then two that were not -playback: one beside the only request for the file since the restart, and the last in the same -millisecond as a page load. The likeliest reading was a player left paused further on, in -another tab or on another device, saving its place as its page reloaded, since every save goes -through `savePos` and it saved whatever the player held whether or not anyone had played it -since. That was never confirmed. Fifteen minutes later the listener was playing on from 41:15 -and heard 2:33 left, which fits that position having been theirs all along. - -`savePos` now saves only once the player has played since its last save, so an idle one never -writes, and only playing counts: the seek to where you left off happens paused, and counting it -would save back whatever the list said, however old. A jump back is also saved at once, where -the ten-second check only ever looked forward. The fixture audio does not decode, so the browser -test stands in for a loaded player and counts what `savePos` sends. - -What was certainly wrong was the length. At 41:15 the item list said 0:08 left and the player -2:33: the feed gives 41:23 for a file that decodes to 43:48. The length the player measures now -goes into `entry_state` beside the position, and wins over the feed's for the 90% line and every -time left. Not into `entries`, where each scan writes the feed's figure back, and per listener, -so one person's player never changes what another sees. - -## 2026-09-15 — Currently Listening, empty for anyone who opens what they play - -Issue #14: an episode 32 minutes into 41 was missing from Currently Listening, which said nothing -was in progress. The filter was "position past five seconds and unread", on the reasoning that -`markPlayed` marks an episode read at 90%. But opening an item marks it read too, and you open an -episode to play it. In production every one of the twelve episodes with a saved position was read, -so the list was empty for everyone. The browser test for it had set the episode unread before -checking, to cope with an earlier test having opened it, and so tested around exactly this. - -Finished now means the saved position is 90% of the episode's length or more, the same line -`markPlayed` draws, and read plays no part. Three of those twelve episodes had no length in their -feed, and with no length there is no telling finished from started, so they would have stayed -listed for good. The player now sends the length it measured with each saved position, and that -fills in a missing one, never replacing a length the feed gave. - -## 2026-09-15 — davewiner's 922 rows, retired at last - -davewiner's OPML subscription left config.toml before `retire_group` existed, so nothing ever -retired the 922 feeds derived from it. The backstop in `subscriptions()` kept them from being -scanned, but the rows stayed, and 56 of the 75 errors stored in production were theirs: stale, and -never going to change. - -Retiring them the obvious way would have done damage. Eleven of the 922 -- xkcd, The Verge, -TechCrunch, Hacker News and others -- had since been given config entries of their own and were -scanned from there, but their rows still said `managed = 1`. `managed_feeds()` returned them, so -`retire_group("davewiner")` would have dropped the ten with no files as derived feeds, deleting -the entries people were reading. `unmanage` exists for exactly this and had never been applied. - -`retire_group` now unmanages a feed that has its own config entry instead of dropping it, which -also covers removing any OPML or Patreon subscription from the page, and the daemon retires every -group whose parent is gone from config when it starts. In production that is davewiner alone: 792 -rows with nothing downloaded forgotten, 119 with files kept as orphaned, 11 unmanaged. One of the -792, `ars-technica-all-content-2`, has a subscriber but neither a config entry nor a file; the -backstop had already hidden it, so nobody could see it to lose it. - -## 2026-09-14 — Settings, for everyone with an account - -A user reported that Settings disappeared shortly after they signed in: it showed for a moment, -then was gone. `#prefs` sat inside the same `.tgroup` as `#logs`, and `api('/api/me')` hid the -whole group -- `$('#admintools').hidden=true` -- the moment it learned the account was not an -admin. Nothing wrong with that check timing; it was hiding the wrong thing. - -The Settings modal is not actually all-or-nothing. `GET /api/settings`, and Export and Import -OPML, carry no admin check server-side -- `export_opml` and `import_opml` work from a user's own -subscriptions, and the schedule/quota page is read-only information, not a control. Only the -`PATCH` that changes those settings, and the Users screen behind it, return 403 for anyone but an -admin. The comment above the old hide -- "scanning, quotas, accounts and the log are the -operator's business" -- was wrong about quotas and half wrong about accounts: reading them is -everyone's; changing them is the operator's. - -`prefsModal()` now branches on `S.me.admin` the way the per-feed settings modal already does for -its URL field: a non-admin gets the schedule and quota as text, Subscriptions (Export/Import) -in full, and no Users section or Save button. Only `#logs` stays hidden, since the log names every -account and every failed sign-in. The browser test for a second account asserted the old -behaviour outright (`#prefs` hidden, not an admin) rather than what the server actually allows; -fixing the UI meant fixing the test's premise too, not just the assertion. - -## 2026-09-12 — Healthy while busy - -After a deploy the container sat at "starting" for a minute, and Docker's health log showed two -`ipx status` probes exceeding their 5-second timeout. The daemon's own log explained it. The first -scan after the start fetched 23 feeds, from 14:10:41 to 14:11:35, and both probes' `status` -commands waited in the job queue behind it; they were answered together at 14:11:35, straight after -`scan_done`. The worker runs one job at a time and `status` was one of its jobs, so any scan or -download longer than about a minute and a half, three 30-second probes, would have had Docker call -a working daemon unhealthy. - -The socket now answers `status` itself, from two short queries, and only real work goes through the -queue. The trade is that healthy now means the daemon answers on its socket and can read its -database; a worker stuck on one job would still pass. Asking a daemon that downloads hour-long -podcasts to be idle within five seconds was never a fair test of whether it was alive. A test holds -the queue full and checks `status` still comes back. - -The first version broadcast the answer, as the queued one had been. Timing `status` during a forced -scan in production showed the catch: `status` is a terminal event, so the `ipx fetch` watching that -scan stopped reading at the first probe and printed the status line as its last, while the scan -carried on. When `status` waited behind the scan it could never arrive first, so this had never -shown. The answer now goes only to the client that asked, and the test checks that another client -hears nothing. - -## 2026-09-12 — Signing in through Authentik, for real - -Ray could not get Authentik's sign-in to reach ipx, following `docs/sso.md`, which had been written -without ever being tried. Looking at the Cloudflare account through its API showed that side was -already complete. Authentik is Zero Trust's OpenID Connect identity provider; the Access application -`ipodderx` allows only it and a list of five addresses; the tunnel `rays-unraid` routes -`ipodderx.sdf1.net` to `192.168.1.130:8099`; DNS is a proxied CNAME to the tunnel. Access's log -showed `rays@sdf1.net` signing in through it. Nothing on Cloudflare was changed, so no other site -was touched. - -The gaps were all at ipx's end: `trusted_header` was empty, `trusted_proxies` held only loopback, -and the account was called `rays` while the header carries `rays@sdf1.net`. - -Finding the address to trust took the most time. The page said `127.0.0.1`, but `cloudflared` runs in -its own container and reaches ipx through the host's published port. ipx logs no peer addresses, so -the address was read from `/proc/net/tcp` inside the ipx container: `192.168.16.1`, the gateway of -`content_default`, where Docker's masquerade puts traffic crossing from another bridge. A request -from Tower's own shell arrived as `192.168.1.130` instead, and a throwaway `busybox` on the default -bridge as `192.168.16.1`: the first was refused with the header, the second believed. LAN machines -keep their own addresses, since Docker forwards published ports with iptables (the userland proxy -only handles loopback). - -Every change, in order, with how to undo it: - -1. **Code**, commit `586d2c0`: `ipx user rename`, deployed. Revert the commit and redeploy to - remove it; nothing depends on it once used. -2. **Account**: `docker exec iPodderX ipx user rename rays rays@sdf1.net`. Same id, so its feeds, - read state, password and admin rights stayed. Undo: `docker exec iPodderX ipx user rename - rays@sdf1.net rays`. Signing in at `/login` now takes the new name. -3. **Config**, `/mnt/fast/appdata/ipodderx/config.toml`, `[web]`: `trusted_header` from `""` to - `"Cf-Access-Authenticated-User-Email"`, and `"192.168.16.1"` added to `trusted_proxies`. The - file as it was is `config.toml.2026-09-12-sso.bak` beside it. Undo: copy the backup back and - `docker compose -f /mnt/fast/arcane/projects/content/compose.yaml restart ipodderx`. -4. **Cloudflare, Docker networks and other containers**: unchanged. The `busybox` test container - was removed when it exited, and its image afterwards. -5. **Authentik**, later the same day, because ipodderx had no tile in its library while Outline - did: a bookmark application `ipodderx` (pk `5854a98e-816a-4c4f-9f27-63e69dc29d1d`), made - through the API with a token of Ray's. No provider and no policy bindings, like Outline's, the - iPodderX icon, and a link to `https://ipodderx.sdf1.net`. It changes nothing about who can sign - in. Undo: delete it under Applications → Applications, or - `DELETE /api/v3/core/applications/ipodderx/`. -6. **Signing out**, later again. Sign out landed on ipx's password page while Access still vouched - for Ray, so it signed nothing out, and the page looked like the wrong login. Cloudflare's - `/cdn-cgi/access/logout` ends the Access session for every Access application at once (there is - no per-application sign-out, and it takes no redirect), and Authentik's end-session only ends - one application's session unless single logout is set up there. Ray chose Access's sign-out. New - `[web] sign_out_url`, set to `/cdn-cgi/access/logout` in production (the file as it was is - `config.toml.2026-09-12-signout.bak`), and `/login` now sends anyone the proxy vouches for on to - `/`. Undo: take the key out and restart; the code does nothing without it. - -What the address trusts is any container on Tower that connects through the host's port, not only -`cloudflared`. Verifying Cloudflare's signed `Cf-Access-Jwt-Assertion` would remove that, and is -the upgrade if it matters. - -## 2026-09-12 — Trimming the state database - -An audit of the database layer, with a read-only copy of production to check it against. The -data was already clean: no tables or indexes left from older versions, 47 free pages after the -column drops earlier the same day, and one stray `entry_state` row. The code had five things: - -- `migrate()` still added eight columns to any table missing them. All eight shipped in 0.2.0 and - upgrades now start from 0.3.0 at the oldest, so the list and its loop went; the `retired` drop - list stays, since a database coming from 0.4.0 still has the old read columns. -- `created` on `users`, `subscriptions` and `sessions` was written by every insert and read by - nothing. They joined `retired`. The old-database test now builds all three tables, foreign keys - included, since `DROP COLUMN` on a table that references another was the part worth proving. -- `Db::subscribed_feed_ids` had no callers, though its doc said the scanner walked it. - `Db::subscriber_count` had one caller asking whether it was above zero, which - `subscriber_counts().contains_key` answers. `Managed.orphaned` was selected and never read. -- `users.created` came back the same afternoon, with `last_login` beside it. Nothing read it, but - when an account was made and when it last signed in is what you want to know when tidying - accounts, and it cannot be recovered later. Both existing accounts got their creation times back - from the backup taken before the drop, and a last sign-in from their newest session in it. - `last_login` is kept to the hour, because the proxy vouches for every request and that would - otherwise be a write each time. - -## 2026-09-12 — Cutting what had outlived its reason - -A whole-repo audit for over-engineering listed twelve things to cut, and all of them went. - -- **Upgrades from before accounts.** `migrate_opml_children` moved OPML feeds that old versions - wrote into `config.toml` out to the database, and ran at every daemon start to do nothing after - the first. Production ran it in 0.3.0; anything older has to pass through 0.4.0. -- **Half of the adoption, and not the other half.** The audit called `adopt_existing_library` a - one-time migration and it was cut whole. It did two jobs: copy the old read state into - `entry_state`, which was dead, and subscribe the first admin to the whole catalogue while nobody - subscribed to anything, which is how a fresh install's first account gets `config.toml`'s feeds. - The browser suite caught it at once, signing in to an empty sidebar; `cargo test` had no idea. - The second job is back as `adopt_catalogue`, with a unit test of its own. -- **The dead `entries` columns.** `read`, `flagged` and `position` moved to `entry_state` with - accounts. The adoption's copy was their last reader, but `record_entry` still wrote them, and - still reset `read` when a title changed, which nothing looked at. Two bugs came from queries - reading them. `migrate()` now drops them from an existing database (SQLite has had `DROP COLUMN` - since 3.35), and a test builds an old table to prove it. On production each drop rewrote the - 66 MB `entries` table, about four seconds apiece, so the first start took thirteen. An - `ipx status` run in that window failed with `migrating schema`: every `ipx` command migrates when - it opens the database, and it collided with the daemon doing the same. A failed `ALTER TABLE` - changes nothing, and the database had been copied to `backup/` first anyway. -- **`interval_mins`**, which `schedule` replaced. An old config that still has the key loads; the - key is ignored, and the config test carries it to keep that true. -- **Three dependencies.** `infer` was only asked whether a file is a torrent, and the check after - it already looked for `d8:announce`, which is what `infer` looks for. `dirs` was three lookups of - `XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME`. `tokio-stream` wrapped the broadcast receiver for - the event stream; `futures_util::stream::unfold` does the same, lagging clients included. -- **Two token generators.** The web token came from a copy of the session-token code, with a - clock fallback on top. It uses `auth::new_session_token` now, and is 64 characters. -- **The icon inlined four times**, 23 KB of base64 each, into both pages. It is `/icon.png` now, - outside the sign-in wall with `/login`, since the sign-in page shows it. -- Also: the `contrib/` systemd units from before the container, `Db::entries` and - `Db::count_entries` that only the tests called, three `logbuf` visitors that repeated the trait's - defaults, and unused state, a helper and dead CSS in the page. - -## 2026-09-11 — A design pass on the web UI - -A review against screenshots of every view in all three themes found that Dark and Light read as a -generic dark dashboard: one pale blue did every job, most labels were bold, and nothing led. The -list it produced, in `TODO.md`, was worked through in one go. What was worth knowing: - -- **Amber means new.** Badges, unread dots and download bars take the icon's EQ amber; blue is left - for the primary action and links. Light's amber was `#b06f10`, which gives white text 4.1:1, - short of AA for 11 px bold. It is `#9a5f0a` now, 5.2:1. -- **EQ bars mark what is playing.** Three `` bars stand at 60, 100 and 40 % and animate only - while `body.playing` is set. The first version left the animation on but paused, expecting each - bar to hold a different frame. The frames it held were within a pixel of each other, and on - screen the bars read as three dots. Under reduced motion one rule drops every animation and - transition, which leaves the bars standing. -- **The focus ring was clipped.** `.tgroup` and `#topbar` both set `overflow:hidden`, so a ring - drawn outside a toolbar button was cut off. Rings inside clipping parents are inset instead. -- **The feed list could not be used from the keyboard at all.** Rows were `
`s and the triangle - a ``, so Tab went from the feed filter to Sign out. Rows now take focus, the triangle is a - `