rays
26802d2b23
- web/src/*.ts: the script that was inline in index.html and login.html, split along its existing sections. Still one scope, concatenated in order, not modules. - web/build.mjs strips the types, puts the script in the page and minifies it with swc; build.rs runs it into OUT_DIR and web.rs include_str!s the result. 137 KB -> 106 KB. - npx tsc -p . type-checks web/src, loosely; the handful of annotations it needed change no behaviour. - The Docker build installs node and swc (npm ci --omit=dev). - Two list requests racing no longer let the older one win, and switching tabs clears the selection it closes, which made a browser test flaky. Closes #23, #24. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
178 lines
10 KiB
Markdown
178 lines
10 KiB
Markdown
# Working on ipodderx-rs
|
|
|
|
Notes for whoever picks this up next. Read [docs/architecture.md](docs/architecture.md) for how the
|
|
thing is built; this file is about working on it without repeating mistakes that have already been
|
|
made here.
|
|
|
|
## Where things are
|
|
|
|
Production is the `iPodderX` container on Tower (192.168.1.130), the `ipodderx` service of the
|
|
Arcane project `content`: `/mnt/fast/arcane/projects/content/compose.yaml`. That file is what runs;
|
|
`docker-compose.yml` in this repo is a copy, and editing it changes nothing in production.
|
|
|
|
| | Host | In the container |
|
|
|---|---|---|
|
|
| Image | `192.168.1.130:5000/ipodderx:latest` | |
|
|
| Config | `/mnt/fast/appdata/ipodderx/config.toml` | `/config/config.toml` |
|
|
| Database | `/mnt/user/ipodderx/state.db` | `/data/state.db` |
|
|
| Downloads | `/mnt/user/ipodderx/downloads` | `/downloads` |
|
|
| Web UI | `192.168.1.130:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel | `0.0.0.0:8099` |
|
|
| Sign-in via the tunnel | Cloudflare Access app `ipodderx`, with Authentik as its identity provider; see [docs/sso.md](docs/sso.md) | trusts `Cf-Access-Authenticated-User-Email` from `192.168.16.1`, the `content_default` gateway |
|
|
|
|
Work to do lives in the Gitea issues at https://git.sdf1.net/rays/ipodderx-rs/issues, not in a
|
|
`TODO.md`. `/src/tea` is logged in: `/src/tea issues list --login git.sdf1.net --repo rays/ipodderx-rs`.
|
|
|
|
Deploying a change is: build and push the image, then pull it and recreate the container.
|
|
|
|
```sh
|
|
docker buildx build --tag 192.168.1.130:5000/ipodderx:latest . --push
|
|
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml pull ipodderx
|
|
docker compose -f /mnt/fast/arcane/projects/content/compose.yaml up -d ipodderx
|
|
docker logs --tail 20 iPodderX
|
|
```
|
|
|
|
**Name the service.** A bare `up -d` recreates every container in `content`, beets and immich
|
|
included. Run `pull` before `up`, because `up` reuses whatever `latest` the host already has.
|
|
|
|
**A build that fails with `429 Too Many Requests` on a base image** is Docker Hub rate-limiting
|
|
this host. There is no Docker Hub login here, and the build asks about `debian:bookworm-slim` and
|
|
`rust:1-slim-bookworm` every time unless they are already stored locally. Pull them from Google's
|
|
mirror and tag them; the build then uses the local copies without asking Docker Hub:
|
|
|
|
```sh
|
|
docker pull mirror.gcr.io/library/debian:bookworm-slim
|
|
docker tag mirror.gcr.io/library/debian:bookworm-slim debian:bookworm-slim
|
|
docker pull mirror.gcr.io/library/rust:1-slim-bookworm
|
|
docker tag mirror.gcr.io/library/rust:1-slim-bookworm rust:1-slim-bookworm
|
|
```
|
|
|
|
Run those again now and then, or the local copies go stale.
|
|
|
|
The healthcheck runs `ipx status` against the control socket, so `(healthy)` in `docker ps` means
|
|
the daemon answers there and can read its database, not just that the web port is up. The socket
|
|
answers `status` itself instead of queuing it behind the worker's current job, so a long scan or
|
|
download does not fail the check; it also means a worker stuck on one job would still pass. The
|
|
container restarts on its own after a reboot.
|
|
|
|
Before the container, ipx ran by hand in code-server, with its files in `/config/.config/ipx/` and
|
|
`/config/.local/share/ipx/`. Those are still there and the container does not read them. If you run
|
|
a daemon by hand for testing, stop it with **`pkill -x ipx`, never `pkill -f ipx`**. `-f` matches
|
|
the shell running the command and kills the session (exit 144). This has happened more than once.
|
|
|
|
## Before you touch the page
|
|
|
|
The page is markup and CSS in `web/index.html` and TypeScript in `web/src/`. `build.rs` runs
|
|
`web/build.mjs`, which uses swc to strip the types, put the script into the page and minify the
|
|
whole thing, and the result is `include_str!`d into the binary. So **every page change needs a
|
|
rebuild** before it is visible, and building needs node and `npm ci` run once.
|
|
|
|
The files in `web/src` are not modules. They are one script split up, concatenated in the order
|
|
`web/build.mjs` lists them, sharing one top-level scope as the single inline script did; a new
|
|
file goes into that list. Top-level names are kept as they are, because markup calls some by
|
|
name (`onclick="closeModal()"`) and the browser tests reach others through `page.evaluate`.
|
|
|
|
After any edit to it:
|
|
|
|
```sh
|
|
npx tsc -p .
|
|
node tests/page-smoke.js
|
|
```
|
|
|
|
The first type-checks `web/src` (loosely: `strict` is off, and `$` returns `any`). The second
|
|
builds the page as shipped and runs its script against a stub DOM, checking every selector it
|
|
wires at load actually exists. That check exists because a patch once anchored on a deleted
|
|
function, `String.replace` silently matched nothing, and the whole UI died with a
|
|
`ReferenceError` while every server-side test passed.
|
|
|
|
Patching that file by guessing an anchor string has failed repeatedly. Read the exact block first
|
|
(`sed -n 'START,ENDp'`), match it verbatim, and assert the replacement happened rather than hoping.
|
|
|
|
## Tests
|
|
|
|
```sh
|
|
cargo test # ~80 tests: parsing, filters, retention, schedules, SQL, per-user state
|
|
npx tsc -p . # type-checks web/src
|
|
node tests/page-smoke.js
|
|
npx playwright test # 40 browser tests against a real daemon on fixture feeds
|
|
```
|
|
|
|
Things about the browser suite that have cost time:
|
|
|
|
* It starts **its own daemon and database** under `/tmp/ipx-ui-test`, wiped once per run. Playwright
|
|
re-imports the config in every worker, so `prepare()` guards on `TEST_WORKER_INDEX` — without
|
|
that guard a worker deleted the database out from under the running daemon, which then kept
|
|
serving from the unlinked inode while everything else saw an empty file.
|
|
* Tests **share that daemon and run in order**. A test that opens an item marks it read and changes
|
|
what later tests see. Write assertions that do not depend on what ran before, or normalise the
|
|
state first.
|
|
* Fixture feeds must not share an enclosure URL, because `enclosures.url` is globally unique and
|
|
whichever feed is scanned first claims it.
|
|
* `webServer` starts **before** `globalSetup`, which is why the fixture config is written at
|
|
config-load time instead.
|
|
|
|
Non-trivial logic leaves one runnable check behind. Pure functions (`merge_policy`, `pick`,
|
|
`matches_keywords`, `parse_interval`) are the easiest place to put it.
|
|
|
|
## Things that are true and easy to get wrong
|
|
|
|
* **`enclosures.url` is globally UNIQUE.** It is the dedupe key and the reason one file serves every
|
|
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
|
|
* **Read state lives in `entry_state`, per user, and nowhere else.** `entries` had `read`, `flagged`
|
|
and `position` columns from before accounts; two bugs came from queries still reading them
|
|
(retention, and the entry pruner), and `migrate()` now drops them.
|
|
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
|
|
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
|
|
derived and never written to config.
|
|
* **One fetch serves everyone**, so scan policy is a union of subscribers' wants (`merge_policy`).
|
|
Anyone wanting an item is enough to fetch it.
|
|
* **The UI hiding a control is not enforcement.** Admin-only actions check `user.is_admin` in the
|
|
handler and return `403`.
|
|
* **A `tokio::select!` only races its branches at the point of selection.** A long download has to
|
|
watch the shutdown channel itself; the daemon ignored SIGTERM for exactly this reason.
|
|
* Only one daemon per socket. Removing the socket file defeats the guard and you get two daemons
|
|
fighting over the database, with the stale one still holding the port.
|
|
* `/api/settings` answering `200` does **not** mean the daemon is well — the web server is a
|
|
different task. `ipx status` checks the control socket and the database; to see the worker
|
|
getting through its jobs, watch for `scan complete` in the log.
|
|
* **Every `ipx` command runs `migrate()` when it opens the database**, the healthcheck's
|
|
`ipx status` included. A migration that rewrites a big table (`DROP COLUMN`) takes seconds on
|
|
production, and a command run meanwhile fails with `migrating schema`. It changes nothing; wait
|
|
for `daemon started` in the log. Copy `state.db` aside before deploying one.
|
|
|
|
## House style
|
|
|
|
Comments explain **why**, not what. If a line looks odd, the comment says what went wrong without
|
|
it. No emoji, no exclamation marks, no "obviously". Prose in the UI and docs is plain English and
|
|
addressed to the person using it.
|
|
|
|
Every change gets one line under `## [Unreleased]` in [CHANGELOG.md](CHANGELOG.md), in its
|
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) group: Added, Changed, Deprecated,
|
|
Removed, Fixed or Security. Say it the way someone using ipx would notice it. When there is more to
|
|
say, such as what was wrong before or what it cost to find out, write it up at the top of
|
|
[docs/history.md](docs/history.md), dated. That record has been more useful than the git log more
|
|
than once.
|
|
|
|
Cutting a release: rename `[Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD` and open a new empty
|
|
`[Unreleased]` above it, bump `version` in `Cargo.toml`, tag the commit `vX.Y.Z`, and update the
|
|
compare links at the bottom of the changelog.
|
|
|
|
Deliberate simplifications get a `ponytail:` comment naming the ceiling and the upgrade path, e.g.
|
|
`// ponytail: global connection mutex, move to a pool if feed count makes it contend`.
|
|
|
|
## Known gaps
|
|
|
|
* Cloudflare's `Cf-Access-Jwt-Assertion` is not verified — ipx trusts the hop plus `trusted_proxies`
|
|
(documented in [docs/sso.md](docs/sso.md)).
|
|
* A feed's `<description>` subtitle is dropped whenever `content:encoded` exists, which loses
|
|
Substack-style subtitles.
|
|
|
|
<!-- rtk-instructions v2 -->
|
|
# Command output
|
|
|
|
Command output here is condensed to save tokens, keeping every signal and
|
|
dropping costly noise. Treat it as the complete result: run commands
|
|
normally, and batch related commands into one call to avoid extra turns.
|
|
Truncated results state their recovery path in their own output. Re-run a
|
|
command as `rtk proxy <cmd>` only when its result is unusable: empty when
|
|
output was clearly expected, contradicting its exit code, or garbled.
|
|
<!-- /rtk-instructions --> |