rays
ab2583fc64
Conditional GET plus an RSS-first, Atom-fallback parser normalising both into one entry model, with iTunes explicit and ttl handling carried over from the Python. Enclosures are recorded as pending; nothing downloads yet. GUID falls back guid -> link -> enclosure url -> title rather than hashing the description as the original did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
119 lines
6.4 KiB
Markdown
119 lines
6.4 KiB
Markdown
# Progress
|
|
|
|
Running record of what has actually landed. Newest entry first.
|
|
The full design and step list live in the plan file at
|
|
`/config/.claude/plans/i-want-to-create-playful-quiche.md`.
|
|
|
|
## Build order
|
|
|
|
- [x] **1. Repo skeleton** — git init (`main`), `cargo init --name ipx`, deps pinned, LICENSE,
|
|
README, this file.
|
|
- [x] **2. `config.rs` + `db.rs`** — TOML config structs + SQLite schema.
|
|
- [x] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries.
|
|
- [ ] **4. `download.rs`** — filename derivation + sanitizer, streaming download, `infer` sniff,
|
|
HTML rejection, move into place, dedupe, keyword/explicit filters, `max_new_per_check`.
|
|
*Done when:* smoke 1, 2, 4 pass.
|
|
- [ ] **5. `retention.rs`** — oldest-first quota + age reaper, `ipx reap [--dry-run]`.
|
|
*Done when:* smoke 6 passes.
|
|
- [ ] **6. `ipc.rs` + daemon** — broadcast event bus, UDS JSON-lines server, TTL scheduler,
|
|
CLI-proxies-to-daemon. *Done when:* smoke 3 passes.
|
|
- [ ] **7. `torrent.rs`** — librqbit, seed to ratio/time, stall abort. *Done when:* smoke 5 passes.
|
|
- [ ] **8. OPML + polish** — import/export, add/rm/status, tracing setup, systemd unit, README.
|
|
|
|
## Smoke tests
|
|
|
|
1. `ipx add <feed>` + `ipx fetch` → file in `download_dir/<Show>/`, row in `enclosures`.
|
|
2. `ipx fetch` again → no re-download, feed skipped for TTL.
|
|
3. `ipx daemon &` + `nc -U $XDG_RUNTIME_DIR/ipx.sock`, send `{"cmd":"fetch"}` → JSON events;
|
|
a concurrent `ipx fetch` proxies to the daemon instead of downloading in parallel.
|
|
4. Delete a downloaded file by hand, `ipx fetch` → NOT re-downloaded.
|
|
5. Torrent enclosure → downloads, moves, stops seeding at the configured ratio/time.
|
|
6. `ipx reap --dry-run` under quota pressure → oldest-first hit list; real run flips rows to
|
|
`reaped`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 3: feed.rs
|
|
|
|
`src/feed.rs`: conditional GET (`If-None-Match` + `If-Modified-Since`, optional basic auth) and a
|
|
RSS-first / Atom-fallback parser normalising both into `ParsedFeed`/`Entry`/`Enclosure`. Feed-level
|
|
`itunes:explicit` overrides the entry level, as the original did. `<ttl>` is captured. RSS
|
|
`content:encoded` wins over `description`. Atom enclosures come only from `rel="enclosure"` links.
|
|
|
|
GUID: the original hashed the title or description when no guid existed; here the chain is
|
|
guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5
|
|
dependency. An entry with none of them has nothing to download and is dropped.
|
|
|
|
`db.rs` gained `http_state`, `record_feed`, `touch_feed`, `set_feed_error`, `record_entry`,
|
|
`record_enclosure`. A changed title/description flips `read` back to 0 — what the original's
|
|
textDiff was ultimately for, minus the `<ins>`/`<del>` markup, which belongs in the UI.
|
|
|
|
`main.rs` gained `ipx fetch [FEED] [--force]`. A failing feed records its error and the scan
|
|
continues.
|
|
|
|
Verified: `cargo test` 10/10. Gate met against a local `python3 -m http.server` serving the
|
|
fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run
|
|
showed both skip paths, `atomcast: not modified` (304) and `testcast: not due for 45m` (the feed's
|
|
own ttl=45 beating `interval_mins = 0`).
|
|
|
|
Deferred: nothing downloads yet — enclosure rows land in state `pending`. That is step 4.
|
|
Next: step 4 — `download.rs`.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 2: config.rs + db.rs
|
|
|
|
`src/config.rs`: serde structs for `[general]`, `[torrent]` and `[feeds.<id>]` with defaults, `~`
|
|
expansion, `IPX_CONFIG` / `IPX_DATA_DIR` overrides, `save()` at mode 0600, `Feed::password()`
|
|
(`password_env` beats a literal `password`), `Torrent::ports()` parsing `"6881-6889"`. A missing
|
|
config file loads as an empty one so a fresh install works. Retention defaults are 0/0
|
|
(unlimited, keep forever) — nothing gets deleted until Ray asks for it.
|
|
|
|
`src/db.rs`: schema exactly as planned, WAL + `busy_timeout`, `Db::open()` idempotent, connection
|
|
behind a `Mutex`, `feed_summary()` for `list`, `now()` helper. `enclosures.url` is UNIQUE — the
|
|
dedupe key that replaces `history.dat`.
|
|
|
|
`src/main.rs`: clap skeleton with `ipx list`. Only the subcommands that work exist; the rest arrive
|
|
with their steps.
|
|
|
|
Verified: `cargo test` 5/5 green (config defaults, port-range fallback incl. reversed range,
|
|
password_env precedence, schema idempotency, enclosure-url uniqueness). Gate met — `ipx --config
|
|
<scratch> list` printed both feeds and created `state.db` once across two runs.
|
|
|
|
Deferred: nothing. Two dead-code warnings (`Config::save`, `Feed::password`) are expected; steps 3
|
|
and 8 consume them.
|
|
Next: step 3 — `feed.rs`.
|
|
|
|
Also added `chrono` 0.4 (std, clock) for RFC-2822 pubDate parsing in step 3.
|
|
|
|
---
|
|
|
|
## 2026-09-09 — Step 1: repo skeleton
|
|
|
|
Repo created at `/src/ipodderx-rs`, default branch `main`, `cargo init --name ipx` (edition 2024,
|
|
rustc 1.95.0). LICENSE (MIT, carrying the 2010 copyright), README with the lineage note, and this
|
|
file. Hello-world `main.rs` builds.
|
|
|
|
Dependency versions pinned today — the later steps are written against these APIs:
|
|
|
|
| crate | version | notes |
|
|
|---|---|---|
|
|
| tokio | 1.53.1 | rt-multi-thread, macros, fs, io-util, net, sync, time, signal |
|
|
| reqwest | 0.13.5 | `default-features = false`; features **`rustls`** (not `rustls-tls` — renamed in 0.13), http2, gzip, stream, json, charset, **`system-proxy`** (env-var proxy pickup is opt-in in 0.13) |
|
|
| rss | 2.1.1 | default features; the `with-syndication` feature name in my notes does not exist — Atom is handled by the separate crate |
|
|
| atom_syndication | 0.12.10 | |
|
|
| rusqlite | 0.40.2 | `bundled` |
|
|
| librqbit | 9.0.1 | `default-features = false`; features `rust-tls`, `http-api-client`. The default `default-tls` feature pulls reqwest/native-tls **and** an OpenSSL sha1 backend (`crypto-hash`), which fails to build without pkg-config/OpenSSL headers. API not yet exercised — step 7 |
|
|
| serde 1.0.229 / serde_json 1.0.151 / toml 1.1.5 | | |
|
|
| clap | 4.6.6 | derive |
|
|
| infer 0.22.0 / dirs 7.0.0 / anyhow 1.0.104 / opml 1.1.6 | | |
|
|
| tracing 0.1.44 / tracing-subscriber 0.3.23 | | env-filter |
|
|
|
|
Deferred: nothing.
|
|
|
|
Gotcha worth keeping: three feature names in the plan were wrong against current crate versions —
|
|
`reqwest/rustls-tls` is now `rustls`, env-var proxy support moved behind `system-proxy`, and
|
|
`rss/with-syndication` does not exist. `librqbit`'s default features drag in OpenSSL; `rust-tls`
|
|
is the fix. Whole tree is rustls-only now, no C TLS dependency.
|
|
Next: step 2 — `config.rs` + `db.rs`. (done)
|