Step 3: feed fetch and parse

Conditional GET plus an RSS-first, Atom-fallback parser normalising both
into one entry model, with iTunes explicit and ttl handling carried over
from the Python. Enclosures are recorded as pending; nothing downloads yet.

GUID falls back guid -> link -> enclosure url -> title rather than hashing
the description as the original did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPyeapneuXrCdojsaiXGbe
This commit is contained in:
2026-09-09 20:34:09 +00:00
parent a13f19136c
commit ab2583fc64
6 changed files with 663 additions and 15 deletions

View File

@@ -9,8 +9,7 @@ The full design and step list live in the plan file at
- [x] **1. Repo skeleton** — git init (`main`), `cargo init --name ipx`, deps pinned, LICENSE,
README, this file.
- [x] **2. `config.rs` + `db.rs`** — TOML config structs + SQLite schema.
- [ ] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries.
*Done when:* `ipx fetch` populates `entries`, second run 304s.
- [x] **3. `feed.rs`** — conditional GET, RSS-then-Atom parse, persist entries.
- [ ] **4. `download.rs`** — filename derivation + sanitizer, streaming download, `infer` sniff,
HTML rejection, move into place, dedupe, keyword/explicit filters, `max_new_per_check`.
*Done when:* smoke 1, 2, 4 pass.
@@ -34,6 +33,34 @@ The full design and step list live in the plan file at
---
## 2026-09-09 — Step 3: feed.rs
`src/feed.rs`: conditional GET (`If-None-Match` + `If-Modified-Since`, optional basic auth) and a
RSS-first / Atom-fallback parser normalising both into `ParsedFeed`/`Entry`/`Enclosure`. Feed-level
`itunes:explicit` overrides the entry level, as the original did. `<ttl>` is captured. RSS
`content:encoded` wins over `description`. Atom enclosures come only from `rel="enclosure"` links.
GUID: the original hashed the title or description when no guid existed; here the chain is
guid → permalink → enclosure URL → title, all stable identifiers, so no hashing and no MD5
dependency. An entry with none of them has nothing to download and is dropped.
`db.rs` gained `http_state`, `record_feed`, `touch_feed`, `set_feed_error`, `record_entry`,
`record_enclosure`. A changed title/description flips `read` back to 0 — what the original's
textDiff was ultimately for, minus the `<ins>`/`<del>` markup, which belongs in the UI.
`main.rs` gained `ipx fetch [FEED] [--force]`. A failing feed records its error and the scan
continues.
Verified: `cargo test` 10/10. Gate met against a local `python3 -m http.server` serving the
fixtures — first run inserted 4 entries + 4 enclosures across an RSS and an Atom feed; second run
showed both skip paths, `atomcast: not modified` (304) and `testcast: not due for 45m` (the feed's
own ttl=45 beating `interval_mins = 0`).
Deferred: nothing downloads yet — enclosure rows land in state `pending`. That is step 4.
Next: step 4 — `download.rs`.
---
## 2026-09-09 — Step 2: config.rs + db.rs
`src/config.rs`: serde structs for `[general]`, `[torrent]` and `[feeds.<id>]` with defaults, `~`