Cargo workspace: crates/ipx and a shared crates/feed (#202)

The first step of the split (#207). The daemon moves to crates/ipx unchanged;
feed.rs becomes crates/feed (ipx-feed), which the directory service (#204)
will share for its metadata refresher and its privacy check.

To stand alone, fetch takes the feed's address and its credentials instead
of ipx's config::Feed (config::Feed::auth gives them), collect_outlines
moves in with the OPML parsing, and the address half of looks_private is
feed::url_looks_private. ipx imports the crate as `feed` at its root, so
every crate::feed path is unchanged. The version is the workspace's, in the
root Cargo.toml.

Checked: cargo test on Postgres (118 + 29), tsc, page smoke, native bridge,
contrast and Playwright (72) pass; the image built from the new Dockerfile
is deployed and healthy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-12 03:30:06 +00:00
parent c2cea0fc5b
commit beded23530
25 changed files with 200 additions and 98 deletions

View File

@@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Add a feed no longer asks for a folder; a feed's folder is its title (#201).
- `docker-compose.yml` is a sample to start from, ipx alone on SQLite with its folders beside the file, instead of a copy of the author's server (#208).
- The source is a Cargo workspace: the daemon in `crates/ipx`, feed fetching and parsing in `crates/feed`. `cargo build --release` still makes `target/release/ipx` (#202).
## [0.12.0] - 2026-10-11

View File

@@ -89,7 +89,7 @@ running the command and kills the session (exit 144), which has happened more th
There are three pages: the app (`web/index.html`), the admin page (`web/admin.html`, sent to
admins only) and sign-in (`web/login.html`). The app and admin pages share one stylesheet,
`web/app.css`, and their script is TypeScript in `web/src/`; `web/build.mjs` lists which files
make up each page's script. `build.rs` runs
make up each page's script. `crates/ipx/build.rs` runs
`web/build.mjs`, which uses swc to strip the types and minify the script into `app.js` (and
`login.js`), and minifies the page, and the results are `include_str!`d into the binary. The page
loads its script as `/app.js?v=<hash of its contents>`, and `/app.css` the same way: the page is
@@ -205,7 +205,7 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
* `/api/settings` answering `200` does **not** mean the daemon is well — the web server is a
different task. `ipx status` checks the control socket and the database; to see the worker
getting through its jobs, watch for `scan complete` in the log.
* **The database goes through SeaORM, and the entities in `src/entity.rs` are the schema.**
* **The database goes through SeaORM, and the entities in `crates/ipx/src/entity.rs` are the schema.**
`Db::open` creates any missing table or index from them (`create_missing`), on every `ipx`
command, the healthcheck's `ipx status` included, so it must never write when nothing is
missing: SeaORM's experimental schema sync dropped and remade an index on every open, the
@@ -236,7 +236,7 @@ a feature added and then reworked in the same release is one line saying what it
Added; something added and taken out again before the release, or a fix for a bug no release ever
had, is not there at all; and "instead of" and "no longer" compare with the last release, not with
last week. Issue numbers stay, all of them, on the line that survives. Then rename `[Unreleased]` to
`## [X.Y.Z] - YYYY-MM-DD` and open a new empty `[Unreleased]` above it, set `version` in `Cargo.toml` to `X.Y.Z` (dropping `-dev`), tag the commit
`## [X.Y.Z] - YYYY-MM-DD` and open a new empty `[Unreleased]` above it, set `version` in the root `Cargo.toml` (`[workspace.package]`) to `X.Y.Z` (dropping `-dev`), tag the commit
`vX.Y.Z`, and update the compare links at the bottom of the changelog. Then, in the next commit,
set `version` to the next patch with `-dev` (after 0.9.0, `0.9.1-dev`), so a build between releases says so
in the logo's tooltip instead of claiming to be the last release. The release that follows can

18
Cargo.lock generated
View File

@@ -1830,6 +1830,7 @@ dependencies = [
"chrono",
"clap",
"futures-util",
"ipx-feed",
"jsonwebtoken",
"librqbit",
"opentelemetry",
@@ -1854,6 +1855,23 @@ dependencies = [
"url",
]
[[package]]
name = "ipx-feed"
version = "0.12.1-dev"
dependencies = [
"anyhow",
"atom_syndication",
"chrono",
"opml",
"quick-xml 0.42.0",
"reqwest",
"rss",
"serde_json",
"tokio",
"tracing",
"url",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"

View File

@@ -1,9 +1,14 @@
[package]
name = "ipx"
[workspace]
members = ["crates/ipx", "crates/feed"]
resolver = "3"
# One version for the whole workspace: what a release sets, and what the logo's tooltip shows.
[workspace.package]
version = "0.12.1-dev"
edition = "2024"
[dependencies]
[workspace.dependencies]
ipx-feed = { path = "crates/feed" }
ammonia = "4.1.4"
anyhow = "1.0.104"
argon2 = "0.6.0"

View File

@@ -11,17 +11,20 @@ WORKDIR /src
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
# Dependencies first, so editing the source does not rebuild librqbit every time.
# Dependencies first, so editing the source does not rebuild librqbit every time: each crate of
# the workspace as an empty stub, its manifest real.
COPY Cargo.toml Cargo.lock ./
RUN mkdir src && echo 'fn main(){}' > src/main.rs \
COPY crates/ipx/Cargo.toml crates/ipx/
COPY crates/feed/Cargo.toml crates/feed/
RUN mkdir -p crates/ipx/src crates/feed/src \
&& echo 'fn main(){}' > crates/ipx/src/main.rs && touch crates/feed/src/lib.rs \
&& cargo build --release --locked \
&& rm -rf src
&& rm -rf crates/ipx/src crates/feed/src
COPY build.rs ./
COPY src ./src
COPY crates ./crates
COPY web ./web
# cargo skips a rebuild if mtimes look untouched; make sure it does not.
RUN touch src/main.rs && cargo build --release --locked
RUN touch crates/ipx/src/main.rs crates/feed/src/lib.rs && cargo build --release --locked
FROM debian:trixie-slim
# ffmpeg is DeadAir's (#171): it finds the silences and copies the frames kept. Debian's adds

19
crates/feed/Cargo.toml Normal file
View File

@@ -0,0 +1,19 @@
[package]
name = "ipx-feed"
version.workspace = true
edition.workspace = true
[dependencies]
anyhow.workspace = true
atom_syndication.workspace = true
chrono.workspace = true
opml.workspace = true
quick-xml.workspace = true
reqwest.workspace = true
rss.workspace = true
serde_json.workspace = true
tracing.workspace = true
url.workspace = true
[dev-dependencies]
tokio.workspace = true

View File

@@ -1,11 +1,10 @@
//! Feed fetching and parsing. Replaces FeedData.__getFeed / __getEntries.
//! Feed fetching and parsing, shared by ipx and the directory (#202). Replaces
//! FeedData.__getFeed / __getEntries.
use anyhow::{Context, Result, anyhow};
use reqwest::StatusCode;
use reqwest::header::{ETAG, IF_MODIFIED_SINCE, IF_NONE_MATCH, LAST_MODIFIED};
use crate::config::Feed as FeedCfg;
#[derive(Debug, Default)]
pub struct ParsedFeed {
pub title: Option<String>,
@@ -67,14 +66,16 @@ pub const FEED_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30)
/// (301 or 308): a publisher that moved its feed, which the catalogue should follow rather
/// than be redirected on every read. A temporary redirect (302, 307) moves nothing.
/// `system-proxy` reads http_proxy/https_proxy.
#[tracing::instrument(skip_all, fields(url = %cfg.url))]
/// `auth` is a username and password, sent only to the feed's own host.
#[tracing::instrument(skip_all, fields(url = %feed_url))]
pub async fn fetch(
client: &reqwest::Client,
cfg: &FeedCfg,
feed_url: &str,
auth: Option<(&str, Option<String>)>,
etag: Option<&str>,
last_modified: Option<&str>,
) -> Result<(Fetched, Option<String>)> {
let start = reqwest::Url::parse(&cfg.url).context("the feed's address")?;
let start = reqwest::Url::parse(feed_url).context("the feed's address")?;
let mut url = start.clone();
let mut permanent = true;
for _ in 0..10 {
@@ -86,10 +87,10 @@ pub async fn fetch(
req = req.header(IF_MODIFIED_SINCE, lm);
}
// The feed's own host only: a redirect elsewhere must not be handed the password.
if let Some(user) = &cfg.username
if let Some((user, password)) = &auth
&& url.host_str() == start.host_str()
{
req = req.basic_auth(user, cfg.password());
req = req.basic_auth(user, password.as_deref());
}
let resp = req.send().await.context("connecting")?;
@@ -231,10 +232,21 @@ pub fn parse_opml(bytes: &[u8]) -> Result<Vec<(String, String)>> {
let doc = opml::OPML::from_str(&text)
.map_err(|e| anyhow!("that does not parse as OPML: {e}"))?;
let mut out = vec![];
crate::collect_outlines(&doc.body.outlines, &mut out);
collect_outlines(&doc.body.outlines, &mut out);
Ok(out)
}
/// OPML nests feeds inside folder outlines, so this walks the whole tree.
pub fn collect_outlines(outlines: &[opml::Outline], out: &mut Vec<(String, String)>) {
for o in outlines {
if let Some(url) = &o.xml_url {
let title = o.title.clone().unwrap_or_else(|| o.text.clone());
out.push((title, url.clone()));
}
collect_outlines(&o.outlines, out);
}
}
/// The <head><title> of an OPML document.
pub fn opml_title(bytes: &[u8]) -> Option<String> {
let text = String::from_utf8_lossy(bytes);
@@ -1121,6 +1133,34 @@ fn picked(answer: &serde_json::Value) -> Result<String> {
.ok_or_else(|| anyhow!("no category in the answer: {answer}"))
}
/// An address that carries a subscriber's own access: a paid-feed host, credentials in the URL,
/// or a key in the query. ipx leaves such a feed out of anything others see, and the directory
/// refuses one sent to it (#202).
///
/// ponytail: a heuristic. A key hidden in the path of a host not in `PAID_HOSTS` gets
/// through; a per-feed private flag is the upgrade (#205).
pub fn url_looks_private(feed_url: &str) -> bool {
let Ok(u) = url::Url::parse(feed_url) else { return true };
// Paid-feed services put the subscriber's key in the path as often as in the query, and a
// long path segment alone proves nothing (acast's public show ids look the same). So a
// feed from one of them is private whatever its URL looks like. Supercast is why this
// exists: `feeds.supercast.com/feeds/<key>` reached the directory before it did.
let host = u.host_str().unwrap_or("");
PAID_HOSTS.iter().any(|h| host == *h || host.ends_with(&format!(".{h}")))
|| !u.username().is_empty()
|| u.password().is_some()
|| u.query_pairs().any(|(k, _)| {
let k = k.to_ascii_lowercase();
["auth", "token", "key", "secret", "pass", "sig", "session", "user", "uid"]
.iter()
.any(|w| k.contains(w))
})
}
/// Services whose feeds are always one subscriber's own.
const PAID_HOSTS: &[&str] =
&["patreon.com", "supercast.com", "supportingcast.fm", "glow.fm", "memberful.com"];
#[cfg(test)]
mod tests {
/// A server answering by path: /old moves for good to /new, /tmp for now, /chain for good
@@ -1157,9 +1197,9 @@ mod tests {
let base = redirecting_server().await;
let client = reqwest::Client::builder().redirect(reqwest::redirect::Policy::none()).build().unwrap();
let get = |path: &str| {
let cfg: crate::config::Feed = serde_json::from_value(serde_json::json!({ "url": format!("{base}{path}") })).unwrap();
let url = format!("{base}{path}");
let client = client.clone();
async move { super::fetch(&client, &cfg, None, None).await }
async move { super::fetch(&client, &url, None, None, None).await }
};
let (got, moved) = get("/old").await.unwrap();
assert!(matches!(got, super::Fetched::Body { .. }));
@@ -1175,7 +1215,7 @@ mod tests {
#[test]
fn parses_rss_with_itunes_extensions() {
let bytes = include_bytes!("../tests/data/rss2.xml");
let bytes = include_bytes!("../../../tests/data/rss2.xml");
let feed = parse(bytes).unwrap();
assert_eq!(feed.title.as_deref(), Some("Test Cast"));
@@ -1381,7 +1421,7 @@ mod tests {
#[test]
fn parses_atom_enclosure_links() {
let bytes = include_bytes!("../tests/data/atom.xml");
let bytes = include_bytes!("../../../tests/data/atom.xml");
let feed = parse(bytes).unwrap();
assert_eq!(feed.title.as_deref(), Some("Atom Cast"));
@@ -1479,8 +1519,8 @@ mod tests {
assert_eq!(feeds[0], ("Alpha".into(), "https://a.example/rss".into()));
// A feed must never be mistaken for a subscription list.
assert!(!is_opml(include_bytes!("../tests/data/rss2.xml")));
assert!(!is_opml(include_bytes!("../tests/data/atom.xml")));
assert!(!is_opml(include_bytes!("../../../tests/data/rss2.xml")));
assert!(!is_opml(include_bytes!("../../../tests/data/atom.xml")));
}
#[test]

37
crates/ipx/Cargo.toml Normal file
View File

@@ -0,0 +1,37 @@
[package]
name = "ipx"
version.workspace = true
edition.workspace = true
[dependencies]
ipx-feed.workspace = true
ammonia.workspace = true
anyhow.workspace = true
argon2.workspace = true
atom_syndication.workspace = true
axum.workspace = true
chrono.workspace = true
clap.workspace = true
futures-util.workspace = true
jsonwebtoken.workspace = true
librqbit.workspace = true
opentelemetry.workspace = true
opentelemetry-otlp.workspace = true
opentelemetry_sdk.workspace = true
opml.workspace = true
percent-encoding.workspace = true
quick-xml.workspace = true
reqwest.workspace = true
rss.workspace = true
sea-orm.workspace = true
serde.workspace = true
serde_json.workspace = true
sha2.workspace = true
tokio.workspace = true
toml.workspace = true
tower.workspace = true
tower-http.workspace = true
tracing.workspace = true
tracing-opentelemetry.workspace = true
tracing-subscriber.workspace = true
url.workspace = true

View File

@@ -1,13 +1,13 @@
//! Builds web/index.html and web/login.html from their TypeScript (web/build.mjs) into OUT_DIR,
//! where src/web.rs include_str!s them. Needs node and `npm ci` run first.
//! where src/web.rs include_str!s them. Needs node and `npm ci` run first, at the repo's root.
use std::process::Command;
fn main() {
println!("cargo:rerun-if-changed=web");
println!("cargo:rerun-if-changed=package-lock.json");
println!("cargo:rerun-if-changed=../../web");
println!("cargo:rerun-if-changed=../../package-lock.json");
let out = std::env::var("OUT_DIR").unwrap();
let status = Command::new("node")
.args(["web/build.mjs", &out])
.args(["../../web/build.mjs", &out])
.status()
.expect("building the web pages needs node on PATH (and `npm ci` run once)");
assert!(status.success(), "web/build.mjs failed; run `node web/build.mjs` to see why");

View File

@@ -137,7 +137,7 @@ mod tests {
const AUD: &str = "aud-tag";
fn jwks() -> JwkSet {
serde_json::from_str(include_str!("../tests/data/access-test.jwks.json")).unwrap()
serde_json::from_str(include_str!("../../../tests/data/access-test.jwks.json")).unwrap()
}
fn token(key: &[u8], alg: Algorithm, claims: serde_json::Value) -> String {
@@ -155,8 +155,8 @@ mod tests {
})
}
const SIGNER: &[u8] = include_bytes!("../tests/data/access-test.der");
const FORGER: &[u8] = include_bytes!("../tests/data/access-forger.der");
const SIGNER: &[u8] = include_bytes!("../../../tests/data/access-test.der");
const FORGER: &[u8] = include_bytes!("../../../tests/data/access-forger.der");
async fn check(keys: &Keys, t: &str) -> Option<String> {
keys.verify_with(TEAM, AUD, t, || async { Ok(jwks()) }).await

View File

@@ -273,6 +273,11 @@ impl Feed {
}
self.password.clone()
}
/// What `feed::fetch` sends the feed's host: a username, and its password if there is one.
pub fn auth(&self) -> Option<(&str, Option<String>)> {
self.username.as_deref().map(|u| (u, self.password()))
}
}
impl Config {

View File

@@ -6,7 +6,8 @@ mod db;
mod deadair;
mod entity;
mod download;
mod feed;
// The crate root's import, so `crate::feed` reaches it from every module as before.
use ipx_feed as feed;
mod ipc;
mod logbuf;
mod metrics;
@@ -908,7 +909,7 @@ async fn add_one(
listed: false,
};
let title = match feed::fetch(&ctx.feed_client, &probe, None, None).await.map(|(got, _)| got) {
let title = match feed::fetch(&ctx.feed_client, &probe.url, None, None, None).await.map(|(got, _)| got) {
// An OPML subscription is named from its own <head><title>, not by trying to
// parse it as a feed and falling back to the hostname.
Ok(feed::Fetched::Body { bytes, .. }) if feed::is_opml(&bytes) => {
@@ -999,7 +1000,7 @@ pub async fn subscribe_opml(
user_id: i64,
) -> Result<(usize, usize)> {
let mut found = vec![];
collect_outlines(&doc.body.outlines, &mut found);
feed::collect_outlines(&doc.body.outlines, &mut found);
let mut cfg = ctx.edit_cfg().await;
let known = subscriptions(ctx).await?;
@@ -1057,17 +1058,6 @@ pub async fn subscribe_opml(
Ok((added, had))
}
/// OPML nests feeds inside folder outlines, so this walks the whole tree.
pub fn collect_outlines(outlines: &[opml::Outline], out: &mut Vec<(String, String)>) {
for o in outlines {
if let Some(url) = &o.xml_url {
let title = o.title.clone().unwrap_or_else(|| o.text.clone());
out.push((title, url.clone()));
}
collect_outlines(&o.outlines, out);
}
}
/// The configuration ipx runs with: config.toml for where things are and who may sign in, the
/// database for the feeds and the server settings (issue #18). The first time a database holds
/// neither, it takes them from config.toml, which is then cut down to the rest, the original kept
@@ -1264,7 +1254,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
return None;
}
let _fetching = metrics::busy(&metrics::STATS.feeds_fetching);
Some(feed::fetch(&client, &feed_cfg, etag.as_deref(), modified.as_deref()).await)
Some(feed::fetch(&client, &feed_cfg.url, feed_cfg.auth(), etag.as_deref(), modified.as_deref()).await)
},
tracing::Span::current(),
))
@@ -1654,7 +1644,7 @@ async fn scan_one(
let (mut fetched, moved) = match prefetched {
Some(got) => got?,
None => feed::fetch(&ctx.feed_client, feed_cfg, state.etag.as_deref(), state.last_modified.as_deref()).await?,
None => feed::fetch(&ctx.feed_client, &feed_cfg.url, feed_cfg.auth(), state.etag.as_deref(), state.last_modified.as_deref()).await?,
};
if let Some(to) = moved {
follow_move(ctx, id, &to).await?;
@@ -1669,7 +1659,7 @@ async fn scan_one(
if matches!(fetched, feed::Fetched::NotModified) && (stored.entries == 0 || stored.image.is_none()) {
tracing::info!(feed = id, "not modified, but nothing stored; refetching without the validator");
ctx.db.clear_validators(id).await?;
fetched = feed::fetch(&ctx.feed_client, feed_cfg, None, None).await?.0;
fetched = feed::fetch(&ctx.feed_client, &feed_cfg.url, feed_cfg.auth(), None, None).await?.0;
}
let (bytes, etag, last_modified) = match fetched {

View File

@@ -663,7 +663,7 @@ fn script(js: &'static str) -> impl IntoResponse {
async fn logo() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "image/svg+xml"), (header::CACHE_CONTROL, "max-age=86400")],
include_str!("../web/logo.svg"),
include_str!("../../../web/logo.svg"),
)
}
@@ -672,7 +672,7 @@ async fn logo() -> impl IntoResponse {
async fn logo_dark() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "image/svg+xml"), (header::CACHE_CONTROL, "max-age=86400")],
include_str!("../web/logo-dark.svg"),
include_str!("../../../web/logo-dark.svg"),
)
}
@@ -680,7 +680,7 @@ async fn logo_dark() -> impl IntoResponse {
async fn icon() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "image/png"), (header::CACHE_CONTROL, "max-age=86400")],
include_bytes!("../web/ipodderx-icon.png").as_slice(),
include_bytes!("../../../web/ipodderx-icon.png").as_slice(),
)
}
@@ -689,7 +689,7 @@ async fn icon() -> impl IntoResponse {
async fn favicon() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "image/png"), (header::CACHE_CONTROL, "max-age=86400")],
include_bytes!("../web/favicon.png").as_slice(),
include_bytes!("../../../web/favicon.png").as_slice(),
)
}
@@ -697,7 +697,7 @@ async fn favicon() -> impl IntoResponse {
async fn favicon_dark() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "image/png"), (header::CACHE_CONTROL, "max-age=86400")],
include_bytes!("../web/favicon-dark.png").as_slice(),
include_bytes!("../../../web/favicon-dark.png").as_slice(),
)
}
@@ -706,7 +706,7 @@ async fn favicon_dark() -> impl IntoResponse {
async fn touch_icon() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "image/png"), (header::CACHE_CONTROL, "max-age=86400")],
include_bytes!("../web/apple-touch-icon.png").as_slice(),
include_bytes!("../../../web/apple-touch-icon.png").as_slice(),
)
}
@@ -716,7 +716,7 @@ async fn touch_icon() -> impl IntoResponse {
async fn inter() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "font/woff2"), (header::CACHE_CONTROL, "max-age=604800")],
include_bytes!("../web/InterVariable.woff2").as_slice(),
include_bytes!("../../../web/InterVariable.woff2").as_slice(),
)
}
@@ -936,34 +936,14 @@ async fn feed_rows(state: &WebState, user: &crate::db::User, only: Option<&str>)
// ---- popular on this server ----
/// A feed that carries a credential is someone's paid or private subscription. Listing it
/// would let anyone signed in subscribe to it and read what they pay for.
///
/// ponytail: a heuristic. A key hidden in the path of a host not in `PAID_HOSTS` gets
/// through; a per-feed `unlisted` flag is the upgrade if that happens again.
/// would let anyone signed in subscribe to it and read what they pay for. The address half is
/// `feed::url_looks_private`, which the directory shares.
fn looks_private(feed: &crate::config::Feed) -> bool {
if feed.username.is_some() || feed.password.is_some() || feed.password_env.is_some() {
return true;
feed.username.is_some()
|| feed.password.is_some()
|| feed.password_env.is_some()
|| crate::feed::url_looks_private(&feed.url)
}
let Ok(u) = url::Url::parse(&feed.url) else { return true };
// Paid-feed services put the subscriber's key in the path as often as in the query, and a
// long path segment alone proves nothing (acast's public show ids look the same). So a
// feed from one of them is private whatever its URL looks like. Supercast is why this
// exists: `feeds.supercast.com/feeds/<key>` reached the directory before it did.
let host = u.host_str().unwrap_or("");
PAID_HOSTS.iter().any(|h| host == *h || host.ends_with(&format!(".{h}")))
|| !u.username().is_empty()
|| u.password().is_some()
|| u.query_pairs().any(|(k, _)| {
let k = k.to_ascii_lowercase();
["auth", "token", "key", "secret", "pass", "sig", "session", "user", "uid"]
.iter()
.any(|w| k.contains(w))
})
}
/// Services whose feeds are always one subscriber's own.
const PAID_HOSTS: &[&str] =
&["patreon.com", "supercast.com", "supportingcast.fm", "glow.fm", "memberful.com"];
/// Only an id, a title, artwork and a count: never a URL, which is where a key would be.
#[derive(Serialize)]

View File

@@ -4,28 +4,32 @@ One binary, `ipx`. `ipx daemon` runs three things in one process: a scheduler, a
control server, and the web UI. Everything else is a CLI that either does the work itself or hands
it to a running daemon.
The repo is a Cargo workspace (#202): `crates/ipx` is the daemon and CLI, and `crates/feed`
(`ipx-feed`) fetches and parses feeds, kept apart because the directory service (#204) shares it.
The version is the workspace's, in the root `Cargo.toml`.
## Modules
| File | Responsibility | What it replaced in the Python |
|---|---|---|
| `src/main.rs` | CLI, dispatch, scan loop, download policy | `iPXAgent.py` |
| `src/config.rs` | TOML load/save, `General`/`Feed`/`Web`, intervals, slugs | `iPXSettings.py`, `feeds.plist` |
| `src/db.rs` | Every query, through SeaORM; creates missing tables | `.ipxd` plists, `history.dat`, `qmcache.dat` |
| `src/entity.rs` | The tables, as SeaORM entities: the schema | — |
| `src/feed.rs` | Conditional GET, RSS/Atom/OPML parsing | `FeedData.__getFeed/__getEntries` |
| `src/download.rs` | Streaming download, naming, type sniffing, placement | `iPXDownloader.getFile` |
| `src/torrent.rs` | librqbit session, seeding limits, stall abort | vendored BitTorrent 4.2.1 |
| `src/retention.rs` | Quota and age sweeps | `iPXQuotaManager.py` |
| `src/ipc.rs` | Event and command types, the socket server | `printMSG` on stdout |
| `src/auth.rs` | Argon2id hashing, session tokens, header names | — |
| `src/web.rs` | axum: HTTP API, auth, SSE, media streaming | — |
| `src/logbuf.rs` | Ring buffer behind the UI's Log view | — |
| `crates/ipx/src/main.rs` | CLI, dispatch, scan loop, download policy | `iPXAgent.py` |
| `crates/ipx/src/config.rs` | TOML load/save, `General`/`Feed`/`Web`, intervals, slugs | `iPXSettings.py`, `feeds.plist` |
| `crates/ipx/src/db.rs` | Every query, through SeaORM; creates missing tables | `.ipxd` plists, `history.dat`, `qmcache.dat` |
| `crates/ipx/src/entity.rs` | The tables, as SeaORM entities: the schema | — |
| `crates/feed/src/lib.rs` | Conditional GET, RSS/Atom/OPML parsing, `url_looks_private` | `FeedData.__getFeed/__getEntries` |
| `crates/ipx/src/download.rs` | Streaming download, naming, type sniffing, placement | `iPXDownloader.getFile` |
| `crates/ipx/src/torrent.rs` | librqbit session, seeding limits, stall abort | vendored BitTorrent 4.2.1 |
| `crates/ipx/src/retention.rs` | Quota and age sweeps | `iPXQuotaManager.py` |
| `crates/ipx/src/ipc.rs` | Event and command types, the socket server | `printMSG` on stdout |
| `crates/ipx/src/auth.rs` | Argon2id hashing, session tokens, header names | — |
| `crates/ipx/src/web.rs` | axum: HTTP API, auth, SSE, media streaming | — |
| `crates/ipx/src/logbuf.rs` | Ring buffer behind the UI's Log view | — |
| `web/index.html` | The app's markup | — |
| `web/admin.html` | The admin page's markup: server settings, accounts, the log. Sent to admins only | — |
| `web/app.css` | The stylesheet both pages share | — |
| `web/src/*.ts` | The page's script, one scope split across files, type-checked by `npx tsc` | — |
| `web/build.mjs` | swc: strips the types into `app.js`/`login.js`, named in the page by a hash of their contents, and minifies | — |
| `build.rs` | Runs `web/build.mjs` into `OUT_DIR`, where `web.rs` `include_str!`s the result | — |
| `crates/ipx/build.rs` | Runs `web/build.mjs` into `OUT_DIR`, where `web.rs` `include_str!`s the result | — |
The page is compiled in, so **editing `web/index.html` or `web/src` needs a rebuild**, and a
build needs node and `npm ci` run once.
@@ -70,7 +74,7 @@ entry_state user_id, feed_id, guid, read, flagged, position
Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position` columns from
before accounts; two bugs came from queries still reading them, and they were dropped in 0.5.
Schema changes: the tables are the entities in `src/entity.rs`, and `Db::open` creates whatever
Schema changes: the tables are the entities in `crates/ipx/src/entity.rs`, and `Db::open` creates whatever
table or index a database is missing from them (`db::create_missing`), with `IF NOT EXISTS`. It
never alters a table that exists, so a new column on one needs its own `ALTER` in
`create_missing`, looked for first and run only when it is missing, as `feeds.description`'s is,