rays/ipx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 18 Wiki Activity

The Directory comes from the directory service (#205)

Browse Source 
ipx asks ipx-directory on its people's behalf, with the token an admin set
in the new Admin, Directory tab, so the page never talks to it and never
sees the token or a feed's address. /api/directory, /api/popular and a
feed's Directory page answer from it; subscribing takes the address from
the directory and goes through Add feed's steps. With no directory set
up, the Directory says so. The local directory code is gone; its privacy
checks are the snapshot builder's now, and their tests moved with them.

With sharing on, ipx sends its snapshot after every scan, whenever the
config changes (set_cfg) and every half hour: each public feed and how
many here subscribe, never an OPML or Patreon creator itself or anything
private. A feed can be marked private in its settings, for a paid feed
whose address does not give it away; anyone subscribed may mark it, only
an admin unmark it. The admin's category for a feed that names none goes
with it, which the directory uses only while the feed names none, so the
category an admin gives a blog still reaches the Directory.

The directory lists only feeds it has read as a feed: the first snapshot
after a start went before the first scan, when an OPML was not yet known
to be one, and the directory listed the OPML. It now fetches what a
snapshot brings in at once instead of within the minute.

The browser suite runs its own ipx-directory on SQLite; the Directory's
tests match feeds by title, since the directory names them by address.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ray Slakinski
2026-10-12 04:31:32 +00:00
parent ed82166745
commit c47aa57909
19 changed files with 710 additions and 246 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
crates/directory/src/lib.rs
View File

@@ -70,6 +70,10 @@ pub struct Item {
pub struct SnapshotEntry {
pub url: String,
pub subscribers: i64,
/// The category the sending server's admin gave a feed that names none itself: used only
/// while the feed names none.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub category: Option<String>,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq)]
@@ -109,6 +113,8 @@ pub struct Directory {
pub store: Store,
tokens: RwLock<Vec<Token>>,
client: reqwest::Client,
/// Wakes the refresher, so a feed a snapshot brings in is fetched now, not in a minute.
wake: tokio::sync::Notify,
}
pub type Shared = Arc<Directory>;
@@ -120,7 +126,7 @@ impl Directory {
.redirect(reqwest::redirect::Policy::none())
.user_agent(concat!("ipx-directory/", env!("CARGO_PKG_VERSION")))
.build()?;
Ok(Arc::new(Self { store, tokens: RwLock::new(tokens), client }))
Ok(Arc::new(Self { store, tokens: RwLock::new(tokens), client, wake: tokio::sync::Notify::new() }))
}
/// Takes the tokens of a config file read again, and drops the snapshots of any that went.
@@ -182,7 +188,10 @@ pub async fn refresher(dir: Shared) {
}
Err(e) => tracing::warn!(error = %format!("{e:#}"), "could not read which feeds are due"),
}
tokio::time::sleep(Duration::from_secs(60)).await;
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(60)) => {}
_ = dir.wake.notified() => {}
}
}
}
@@ -333,6 +342,7 @@ async fn snapshot(
return Err(ApiError(StatusCode::PAYLOAD_TOO_LARGE, format!("a snapshot lists at most {MAX_SNAPSHOT} feeds")));
}
let r = dir.store.snapshot(&me.name, &entries).await?;
dir.wake.notify_one();
tracing::info!(
instance = %me.name, accepted = r.accepted, refused_private = r.refused_private,
refused_blocked = r.refused_blocked, refused_invalid = r.refused_invalid, "snapshot"
@@ -381,7 +391,7 @@ mod tests {
use axum::body::Body;
use tower::ServiceExt;
async fn app() -> Router {
async fn app() -> (Router, Shared) {
let tokens = ["ro", "rw", "rwa"]
.iter()
.map(|n| Token {
@@ -390,7 +400,8 @@ mod tests {
sha256: config::hash(&format!("secret-{n}")),
})
.collect();
router(Directory::new(Store::scratch().await, tokens).unwrap())
let dir = Directory::new(Store::scratch().await, tokens).unwrap();
(router(dir.clone()), dir)
}
async fn call(app: &Router, method: &str, path: &str, token: Option<&str>, body: Option<serde_json::Value>) -> (u16, serde_json::Value) {
@@ -411,8 +422,8 @@ mod tests {
#[tokio::test]
async fn each_access_level_does_what_it_may_and_no_more() {
let app = app().await;
let snap = serde_json::json!([{ "url": "https://x.test/a.xml", "subscribers": 2 }]);
let (app, dir) = app().await;
let snap = serde_json::json!([{ "url": "https://x.test/a.xml", "subscribers": 2, "category": "Comedy" }]);
assert_eq!(call(&app, "GET", "/v1/feeds", None, None).await.0, 401);
assert_eq!(call(&app, "GET", "/v1/feeds", Some("wrong"), None).await.0, 401);
@@ -424,9 +435,11 @@ mod tests {
let (s, r) = call(&app, "POST", "/v1/snapshot", Some("rw"), Some(snap.clone())).await;
assert_eq!((s, r["accepted"].as_u64()), (200, Some(1)));
store::tests::fetch_all(&dir.store).await;
let (_, list) = call(&app, "GET", "/v1/feeds", Some("ro"), None).await;
let id = list[0]["id"].as_str().unwrap().to_owned();
assert_eq!(list[0]["subscribers"], 2);
assert_ne!(list[0]["category"], "Comedy", "the feed's own category wins over the sender's");
assert_eq!(call(&app, "GET", &format!("/v1/feeds/{id}"), Some("ro"), None).await.0, 200);
assert_eq!(call(&app, "GET", "/v1/feeds/nope", Some("ro"), None).await.0, 404);
assert_eq!(call(&app, "GET", "/v1/popular?limit=5", Some("ro"), None).await.1[0]["id"], id.as_str());
@@ -453,6 +466,7 @@ mod tests {
let snap = serde_json::json!([{ "url": "https://x.test/a.xml", "subscribers": 1 }]);
call(&app, "POST", "/v1/snapshot", Some("a"), Some(snap.clone())).await;
call(&app, "POST", "/v1/snapshot", Some("b"), Some(snap)).await;
store::tests::fetch_all(&dir.store).await;
assert_eq!(call(&app, "GET", "/v1/feeds", Some("a"), None).await.1[0]["subscribers"], 2);
dir.set_tokens(vec![t("a")]).await.unwrap();