rays/ipodderx-rs
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

SeaORM: feeds and scanning; rusqlite gone

Browse Source 
The last nineteen functions move to SeaORM: recording feeds, items and
enclosures, managed OPML feeds, folding WordPress's repeated files, and handing a
Patreon creator's files to its shows. Two SQLite-only forms go: GLOB becomes a
LIKE with the underscore escaped (broader, harmlessly: the fold still keys on
`_=` and digits), and UPDATE OR IGNORE becomes an UPDATE ... WHERE NOT EXISTS.
The two transactions are SeaORM transactions.

With nothing left on it, rusqlite goes, with the SQL schema and migrate(). The
entities are the schema: create_missing makes whatever tables and indexes a
database lacks, from them, with CREATE ... IF NOT EXISTS. Production's schema
already has every column migrate() added and none it dropped.

Not SeaORM's schema sync, used until now: despite its docs it drops a unique
index the entities do not describe, so it dropped users_name_lower on every open.
Every `ipx` command then took a write lock, and against a daemon busy writing,
`ipx status` -- the healthcheck -- failed 7 times in 15 where the old code
failed none. Now 15 in 15, as before. On Postgres it would not have started.

WAL is set only when a file is not already in it: setting it takes a lock that
cannot wait out a busy daemon.

Checked on copies of production: a forced scan of all 162 feeds against the real
feeds with no database errors; the feed list, filters, sorts, search and the
reaper's candidates against the old code on the same data, earlier in the
branch. The column comments from the SQL schema move to the entities.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Ray Slakinski
2026-09-18 19:10:59 +00:00
parent a68bfb179b
commit 611716d8b7
8 changed files with 485 additions and 733 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
src/web.rs
View File

@@ -672,7 +672,7 @@ async fn feeds(
let cfg = state.ctx.cfg();
// Config entries plus the feeds derived from OPML subscriptions -- the catalogue.
// What comes back is only the part of it this person subscribes to.
let subs = crate::subscriptions(&state.ctx)?;
let subs = crate::subscriptions(&state.ctx).await?;
let mine: std::collections::HashMap<String, crate::db::Sub> = state
.ctx
.db
@@ -689,8 +689,8 @@ async fn feeds(
// the same fallback the scanner uses (`Db::subscribers`).
let up = feed.group.as_deref().and_then(|g| mine.get(g));
let Some(mine) = mine.get(id) else { continue };
let s = state.ctx.db.feed_summary(id)?;
let st = state.ctx.db.http_state(id)?;
let s = state.ctx.db.feed_summary(id).await?;
let st = state.ctx.db.http_state(id).await?;
out.push(FeedRow {
id: id.clone(),
url: feed.url.clone(),
@@ -806,7 +806,7 @@ async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
db.subscriptions_for(user_id).await?.into_iter().map(|s| s.feed_id).collect();
let counts = db.subscriber_counts().await?;
let media = db.media_feeds().await?;
let catalogue = crate::subscriptions(&state.ctx)?;
let catalogue = crate::subscriptions(&state.ctx).await?;
let by_id: std::collections::HashMap<&str, &crate::config::Feed> =
catalogue.iter().map(|s| (s.id.as_str(), &s.cfg)).collect();
let is_folder: std::collections::HashSet<&str> =
@@ -823,7 +823,7 @@ async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
{
continue;
}
let sum = db.feed_summary(&s.id)?;
let sum = db.feed_summary(&s.id).await?;
let subscribed = mine.contains(&s.id);
out.push(PopularRow {
id: s.id.clone(),
@@ -1219,7 +1219,7 @@ async fn add_feed(
let url = crate::feed::expand_input(&body.url);
// Someone else may already have it. Then adding costs nothing: no second fetch, no
// second copy on disk, just another name against the same feed.
if let Some(existing) = crate::subscriptions(&state.ctx)?
if let Some(existing) = crate::subscriptions(&state.ctx).await?
.into_iter()
.find(|s| crate::feed::same_feed(&s.cfg.url, &url))
{
@@ -1341,13 +1341,13 @@ async fn patch_feed(
// Derived feeds have no config entry. Editing one is the moment it earns a real
// entry: promote it, so the config holds your decisions and nothing else.
if !cfg.feeds.contains_key(&id) {
let subs = crate::subscriptions(&state.ctx)?;
let subs = crate::subscriptions(&state.ctx).await?;
let found = subs
.iter()
.find(|s| s.id == id)
.ok_or_else(|| ApiError::bad_request(format!("no feed with id {id:?}")))?;
cfg.feeds.insert(id.clone(), found.cfg.clone());
state.ctx.db.unmanage(&id)?;
state.ctx.db.unmanage(&id).await?;
}
let checked = match &body.url {
@@ -1388,7 +1388,7 @@ async fn patch_feed(
if url_changed {
// Refreshing a rotated auth token is the common case; entries and download history
// are keyed by feed id, so they survive the change.
state.ctx.db.clear_validators(&id)?;
state.ctx.db.clear_validators(&id).await?;
}
Ok(StatusCode::NO_CONTENT)
}
@@ -1401,7 +1401,7 @@ async fn remove_feed(
// Unsubscribing is personal: it takes the feed off your list and leaves everyone
// else's alone.
state.ctx.db.unsubscribe(user.id, &id).await?;
for child in crate::subscriptions(&state.ctx)?
for child in crate::subscriptions(&state.ctx).await?
.iter()
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
{
@@ -1417,7 +1417,7 @@ async fn remove_feed(
if cfg.feeds.remove(&id).is_none() {
// A derived feed: forget it here, though the OPML will list it again on the next
// read unless you unsubscribe from the OPML itself.
state.ctx.db.drop_managed(&id)?;
state.ctx.db.drop_managed(&id).await?;
return Ok(StatusCode::NO_CONTENT);
}
cfg.save(&state.config_path)?;
@@ -1610,7 +1610,7 @@ async fn read_all(
// A subscription's own row has no entries, so marking it read means everything under it.
let mut ids = vec![id.clone()];
ids.extend(
crate::subscriptions(&state.ctx)?
crate::subscriptions(&state.ctx).await?
.into_iter()
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
.map(|s| s.id),
@@ -1676,7 +1676,7 @@ async fn export_opml(
}),
..Default::default()
};
for s in crate::subscriptions(&state.ctx)? {
for s in crate::subscriptions(&state.ctx).await? {
// A feed from an OPML subscription comes back with the OPML itself.
if s.managed || !mine.contains(&s.id) {
continue;
@@ -1684,7 +1684,7 @@ async fn export_opml(
let title = state
.ctx
.db
.feed_summary(&s.id)
.feed_summary(&s.id).await
.ok()
.and_then(|sum| sum.title)
.unwrap_or_else(|| s.id.clone());