Give SQLite several connections, so requests are answered at once (#136)

SeaORM gives a SQLite pool one connection unless told otherwise, and connect() did not say, so
every request, every scan write and every read went through it in turn. Twenty-five people
browsing in the load tests (#133) got 50 requests a second out of 12 cores, the fastest answers
in 3-30ms and the medians ten times that.

SQLite now gets eight. In WAL readers run beside the one writer. Every transaction here writes
first, so it takes the write lock or waits out the busy timeout for it; SQLite refuses at once
only a transaction holding a snapshot a write has moved past, and none here does. Db::memory
switches its file to WAL too, as Db::open does, so the tests share it as the daemon does. A test
holds a write open and reads beside it, and fails with one connection.

The same browsing: a feed's items at p95 10ms where it was 133, a search 22 where 119, the feed
list 89 where 242. The load tests' SQLite budgets are tightened to match; the Rust tests take
2.8s where they took 7.5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-05 19:55:22 +00:00
parent a553d050ca
commit c9964778cb
7 changed files with 54 additions and 22 deletions

View File

@@ -131,6 +131,15 @@ async fn connect(location: &str) -> Result<sea_orm::DatabaseConnection> {
// commits, about the last 0.6s, and never corrupts anything. Added to whatever options the
// URL sets, so it holds for the tests' search_path too.
opts.map_sqlx_postgres_opts(|o| o.options([("synchronous_commit", "off")]));
// Several connections, where SeaORM gives SQLite one unless told: every request, every scan
// write and every read went through it in turn, and twenty-five people browsing waited in
// line for it at 50 requests a second (#136). In WAL readers run beside the one writer. Each
// transaction here writes first, so it takes the write lock or waits out the busy timeout for
// it; SQLite refuses at once only a transaction that read a snapshot a write has since moved
// past. Postgres keeps sqlx's ten.
if !is_postgres(location) {
opts.max_connections(8);
}
sea_orm::Database::connect(opts)
.await
.with_context(|| format!("opening {}", redact(location)))
@@ -319,6 +328,8 @@ impl Db {
}
let path = std::env::temp_dir().join(format!("ipx-test-{pid}-{n}.db"));
let orm = connect(&path.display().to_string()).await?;
// As Db::open has it, so the tests' connections share the file as the daemon's do.
orm.execute_unprepared("PRAGMA journal_mode = WAL").await?;
create_missing(&orm).await?;
Ok(Self { orm, tmp: Some(path) })
}
@@ -2111,6 +2122,27 @@ pub fn now() -> i64 {
mod tests {
use super::*;
#[tokio::test]
async fn a_read_does_not_wait_for_a_write_in_progress() {
use sea_orm::TransactionTrait;
let db = Db::memory().await.unwrap();
// A write held open, as a scan's or an import's is while it runs.
let tx = db.orm.begin().await.unwrap();
tx.execute_unprepared("INSERT INTO feeds (id, url) VALUES ('held', 'http://x/held')").await.unwrap();
// With one connection, which SQLite had (#136), this waited for the transaction to end.
let read = tokio::time::timeout(std::time::Duration::from_secs(2), db.feed_summary("other")).await;
assert!(read.is_ok(), "a read waited behind a write");
tx.commit().await.unwrap();
// And writers at once all get their turn, not a refusal.
let writes = (0..16).map(|i| {
let db = &db;
async move { db.record_feed(&format!("f{i}"), "http://x/f", Some("F"), None, None, None, None, None, None).await }
});
for r in futures_util::future::join_all(writes).await {
r.unwrap();
}
}
#[tokio::test]
async fn a_commit_does_not_wait_for_the_disk() {
let db = Db::memory().await.unwrap();