Give SQLite several connections, so requests are answered at once (#136)

SeaORM gives a SQLite pool one connection unless told otherwise, and connect() did not say, so
every request, every scan write and every read went through it in turn. Twenty-five people
browsing in the load tests (#133) got 50 requests a second out of 12 cores, the fastest answers
in 3-30ms and the medians ten times that.

SQLite now gets eight. In WAL readers run beside the one writer. Every transaction here writes
first, so it takes the write lock or waits out the busy timeout for it; SQLite refuses at once
only a transaction holding a snapshot a write has moved past, and none here does. Db::memory
switches its file to WAL too, as Db::open does, so the tests share it as the daemon does. A test
holds a write open and reads beside it, and fails with one connection.

The same browsing: a feed's items at p95 10ms where it was 133, a search 22 where 119, the feed
list 89 where 242. The load tests' SQLite budgets are tightened to match; the Rust tests take
2.8s where they took 7.5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-05 19:55:22 +00:00
parent a553d050ca
commit c9964778cb
7 changed files with 54 additions and 22 deletions

View File

@@ -6,7 +6,7 @@ import { BASE, FEEDS, ITEMS, as, me, pick, feedId, p95 } from './lib.js';
// a feed, a search, the Directory and a feed's page in it. Each answer's budget is well above what
// it takes now, so it fails on a query that has started asking once per feed -- the Directory
// asked the database three questions a feed until 0.10.0 -- not on a slow minute. Fifty measured
// the queue for SQLite's one connection (#136) more than any query.
// the queue for what was SQLite's one connection (#136) more than any query.
export const options = {
scenarios: {
evening: {
@@ -17,14 +17,14 @@ export const options = {
thresholds: {
http_req_failed: ['rate==0'],
checks: ['rate==1'],
// About twice each one's p95 on 2026-10-05 on Tower. SQLite: 236, 118, 133, 119, 259, 312ms;
// Postgres: 54, 34, 24, 44, 71, 88ms.
'http_req_duration{name:feeds}': p95(500, 150),
'http_req_duration{name:all-entries}': p95(300, 100),
'http_req_duration{name:feed-entries}': p95(300, 100),
'http_req_duration{name:search}': p95(300, 100),
'http_req_duration{name:directory}': p95(600, 200),
'http_req_duration{name:listed}': p95(700, 200),
// About twice each one's p95 on 2026-10-05 on Tower, and no less than 50ms, where a few ms
// either way is noise. SQLite: 89, 17, 10, 22, 112, 126ms; Postgres: 54, 34, 24, 44, 71, 88ms.
'http_req_duration{name:feeds}': p95(200, 150),
'http_req_duration{name:all-entries}': p95(50, 100),
'http_req_duration{name:feed-entries}': p95(50, 100),
'http_req_duration{name:search}': p95(60, 100),
'http_req_duration{name:directory}': p95(250, 200),
'http_req_duration{name:listed}': p95(300, 200),
},
};

View File

@@ -13,7 +13,7 @@ export const as = (name, tag) => ({
/// The listener this virtual user is, one of the hundred run.js seeded, thirty feeds each.
export const me = () => `load-${(__VU - 1) % USERS}`;
export const pick = a => a[Math.floor(Math.random() * a.length)];
/// A p95 budget, in ms, for whichever database the daemon is on (run.js --postgres). Postgres's
/// are tighter: SQLite's carry the queue for its one connection (#136).
/// A p95 budget, in ms, for whichever database the daemon is on (run.js --postgres): SQLite reads
/// a file beside the daemon, Postgres answers over the network, and each is quicker at something.
export const p95 = (sqlite, postgres) => [`p(95)<${__ENV.DB === 'postgres' ? postgres : sqlite}`];
export const feedId = n => `gen-${String(n).padStart(4, '0')}`;

View File

@@ -15,12 +15,11 @@ export const options = {
thresholds: {
http_req_failed: ['rate==0'],
checks: ['rate==1'],
// About twice each one's p95 on 2026-10-05 on Tower. SQLite: 53, 786 and 383ms; Postgres: 9,
// 225 and 153ms. Marking read answers with the feed's row, counts and all, and on SQLite
// waits behind the scans' writes for its one connection (#136), hence its budget.
'http_req_duration{name:position}': p95(300, 100),
'http_req_duration{name:flags}': p95(1500, 500),
'http_req_duration{name:read-back}': p95(800, 400),
// About twice each one's p95 on 2026-10-05 on Tower. SQLite: 21, 277 and 134ms; Postgres: 9,
// 225 and 153ms. Marking read answers with the feed's row, counts and all, hence its budget.
'http_req_duration{name:position}': p95(100, 100),
'http_req_duration{name:flags}': p95(600, 500),
'http_req_duration{name:read-back}': p95(300, 400),
},
};

View File

@@ -13,8 +13,8 @@
//
// The daemon is on SQLite unless --postgres, which puts it in IPX_TEST_DATABASE_URL, the
// database the Rust tests use on Postgres too (ipodderx_test on the server production's is on, in
// /src/.envrc), in a schema of its own, ipx_load, made afresh each run (#139). SQLite's numbers
// carry its one connection (#136); Postgres's are what production would see.
// /src/.envrc), in a schema of its own, ipx_load, made afresh each run (#139). Postgres's numbers
// are what production would see.
// IPX_LOAD_FEEDS sets the catalogue's size (1500, near production's), IPX_LOAD_USERS the
// listeners (100), IPX_LOAD_LOG=1 shows the daemon's log.
const http = require('http');