On SQLite every request and every scan write share one database connection #136

Closed
opened 2026-10-05 10:19:37 -07:00 by rays · 1 comment
Owner

Found by the load tests (#133) on 2026-10-05. With fifty people browsing at once against a daemon on SQLite, the fastest answers took 3-30ms and the medians 300-850ms, at about 52 requests a second on 12 cores: requests queue for one resource rather than run slowly.

SeaORM gives a SQLite pool one connection unless told otherwise (sea-orm 2.0.3, driver/sqlx_sqlite.rs: max_connections(1)), and connect() does not say, so every request, every scan write and every read of the web server goes through it in turn. Postgres's pool is sqlx's default of 10, so production is not affected.

Raising it is not one line: in WAL mode readers run alongside a writer, but a transaction that reads and then writes (import_config, store_config) gets SQLITE_BUSY at once, busy timeout or not, when another connection holds the write lock. Those would need BEGIN IMMEDIATE first.

Found by the load tests (#133) on 2026-10-05. With fifty people browsing at once against a daemon on SQLite, the fastest answers took 3-30ms and the medians 300-850ms, at about 52 requests a second on 12 cores: requests queue for one resource rather than run slowly. SeaORM gives a SQLite pool one connection unless told otherwise (sea-orm 2.0.3, driver/sqlx_sqlite.rs: max_connections(1)), and connect() does not say, so every request, every scan write and every read of the web server goes through it in turn. Postgres's pool is sqlx's default of 10, so production is not affected. Raising it is not one line: in WAL mode readers run alongside a writer, but a transaction that reads and then writes (import_config, store_config) gets SQLITE_BUSY at once, busy timeout or not, when another connection holds the write lock. Those would need BEGIN IMMEDIATE first.
rays added the bug label 2026-10-05 10:19:37 -07:00
Author
Owner

Fixed in c996477: SQLite gets eight connections; every transaction writes first, so none is refused for a stale snapshot, and Db::memory is WAL as Db::open is. Load tests on SQLite: a feed's items p95 133ms to 10ms, search 119 to 22, the feed list 242 to 89. Released in 0.10.1, deployed 2026-10-05 (production is on Postgres).

Fixed in c996477: SQLite gets eight connections; every transaction writes first, so none is refused for a stale snapshot, and Db::memory is WAL as Db::open is. Load tests on SQLite: a feed's items p95 133ms to 10ms, search 119 to 22, the feed list 242 to 89. Released in 0.10.1, deployed 2026-10-05 (production is on Postgres).
rays closed this issue 2026-10-05 12:58:46 -07:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rays/ipx#136