Database connections kept open; a click's wait is measured (#241, #245, #246)

#241: on production, All Subscriptions' query runs in 31 ms, unread oldest-first in 30, the
feed list's count in 5, generic plans included (EXPLAIN ANALYZE, BUFFERS, read only), against
0.3-1.6 s in ipx. A new Postgres connection takes 80-330 ms to open, and a page load asks for
more than are idle: four of six slow loads came as the pool grew (3 to 6, 2 to 6, 3 to 5, 6 to
8), and 20 requests at once took a median 414 ms while the pool grew from 8 to 20. The daemon
and the directory's server now keep their whole pool open (min = max); a command such as `ipx
status`, and the tests, which share the server's 100 connections, open as they need.

#246: the page reports list_shown, the ms from asking for a list to drawing it, with the view
and filter, under the click's trace; background refreshes are not timed. Timings skip report()'s
once-a-minute limit, which is for errors in a loop. A dashboard panel and table, and the prod
check, read it.

#245: the load test's browse scenario subscribes by OPML and leaves the Directory, which is
ipx-directory's since #204, to it.

CLAUDE.md records the owner's leave to EXPLAIN (ANALYZE, BUFFERS) any query on production,
read only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-15 14:52:14 +00:00
parent 8f19d88fc9
commit 62a3241139
16 changed files with 89 additions and 30 deletions

View File

@@ -154,7 +154,7 @@ fn is_postgres(location: &str) -> bool {
/// sqlx's defaults for SQLite are what ipx wants, foreign keys on and a five-second wait for a
/// lock, which is what rusqlite was set to, but for one.
async fn connect(location: &str) -> Result<sea_orm::DatabaseConnection> {
async fn connect(location: &str, keep_open: bool) -> Result<sea_orm::DatabaseConnection> {
let mut opts = sea_orm::ConnectOptions::new(url_for(location));
opts.sqlx_logging(false);
// NORMAL, SQLite's own advice for WAL: synced at checkpoints, not after every commit. FULL,
@@ -177,6 +177,13 @@ async fn connect(location: &str) -> Result<sea_orm::DatabaseConnection> {
// past. Postgres gets twenty, where sqlx's ten filled in bursts: a page load asks for dozens
// of covers at once, each with a lookup (#200). The server allows 100, shared with other apps.
opts.max_connections(if is_postgres(location) { 20 } else { 8 });
// The daemon keeps every connection open (#241): a Postgres connection takes 80-330 ms to
// open, its password exchange most of it, and a page load asked for more than were idle, so
// the feed list and item list waited on new ones, 300-900 ms where their queries took 5-30.
// Not a command such as `ipx status`, nor a test, which would hold twenty for one question.
if keep_open && is_postgres(location) {
opts.min_connections(20);
}
sea_orm::Database::connect(opts)
.await
.with_context(|| format!("opening {}", redact(location)))
@@ -307,13 +314,14 @@ pub struct FeedSummary {
impl Db {
/// Opens the database at `location` (see `location()`): a postgres:// URL, or a SQLite file.
pub async fn open(location: &str) -> Result<Self> {
/// `keep_open` for the daemon, which keeps its whole pool connected.
pub async fn open(location: &str, keep_open: bool) -> Result<Self> {
if !is_postgres(location)
&& let Some(dir) = Path::new(location).parent()
{
std::fs::create_dir_all(dir).with_context(|| format!("creating {}", dir.display()))?;
}
let orm = connect(location).await?;
let orm = connect(location, keep_open).await?;
if !is_postgres(location) {
// WAL, so the healthcheck's `ipx status` reads while the daemon writes. It is a
// setting of the file, kept once made, and making it takes a lock that cannot wait out
@@ -349,7 +357,7 @@ impl Db {
let n = N.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let pid = std::process::id();
if let Some(url) = std::env::var("IPX_TEST_DATABASE_URL").ok().filter(|u| !u.is_empty()) {
let admin = connect(&url).await?;
let admin = connect(&url, false).await?;
// Schemas an earlier run left behind: anything of ours not made by this process.
let old = admin
.query_all_raw(Statement::from_string(
@@ -369,13 +377,13 @@ impl Db {
// Every connection in the pool starts in it, so each test sees only its own tables.
let sep = if url.contains('?') { '&' } else { '?' };
let orm =
connect(&format!("{url}{sep}options=-c%20search_path%3D{schema}%20-c%20client_min_messages%3Dwarning"))
connect(&format!("{url}{sep}options=-c%20search_path%3D{schema}%20-c%20client_min_messages%3Dwarning"), false)
.await?;
create_missing(&orm).await?;
return Ok(Self { orm, tmp: None });
}
let path = std::env::temp_dir().join(format!("ipx-test-{pid}-{n}.db"));
let orm = connect(&path.display().to_string()).await?;
let orm = connect(&path.display().to_string(), false).await?;
// As Db::open has it, so the tests' connections share the file as the daemon's do.
orm.execute_unprepared("PRAGMA journal_mode = WAL").await?;
create_missing(&orm).await?;