rays/ipodderx-rs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Trim the state database; Popular lists feeds the way Directory does

Browse Source 
Drops the created columns on users, subscriptions and sessions, which were
written by every insert and read by nothing, and migrate()'s add list, whose
columns all predate 0.3.0. Removes Db::subscribed_feed_ids (no callers),
Db::subscriber_count (one caller wanting > 0) and Managed.orphaned (never
read). The old-database test now builds the tables with foreign keys on.

Popular now lists the feeds inside an OPML or a Patreon creator, never the
collection, as Directory does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAC7sLVqfKmY6rsTLXzNgk
This commit is contained in:
Ray Slakinski
2026-09-12 13:15:55 +00:00
parent 457a58dcc5
commit a958f7cb37
10 changed files with 159 additions and 157 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
CHANGELOG.md
View File

@@ -12,8 +12,10 @@ The long form, with what was wrong before and how it was found, is in
### Changed
- Directory lists the feeds inside an OPML one by one, and no longer the OPML itself, so you can
subscribe to just the shows you want. Popular still counts an OPML as one feed.
- Directory and Popular list the feeds inside an OPML one by one, and no longer the OPML itself,
so you can subscribe to just the shows you want.
- The database no longer records when accounts, subscriptions and sign-ins were created. Nothing
ever read it, and an existing database drops the columns on its next start.
## [0.5.1] - 2026-09-12

47
TODO.md
View File

@@ -1,30 +1,27 @@
# To do
## Cut what is no longer needed
## Trim the state database
From a whole-repo audit for over-engineering on 2026-09-12. Biggest cut first.
From an audit of the database layer and a read-only copy of production on 2026-09-12. The data
itself was clean: no leftover tables or indexes, 47 free pages, one stray `entry_state` row.
Check each against the code before cutting it.
- [x] **Pre-accounts adoption and the dead `entries` columns.** The copy of the old read state into
`entry_state` and the `entries.read`, `flagged` and `position` columns are gone. Its other half,
subscribing the first admin to the catalogue, was not dead and stays as `adopt_catalogue`.
(`src/db.rs`, `src/main.rs`)
- [x] **`contrib/` systemd units.** From before the container; nothing points at them.
- [x] **`migrate_opml_children`.** A one-time move of OPML children out of `config.toml` that has
run. Delete it and its call. (`src/main.rs`)
- [x] **The legacy `interval_mins` key.** Production uses `schedule`. Delete the field, the fallback
in `General::interval` and its test. (`src/config.rs`)
- [x] **`Db::entries` and `Db::count_entries`.** One-line wrappers only the tests call; the tests
call `entries_in` and `count_in` instead. (`src/db.rs`)
- [x] **`web::generate_token`.** Repeats `auth::new_session_token`. Use that. (`src/web.rs`)
- [x] **Page leftovers.** `globalEvery`, `S.busy`, `S.limit`, `unitOptions`' `firstLabel`, `--r`,
`.ep.open`, the phone `.ep .art`, the duplicate phone `.fhead.slim{flex-wrap}`, the second
`#sidebar{z-index}`, and the `on()` helper. (`web/index.html`)
- [x] **`logbuf` visitors.** `record_i64`, `record_u64` and `record_bool` repeat what `Visit`'s
defaults already do through `record_debug`. (`src/logbuf.rs`)
- [x] **The `infer` dependency.** Its torrent check is the `d8:announce` test on the next line.
- [x] **The `dirs` dependency.** `XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `HOME` from `std::env`.
- [x] **The `tokio-stream` dependency.** `futures_util::stream::unfold` over the broadcast receiver.
- [x] **The icon inlined four times.** About 94 KB of base64 across both pages; serve it once as
`/icon.png` from `include_bytes!`, open without signing in like `/login`.
- [x] **`migrate()`'s add list.** All eight columns arrived in 0.2.0, and 0.5.0 only supports
upgrades from 0.3.0 on. Drop the list and its loop; keep the `retired` drop loop, which a
database coming from 0.4.0 still needs. (`src/db.rs`)
- [x] **`Db::subscribed_feed_ids`.** No callers; its doc says the scanner walks it, and it does not.
(`src/db.rs`)
- [x] **`Db::subscriber_count`.** One caller, which only asks whether it is above zero:
`subscriber_counts()?.contains_key(&id)`. (`src/db.rs`, `src/web.rs`)
- [x] **The `created` columns** on `users`, `subscriptions` and `sessions`. Written on every insert,
never read. Add them to `retired` and drop them from the inserts. (`src/db.rs`)
- [x] **`Managed.orphaned`.** Selected by `managed_feeds()` on every call and never read;
`FeedSummary.orphaned` is what the UI uses. (`src/db.rs`)
After these: `cargo test`, `node tests/page-smoke.js`, `npx playwright test`.
## Popular
- [x] **Popular lists feeds the way Directory does**: the feeds inside an OPML or a Patreon
creator, never the collection itself. (`src/web.rs`, `web/index.html`)
After these: `cargo test`, `node tests/page-smoke.js`, `npx playwright test`. Copy `state.db`
aside before deploying: `migrate()` drops columns on the first start.

17
docs/architecture.md
View File

@@ -50,10 +50,10 @@ entries feed_id, guid, title, link, published, description, first_seen,
image, duration, episode, season PK (feed_id, guid)
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
bytes_done, downloaded_at, last_error
users id, name, pass_hash, is_admin, created
sessions token, user_id, created, seen
users id, name, pass_hash, is_admin
sessions token, user_id, seen
subscriptions user_id, feed_id, keywords, auto_download, allow_explicit,
max_new_per_check, created PK (user_id, feed_id)
max_new_per_check PK (user_id, feed_id)
entry_state user_id, feed_id, guid, read, flagged, position
PK (user_id, feed_id, guid)
```
@@ -62,9 +62,12 @@ Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position
before accounts; two bugs came from queries still reading them, and `migrate()` drops them from an
older database.
Schema changes: add the table or column to `SCHEMA`, and for a column also to the list in
`migrate()`, which does `PRAGMA table_info` then `ALTER TABLE ADD COLUMN`. `Db::memory()` runs the
same path as `Db::open`, so a migration-only column cannot pass tests while missing in production.
Schema changes: add the table or column to `SCHEMA`. `CREATE TABLE IF NOT EXISTS` leaves a table
that already exists alone, so a new column on one also needs an `ALTER TABLE ADD COLUMN` in
`migrate()`, checked with `PRAGMA table_info` the way its `retired` list is for drops. None is
needed today: every column added so far predates 0.3.0, the oldest version an upgrade may start
from. `Db::memory()` runs the same path as `Db::open`, so a migration cannot pass the tests while
missing in production.
## Control socket
@@ -115,7 +118,7 @@ else a `401`.
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
| `POST /api/fetch` | |
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds with an OPML counted as one, and every listable feed A to Z with an OPML's feeds in place of the OPML, with everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with an OPML's feeds in place of the OPML and everyone counted (id, title, art, count, whether it is yours; never a URL, never a private feed); subscribe by id |
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
| `GET /api/events` | SSE, the same broadcast the socket carries |

16
docs/history.md
View File

@@ -6,6 +6,22 @@ reasoning lives. New write-ups go at the top.
See [README.md](../README.md) for what the thing is.
## 2026-09-12 — Trimming the state database
An audit of the database layer, with a read-only copy of production to check it against. The
data was already clean: no tables or indexes left from older versions, 47 free pages after the
column drops earlier the same day, and one stray `entry_state` row. The code had five things:
- `migrate()` still added eight columns to any table missing them. All eight shipped in 0.2.0 and
upgrades now start from 0.3.0 at the oldest, so the list and its loop went; the `retired` drop
list stays, since a database coming from 0.4.0 still has the old read columns.
- `created` on `users`, `subscriptions` and `sessions` was written by every insert and read by
nothing. They joined `retired`. The old-database test now builds all three tables, foreign keys
included, since `DROP COLUMN` on a table that references another was the part worth proving.
- `Db::subscribed_feed_ids` had no callers, though its doc said the scanner walked it.
`Db::subscriber_count` had one caller asking whether it was above zero, which
`subscriber_counts().contains_key` answers. `Managed.orphaned` was selected and never read.
## 2026-09-12 — Cutting what had outlived its reason
A whole-repo audit for over-engineering listed twelve things to cut, and all of them went.

7
docs/users.md
View File

@@ -73,10 +73,9 @@ re-subscribing does not pull the back catalogue again.
**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
included. Directory lists every one of them A to Z. Your own feeds are marked Subscribed.
It shows a title, artwork and a count, never a URL or who reads it. An OPML subscription is one
feed in Popular, since everyone subscribed to it counts for every feed inside; Directory lists the
feeds inside it one by one instead, and never the OPML, so you can take just the shows you want.
Anything that looks private is left out: a login configured for the feed, credentials in its URL,
It shows a title, artwork and a count, never a URL or who reads it. An OPML subscription is listed
as the feeds inside it, one by one, and never the OPML itself, so you can take just the shows you
want. Anything that looks private is left out: a login configured for the feed, credentials in its URL,
or a key such as `auth=` or `token=` in the query, or a feed from a paid-feed service such as
Patreon or Supercast, which put the key in the path, and any feed inside an OPML that looks private
itself. Those are someone's paid subscriptions, and listing them would let anyone here read what

171
src/db.rs
View File

@@ -72,8 +72,7 @@ CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE COLLATE NOCASE,
pass_hash TEXT,
is_admin INTEGER NOT NULL DEFAULT 0,
created INTEGER NOT NULL
is_admin INTEGER NOT NULL DEFAULT 0
);
-- What one person wants from a feed. The feed, its items and its files are shared; this
@@ -85,7 +84,6 @@ CREATE TABLE IF NOT EXISTS subscriptions (
auto_download INTEGER,
allow_explicit INTEGER,
max_new_per_check INTEGER,
created INTEGER NOT NULL,
PRIMARY KEY (user_id, feed_id)
);
@@ -104,7 +102,6 @@ CREATE TABLE IF NOT EXISTS entry_state (
CREATE TABLE IF NOT EXISTS sessions (
token TEXT PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created INTEGER NOT NULL,
seen INTEGER NOT NULL
);
";
@@ -136,40 +133,30 @@ pub struct Managed {
pub url: String,
pub title: Option<String>,
pub group_id: String,
pub orphaned: bool,
}
/// Adds the columns later versions introduced and drops the ones they retired. CREATE TABLE IF
/// NOT EXISTS does nothing to a table that already exists, so an installed database needs both.
/// Drops the columns later versions retired. CREATE TABLE IF NOT EXISTS leaves a table that
/// already exists alone, so an installed database needs this done explicitly. A new column on an
/// existing table would need an ALTER TABLE ADD COLUMN here too; none does yet, since every one
/// added so far predates 0.3.0, the oldest version an upgrade may start from.
fn migrate(conn: &Connection) -> Result<()> {
let wanted: &[(&str, &str, &str)] = &[
("feeds", "image", "TEXT"),
("feeds", "orphaned", "INTEGER NOT NULL DEFAULT 0"),
("feeds", "group_id", "TEXT"),
("feeds", "managed", "INTEGER NOT NULL DEFAULT 0"),
("entries", "image", "TEXT"),
("entries", "duration", "INTEGER"),
("entries", "episode", "INTEGER"),
("entries", "season", "INTEGER"),
let retired: &[(&str, &str)] = &[
// Read state from before accounts, long since moved to entry_state. Two bugs came from
// queries still reading these after they stopped meaning anything.
("entries", "read"),
("entries", "flagged"),
("entries", "position"),
// Written by every insert and read by nothing.
("users", "created"),
("subscriptions", "created"),
("sessions", "created"),
];
// Read state from before accounts, long since moved to entry_state. Two bugs came from
// queries still reading these after they stopped meaning anything, so they go.
let retired: &[(&str, &str)] = &[("entries", "read"), ("entries", "flagged"), ("entries", "position")];
let has = |table: &str, column: &str| -> Result<bool> {
let mut stmt = conn.prepare(&format!("PRAGMA table_info({table})"))?;
let names = stmt
.query_map([], |r| r.get::<_, String>(1))?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(names.iter().any(|c| c == column))
};
for (table, column, ty) in wanted {
if !has(table, column)? {
tracing::info!(table, column, "adding column");
conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {ty}"))?;
}
}
for (table, column) in retired {
if has(table, column)? {
let names: Vec<String> = conn
.prepare(&format!("PRAGMA table_info({table})"))?
.query_map([], |r| r.get(1))?
.collect::<rusqlite::Result<_>>()?;
if names.iter().any(|c| c == column) {
tracing::info!(table, column, "dropping column");
conn.execute_batch(&format!("ALTER TABLE {table} DROP COLUMN {column}"))?;
}
@@ -829,15 +816,14 @@ impl Db {
}
for id in catalogue {
conn.execute(
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
params![user_id, id, now()],
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id) VALUES (?1, ?2)",
params![user_id, id],
)?;
}
// Feeds that exist only in the database (OPML children) count too.
conn.execute(
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created)
SELECT ?1, id, ?2 FROM feeds",
params![user_id, now()],
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id) SELECT ?1, id FROM feeds",
params![user_id],
)?;
Ok(catalogue.len())
}
@@ -954,8 +940,8 @@ impl Db {
pub fn subscribe(&self, user_id: i64, feed_id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id, created) VALUES (?1, ?2, ?3)",
params![user_id, feed_id, now()],
"INSERT OR IGNORE INTO subscriptions (user_id, feed_id) VALUES (?1, ?2)",
params![user_id, feed_id],
)?;
Ok(())
}
@@ -969,16 +955,6 @@ impl Db {
Ok(())
}
/// How many people want this feed. Nobody means it stops being scanned.
pub fn subscriber_count(&self, feed_id: &str) -> Result<i64> {
let conn = self.conn.lock().unwrap();
Ok(conn.query_row(
"SELECT count(*) FROM subscriptions WHERE feed_id = ?1",
[feed_id],
|r| r.get(0),
)?)
}
/// Overwrites one person's settings for a feed. A None field means: follow the feed.
pub fn set_subscription(&self, user_id: i64, sub: &Sub) -> Result<()> {
let conn = self.conn.lock().unwrap();
@@ -989,8 +965,8 @@ impl Db {
.transpose()?;
conn.execute(
"INSERT INTO subscriptions
(user_id, feed_id, keywords, auto_download, allow_explicit, max_new_per_check, created)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
(user_id, feed_id, keywords, auto_download, allow_explicit, max_new_per_check)
VALUES (?1, ?2, ?3, ?4, ?5, ?6)
ON CONFLICT(user_id, feed_id) DO UPDATE SET
keywords = excluded.keywords,
auto_download = excluded.auto_download,
@@ -1003,29 +979,18 @@ impl Db {
sub.auto_download.map(|v| v as i64),
sub.allow_explicit.map(|v| v as i64),
sub.max_new_per_check,
now()
],
)?;
Ok(())
}
/// Feeds with at least one subscriber. What the scanner walks.
pub fn subscribed_feed_ids(&self) -> Result<Vec<String>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare("SELECT DISTINCT feed_id FROM subscriptions")?;
let out = stmt
.query_map([], |r| r.get::<_, String>(0))?
.collect::<rusqlite::Result<Vec<_>>>()?;
Ok(out)
}
// ---- users and sessions ----
pub fn create_user(&self, name: &str, pass_hash: Option<&str>, admin: bool) -> Result<i64> {
let conn = self.conn.lock().unwrap();
conn.execute(
"INSERT INTO users (name, pass_hash, is_admin, created) VALUES (?1, ?2, ?3, ?4)",
params![name, pass_hash, admin as i64, now()],
"INSERT INTO users (name, pass_hash, is_admin) VALUES (?1, ?2, ?3)",
params![name, pass_hash, admin as i64],
)?;
Ok(conn.last_insert_rowid())
}
@@ -1093,7 +1058,7 @@ impl Db {
pub fn create_session(&self, user_id: i64, token: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute(
"INSERT INTO sessions (token, user_id, created, seen) VALUES (?1, ?2, ?3, ?3)",
"INSERT INTO sessions (token, user_id, seen) VALUES (?1, ?2, ?3)",
params![token, user_id, now()],
)?;
Ok(())
@@ -1195,7 +1160,7 @@ impl Db {
.optional()?)
}
/// Read and starred, per person. The row is created on first touch.
/// Read and kept, per person. The row is created on first touch.
pub fn set_entry_flag(
&self,
user_id: i64,
@@ -1270,7 +1235,7 @@ impl Db {
pub fn managed_feeds(&self) -> Result<Vec<Managed>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT id, url, title, group_id, orphaned FROM feeds
"SELECT id, url, title, group_id FROM feeds
WHERE managed = 1 AND group_id IS NOT NULL ORDER BY coalesce(title, id)",
)?;
Ok(stmt
@@ -1280,7 +1245,6 @@ impl Db {
url: r.get(1)?,
title: r.get(2)?,
group_id: r.get(3)?,
orphaned: r.get::<_, i64>(4)? != 0,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?)
@@ -1423,8 +1387,8 @@ mod tests {
fn every_sort_column_runs_and_orders_both_ways() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(1,'g',0);
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1);
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'f'),(1,'g');
INSERT INTO feeds (id, url, title) VALUES ('f','u','Zebra'),('g','v','Aardvark');
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
('f','a','banana',100),('g','b','Apple',200),('f','c','cherry',300);
@@ -1457,8 +1421,8 @@ mod tests {
fn deleting_a_shared_file_asks_about_everyone_else() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0),(3,'kit',0,0);
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0),(3,'f',0);
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0),(3,'kit',0);
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'f'),(2,'f'),(3,'f');
INSERT INTO entries (feed_id, guid, first_seen) VALUES ('f','a',0);
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
(1,'f','a','u1','/tmp/a','done');",
@@ -1483,7 +1447,7 @@ mod tests {
fn read_state_belongs_to_one_person() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0);
INSERT INTO entries (feed_id, guid, title, first_seen) VALUES
('f','a','One',100),('f','b','Two',200);",
)
@@ -1526,27 +1490,52 @@ mod tests {
}
#[test]
fn an_old_database_loses_the_retired_read_columns() {
fn an_old_database_loses_its_retired_columns() {
let conn = Connection::open_in_memory().unwrap();
// As open() has it: a DROP COLUMN on a table that references another is the part worth
// proving, and it has to work with the foreign keys switched on.
conn.pragma_update(None, "foreign_keys", "ON").unwrap();
conn.execute_batch(
"CREATE TABLE entries (feed_id TEXT NOT NULL, guid TEXT NOT NULL,
first_seen INTEGER NOT NULL, read INTEGER NOT NULL DEFAULT 0,
flagged INTEGER NOT NULL DEFAULT 0, position INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (feed_id, guid));",
PRIMARY KEY (feed_id, guid));
CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE COLLATE NOCASE,
pass_hash TEXT, is_admin INTEGER NOT NULL DEFAULT 0, created INTEGER NOT NULL);
CREATE TABLE subscriptions (
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
feed_id TEXT NOT NULL, created INTEGER NOT NULL, PRIMARY KEY (user_id, feed_id));
CREATE TABLE sessions (token TEXT PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created INTEGER NOT NULL, seen INTEGER NOT NULL);
INSERT INTO users VALUES (1, 'ray', NULL, 1, 0);
INSERT INTO subscriptions VALUES (1, 'f', 0);
INSERT INTO sessions VALUES ('t', 1, 0, 0);",
)
.unwrap();
// The same order as open(): the schema leaves the old table alone, migrate() fixes it.
// The same order as open(): the schema leaves the old tables alone, migrate() fixes them.
conn.execute_batch(SCHEMA).unwrap();
migrate(&conn).unwrap();
let cols: Vec<String> = conn
.prepare("PRAGMA table_info(entries)")
.unwrap()
.query_map([], |r| r.get(1))
.unwrap()
.collect::<rusqlite::Result<_>>()
for (table, gone) in [
("entries", &["read", "flagged", "position"][..]),
("users", &["created"][..]),
("subscriptions", &["created"][..]),
("sessions", &["created"][..]),
] {
let cols: Vec<String> = conn
.prepare(&format!("PRAGMA table_info({table})"))
.unwrap()
.query_map([], |r| r.get(1))
.unwrap()
.collect::<rusqlite::Result<_>>()
.unwrap();
assert!(!cols.iter().any(|c| gone.contains(&c.as_str())), "{table}: {cols:?}");
}
// And the rows come through it.
let kept: i64 = conn
.query_row("SELECT count(*) FROM subscriptions JOIN sessions USING (user_id)", [], |r| r.get(0))
.unwrap();
assert!(!cols.iter().any(|c| ["read", "flagged", "position"].contains(&c.as_str())), "{cols:?}");
assert!(cols.iter().any(|c| c == "image"), "and it still gains the newer ones");
assert_eq!(kept, 1);
}
#[test]
@@ -1554,7 +1543,7 @@ mod tests {
// Cutting this along with the dead read columns left the browser suite's admin with an
// empty sidebar: it is how a fresh install's first account gets config.toml's feeds.
let db = Db::memory().unwrap();
db.exec_for_test("INSERT INTO users (id, name, is_admin, created) VALUES (1,'admin',1,0);")
db.exec_for_test("INSERT INTO users (id, name, is_admin) VALUES (1,'admin',1);")
.unwrap();
let subs = || -> i64 {
db.conn.lock().unwrap().query_row("SELECT count(*) FROM subscriptions", [], |r| r.get(0)).unwrap()
@@ -1582,7 +1571,7 @@ mod tests {
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
(1,'f','b','u1','/tmp/b','done');
-- Read and starred belong to a person now, so say which one.
INSERT INTO users (id, name, is_admin, created) VALUES (7,'reader',1,0);
INSERT INTO users (id, name, is_admin) VALUES (7,'reader',1);
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
(7,'f','b',1,0),
(7,'f','c',1,1);",
@@ -1652,7 +1641,7 @@ mod tests {
fn a_show_takes_over_what_its_creator_held() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1);
INSERT INTO enclosures (id, feed_id, guid, url, state, path, last_error) VALUES
(1,'creator','a','u1','done','/x/a.mp3',NULL),
(2,'creator','b','u2','skipped',NULL,'explicit'),
@@ -1688,9 +1677,9 @@ mod tests {
fn a_feed_in_a_group_follows_your_settings_on_the_group() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
INSERT INTO subscriptions (user_id, feed_id, allow_explicit, created) VALUES
(1,'group',1,0),(1,'show',NULL,0),(2,'group',1,0),(2,'show',0,0);",
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0);
INSERT INTO subscriptions (user_id, feed_id, allow_explicit) VALUES
(1,'group',1),(1,'show',NULL),(2,'group',1),(2,'show',0);",
)
.unwrap();
let explicit = |group| -> Vec<Option<bool>> {

6
src/retention.rs
View File

@@ -154,8 +154,8 @@ mod tests {
// One file serves both subscribers, so it takes both of them to release it.
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0),(2,'sam',0,0);
INSERT INTO subscriptions (user_id, feed_id, created) VALUES (1,'f',0),(2,'f',0);
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0);
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'f'),(2,'f');
INSERT INTO entries (feed_id, guid, first_seen) VALUES
('f', 'keep', 0),
('f', 'half', 0),
@@ -189,7 +189,7 @@ mod tests {
fn prune_keeps_entries_that_still_have_a_file() {
let db = Db::memory().unwrap();
db.exec_for_test(
"INSERT INTO users (id, name, is_admin, created) VALUES (1,'ray',1,0);
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1);
INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',1);
INSERT INTO entries (feed_id, guid, first_seen) VALUES
('f', 'has-file', 100),

31
src/web.rs
View File

@@ -579,14 +579,10 @@ struct PopularRow {
}
/// Every feed that may be listed, with everyone counted, you included, most subscribers
/// first. Popular is the top of one list and the directory is all of the other, and between
/// them they are all that `subscribe_popular` will subscribe you to.
///
/// `folders` says how an OPML appears. Popular lists the OPML once and not the feeds inside:
/// everyone subscribed to it counts for every one of them, and eighty of those would bury
/// everything anyone chose on purpose. The directory, which is for finding a show, lists the
/// feeds inside and never the OPML.
fn popular(state: &WebState, user_id: i64, folders: bool) -> Result<Vec<PopularRow>> {
/// first. Popular is the top of it, the directory is all of it, and it is all that
/// `subscribe_popular` will subscribe you to. An OPML or a Patreon creator is listed as the
/// feeds inside it and never itself: both lists are for finding a show.
fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
let db = &state.ctx.db;
let mine: std::collections::HashSet<String> =
db.subscriptions_for(user_id)?.into_iter().map(|s| s.feed_id).collect();
@@ -599,11 +595,13 @@ fn popular(state: &WebState, user_id: i64, folders: bool) -> Result<Vec<PopularR
let mut out = vec![];
for s in &catalogue {
let n = counts.get(&s.id).copied().unwrap_or(0);
let inside = s.managed || s.cfg.group.is_some();
let hidden = if folders { inside } else { is_folder.contains(s.id.as_str()) };
// A feed inside an OPML that looks private is as private as the OPML.
let folder = s.cfg.group.as_deref().and_then(|g| by_id.get(g));
if n == 0 || hidden || looks_private(&s.cfg) || folder.is_some_and(|f| looks_private(f)) {
if n == 0
|| is_folder.contains(s.id.as_str())
|| looks_private(&s.cfg)
|| folder.is_some_and(|f| looks_private(f))
{
continue;
}
let sum = db.feed_summary(&s.id)?;
@@ -618,7 +616,7 @@ async fn get_popular(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<Vec<PopularRow>>, ApiError> {
let mut rows = popular(&state, user.id, true)?;
let mut rows = popular(&state, user.id)?;
rows.truncate(10);
Ok(Json(rows))
}
@@ -628,7 +626,7 @@ async fn get_directory(
State(state): State<WebState>,
user: crate::db::User,
) -> Result<Json<Vec<PopularRow>>, ApiError> {
let mut rows = popular(&state, user.id, false)?;
let mut rows = popular(&state, user.id)?;
rows.sort_by_key(sort_name);
Ok(Json(rows))
}
@@ -637,15 +635,14 @@ fn sort_name(p: &PopularRow) -> String {
p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase()
}
/// Subscribes by id, because the lists never show a URL. Checked against the same lists, so
/// Subscribes by id, because the list never shows a URL. Checked against the same list, so
/// a guessed id cannot reach a private feed.
async fn subscribe_popular(
State(state): State<WebState>,
user: crate::db::User,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, ApiError> {
let listed = |folders| popular(&state, user.id, folders).map(|rows| rows.iter().any(|p| p.id == id));
if !(listed(true)? || listed(false)?) {
if !popular(&state, user.id)?.iter().any(|p| p.id == id) {
return Err(ApiError::bad_request(format!("{id:?} is not in the directory")));
}
state.ctx.db.subscribe(user.id, &id)?;
@@ -1095,7 +1092,7 @@ async fn remove_feed(
{
state.ctx.db.unsubscribe(user.id, &child.id)?;
}
if state.ctx.db.subscriber_count(&id)? > 0 {
if state.ctx.db.subscriber_counts()?.contains_key(&id) {
return Ok(StatusCode::NO_CONTENT);
}

13
tests/ui/app.spec.js
View File

@@ -613,8 +613,8 @@ test('Popular lists what everyone here reads, but never a private feed', async (
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
const offered = piper.locator('#popular .childrow');
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
// An OPML's own feeds ride on the OPML, and a key in a URL marks someone's paid feed.
await expect(offered.filter({ hasText: /Grouped Show|grouped-show/ })).toHaveCount(0);
// An OPML is listed as the feeds inside it, and a key in a URL marks someone's paid feed.
await expect(offered.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
// No URL reaches the page at all, so neither can a key, and the server holds the same line.
@@ -623,8 +623,8 @@ test('Popular lists what everyone here reads, but never a private feed', async (
expect(listed).not.toContain('.xml');
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
// The directory is every listed feed A to Z, with an OPML's feeds in place of the OPML.
// Popular is the most subscribed, with an OPML as one feed, so its feeds cannot bury the rest.
// Popular is the top ten of the directory, and the directory is every listed feed A to Z,
// with an OPML's feeds in place of the OPML in both.
const dir = await (await piper.request.get('/api/directory')).json();
const top = await (await piper.request.get('/api/popular')).json();
const names = dir.map(p => (p.title || p.id).toLowerCase());
@@ -632,9 +632,8 @@ test('Popular lists what everyone here reads, but never a private feed', async (
const ids = dir.map(p => p.id);
expect(ids).not.toContain('test-subscriptions');
expect(ids).toEqual(expect.arrayContaining(['grouped-show', 'aardvark-radio']));
expect(top.length).toBeLessThanOrEqual(10);
expect(top.map(t => t.id)).not.toContain('grouped-show');
expect(top.filter(t => t.id !== 'test-subscriptions').every(t => ids.includes(t.id))).toBe(true);
expect(top.length).toBe(Math.min(10, dir.length));
expect(top.every(t => ids.includes(t.id))).toBe(true);
expect(ids).not.toContain('paid-show');
// Subscribe from the directory this time; the popular list shares the same rows.

2
web/index.html
View File

@@ -777,7 +777,7 @@ const VIEWS={
':directory':{title:'Directory',icon:ICON.directory,url:'/api/directory',
blurb:'Every feed anyone on this server subscribes to, A to Z. The feeds inside an OPML are listed one by one, not the OPML.'},
':popular':{title:'Popular',icon:ICON.popular,url:'/api/popular',
blurb:'The ten feeds with the most subscribers here. An OPML counts as one feed.'},
blurb:'The ten feeds with the most subscribers here. The feeds inside an OPML count one by one, not the OPML.'},
':all':{title:'All Subscriptions',icon:ICON.all},
};
function renderFeeds(){