Commit Graph

30 Commits

Author SHA1 Message Date
f22c1993b1 Import type names instead of writing their paths (#355)
The style guide imports types and uses them unqualified. 222 places
wrote a path such as std::collections::HashMap or std::time::Duration
inline; each file now imports the type once, one use a module, and
names it bare. Done with a script, file by file, only where the name is
unambiguous there:
- one path for the name throughout the file, and not a name the file
  already binds otherwise (its own types, other imports, a function's
  local use);
- not a name generic enough to mislead bare (Error, Result, Value,
  Mutex, Client, Request, Response, Command, Event, Context, Map, Ok);
- only code a normal build compiles, so no import is unused outside
  tests or names a crate only tests depend on.

193 paths stay, those cases above. No behaviour change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:19:10 +00:00
41affc8d8e Follow the Rust style guide's smaller rules (#353)
Measured against /src/styleguide/rustguide.md:
- Each use statement imports from one module, and free functions are
  called through theirs (db::step, db::now, jsonwebtoken::decode,
  crate::feed::redact, unix::signal) rather than imported bare.
- unsafe_code is forbidden workspace-wide. The one unsafe was a test's
  set_var, which is unsound while other tests' threads read the
  environment; the test now reads PATH, which every run has.
- Locks say why they may panic: expect("a panic elsewhere poisoned this
  lock") in place of 25 unwraps, and unique_slug's endless search says
  it cannot end.
- No combinator chains in for headers: each is a named variable first.
- #[must_use] on the 27 functions clippy's must_use_candidate names.
- A literal bool passed where its meaning is not plain gets its name
  (/* keep_open */ true, /* piece */ false, ...).

No behaviour changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:07:09 +00:00
e1291c1627 Count unread feed by feed (#352)
After #347 an unread count read only the items since each feed's
unread_from, but Postgres guessed a range join across every feed
keeps a third of its rows (17,000 where there are 200), so it hashed
all 22,949 of the person's entry_state rows. Counted in a subquery per
subscription, the feed and its unread_from are constants there, and
it looks up the few items since and their read state by key: on
production the feed list's unread counts went from 19.7 to 14.8 ms and
All Subscriptions' unread count from 19 to 13.

The feed list's unread counts and count_of for Unread without a search
(All Subscriptions, a feed, an OPML's shows) use it; a page of unread
items still joins, since it needs rows in order, not a count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:14:36 +00:00
a11e3b6367 Unread lists start from each person's oldest unread item (#347)
All Subscriptions' Unread page, its count and the feed list's unread
counts read every item of the person's feeds and hashed all their
entry_state rows to find the few unread: 11,727 items and 22,949 rows
for 44, 27-32 ms on production, growing with the back catalogue. A
NOT EXISTS rewrite of the same scan was no faster.

subscriptions.unread_from says every item of the feed first seen
before it is read by this person. Unread queries join the subscription
on e.first_seen >= coalesce(unread_from, 0), and the new index
entries_seen (feed_id, first_seen) makes that a range per feed.

Nothing has to keep a list in step, so nothing can drift: the reap job
and daemon start move it up to the oldest unread item (hidden ones
too, since a block list can be lifted), never past an hour ago, so an
item a scan is storing meanwhile is not passed over. The one way an
older item becomes unread, marking it so, clears the person's value
for that feed. NULL, or stale, only means reading more.

The filter test now subscribes its reader, as every list's reader is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:04:32 +00:00
c4878731c6 Count a Signal's items and unread in one pass (#348)
signal_rows ran the same correlated subquery over signal_verdicts,
entries, subscriptions and entry_state twice a Signal, once for items
and once for unread. One grouped subquery now gives both: count(*) and
a sum of the unread, joined to the person's Signals. Same results on
SQLite and Postgres (the existing counts test runs on both).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:30:30 +00:00
21fa0ae476 Run the item page's count and the feed list's queries together (#346)
entry_page awaited the page's query and then its count, about 40 and
24 ms on /api/entries, the busiest route; feed_list awaited five
independent queries in turn (entries, downloaded, unread, blocked,
feeds), 59 ms of a 67 ms /api/feeds, with the pool's connections idle.
Both now use tokio::try_join!. On SQLite they queue on its connection
as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:28:09 +00:00
aa430f6d50 A page's files are looked up by feed and item, not item alone (#344)
entries_where found a page's enclosures with guid IN (...). The only
index is enclosures_entry (feed_id, guid), so Postgres read the whole
of it: 587k rows, 9,150 buffers and 250 ms for user 2's 50 newest, on
every /api/entries, feed page, Signal page and single item. With the
page's feed ids as well it is an index lookup, 0.5 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:24:29 +00:00
37c08195f1 A new password signs the account out everywhere (#334)
ipx user passwd, the one way to change a password and the step taken
when one may be known to someone else, only replaced the hash. Every
session stayed, and session_user moves `seen` on each use, so a
session someone kept using never reached the idle cutoff.

set_password now deletes the account's sessions too. API tokens stay:
a reset that broke every script would be a reason not to reset, and
the owner lists and removes tokens in Settings. The command says how
many still work.

Also corrects the first-start log line from #321: there is no way to
change a password in the page, so it names the command alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:45:41 +00:00
511246acb2 More step spans where flame graphs showed time with nothing under it (#350)
A week of traces folded with query.py flame left whole requests opaque:
entries_in at 46-82 ms a request with the page's query and its enclosures
together, /api/art 37 ms, /api/status 95 ms, PATCH /api/feeds/{id} 72 ms and
DELETE 45 ms. Each now has a step around what it waits on: the page and its
enclosures; the artwork cache, the check that a feed names it, the fetch and the
write; the status counts; and in changing or leaving a feed, each database call,
the wait for the catalogue's lock and storing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:45:58 +00:00
44afc08b9d Search treats % and _ as themselves (#296)
The changelog lines for #292, #293 and #295 ride along in this commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:12:58 +00:00
b54b47897b Keep a feed out of Signals, and a feed's settings in named parts (#289, #290)
hide_from_signals on a subscription, on an OPML or creator covering every
show in it, as hide_from_all does for All Subscriptions: the judge does
not put its items to Jev, and each Signal's list and counts leave them
out. shown_in() is the one SQL rule for both.

The settings dialog had grown one captioned setting at a time into a
single column, one part boxed and the rest not. It is in named parts now,
the name in a column beside the controls and above them on a phone, ruled
apart; All Subscriptions and Signals carry the sidebar's icons. The
address, the same for everyone, moves to the foot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:37:49 +00:00
b89c371662 Keep a feed, an OPML or a Patreon creator out of All Subscriptions (#288)
A switch in the feed's settings, In All Subscriptions, on its
subscription as hide_from_all. All Subscriptions' list and count leave
out a feed hidden on its own or through the OPML it is in, and the page's
unread count does the same. Signals read every feed as before, and
Currently Listening, which is All Subscriptions' in-progress filter, keeps
an episode under way whatever the feed's setting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:19:54 +00:00
d5ffe5d042 Switch an item between its HTML and its Markdown (#285)
A feed that sends <source:markdown> beside its HTML (Scripting News) has its
Markdown kept in entries.markdown, added to existing databases by an ALTER.
A stored item gets it when a scan finds it told differently, the Markdown
counting as a change, the way corrections arrive (#141). The API sends it as
written and made HTML, cleaned by ammonia like the description.

The item's page has a choice between its meta line and its buttons: HTML,
Markdown or Markdown source, remembered in the browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 18:39:16 +00:00
21c786f48a Artwork without a scan of entries, and the Directory's artwork let through (#272, #273)
/api/art asked whether a feed or item names the image before anything else,
cache hits included, and entries.image had no index: a sequential scan of
entries, 571 ms on production, 50 at once for a page of images. A kept image
is served first now, since it was only kept once let through, and entries.image
has an index: hash on Postgres, whose btree refuses keys over about 2.7 KB.

The Directory lists podcasts nobody here subscribes to, and their artwork was
refused, so the page drew initials. ipx remembers the images the directory's
listings and pages carry and lets those through too; any other address is
still refused, so /api/art cannot be used to fetch arbitrary URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 18:31:20 +00:00
5cf7a28c44 An OPML's page lists its shows' items together, not the shows (#283)
GET /api/feeds/{id}/entries for an OPML or a Patreon creator now lists the
items of the shows in it the person subscribes to, as All Subscriptions does
every feed's. The shows are looked up first and queried by list: an OR with a
subquery, or an IN over one, made a single feed's page a sequential scan of
entries (clarkesworld 10 ms to 26 ms or a second, EXPLAIN on production), so a
single feed keeps its `feed_id = $1`.

The page draws the OPML with its own header and the item list, tabs, search
and sort of any feed, with the feed column showing which show each is from.
A show's scan or change refreshes its OPML's list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 17:41:38 +00:00
c90a66a24f Clippy clean, and rustfmt and clippy part of the checks
cargo clippy --workspace --all-targets had 24 warnings; its own fixes took
most, the rest by hand: two type aliases for long types, a struct built
whole, a checked division, a doc comment's stray blank line. Its fix for
the stale-feed fold was wrong: `any` stops at the first feed it removes,
where the fold removed every one, so that is a loop instead.

A pre-commit hook in .githooks refuses unformatted Rust; clippy, being a
build, is in CLAUDE.md's checks with -D warnings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 21:59:37 +00:00
692a510217 Format the Rust code with rustfmt
The code was formatted by hand, and `cargo fmt --check` differed in 866
places. rustfmt.toml keeps the house's long lines (120) and short
expressions on one line, which halves what changes; this commit is only
what rustfmt does with it, and is named in .git-blame-ignore-revs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 21:55:41 +00:00
2c4ba7896e Keep a show from an OPML when the OPML drops it (#280)
A show inside a subscribed OPML went when the OPML stopped listing it
(orphaned only if it had downloads), and went for you when you left the
OPML. Keep, in its settings, is a per-person flag on the subscription:
when the OPML drops a kept show, or nobody subscribes to the OPML any more,
it becomes a catalogue entry outside the OPML, taking the OPML's settings
while it is there, and stays on the lists of those who kept it. Whoever had
it only through the OPML loses it as before. Leaving the OPML yourself
leaves what you kept on your list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 21:37:01 +00:00
4eba746ba4 Signals' metrics by person and whose key paid (#278)
With the server paying for everyone's Signals, its bill had no names.
ipx_signals_judged_total, ipx_signals_hits_total and ipx_jev_tokens_total
now carry user (the account's name) and key (server or own); a call
shared on the server's key is split by each person's share of its
questions. Icon picks bill the person on the server's key, key checks on
their own. The dashboard has tokens and items by person and a Who used
what table over its range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 15:26:50 +00:00
5130f754cc Analyse entries once when entries_recent has no statistics (#274)
Deployed, the judge's query still read every entry: an expression index
has no statistics until its table is analysed, and Postgres, guessing a
third of entries were recent, chose a scan (898 ms) over entries_recent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 14:17:56 +00:00
2077ea5810 Signals: index the judge's query, six calls at once, one call per item on the server's key, tokens counted, a week's look-back an admin sets, edits that ask again only what was picked out (#274, #275, #276)
The judge's query read all of entries (605 MB, 566 ms) on every wake; an
index on each feed's items by date, as the query asks, and the rechecks
in a half of their own. Calls ran one at a time, about 150 ms each, so a
new Signal's month of about 8,000 items took twenty minutes: six in flight
now. On the server's key everyone's Signals on an item share one call.
TypeSafe's token counts go to /metrics and each jev span.

An item's text is cut at 1,000 characters instead of 2,000: on 55 real
items of eight podcast feeds and six Signals, on the development key,
1,000 changed none of 330 verdicts (nor did the same request twice) and
saved 12% of the input tokens, 1,266 to 1,112 a call; 500 changed one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 14:13:25 +00:00
9a41641eaa Signals: an admin chooses whether the server's TypeSafe key pays for everyone (#271)
With signals_on_server set and TYPESAFE_KEY present, the judge asks Jev
about everyone's Signals on the server's key; otherwise each person turns
Signals on in Settings with a key of their own, which is now checked with
TypeSafe before it is saved. The Signals folder shows only when they are on.

The browser suite's daemon inherited the shell's TYPESAFE_KEY, so fixture
feeds were sent to the real TypeSafe to be categorised; it is blanked now,
and an empty key counts as none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 13:18:03 +00:00
2ed8781604 Jev picks a Signal's icon, on the server's key, and each Signal counts its items (#263)
Jev chooses among options and cannot name one, so the page searches Font Awesome's index for each
word of the Signal's name, words and sentence, the best six each, and offers the likeliest 40.
POST /api/signals/icon asks a choice question on the server's TYPESAFE_KEY and answers with one
of those offered, never another. "Let Jev pick" in the icon picker, and a Signal made without an
icon gets one a moment after. Tried against TypeSafe: for AI Development, laptop-code 0.45 just
ahead of robot 0.40.

Each Signal's row says "12 items · 3 unread", as a feed's row counts its own, in place of its
words; /api/signals gives `items` beside `unread`, by the same rules as its list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 19:09:26 +00:00
b18c92d1f5 A Signal's own icon, from all of Font Awesome Free, searchable (#261)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 18:36:08 +00:00
ce485a82ac Signals judge one subject, with a sentence of your own, not any one word (#259)
Jev was asked "Is this item about any of these: ai, development, programming?", so any one broad
word was enough and the Signal's name, which says what is meant, was never sent. AI Development
held 54 items: Sega pulling a game remaster (0.90), a Surface Laptop review (0.95), a C++ tutorial
(0.99). It is now asked about one subject, called by the Signal's name, described by its words
and, when those are not enough, its owner's sentence. Scored against TypeSafe over those 54 with
"AI tools for writing software: coding agents, AI dev tools, vibe coding, AI in code review.":
5 stay, Linux kernel AI code review highest at 0.94; the remaster 0.06, the C++ tutorial 0.02.

signals.description is new, and its first start deletes every verdict, which answered the old
question, so everything in the window is asked again. Any change to a Signal's name, words or
sentence drops its verdicts, since each is part of the question. The queue query names the
Signal's sentence apart from the item's description, which share a row; the test caught the
item's winning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 18:12:29 +00:00
70990c70c7 Signals: feeds made from your subscriptions, picked by Jev (#254)
A Signal is a name and keywords. A task beside DeadAir's asks TypeSafe's Jev, on the owner's own
key, about each item of their subscribed feeds from the last 30 days and every new one: one call
an item, one noul question for each of their Signals still to hear about it, 0.5 or more a fit.
Every verdict is kept, fitting or not, in signal_verdicts, so nothing is asked twice, and new
keywords drop a Signal's verdicts to be judged afresh. A key TypeSafe refuses (401/402/403) is
passed over until its owner saves another, so a bad key is not sent the whole queue.

A Signal's items come through entries_where with an EXISTS on its hits, so filters, search, sort
and read state are an ordinary list's, and reading an item in a Signal reads it in its feed.

The page shows Signals under the places, with New signal, and Settings takes the TypeSafe key,
which is never sent back. The browser suite stubs Jev in the fixture server (IPX_TYPESAFE_URL).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 16:54:50 +00:00
6fde4b73cd Currently Listening and Pinned start from the person's own rows (#249)
coalesce(s.position, 0) > 5 and coalesce(s.flagged, false) hid from Postgres that a match needs
a row of entry_state, so it kept the outer join and scanned every item: Currently Listening read
all 574,000 (74,733 blocks from disk), 970 ms of its 1.3 s. s.position > 5 and s.flagged mean the
same, a missing row being neither, and let it start from the person's own rows: 2.2 ms, and
Pinned 32 ms to 2.3 (EXPLAIN ANALYZE, BUFFERS on production, read only). Unread keeps its
coalesce: no row is unread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 15:27:25 +00:00
62a3241139 Database connections kept open; a click's wait is measured (#241, #245, #246)
#241: on production, All Subscriptions' query runs in 31 ms, unread oldest-first in 30, the
feed list's count in 5, generic plans included (EXPLAIN ANALYZE, BUFFERS, read only), against
0.3-1.6 s in ipx. A new Postgres connection takes 80-330 ms to open, and a page load asks for
more than are idle: four of six slow loads came as the pool grew (3 to 6, 2 to 6, 3 to 5, 6 to
8), and 20 requests at once took a median 414 ms while the pool grew from 8 to 20. The daemon
and the directory's server now keep their whole pool open (min = max); a command such as `ipx
status`, and the tests, which share the server's 100 connections, open as they need.

#246: the page reports list_shown, the ms from asking for a list to drawing it, with the view
and filter, under the click's trace; background refreshes are not timed. Timings skip report()'s
once-a-minute limit, which is for errors in a loop. A dashboard panel and table, and the prod
check, read it.

#245: the load test's browse scenario subscribes by OPML and leaves the Directory, which is
ipx-directory's since #204, to it.

CLAUDE.md records the owner's leave to EXPLAIN (ANALYZE, BUFFERS) any query on production,
read only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:52:14 +00:00
8f19d88fc9 Compressed answers, a directory twice as patient, fewer whole re-reads (#239, #242, #243, #244)
Measured on 2026-10-15: the feed list and item lists went out uncompressed at over 100 KB; ipx
read 2.6 GB of feed bodies a day and the directory 3.9 GB.

- ipx-web compresses with zstd or gzip, leaving out /api/events, which a compressor would hold
  back, and /media (#239).
- A body identical to the last one (SHA-256 in feeds.body_hash) is taken as a 304, unless
  something is missing a read would fill or the check is forced; and a feed without a schedule
  of its own whose body has been the same waits a twelfth of that, up to four hours, beside the
  failure backoff (#242). 35 Patreon show feeds sent ~300 KB each hourly, about 250 MB a day.
- The directory checks each feed every twelve hours, not six (#243).
- reqwest accepts brotli and zstd too (#244).

#240 (cache the feed list's counts) is left open: at production's size on the test Postgres
that count takes 12 ms and All Subscriptions 32 ms, against 366 and 300 ms in production, so
the cause is production's database, not the query (#241).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 13:54:00 +00:00
beded23530 Cargo workspace: crates/ipx and a shared crates/feed (#202)
The first step of the split (#207). The daemon moves to crates/ipx unchanged;
feed.rs becomes crates/feed (ipx-feed), which the directory service (#204)
will share for its metadata refresher and its privacy check.

To stand alone, fetch takes the feed's address and its credentials instead
of ipx's config::Feed (config::Feed::auth gives them), collect_outlines
moves in with the OPML parsing, and the address half of looks_private is
feed::url_looks_private. ipx imports the crate as `feed` at its root, so
every crate::feed path is unchanged. The version is the workspace's, in the
root Cargo.toml.

Checked: cargo test on Postgres (118 + 29), tsc, page smoke, native bridge,
contrast and Playwright (72) pass; the image built from the new Dockerfile
is deployed and healthy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 03:30:06 +00:00