Commit Graph

105 Commits

Author SHA1 Message Date
ecff2f8dcc The directory's lib.rs splits into leaf modules (#356)
The crate root held the wire types, Directory with its refresher and
reloader, and every /v1 handler. It is now the module list and the
re-exports ipx and the directory's main use, so their paths are as
before:

  wire.rs       Listing, FeedPage, SnapshotEntry, ... and MAX_SNAPSHOT
  directory.rs  Directory, Shared, the refresher and the reloader
  api.rs        the router, the token check and the /v1 handlers

Directory's fields and methods are pub(crate) for the handlers beside
it. Its two tests that read those fields went with it, on a scratch
Directory of their own; the API tests stay with the API. Every
definition and doc comment line of the old lib.rs is in exactly one of
the new files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:48:05 +00:00
ff5d786602 ipx-feed's lib.rs splits into leaf modules (#356)
lib.rs held 2,000 lines: fetching, parsing, OPML, Patreon, discovery,
artwork, categories, Jev and redaction, and their tests. It is now the
module list and the re-exports, so ipx and the directory call
feed::fetch, feed::parse and the rest as before:

  fetch.rs      fetch and its limits    parse.rs     RSS and Atom
  outlines.rs   OPML (not opml.rs, which would shadow the opml crate)
  patreon.rs    a creator's shows       discover.rs  finding a feed
  wordpress.rs  WordPress.com's API     failure.rs   failures in words
  privacy.rs    redact, looks_private   artwork.rs   https art, icons
  category.rs   Apple's categories      jev.rs       Jev's category

Moved verbatim, private items pub(crate) where a sibling calls them,
calls across modules module::function. Every definition and doc comment
line of the old lib.rs is in exactly one of the new files; each test
went with what it tests, and all 40 run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:41:14 +00:00
4f4fd4c3fe ipx's main.rs splits into leaf modules (#356)
main.rs held 3,200 lines: the command line, the shared context, the
daemon, scans, downloads, the catalogue and OPML, and their tests. The
style guide keeps a root module to declarations and a little glue, the
code in leaf modules named for what they hold:

  cli.rs        the commands, run and user_cmd, ago and duration
  ctx.rs        Ctx and CfgEdit, and Ctx::new, which main() built inline
  daemon.rs     the daemon's loop, web server, shutdown, status, reload
  scan.rs       due_at/due_after, scan_one, policy, downloads, torrents
  catalogue.rs  add/rm/leave/remove_user, subscriptions, retiring lists
  lists.rs      OPML import and syncing an OPML's or a creator's feeds
                (not opml.rs, which would shadow the opml crate)
  logging.rs    logging::init and the JSON lines with their trace
  test_util.rs  the context, feeds and feed server tests share

first_password moves to auth.rs and reap to retention.rs. Each test went
with what it tests; all 151 still run. main.rs is the module list and a
main() that hands the command on, 113 lines.

Moved verbatim, then made pub(crate) where a sibling calls it, with
calls across modules written module::function as the guide asks.
Everything the old main.rs defined is defined once in the new files.

With Ctx no longer a pub type at the crate root, Db's pub methods stop
counting as public API, and the compiler sees what nothing outside tests
calls: entries_in, count_in, unread_count and feed_description are now
#[cfg(test)], and media_feeds, called by nothing, is gone.

ipx depends on opentelemetry_sdk directly now (it was already in the
lock through ipx-directory), for logging::init's signature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:35:24 +00:00
f22c1993b1 Import type names instead of writing their paths (#355)
The style guide imports types and uses them unqualified. 222 places
wrote a path such as std::collections::HashMap or std::time::Duration
inline; each file now imports the type once, one use a module, and
names it bare. Done with a script, file by file, only where the name is
unambiguous there:
- one path for the name throughout the file, and not a name the file
  already binds otherwise (its own types, other imports, a function's
  local use);
- not a name generic enough to mislead bare (Error, Result, Value,
  Mutex, Client, Request, Response, Command, Event, Context, Map, Ok);
- only code a normal build compiles, so no import is unused outside
  tests or names a crate only tests depend on.

193 paths stay, those cases above. No behaviour change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:19:10 +00:00
d33d83043d Doc comments on ipx-feed's and ipx-directory's public items (#354)
The style guide asks for doc comments on a library's public API. Every
type, function, constant and static the two libraries export now has
one, saying what it is for or what it does that its name does not:
fetch follows redirects itself so credentials stay on the feed's host,
Config::parse refuses repeated token names, Store::due_one is the
admin's Refresh. Fields and variants whose names say it all have none,
as the guide allows.

category_path's comment had slid above categories(), which then had
two; each has its own again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:12:14 +00:00
41affc8d8e Follow the Rust style guide's smaller rules (#353)
Measured against /src/styleguide/rustguide.md:
- Each use statement imports from one module, and free functions are
  called through theirs (db::step, db::now, jsonwebtoken::decode,
  crate::feed::redact, unix::signal) rather than imported bare.
- unsafe_code is forbidden workspace-wide. The one unsafe was a test's
  set_var, which is unsound while other tests' threads read the
  environment; the test now reads PATH, which every run has.
- Locks say why they may panic: expect("a panic elsewhere poisoned this
  lock") in place of 25 unwraps, and unique_slug's endless search says
  it cannot end.
- No combinator chains in for headers: each is a named variable first.
- #[must_use] on the 27 functions clippy's must_use_candidate names.
- A literal bool passed where its meaning is not plain gets its name
  (/* keep_open */ true, /* piece */ false, ...).

No behaviour changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:07:09 +00:00
59c7f5f213 Back to -dev after ipx 1.5.0, ipx-directory 1.4.0 and ipx-web 1.3.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:48:48 +00:00
0be8eb5cdf ipx 1.5.0, ipx-directory 1.4.0 and ipx-web 1.3.0
The security audit's 26 findings, three dependency advisories and the
performance issues found in traces (#318-#352), each already deployed
as a -dev build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:41:57 +00:00
e1291c1627 Count unread feed by feed (#352)
After #347 an unread count read only the items since each feed's
unread_from, but Postgres guessed a range join across every feed
keeps a third of its rows (17,000 where there are 200), so it hashed
all 22,949 of the person's entry_state rows. Counted in a subquery per
subscription, the feed and its unread_from are constants there, and
it looks up the few items since and their read state by key: on
production the feed list's unread counts went from 19.7 to 14.8 ms and
All Subscriptions' unread count from 19 to 13.

The feed list's unread counts and count_of for Unread without a search
(All Subscriptions, a feed, an OPML's shows) use it; a page of unread
items still joins, since it needs rows in order, not a count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:14:36 +00:00
a11e3b6367 Unread lists start from each person's oldest unread item (#347)
All Subscriptions' Unread page, its count and the feed list's unread
counts read every item of the person's feeds and hashed all their
entry_state rows to find the few unread: 11,727 items and 22,949 rows
for 44, 27-32 ms on production, growing with the back catalogue. A
NOT EXISTS rewrite of the same scan was no faster.

subscriptions.unread_from says every item of the feed first seen
before it is read by this person. Unread queries join the subscription
on e.first_seen >= coalesce(unread_from, 0), and the new index
entries_seen (feed_id, first_seen) makes that a range per feed.

Nothing has to keep a list in step, so nothing can drift: the reap job
and daemon start move it up to the oldest unread item (hidden ones
too, since a block list can be lifted), never past an hour ago, so an
item a scan is storing meanwhile is not passed over. The one way an
older item becomes unread, marking it so, clears the person's value
for that feed. NULL, or stale, only means reading more.

The filter test now subscribes its reader, as every list's reader is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:04:32 +00:00
cb4ee5a570 Kept artwork off the async threads, and one fetch per missing image (#345 #349)
#345: a cache hit made four blocking std::fs calls on a tokio worker,
and the cache was on shfs (FUSE). A page asks for about fifty images at
once, so hits ran 11 ms at p50 and over a second at p90. Reading,
keeping and the reap job's trim now run in spawn_blocking. Production's
cache also moves to /mnt/fast in the compose file, outside the repo.

#349: two requests for the same uncached image both fetched it from its
publisher. A per-address lock now makes the second wait for the first,
then find the image kept. The lock is forgotten once its last holder is
done, whichever way the request ended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:33:41 +00:00
c4878731c6 Count a Signal's items and unread in one pass (#348)
signal_rows ran the same correlated subquery over signal_verdicts,
entries, subscriptions and entry_state twice a Signal, once for items
and once for unread. One grouped subquery now gives both: count(*) and
a sum of the unread, joined to the person's Signals. Same results on
SQLite and Postgres (the existing counts test runs on both).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:30:30 +00:00
21fa0ae476 Run the item page's count and the feed list's queries together (#346)
entry_page awaited the page's query and then its count, about 40 and
24 ms on /api/entries, the busiest route; feed_list awaited five
independent queries in turn (entries, downloaded, unread, blocked,
feeds), 59 ms of a 67 ms /api/feeds, with the pool's connections idle.
Both now use tokio::try_join!. On SQLite they queue on its connection
as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:28:09 +00:00
aa430f6d50 A page's files are looked up by feed and item, not item alone (#344)
entries_where found a page's enclosures with guid IN (...). The only
index is enclosures_entry (feed_id, guid), so Postgres read the whole
of it: 587k rows, 9,150 buffers and 250 ms for user 2's 50 newest, on
every /api/entries, feed page, Signal page and single item. With the
page's feed ids as well it is an index lookup, 0.5 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:24:29 +00:00
543d35f02c A snapshot's subscriber counts are bounded above (#342)
Store::snapshot clamped a count only from below, and LISTING sums each
feed's counts across instances, cast to BIGINT. Two instances sending
i64::MAX for a feed overflowed it (Postgres: bigint out of range;
SQLite: integer overflow), and list() and popular() failed for every
feed: the public pages 503, /v1/feeds and /v1/popular 500, until an
admin dropped the instance. Counts are now clamped to a million.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:17:43 +00:00
0de0fb4332 Bound icon picks per person, and keep people's Signals apart (#340 #343)
#340: POST /api/signals/icon spends the server's TYPESAFE_KEY for
anyone signed in, whatever signals_on_server says, and is not tied to
a Signal, so a loop on it ran up the bill and, through the shared pace
and a 429's back-off, held back everyone's judging. Each person now has
20 picks an hour; past that it is a 429 the page shows as a toast.

#343: on the server's key, calls() put everyone's Signals on an item in
one call (#275), so one person's free-text description was part of the
question that decided another's verdicts. Each person's Signals on an
item are now a call of their own. The saving #275 made is lost only
where two people have Signals on the same item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:13:02 +00:00
79b496bdfb A password check holds its turn until it finishes (#324)
check_password took a permit from CHECKS in the async function and
ran Argon2 in spawn_blocking. A client that hung up dropped the
future and with it the permit, while the blocking check ran on, so a
flood of sign-ins that disconnected at once ran unbounded Argon2 work,
the very thing CHECKS was added for (#137).

The permit is now owned (CHECKS is an Arc) and moves into the blocking
closure, so it is freed when the check ends, whoever stopped waiting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:03:19 +00:00
a20fade8ad OPML uploads are capped at 2,000 feeds and checked by address once (#338)
subscribe_opml, reachable by anyone signed in through POST /api/opml,
held the catalogue lock while, for each outline, it searched every
subscription and the whole catalogue for its address and probed
unique_slug's suffixes in order. 2 MB holds about 40,000 outlines, so
one upload was quadratic work with every other catalogue change
waiting, and its feeds stayed in the catalogue for good.

opml_feeds lists a document's feeds and refuses more than MAX_LISTED,
the limit a subscribed list already has (#337): a 400 from the web, a
message from `ipx opml`. subscribe_opml takes that list and looks
addresses up in a map built once, the first feed with an address
winning as the search in order did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:00:29 +00:00
daf95a25cd Only a subscriber or an admin marks a feed private (#333)
patch_feed let private = true through for anyone signed in, as the
field's doc comment meant for subscribers, but never checked that the
caller subscribed. Ids are slugs of titles, so anyone could mark
another person's feed, or a whole OPML (whose shows go with it),
private, keeping it out of the directory, and promote a derived feed
they did not read out of its OPML; only an admin could undo either.

A non-admin now needs a subscription to the feed, or gets 404, as the
per-person settings above already do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:56:36 +00:00
91de1114d8 Live updates go to those who read the feed, while their account exists (#325 #335)
/api/events sent every daemon event to every open page. Event::redacted
strips the query and the password from addresses, not the path, so a
paid feed's key in its path (feeds.supercast.com/feeds/<key>) reached
everyone signed in through its FeedError or FeedMoved (#325). And the
stream checked who was asking once, when it opened: a removed account's
page kept getting events until it closed (#335).

Each stream now looks its account and subscriptions up again at most
every 30 s, as an event arrives, ends when the account is gone, and
sends an event about a feed only to someone subscribed to it. Admins
get everything, for the log; Reaped, a file path, goes to admins only;
SignalHits to the person it is for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:51:12 +00:00
37c08195f1 A new password signs the account out everywhere (#334)
ipx user passwd, the one way to change a password and the step taken
when one may be known to someone else, only replaced the hash. Every
session stayed, and session_user moves `seen` on each use, so a
session someone kept using never reached the idle cutoff.

set_password now deletes the account's sessions too. API tokens stay:
a reset that broke every script would be a reason not to reset, and
the owner lists and removes tokens in Settings. The command says how
many still work.

Also corrects the first-start log line from #321: there is no way to
change a password in the page, so it names the command alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:45:41 +00:00
d1149e1236 Requests that change something are refused from another site (#336)
Six POST routes take no body, so a cross-origin form or no-cors fetch
reached them with no preflight: Sign out, Read all (one feed's and
every feed's), Download, Refresh a listing and subscribing to a
Directory feed. The ipx cookies are SameSite=Lax, which stops a form
on an unrelated site but not one on a sibling subdomain, and the
Cloudflare Access path does not use ipx's cookies at all.

cross_site_refused wraps every route, login included: anything but
GET, HEAD and OPTIONS is a 403 when Sec-Fetch-Site is not same-origin
or none, or, from a browser too old to send it, when Origin is not the
Host. A request with neither header is not a browser's (a script with
a token, the healthcheck, k6) and goes through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:33:39 +00:00
46fe0dddbf directory.toml's fetch_internal lets the refresher reach listed addresses (#329)
3bfd8f8 gave the refresher the guard with no exceptions, which broke
the browser suite: its directory refreshes fixture feeds on 127.0.0.1,
and five Directory tests waited on listings that never filled. The
cargo tests passed because none of them fetches over HTTP.

directory.toml takes fetch_internal as config.toml does, empty by
default; the suite's lists 127.0.0.1. Production's has none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:33:30 +00:00
d35b8054b4 Bound the scan work one feed's author can ask for (#339 #337)
Both run on the daemon's single worker, so every other feed's check
and download waited behind them.

#339: prefer_https asked each new artwork host whether it served the
image on https, up to 20 s each, one after another, and the per-scan
cache is keyed by the authority, which ports and user names multiply
for one server. It now asks at most eight hosts a read; past that an
http image stays http, for /api/art to fetch, as one that fails does.

#337: a list (OPML or Patreon) may bring in 2,000 feeds; past that
sync_group refuses it whole, so the feed shows an error and nothing
changes. davewiner's, the largest seen, had 922. sync_group also read
the whole feeds table and rebuilt an id map with a Feed cloned per
entry for every new feed; it now reads the table once and checks ids
against sets, and unique_slug takes a predicate rather than a map.
same_feed skips parsing two URLs unless both are Patreon's, since only
those can differ and match.

sync_group and sync_opml lose the parent argument, which nothing used
once promote_kept stopped taking it (#332).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:18:32 +00:00
3bfd8f848b The directory's refresher refuses internal addresses (#329)
The refresher fetches every feed a snapshot names, and anyone with a
write token sends snapshots; a listed feed's redirect goes wherever its
publisher likes. Nothing checked the destination, so the directory
fetched loopback, its Docker network and the LAN, and what answered
became a listing's title and description on its public pages.

Its client now has ipx-feed's guard (#320) with nothing allowed: no
feed the directory lists is on this network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:10:32 +00:00
fc04a741a6 The directory's public pages read the list once a minute (#341)
/, /all, /search and /category are anyone's to ask for (#215), and each
called Store::list: the whole catalogue, joined with the snapshots and
grouped, on the 10-connection pool the refresher and every server's
/v1 calls share, on the Postgres that holds production ipx's database.
The pages' Cache-Control did nothing for a request with a query string
of its own, such as /search?q=<random>.

Directory::pages_list keeps the list for a minute behind a mutex held
through the read, so a burst of misses waits on one query rather than
each sending its own. Filtering and rendering still happen per request,
in memory. /v1/feeds, behind a token, still reads the database.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:06:19 +00:00
80f12cda70 A feed's credentials go to its own host alone (#327 #328)
feed::fetch already sent basic auth only to the feed's own host, but
two paths around it did not:

- download::download attached the feed's username and password to
  every enclosure, on whatever host the feed named: a CDN, or the
  measurement and ad-insertion prefixes podcasts route through (#328).
- subscriptions() and promote_kept() gave every feed an OPML or Patreon
  list derives the list's credentials, and fetch's same-host check then
  compared against the derived feed's own URL, so the list's password
  went to every host the OPML named, on every scan (#327).

config::Feed::auth_for and credentials_for compare hosts once, and
each of the three uses them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 14:02:10 +00:00
4f7dab9c04 A kept show keeps its list's private mark and credentials (#332)
promote_kept copied the list's settings from the current config, but
leave, rm and the directory cleanup remove the list from config before
calling retire_group, so the lookup found nothing: the promoted show
got private = false and no username or password, and the next snapshot
sent its URL to the directory. Only sync_group, which promotes while
the list is still there, kept them.

The callers now pass the list's settings as they were before removal.
Where they are no longer known (retire_stranded, for a list removed
long ago), the promoted show is private: a URL kept back by mistake
can be shared by hand, one shared by mistake cannot be taken back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:59:02 +00:00
cc3b8742bc The first admin gets a random password, not admin / ipodderx (#321)
A database with no accounts created admin with the hard-coded password
ipodderx, published in the README, the docs and the sample compose
file, and the log line announcing it printed the password. /api/login
is open to anyone who reaches the port, and nothing made anyone change
it, so a fresh install was an admin account anyone could sign into.

Now the password is IPX_ADMIN_PASSWORD when set, for scripted installs,
or 24 random hex characters written to admin-password beside
config.toml, created 0600 so it is never readable by others even for a
moment. The log says where it is, never what it is, as it does for the
web token (#40). Existing databases are untouched: seeding runs only
with no accounts at all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:55:34 +00:00
c99f481124 Update rustls-webpki, webpki-roots and ctutils for three advisories (#351)
cargo deny check failed on main:
- RUSTSEC-2026-0382, rustls-webpki 0.103.15: verification may accept
  distrusted intermediates. Now 0.103.16.
- RUSTSEC-2026-0381, webpki-roots 1.0.9: bundled roots trust distrusted
  emSign intermediates. Now 1.0.11. It surfaced once the first was fixed.
- RUSTSEC-2026-0385, ctutils 0.4.2: non-constant-time masks and u128
  comparisons. Now 0.4.5.

All transitive; Cargo.lock only. cargo deny check passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:43:09 +00:00
a20062d0f7 Fetches refuse internal addresses unless fetch_internal lists them (#320)
ipx's clients had no destination policy and followed redirects
anywhere. Adding a feed returned the status or first line of whatever
answered at the address; an enclosure declared as audio was downloaded
whole from any URL and served back through /media; /api/art fetched
any image a feed named. Feeds come from publishers and anyone signed
in, so loopback, the Docker network and the LAN were all reachable.

ipx-feed's guard is a connector layer: after the name resolves and the
connection opens, before the request is written, it looks at the
peer's address and refuses anything off the public internet unless
config.toml's [general] fetch_internal lists it. Being on the
connection, it covers every request on ctx.client and feed_client,
each redirect hop, IP literals (which never reach a DNS resolver) and
a name that resolves differently the second time. reqwest's rustls
connection reports the peer address; a test run by hand checks HTTPS
gets through, since were it lost every https fetch would be refused.

The directory is on the server's own network by design
(http://ipx-directory:8100), so it moves to local_client, unguarded,
set by an admin. The browser suite and load tests serve their feeds on
loopback and list 127.0.0.1.

Not covered: librqbit's own connections (trackers, peers), which do
not go through these clients.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:27:40 +00:00
6d71613e19 ipx fetches a .torrent itself, capped at 10 MB (#330)
Handed an http(s) URL, librqbit 9.0.1 fetched the .torrent with its own
reqwest client (no timeout) and response.bytes(), buffering the whole
body: a publisher could serve one without end, for up to the 30-minute
metadata budget, two torrents at a time, on a host with no swap.

Now ipx fetches it on ctx.client (connect and read timeouts), stops past
10 MB, holds it to the stall budget too so a trickle cannot keep a slot,
and gives librqbit the bytes. Magnet links go to librqbit as before.

Not fixed here: librqbit reads each HTTP tracker announce the same way,
and offers no setting to bound it short of disable_trackers, which would
break torrents that need their tracker. #330 stays open for that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:15:17 +00:00
f398211fd6 OPML nested past 100 levels is refused before it is parsed (#322)
The opml crate's Outline reader (hard-xml's derive) recurses once per
<outline> level with no limit. A 2 MB upload holds tens of thousands of
levels, enough to overflow a tokio worker's 2 MiB stack, and a stack
overflow aborts the process, not the task. A fetched OPML (up to 50 MB)
reached the same parse on every scan, so a publisher could keep the
daemon in a crash loop.

feed::read_opml counts nesting with quick-xml's streaming reader, which
does not recurse, and refuses past 100 before handing the text to the
opml crate. Every place that parsed OPML goes through it: the upload,
`ipx opml`, a fetched OPML's feeds and its title. The limit also bounds
collect_outlines' recursion and the drop of the nested Vec.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:11:21 +00:00
62a2456dde Downloads open in the browser only when they cannot run script (#319)
/media served a downloaded file with ServeFile, which types it by its
extension, and the file and its name are the publisher's. An enclosure
saved as .svg or .html, which download::sniff lets through unless it
starts <html or <!doctype html, opened from "View in a new tab" ran its
script on iPX's origin with the viewer's session.

Now every /media response is nosniff, anything but audio, video, a
non-SVG image, a PDF or plain text is sent as an attachment, and all
but PDFs get the sandbox CSP artwork already had. PDFs go without it
because Chrome refuses to render one in a sandboxed document; its
viewer does not run the PDF's script as the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:01:00 +00:00
ec3dedf113 Torrents download into a folder of their own (#318 #331)
The recorded path was output_folder.join(handle.name()). The name is the
publisher's: librqbit checks it for traversal only when it chooses the
subfolder itself, and ipx passes output_folder, so a multi-file torrent
named "/config" or ".." pointed Delete, the reaper and /media at that
path. With output_folder set, librqbit also puts a multi-file torrent's
files straight into it, not under the name, so the joined path was not
where the files were even for an honest torrent.

Each enclosure now gets torrent-<id> inside its feed's folder, and that
folder is the path: playable() already picks the largest file under a
folder and remove() deletes one (#158). The same folder keeps
overwrite: true from replacing another enclosure's files of the same
name. A torrent half-downloaded before this restarts in its new folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:55:35 +00:00
511246acb2 More step spans where flame graphs showed time with nothing under it (#350)
A week of traces folded with query.py flame left whole requests opaque:
entries_in at 46-82 ms a request with the page's query and its enclosures
together, /api/art 37 ms, /api/status 95 ms, PATCH /api/feeds/{id} 72 ms and
DELETE 45 ms. Each now has a step around what it waits on: the page and its
enclosures; the artwork cache, the check that a feed names it, the fetch and the
write; the status counts; and in changing or leaving a feed, each database call,
the wait for the catalogue's lock and storing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:45:58 +00:00
4af12363bc Read a WordPress.com site's posts from its API when the feed is behind a browser check (#317)
pelgranepress.com is on WordPress.com, whose CDN answers /feed/, /feed/atom/ and ?feed=rss2
with HTTP 403 and "Checking your browser...", a JavaScript challenge, whatever the User-Agent.
The checks that hit it failed for hours (8 in the day before this), and the publisher has
not changed anything.

The API at public-api.wordpress.com/wp/v2/sites/<host>/posts has the same posts with no check,
and each post's guid is the feed's own (https://host/?p=<id>), so the items read from it are
the ones already stored. fetch recognises the challenge by its /__challenge/ script, asks the
API instead, and parse reads WordPress's posts JSON as a feed. It sends no title or image for
the feed, which record_feed keeps as they were.

shortcut: the API's default ten posts, as many as WordPress puts in a feed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-23 12:06:22 +00:00
c8afcec73d Back to -dev after ipx 1.4.0, ipx-directory 1.3.0 and ipx-web 1.2.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:46:58 +00:00
5bad68670a Release ipx 1.4.0, ipx-directory 1.3.0 and ipx-web 1.2.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:40:00 +00:00
44afc08b9d Search treats % and _ as themselves (#296)
The changelog lines for #292, #293 and #295 ride along in this commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:12:58 +00:00
a21472730e A feed is read up to 50 MB (#295)
resp.bytes() read any size whole: an episode's MP3 given as a feed was
held in memory at every check. Feeds here are a few MB at most.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:12:55 +00:00
7af6153d18 A clean-up with the downloads folder gone forgets no file (#293)
missing_files took every path that did not exist() as deleted by hand.
With /downloads out of reach (the array stopped, shfs dropped) that is
every file, and each would have been marked reaped at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:12:52 +00:00
c1d8983ca1 Download addresses lose their query before they go out (#292)
Progress and DownloadDone went to every open page and the log with the
file's whole address; a Patreon file's carries its signature. Seen in
production's log for gcn-news-minute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:12:48 +00:00
b54b47897b Keep a feed out of Signals, and a feed's settings in named parts (#289, #290)
hide_from_signals on a subscription, on an OPML or creator covering every
show in it, as hide_from_all does for All Subscriptions: the judge does
not put its items to Jev, and each Signal's list and counts leave them
out. shown_in() is the one SQL rule for both.

The settings dialog had grown one captioned setting at a time into a
single column, one part boxed and the rest not. It is in named parts now,
the name in a column beside the controls and above them on a phone, ruled
apart; All Subscriptions and Signals carry the sidebar's icons. The
address, the same for everyone, moves to the foot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:37:49 +00:00
b89c371662 Keep a feed, an OPML or a Patreon creator out of All Subscriptions (#288)
A switch in the feed's settings, In All Subscriptions, on its
subscription as hide_from_all. All Subscriptions' list and count leave
out a feed hidden on its own or through the OPML it is in, and the page's
unread count does the same. Signals read every feed as before, and
Currently Listening, which is All Subscriptions' in-progress filter, keeps
an episode under way whatever the feed's setting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:19:54 +00:00
6daab6989e A list's trace says which list it was (#287)
The prod check of 2026-10-19 found /api/entries requests of 0.5-0.8 s
overnight beside others of 80-100 ms on an idle host, the time all in
entries_in, and nothing in the trace said which filter, sort or page
they were. The step now carries them, and whether a search, not its words.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:05:50 +00:00
d5ffe5d042 Switch an item between its HTML and its Markdown (#285)
A feed that sends <source:markdown> beside its HTML (Scripting News) has its
Markdown kept in entries.markdown, added to existing databases by an ALTER.
A stored item gets it when a scan finds it told differently, the Markdown
counting as a change, the way corrections arrive (#141). The API sends it as
written and made HTML, cleaned by ammonia like the description.

The item's page has a choice between its meta line and its buttons: HTML,
Markdown or Markdown source, remembered in the browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 18:39:16 +00:00
d16460267f Changelog: #284 reaches stored items too, on the feed's next whole read
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 18:33:32 +00:00
21c786f48a Artwork without a scan of entries, and the Directory's artwork let through (#272, #273)
/api/art asked whether a feed or item names the image before anything else,
cache hits included, and entries.image had no index: a sequential scan of
entries, 571 ms on production, 50 at once for a page of images. A kept image
is served first now, since it was only kept once let through, and entries.image
has an index: hash on Postgres, whose btree refuses keys over about 2.7 KB.

The Directory lists podcasts nobody here subscribes to, and their artwork was
refused, so the page drew initials. ipx remembers the images the directory's
listings and pages carry and lets those through too; any other address is
still refused, so /api/art cannot be used to fetch arbitrary URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 18:31:20 +00:00
3a8d564eb7 Items written in Markdown show formatted (#284)
A description with Markdown and no HTML tag is rendered to HTML with
pulldown-cmark, and an item with no text but RSS's <source:markdown>
(https://source.scripting.com/, as Scripting News sends beside its HTML) uses
that. HTML is left alone, and plain text without Markdown in it stays as it
was, so a line like "1984. ..." is not made a list. ammonia cleans the result
as any feed's HTML. Stored items keep what they have: the scanner does not
rewrite an item it has.

Reference: http://scripting.com/2026/10/06/005559.html

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 18:05:05 +00:00