Commit Graph

3 Commits

Author SHA1 Message Date
692a510217 Format the Rust code with rustfmt
The code was formatted by hand, and `cargo fmt --check` differed in 866
places. rustfmt.toml keeps the house's long lines (120) and short
expressions on one line, which halves what changes; this commit is only
what rustfmt does with it, and is named in .git-blame-ignore-revs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 21:55:41 +00:00
1949f3e493 Compression across the board: directory answers, uploads, metrics, traces, the page (#253)
Measured 2026-10-16, uncompressed until now: the directory's answers (the feed list ipx reads,
603 KB of JSON, ~134 ms of a 159 ms Directory request), ipx's snapshot upload (~100 KB each half
hour), both /metrics (4.7 and 0.9 KB every 15 s) and every trace export to Tempo. Already
compressed: ipx-web's answers (zstd, #239), the home page and the directory's public pages
through Cloudflare (brotli, zstd).

- The directory compresses with brotli, zstd or gzip as asked, and unpacks a gzipped request.
- ipx sends its snapshot gzipped; the directory is deployed first.
- Both /metrics answer gzipped, as Prometheus asks.
- opentelemetry-otlp's gzip-http: OTEL_EXPORTER_OTLP_COMPRESSION=gzip compresses the export, set
  for all three in production's compose file; Caddy's exporter reads the same variable.
- web/build.mjs writes .br (quality 11) and .gz (level 9) beside each text file, and Caddy serves
  them in place (file_server precompressed): app.js 30 KB where zstd on the fly gave 37, app.css
  11 where 14. A .br or .gz asked for by name is refused, or /admin.js.br would be the admin
  page's script without the check that keeps it to admins; in a handle of its own, since a bare
  respond runs after the handles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 14:02:39 +00:00
45c9f9252b mimalloc for ipx and the directory, and metrics and traces for memory (#224)
ipx's resident memory climbed from about 40 MB to 500-850 MB each night while scans ran five
times as often, and stayed there until a restart; the directory sat at 1.25 GiB idle. glibc's
allocator keeps what each thread's arena freed. mimalloc gives freed pages back, so resident
memory now follows what is in use. jemalloc was the first choice for its in-use figures, but its
upstream was archived in 2025.

Declared once in the directory crate, which ipx builds in, so both binaries get it.

What was missing to see it: the directory served no /metrics (now on metrics_bind, a port apart
from the tunnel's), nothing counted the feed bytes fetched, and the fetch span had no body size.
The feed crate's spans never reached Tempo either: both trace exporters filtered to their own
crate, so ipx_feed's fetch and parse spans were dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 17:15:01 +00:00