Traces keep feed and file addresses without their query or password (#225)

The fetch span and the download span recorded the whole address, so a private feed's
credential (Patreon's ?auth=) went to Tempo with every scan, readable by anyone with
Grafana. feed::redact drops the query, fragment and user:password; a token in the path
would still go, noted where it is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-13 17:54:08 +00:00
parent 45c9f9252b
commit 382f51a480
4 changed files with 35 additions and 2 deletions

View File

@@ -14,6 +14,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- With `metrics_bind` set, the directory serves Prometheus metrics on a port of its own: memory, open files, and the feeds it has fetched and their bytes. A feed check's trace says how large the feed was (#224). - With `metrics_bind` set, the directory serves Prometheus metrics on a port of its own: memory, open files, and the feeds it has fetched and their bytes. A feed check's trace says how large the feed was (#224).
### Security
- A feed check's trace keeps the feed's address without its query or password (#225).
### Changed ### Changed
- The directory uses mimalloc instead of glibc's allocator, so it gives back memory it has freed instead of holding over a gigabyte while idle (#224). - The directory uses mimalloc instead of glibc's allocator, so it gives back memory it has freed instead of holding over a gigabyte while idle (#224).

View File

@@ -73,7 +73,24 @@ pub static FETCHED_BYTES: std::sync::atomic::AtomicU64 = std::sync::atomic::Atom
/// than be redirected on every read. A temporary redirect (302, 307) moves nothing. /// than be redirected on every read. A temporary redirect (302, 307) moves nothing.
/// `system-proxy` reads http_proxy/https_proxy. /// `system-proxy` reads http_proxy/https_proxy.
/// `auth` is a username and password, sent only to the feed's own host. /// `auth` is a username and password, sent only to the feed's own host.
#[tracing::instrument(skip_all, fields(url = %feed_url, http.response.body.size))] /// An address as a trace may keep it: without its query or user and password, where a private
/// feed carries its credential (Patreon's `?auth=`), which Tempo kept for anyone reading it (#225).
/// shortcut: a token in the path (`/feed/<token>.xml`) stays; strip paths too if a feed does that.
pub fn redact(url: &str) -> String {
match url::Url::parse(url) {
Ok(mut u) => {
u.set_query(None);
u.set_fragment(None);
let _ = u.set_username("");
let _ = u.set_password(None);
u.to_string()
}
// Not an address ipx could fetch; keep what comes before anything that could be a query.
Err(_) => url.split(['?', '#', '@']).next().unwrap_or_default().to_owned(),
}
}
#[tracing::instrument(skip_all, fields(url = %redact(feed_url), http.response.body.size))]
pub async fn fetch( pub async fn fetch(
client: &reqwest::Client, client: &reqwest::Client,
feed_url: &str, feed_url: &str,
@@ -1173,6 +1190,14 @@ const PAID_HOSTS: &[&str] =
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
#[test]
fn redact_drops_what_carries_a_credential() {
assert_eq!(super::redact("https://www.patreon.com/rss/x?auth=SECRET&show=1"), "https://www.patreon.com/rss/x");
assert_eq!(super::redact("https://me:pw@example.com/feed.xml#top"), "https://example.com/feed.xml");
assert_eq!(super::redact("https://example.com/feed.xml"), "https://example.com/feed.xml");
assert_eq!(super::redact("not a url?token=SECRET"), "not a url");
}
#[test] #[test]
fn an_address_carrying_someones_access_looks_private() { fn an_address_carrying_someones_access_looks_private() {
let private = super::url_looks_private; let private = super::url_looks_private;

View File

@@ -16,6 +16,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `/metrics` counts the feeds fetched and the bytes of their bodies (`ipx_feed_fetches_total`, `ipx_feed_bytes_total`), and a feed's trace says how large its body was (#224). - `/metrics` counts the feeds fetched and the bytes of their bodies (`ipx_feed_fetches_total`, `ipx_feed_bytes_total`), and a feed's trace says how large its body was (#224).
### Security
- Traces no longer keep a feed's or file's address with its query or password, where private feeds carry their credentials, such as Patreon's `auth` (#225).
### Changed ### Changed
- ipx uses mimalloc instead of glibc's allocator, so memory a night of scans needed goes back to the system instead of staying held until a restart (#224). - ipx uses mimalloc instead of glibc's allocator, so memory a night of scans needed goes back to the system instead of staying held until a restart (#224).

View File

@@ -2139,7 +2139,7 @@ fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Pol
/// Downloads one file and records it. None when it turned out to be a torrent, handed off to run /// Downloads one file and records it. None when it turned out to be a torrent, handed off to run
/// on its own, which records it when it finishes. /// on its own, which records it when it finishes.
#[tracing::instrument(name = "download", skip_all, fields(feed = feed_id, enclosure = enclosure, url = url))] #[tracing::instrument(name = "download", skip_all, fields(feed = feed_id, enclosure = enclosure, url = %feed::redact(url)))]
async fn fetch_one( async fn fetch_one(
ctx: &Arc<Ctx>, ctx: &Arc<Ctx>,
feed_id: &str, feed_id: &str,