ipx fetches a .torrent itself, capped at 10 MB (#330)

Handed an http(s) URL, librqbit 9.0.1 fetched the .torrent with its own
reqwest client (no timeout) and response.bytes(), buffering the whole
body: a publisher could serve one without end, for up to the 30-minute
metadata budget, two torrents at a time, on a host with no swap.

Now ipx fetches it on ctx.client (connect and read timeouts), stops past
10 MB, holds it to the stall budget too so a trickle cannot keep a slot,
and gives librqbit the bytes. Magnet links go to librqbit as before.

Not fixed here: librqbit reads each HTTP tracker announce the same way,
and offers no setting to bound it short of disable_trackers, which would
break torrents that need their tracker. #330 stays open for that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-23 13:15:17 +00:00
parent f398211fd6
commit 6d71613e19
3 changed files with 47 additions and 2 deletions

View File

@@ -25,6 +25,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- An episode that comes as a torrent downloads into a folder of its own, `torrent-<id>` in its podcast's folder. A torrent could name a path anywhere on the server for Delete, the cleanup and playback to act on, and could overwrite other episodes' files of the same name (#318, #331). - An episode that comes as a torrent downloads into a folder of its own, `torrent-<id>` in its podcast's folder. A torrent could name a path anywhere on the server for Delete, the cleanup and playback to act on, and could overwrite other episodes' files of the same name (#318, #331).
- A downloaded file opens in the browser only when it is audio, video, an image other than SVG, a PDF or plain text, and is saved otherwise. An SVG or web page a podcast sent ran its script as iPX, signed in as whoever opened it (#319). - A downloaded file opens in the browser only when it is audio, video, an image other than SVG, a PDF or plain text, and is saved otherwise. An SVG or web page a podcast sent ran its script as iPX, signed in as whoever opened it (#319).
- An OPML nested more than 100 levels deep is refused, uploaded or fetched. A deep enough one stopped iPX, and a subscribed one stopped it again each time it restarted (#322). - An OPML nested more than 100 levels deep is refused, uploaded or fetched. A deep enough one stopped iPX, and a subscribed one stopped it again each time it restarted (#322).
- A `.torrent` file larger than 10 MB, or slower than the torrent stall limit, is refused. iPX read one into memory however large it was (#330).
## [1.4.0] - 2026-10-19 ## [1.4.0] - 2026-10-19

View File

@@ -2401,7 +2401,7 @@ async fn torrent_one(
let cfg = ctx.cfg(); let cfg = ctx.cfg();
ctx.torrents() ctx.torrents()
.await? .await?
.fetch(&cfg, url, &torrent::own_folder(dest_dir, enclosure), |done, total| { .fetch(&cfg, &ctx.client, url, &torrent::own_folder(dest_dir, enclosure), |done, total| {
if let Some(pct) = (done * 100).checked_div(total).map(|p| p as i64) if let Some(pct) = (done * 100).checked_div(total).map(|p| p as i64)
&& pct > last_pct && pct > last_pct
{ {

View File

@@ -47,6 +47,7 @@ impl Torrents {
pub async fn fetch( pub async fn fetch(
&self, &self,
cfg: &Config, cfg: &Config,
client: &reqwest::Client,
url: &str, url: &str,
dest_dir: &Path, dest_dir: &Path,
mut on_progress: impl FnMut(u64, u64), mut on_progress: impl FnMut(u64, u64),
@@ -54,6 +55,15 @@ impl Torrents {
tokio::fs::create_dir_all(dest_dir).await?; tokio::fs::create_dir_all(dest_dir).await?;
let limit = stall_limit(cfg); let limit = stall_limit(cfg);
let source = if url.starts_with("http://") || url.starts_with("https://") {
// Under the stall budget as well: a body sent a byte a minute stays inside the cap.
let file = tokio::time::timeout(limit, torrent_file(client, url))
.await
.map_err(|_| anyhow::anyhow!("no .torrent after {} minutes, gave up", limit.as_secs() / 60))??;
AddTorrent::from_bytes(file)
} else {
AddTorrent::from_url(url)
};
// Resolving a magnet's metadata happens inside add_torrent, and against a dead // Resolving a magnet's metadata happens inside add_torrent, and against a dead
// swarm it never returns. The stall budget has to cover this phase too, or a // swarm it never returns. The stall budget has to cover this phase too, or a
@@ -61,7 +71,7 @@ impl Torrents {
let added = tokio::time::timeout( let added = tokio::time::timeout(
limit, limit,
self.session.add_torrent( self.session.add_torrent(
AddTorrent::from_url(url), source,
Some(AddTorrentOptions { Some(AddTorrentOptions {
output_folder: Some(dest_dir.to_string_lossy().into_owned()), output_folder: Some(dest_dir.to_string_lossy().into_owned()),
overwrite: true, overwrite: true,
@@ -137,6 +147,23 @@ impl Torrents {
} }
} }
/// A .torrent file is metadata: a few hundred KB even for a large torrent.
const MAX_TORRENT_FILE: usize = 10 << 20;
/// Fetches a .torrent file, up to `MAX_TORRENT_FILE`. Handed the URL, librqbit read the whole body
/// into memory with no limit and no timeout, so a feed could serve one without end (#330).
async fn torrent_file(client: &reqwest::Client, url: &str) -> Result<Vec<u8>> {
let mut resp = client.get(url).send().await.context("fetching the .torrent")?.error_for_status()?;
let mut out = Vec::new();
while let Some(c) = resp.chunk().await.context("reading the .torrent")? {
out.extend_from_slice(&c);
if out.len() > MAX_TORRENT_FILE {
bail!("the .torrent is larger than {} MB, too large to be one", MAX_TORRENT_FILE >> 20);
}
}
Ok(out)
}
/// The folder an enclosure's torrent downloads into, one per enclosure inside the feed's folder. /// The folder an enclosure's torrent downloads into, one per enclosure inside the feed's folder.
/// With `overwrite` on, a torrent written straight into the feed's folder could replace another /// With `overwrite` on, a torrent written straight into the feed's folder could replace another
/// enclosure's files of the same name, another feed's too when they share a folder (#331). /// enclosure's files of the same name, another feed's too when they share a folder (#331).
@@ -164,6 +191,23 @@ mod tests {
assert!(ratio(0, 0).is_infinite(), "must not divide by zero or seed forever"); assert!(ratio(0, 0).is_infinite(), "must not divide by zero or seed forever");
} }
#[tokio::test]
async fn an_endless_torrent_file_stops_at_the_cap() {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let Ok((mut sock, _)) = listener.accept().await else { return };
let _ = sock.read(&mut [0u8; 2048]).await;
// No length, no end: a body that goes on until the reader stops.
let _ = sock.write_all(b"HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n").await;
let chunk = vec![b'd'; 64 << 10];
while sock.write_all(&chunk).await.is_ok() {}
});
let err = torrent_file(&reqwest::Client::new(), &format!("http://{addr}/ep.torrent")).await.unwrap_err();
assert!(err.to_string().contains("too large"), "{err:#}");
}
#[test] #[test]
fn own_folder_is_one_folder_inside_the_feeds() { fn own_folder_is_one_folder_inside_the_feeds() {
let feed = Path::new("/downloads/Show"); let feed = Path::new("/downloads/Show");