ipx fetches a .torrent itself, capped at 10 MB (#330)
Handed an http(s) URL, librqbit 9.0.1 fetched the .torrent with its own reqwest client (no timeout) and response.bytes(), buffering the whole body: a publisher could serve one without end, for up to the 30-minute metadata budget, two torrents at a time, on a host with no swap. Now ipx fetches it on ctx.client (connect and read timeouts), stops past 10 MB, holds it to the stall budget too so a trickle cannot keep a slot, and gives librqbit the bytes. Magnet links go to librqbit as before. Not fixed here: librqbit reads each HTTP tracker announce the same way, and offers no setting to bound it short of disable_trackers, which would break torrents that need their tracker. #330 stays open for that. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -25,6 +25,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
- An episode that comes as a torrent downloads into a folder of its own, `torrent-<id>` in its podcast's folder. A torrent could name a path anywhere on the server for Delete, the cleanup and playback to act on, and could overwrite other episodes' files of the same name (#318, #331).
|
- An episode that comes as a torrent downloads into a folder of its own, `torrent-<id>` in its podcast's folder. A torrent could name a path anywhere on the server for Delete, the cleanup and playback to act on, and could overwrite other episodes' files of the same name (#318, #331).
|
||||||
- A downloaded file opens in the browser only when it is audio, video, an image other than SVG, a PDF or plain text, and is saved otherwise. An SVG or web page a podcast sent ran its script as iPX, signed in as whoever opened it (#319).
|
- A downloaded file opens in the browser only when it is audio, video, an image other than SVG, a PDF or plain text, and is saved otherwise. An SVG or web page a podcast sent ran its script as iPX, signed in as whoever opened it (#319).
|
||||||
- An OPML nested more than 100 levels deep is refused, uploaded or fetched. A deep enough one stopped iPX, and a subscribed one stopped it again each time it restarted (#322).
|
- An OPML nested more than 100 levels deep is refused, uploaded or fetched. A deep enough one stopped iPX, and a subscribed one stopped it again each time it restarted (#322).
|
||||||
|
- A `.torrent` file larger than 10 MB, or slower than the torrent stall limit, is refused. iPX read one into memory however large it was (#330).
|
||||||
|
|
||||||
## [1.4.0] - 2026-10-19
|
## [1.4.0] - 2026-10-19
|
||||||
|
|
||||||
|
|||||||
@@ -2401,7 +2401,7 @@ async fn torrent_one(
|
|||||||
let cfg = ctx.cfg();
|
let cfg = ctx.cfg();
|
||||||
ctx.torrents()
|
ctx.torrents()
|
||||||
.await?
|
.await?
|
||||||
.fetch(&cfg, url, &torrent::own_folder(dest_dir, enclosure), |done, total| {
|
.fetch(&cfg, &ctx.client, url, &torrent::own_folder(dest_dir, enclosure), |done, total| {
|
||||||
if let Some(pct) = (done * 100).checked_div(total).map(|p| p as i64)
|
if let Some(pct) = (done * 100).checked_div(total).map(|p| p as i64)
|
||||||
&& pct > last_pct
|
&& pct > last_pct
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -47,6 +47,7 @@ impl Torrents {
|
|||||||
pub async fn fetch(
|
pub async fn fetch(
|
||||||
&self,
|
&self,
|
||||||
cfg: &Config,
|
cfg: &Config,
|
||||||
|
client: &reqwest::Client,
|
||||||
url: &str,
|
url: &str,
|
||||||
dest_dir: &Path,
|
dest_dir: &Path,
|
||||||
mut on_progress: impl FnMut(u64, u64),
|
mut on_progress: impl FnMut(u64, u64),
|
||||||
@@ -54,6 +55,15 @@ impl Torrents {
|
|||||||
tokio::fs::create_dir_all(dest_dir).await?;
|
tokio::fs::create_dir_all(dest_dir).await?;
|
||||||
|
|
||||||
let limit = stall_limit(cfg);
|
let limit = stall_limit(cfg);
|
||||||
|
let source = if url.starts_with("http://") || url.starts_with("https://") {
|
||||||
|
// Under the stall budget as well: a body sent a byte a minute stays inside the cap.
|
||||||
|
let file = tokio::time::timeout(limit, torrent_file(client, url))
|
||||||
|
.await
|
||||||
|
.map_err(|_| anyhow::anyhow!("no .torrent after {} minutes, gave up", limit.as_secs() / 60))??;
|
||||||
|
AddTorrent::from_bytes(file)
|
||||||
|
} else {
|
||||||
|
AddTorrent::from_url(url)
|
||||||
|
};
|
||||||
|
|
||||||
// Resolving a magnet's metadata happens inside add_torrent, and against a dead
|
// Resolving a magnet's metadata happens inside add_torrent, and against a dead
|
||||||
// swarm it never returns. The stall budget has to cover this phase too, or a
|
// swarm it never returns. The stall budget has to cover this phase too, or a
|
||||||
@@ -61,7 +71,7 @@ impl Torrents {
|
|||||||
let added = tokio::time::timeout(
|
let added = tokio::time::timeout(
|
||||||
limit,
|
limit,
|
||||||
self.session.add_torrent(
|
self.session.add_torrent(
|
||||||
AddTorrent::from_url(url),
|
source,
|
||||||
Some(AddTorrentOptions {
|
Some(AddTorrentOptions {
|
||||||
output_folder: Some(dest_dir.to_string_lossy().into_owned()),
|
output_folder: Some(dest_dir.to_string_lossy().into_owned()),
|
||||||
overwrite: true,
|
overwrite: true,
|
||||||
@@ -137,6 +147,23 @@ impl Torrents {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A .torrent file is metadata: a few hundred KB even for a large torrent.
|
||||||
|
const MAX_TORRENT_FILE: usize = 10 << 20;
|
||||||
|
|
||||||
|
/// Fetches a .torrent file, up to `MAX_TORRENT_FILE`. Handed the URL, librqbit read the whole body
|
||||||
|
/// into memory with no limit and no timeout, so a feed could serve one without end (#330).
|
||||||
|
async fn torrent_file(client: &reqwest::Client, url: &str) -> Result<Vec<u8>> {
|
||||||
|
let mut resp = client.get(url).send().await.context("fetching the .torrent")?.error_for_status()?;
|
||||||
|
let mut out = Vec::new();
|
||||||
|
while let Some(c) = resp.chunk().await.context("reading the .torrent")? {
|
||||||
|
out.extend_from_slice(&c);
|
||||||
|
if out.len() > MAX_TORRENT_FILE {
|
||||||
|
bail!("the .torrent is larger than {} MB, too large to be one", MAX_TORRENT_FILE >> 20);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
/// The folder an enclosure's torrent downloads into, one per enclosure inside the feed's folder.
|
/// The folder an enclosure's torrent downloads into, one per enclosure inside the feed's folder.
|
||||||
/// With `overwrite` on, a torrent written straight into the feed's folder could replace another
|
/// With `overwrite` on, a torrent written straight into the feed's folder could replace another
|
||||||
/// enclosure's files of the same name, another feed's too when they share a folder (#331).
|
/// enclosure's files of the same name, another feed's too when they share a folder (#331).
|
||||||
@@ -164,6 +191,23 @@ mod tests {
|
|||||||
assert!(ratio(0, 0).is_infinite(), "must not divide by zero or seed forever");
|
assert!(ratio(0, 0).is_infinite(), "must not divide by zero or seed forever");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn an_endless_torrent_file_stops_at_the_cap() {
|
||||||
|
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||||
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||||
|
let addr = listener.local_addr().unwrap();
|
||||||
|
tokio::spawn(async move {
|
||||||
|
let Ok((mut sock, _)) = listener.accept().await else { return };
|
||||||
|
let _ = sock.read(&mut [0u8; 2048]).await;
|
||||||
|
// No length, no end: a body that goes on until the reader stops.
|
||||||
|
let _ = sock.write_all(b"HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n").await;
|
||||||
|
let chunk = vec![b'd'; 64 << 10];
|
||||||
|
while sock.write_all(&chunk).await.is_ok() {}
|
||||||
|
});
|
||||||
|
let err = torrent_file(&reqwest::Client::new(), &format!("http://{addr}/ep.torrent")).await.unwrap_err();
|
||||||
|
assert!(err.to_string().contains("too large"), "{err:#}");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn own_folder_is_one_folder_inside_the_feeds() {
|
fn own_folder_is_one_folder_inside_the_feeds() {
|
||||||
let feed = Path::new("/downloads/Show");
|
let feed = Path::new("/downloads/Show");
|
||||||
|
|||||||
Reference in New Issue
Block a user