Compare commits
18 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 5f6bdbfbcb | |||
| 8849ba6bef | |||
| 1cafd8d6e3 | |||
| f09bb4a11c | |||
| 973ebdd33a | |||
| bc7491a377 | |||
| c8df148546 | |||
| c1c06229c8 | |||
| bc53e0f730 | |||
| bd8f6ab855 | |||
| 611716d8b7 | |||
| a68bfb179b | |||
| 6bf1ad6b31 | |||
| d7f8f2df1d | |||
| 4927677e66 | |||
| 484aaa1849 | |||
| c99e17bd80 | |||
| e312f11bb1 |
23
CHANGELOG.md
23
CHANGELOG.md
@@ -5,11 +5,27 @@ All notable changes to this project will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
The long form, with what was wrong before and how it was found, is in
|
||||
[docs/history.md](docs/history.md).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.8.0] - 2026-09-18
|
||||
|
||||
### Added
|
||||
|
||||
- ipx can keep its data in Postgres: set `IPX_DATABASE_URL` to a `postgres://` URL. Without it,
|
||||
it is the SQLite `state.db` as before. `ipx copy-db <state.db>` moves an existing database
|
||||
across, everything in one go.
|
||||
- The catalogue of feeds and the server settings the admin page edits are kept in the database
|
||||
rather than config.toml, which keeps where things are, the torrent settings and who may sign
|
||||
in. The first start takes them from config.toml and trims it, keeping the original as
|
||||
`config.toml.pre-database`; feeds added to config.toml after that are ignored, with a warning.
|
||||
|
||||
### Changed
|
||||
|
||||
- The database is reached through SeaORM, which is what lets it be SQLite or Postgres; on SQLite
|
||||
nothing you see changes. On Postgres, sorting by title or feed follows the language's order (an
|
||||
accented letter beside the plain one) rather than raw bytes. A database from before 0.7 has to
|
||||
be opened by a 0.7 release first, which brings its tables up to date.
|
||||
|
||||
## [0.7.0] - 2026-09-18
|
||||
|
||||
### Added
|
||||
@@ -487,6 +503,7 @@ The long form, with what was wrong before and how it was found, is in
|
||||
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
|
||||
|
||||
[unreleased]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.6.1...main
|
||||
[0.8.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.7.0...v0.8.0
|
||||
[0.7.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.6.1...v0.7.0
|
||||
[0.6.1]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.6.0...v0.6.1
|
||||
[0.6.0]: https://git.sdf1.net/rays/ipodderx-rs/compare/v0.5.5...v0.6.0
|
||||
|
||||
39
CLAUDE.md
39
CLAUDE.md
@@ -13,8 +13,9 @@ Arcane project `content`: `/mnt/fast/arcane/projects/content/compose.yaml`. That
|
||||
| | Host | In the container |
|
||||
|---|---|---|
|
||||
| Image | `192.168.1.130:5000/ipodderx:latest` | |
|
||||
| Config | `/mnt/fast/appdata/ipodderx/config.toml` | `/config/config.toml` |
|
||||
| Database | `/mnt/user/ipodderx/state.db` | `/data/state.db` |
|
||||
| Config | `/mnt/fast/appdata/ipodderx/config.toml`: bind address, token, trusted proxies, torrent, paths. The feeds and server settings are in the database | `/config/config.toml` |
|
||||
| Database | Postgres 18, database `ipodderx`, login `ipodderx`, on the `postgres` container of the Arcane project `databases` (`192.168.1.130:5433`). The URL is in `ipodderx.env` beside the compose file (`/mnt/fast/arcane/projects/content/ipodderx.env`, mode 600), passed to the container as `IPX_DATABASE_URL`. A relative `env_file`: Arcane runs compose in its own container, where `/mnt/fast/appdata` does not exist | |
|
||||
| Old database | `/mnt/user/ipodderx/state.db`, SQLite, used until the move to Postgres on 2026-09-18 and kept for rollback | `/data/state.db` |
|
||||
| Downloads | `/mnt/user/ipodderx/downloads` | `/downloads` |
|
||||
| Web UI | `192.168.1.130:8099`, also `ipodderx.sdf1.net` via a Cloudflare tunnel | `0.0.0.0:8099` |
|
||||
| Sign-in via the tunnel | Cloudflare Access app `ipodderx`, with Authentik as its identity provider; see [docs/sso.md](docs/sso.md) | trusts `Cf-Access-Authenticated-User-Email` from `192.168.16.1`, the `content_default` gateway |
|
||||
@@ -125,10 +126,18 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
|
||||
subscriber. Two feeds publishing the same URL means only the first one scanned shows it.
|
||||
* **Read state lives in `entry_state`, per user, and nowhere else.** `entries` had `read`, `flagged`
|
||||
and `position` columns from before accounts; two bugs came from queries still reading them
|
||||
(retention, and the entry pruner), and `migrate()` now drops them.
|
||||
* **The catalogue is config.toml; the subscriptions are in the database.** A feed exists once;
|
||||
(retention, and the entry pruner), and they were dropped in 0.5.
|
||||
* **The catalogue and the server settings are in the database, not config.toml** (issue #18):
|
||||
tables `catalogue` (each feed's `config::Feed` as JSON) and `settings` (`general`:
|
||||
`config::Stored`). ipx still runs from one in-memory `Config`, config.toml for where things are
|
||||
and who gets in, the database for the rest (`assemble_config`); a change goes through
|
||||
`Ctx::store_cfg`, never a write to the file. The first start on a database without them imports
|
||||
config.toml's and trims the file, keeping `config.toml.pre-database`. A feed exists once;
|
||||
`subscriptions(user_id, feed_id)` says who wants it and with what settings. OPML children are
|
||||
derived and never written to config.
|
||||
derived and never in the catalogue.
|
||||
* **Postgres connections ask for no notices** (`client_min_messages=warning`, `db::url_for`).
|
||||
Postgres sends one for every `CREATE ... IF NOT EXISTS` on something existing, sqlx logs each,
|
||||
and tracing-subscriber's per-layer filters then dropped the next line ipx logged.
|
||||
* **One fetch serves everyone**, so scan policy is a union of subscribers' wants (`merge_policy`).
|
||||
Anyone wanting an item is enough to fetch it.
|
||||
* **The UI hiding a control is not enforcement.** Admin-only actions check `user.is_admin` in the
|
||||
@@ -140,10 +149,16 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
|
||||
* `/api/settings` answering `200` does **not** mean the daemon is well — the web server is a
|
||||
different task. `ipx status` checks the control socket and the database; to see the worker
|
||||
getting through its jobs, watch for `scan complete` in the log.
|
||||
* **Every `ipx` command runs `migrate()` when it opens the database**, the healthcheck's
|
||||
`ipx status` included. A migration that rewrites a big table (`DROP COLUMN`) takes seconds on
|
||||
production, and a command run meanwhile fails with `migrating schema`. It changes nothing; wait
|
||||
for `daemon started` in the log. Copy `state.db` aside before deploying one.
|
||||
* **The database goes through SeaORM, and the entities in `src/entity.rs` are the schema.**
|
||||
`Db::open` creates any missing table or index from them (`create_missing`), on every `ipx`
|
||||
command, the healthcheck's `ipx status` included, so it must never write when nothing is
|
||||
missing: SeaORM's experimental schema sync dropped and remade an index on every open, the
|
||||
write lock that took made `ipx status` time out behind a busy daemon, and it was removed for
|
||||
it. A new column on an existing table needs its own `ALTER`; nothing adds one for you.
|
||||
* **SQL written by hand in `db.rs` has to run on SQLite and Postgres both** (issue #18): `$1`
|
||||
parameters, bound only if used; `ON CONFLICT`, not `INSERT OR IGNORE`; yes/no columns tested
|
||||
as themselves (`NOT coalesce(s.read, false)`) and written as `true`/`false`, never compared to
|
||||
1; no `rowid`, `GLOB` or `UPDATE OR IGNORE`. `Args` in `db.rs` builds the parameters.
|
||||
|
||||
## House style
|
||||
|
||||
@@ -154,9 +169,9 @@ addressed to the person using it.
|
||||
Every change gets one line under `## [Unreleased]` in [CHANGELOG.md](CHANGELOG.md), in its
|
||||
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) group: Added, Changed, Deprecated,
|
||||
Removed, Fixed or Security. Say it the way someone using ipx would notice it. When there is more to
|
||||
say, such as what was wrong before or what it cost to find out, write it up at the top of
|
||||
[docs/history.md](docs/history.md), dated. That record has been more useful than the git log more
|
||||
than once.
|
||||
say, such as what was wrong before or what it cost to find out, it goes in the commit message's
|
||||
body, where `git log` and `git blame` find it beside the change. (There was a long-form
|
||||
`docs/history.md` until 0.7.0; it grew too large to be useful and was removed. It is in git.)
|
||||
|
||||
Cutting a release: rename `[Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD` and open a new empty
|
||||
`[Unreleased]` above it, bump `version` in `Cargo.toml`, tag the commit `vX.Y.Z`, and update the
|
||||
|
||||
875
Cargo.lock
generated
875
Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "ipx"
|
||||
version = "0.7.0"
|
||||
version = "0.8.0"
|
||||
edition = "2024"
|
||||
|
||||
[dependencies]
|
||||
@@ -18,7 +18,7 @@ percent-encoding = "2.3.2"
|
||||
quick-xml = { version = "0.42.0", features = ["escape-html"] }
|
||||
reqwest = { version = "0.13.5", default-features = false, features = ["rustls", "http2", "gzip", "stream", "json", "charset", "system-proxy"] }
|
||||
rss = "2.1.1"
|
||||
rusqlite = { version = "0.40.2", features = ["bundled"] }
|
||||
sea-orm = { version = "2.0.3", default-features = false, features = ["sqlx-sqlite", "sqlx-postgres", "runtime-tokio-rustls", "macros", "with-json", "sqlite-use-returning-for-3_35"] }
|
||||
serde = { version = "1.0.229", features = ["derive"] }
|
||||
serde_json = "1.0.151"
|
||||
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
|
||||
|
||||
@@ -63,7 +63,6 @@ The UI is plain HTTP, so put TLS in front of it if it is reachable from outside
|
||||
| [docs/sso.md](docs/sso.md) | Signing in through Cloudflare Zero Trust or Authentik |
|
||||
| [docs/architecture.md](docs/architecture.md) | How it works: modules, schema, control socket, HTTP API |
|
||||
| [CHANGELOG.md](CHANGELOG.md) | What changed, by release |
|
||||
| [docs/history.md](docs/history.md) | How it was built, with what was wrong and why |
|
||||
| [CLAUDE.md](CLAUDE.md) | Notes for working on the code, including how production is deployed |
|
||||
|
||||
## Tests
|
||||
|
||||
@@ -8,6 +8,9 @@ services:
|
||||
PGID: "100"
|
||||
TZ: "America/Toronto"
|
||||
IPX_LOG: "ipx=info"
|
||||
# IPX_DATABASE_URL=postgres://... to use Postgres; without it, /data/state.db (SQLite).
|
||||
env_file:
|
||||
- ipodderx.env # relative: Arcane resolves it inside its own container
|
||||
ports:
|
||||
- "8099:8099" # web UI
|
||||
- "6881:6881/tcp" # BitTorrent peers
|
||||
|
||||
@@ -10,7 +10,8 @@ it to a running daemon.
|
||||
|---|---|---|
|
||||
| `src/main.rs` | CLI, dispatch, scan loop, download policy | `iPXAgent.py` |
|
||||
| `src/config.rs` | TOML load/save, `General`/`Feed`/`Web`, intervals, slugs | `iPXSettings.py`, `feeds.plist` |
|
||||
| `src/db.rs` | SQLite schema, migrations, every query | `.ipxd` plists, `history.dat`, `qmcache.dat` |
|
||||
| `src/db.rs` | Every query, through SeaORM; creates missing tables | `.ipxd` plists, `history.dat`, `qmcache.dat` |
|
||||
| `src/entity.rs` | The tables, as SeaORM entities: the schema | — |
|
||||
| `src/feed.rs` | Conditional GET, RSS/Atom/OPML parsing | `FeedData.__getFeed/__getEntries` |
|
||||
| `src/download.rs` | Streaming download, naming, type sniffing, placement | `iPXDownloader.getFile` |
|
||||
| `src/torrent.rs` | librqbit session, seeding limits, stall abort | vendored BitTorrent 4.2.1 |
|
||||
@@ -65,14 +66,14 @@ entry_state user_id, feed_id, guid, read, flagged, position
|
||||
```
|
||||
|
||||
Read state is `entry_state` alone. `entries` had `read`, `flagged` and `position` columns from
|
||||
before accounts; two bugs came from queries still reading them, and `migrate()` drops them from an
|
||||
older database.
|
||||
before accounts; two bugs came from queries still reading them, and they were dropped in 0.5.
|
||||
|
||||
Schema changes: add the table or column to `SCHEMA`. `CREATE TABLE IF NOT EXISTS` leaves a table
|
||||
that already exists alone, so a new column on one also goes in `migrate()`'s `wanted` list, and a
|
||||
retired one in its `retired` list; both are checked with `PRAGMA table_info`. Columns from before
|
||||
0.3.0, the oldest version an upgrade may start from, need no entry. `Db::memory()` runs the same
|
||||
path as `Db::open`, so a migration cannot pass the tests while missing in production.
|
||||
Schema changes: the tables are the entities in `src/entity.rs`, and `Db::open` creates whatever
|
||||
table or index a database is missing from them (`db::create_missing`), with `IF NOT EXISTS`. It
|
||||
never alters a table that exists, so a new column on one needs its own `ALTER` in
|
||||
`create_missing`, or `sea-orm-migration` once there are several. `Db::memory()` builds its
|
||||
database the same way, so the tests run on the schema production gets. A database from before
|
||||
0.7 takes its last columns from the old `migrate()`, so it upgrades through a 0.7 release first.
|
||||
|
||||
## Control socket
|
||||
|
||||
|
||||
@@ -1,7 +1,19 @@
|
||||
# Configuration
|
||||
|
||||
One TOML file, read at startup and re-read whenever the web UI writes to it — most changes take
|
||||
effect without a restart. Default location `$XDG_CONFIG_HOME/ipx/config.toml`
|
||||
Two places. **config.toml** holds what ipx needs before it reaches its database, and what decides
|
||||
who gets in: where things are (`download_dir`, `socket`, `organize`), `[torrent]` and `[web]`.
|
||||
**The database** holds the catalogue of feeds (`[feeds.<id>]` below) and the server settings the
|
||||
admin page edits (`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check`,
|
||||
`media_types`). Change those in the web UI, or with `ipx add`, `ipx rm` and `ipx import`; they
|
||||
take effect without a restart.
|
||||
|
||||
The first time ipx meets a database that holds no catalogue, it takes the feeds and those
|
||||
settings from config.toml, then rewrites config.toml without them, keeping the original beside it
|
||||
as `config.toml.pre-database`. After that, feeds or those settings written into config.toml are
|
||||
ignored, with a warning in the log saying so. The sections below describe them as they were
|
||||
written in config.toml, which is still how a fresh install begins.
|
||||
|
||||
config.toml's default location is `$XDG_CONFIG_HOME/ipx/config.toml`
|
||||
(`~/.config/ipx/config.toml`), overridden with `--config` or `$IPX_CONFIG`.
|
||||
|
||||
| What | Where | Override |
|
||||
@@ -11,8 +23,10 @@ effect without a restart. Default location `$XDG_CONFIG_HOME/ipx/config.toml`
|
||||
| Control socket | `$XDG_RUNTIME_DIR/ipx.sock` | `[general] socket` |
|
||||
| Downloads | `[general] download_dir` | — |
|
||||
|
||||
`~` is expanded in paths. The database is SQLite in WAL mode; back it up by copying `state.db`
|
||||
while the daemon is stopped, or with `sqlite3 state.db .backup`.
|
||||
`~` is expanded in paths. The database is SQLite in WAL mode unless `IPX_DATABASE_URL` names a
|
||||
Postgres database instead. Back SQLite up by copying `state.db` while the daemon is stopped, or
|
||||
with `sqlite3 state.db .backup`; back Postgres up with `pg_dump`. `ipx copy-db <state.db>` copies a
|
||||
SQLite database into the empty Postgres one `IPX_DATABASE_URL` names.
|
||||
|
||||
## `[general]`
|
||||
|
||||
@@ -28,6 +42,9 @@ max_new_per_check = 3 # per feed, per scan. 0 = unlimited
|
||||
media_types = ["audio", "video"]
|
||||
```
|
||||
|
||||
`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check` and `media_types` move into the
|
||||
database as described above; `download_dir`, `socket` and `organize` stay in config.toml.
|
||||
|
||||
* **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
|
||||
be polled *less* often, and a per-feed `schedule` overrides both. Admin-only from the UI.
|
||||
* **`organize`** — `feed` files downloads under the feed's folder; `date` under `YYYY-MM-DD`.
|
||||
@@ -88,8 +105,9 @@ itself carry a credential. Put TLS in front of it if that matters.
|
||||
|
||||
## `[feeds.<id>]`
|
||||
|
||||
The table key is the feed id: stable, human-readable, and used in paths and the API. `ipx add`
|
||||
derives it from the feed title.
|
||||
Kept in the database once ipx has moved them in: a feed's settings are changed in the web UI, and
|
||||
feeds come and go with `ipx add`, `ipx rm` and `ipx import`. The table key is the feed id: stable,
|
||||
human-readable, and used in paths and the API. `ipx add` derives it from the feed title.
|
||||
|
||||
```toml
|
||||
[feeds.atp]
|
||||
@@ -107,8 +125,8 @@ With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
|
||||
config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
|
||||
and are the same for everyone. See [users.md](users.md).
|
||||
|
||||
Feeds derived from a subscribed OPML are **not** written here: the OPML is the source of truth and
|
||||
they are re-derived on every scan. Editing one in the UI promotes it to a real config entry.
|
||||
Feeds derived from a subscribed OPML are **not** in the catalogue: the OPML is the source of truth
|
||||
and they are re-derived on every scan. Editing one in the UI promotes it to a catalogue entry.
|
||||
|
||||
## Environment
|
||||
|
||||
@@ -116,6 +134,8 @@ they are re-derived on every scan. Editing one in the UI promotes it to a real c
|
||||
|---|---|
|
||||
| `IPX_CONFIG` | Config file path |
|
||||
| `IPX_DATA_DIR` | Directory holding `state.db` |
|
||||
| `IPX_DATABASE_URL` | A `postgres://user:password@host:port/database` URL: use that database instead of `state.db` |
|
||||
| `IPX_TEST_DATABASE_URL` | For `cargo test`: run the database tests on this Postgres database too, each in a schema of its own |
|
||||
| `IPX_LOG` | What reaches stderr (`ipx=debug`, `ipx::scan=debug`, …) |
|
||||
| `IPX_UI_LOG` | What the in-process log buffer captures for the UI's Log view |
|
||||
| `http_proxy` / `https_proxy` | Honoured for feed and enclosure fetches |
|
||||
|
||||
1732
docs/history.md
1732
docs/history.md
File diff suppressed because it is too large
Load Diff
109
src/config.rs
109
src/config.rs
@@ -261,21 +261,86 @@ impl Config {
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
pub fn save(&self, path: &Path) -> Result<()> {
|
||||
if let Some(dir) = path.parent() {
|
||||
std::fs::create_dir_all(dir)
|
||||
.with_context(|| format!("creating {}", dir.display()))?;
|
||||
}
|
||||
|
||||
/// What the database keeps of the configuration (issue #18): the server settings the admin page
|
||||
/// edits, and, beside them in `Db::stored_config`, the catalogue of feeds. The rest -- where
|
||||
/// things are, who may sign in, the torrent session -- is needed before the database is reached,
|
||||
/// or decides who gets in, and stays in config.toml.
|
||||
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize)]
|
||||
pub struct Stored {
|
||||
pub schedule: String,
|
||||
pub max_total_gb: f64,
|
||||
pub max_age_days: u64,
|
||||
pub max_new_per_check: usize,
|
||||
pub media_types: Vec<String>,
|
||||
}
|
||||
|
||||
/// `[general]` keys that live in the database once it holds the configuration.
|
||||
const STORED_KEYS: [&str; 5] = ["schedule", "max_total_gb", "max_age_days", "max_new_per_check", "media_types"];
|
||||
|
||||
impl Stored {
|
||||
pub fn of(cfg: &Config) -> Self {
|
||||
let g = &cfg.general;
|
||||
Self {
|
||||
schedule: g.schedule.clone(),
|
||||
max_total_gb: g.max_total_gb,
|
||||
max_age_days: g.max_age_days,
|
||||
max_new_per_check: g.max_new_per_check,
|
||||
media_types: g.media_types.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn apply(self, cfg: &mut Config) {
|
||||
let g = &mut cfg.general;
|
||||
g.schedule = self.schedule;
|
||||
g.max_total_gb = self.max_total_gb;
|
||||
g.max_age_days = self.max_age_days;
|
||||
g.max_new_per_check = self.max_new_per_check;
|
||||
g.media_types = self.media_types;
|
||||
}
|
||||
}
|
||||
|
||||
impl Config {
|
||||
/// config.toml as it is kept once the database holds the feeds and server settings: the same
|
||||
/// file without `[feeds]` or the `[general]` keys in `Stored`.
|
||||
pub fn save_bootstrap(&self, path: &Path) -> Result<()> {
|
||||
let mut v = toml::Value::try_from(self)?;
|
||||
if let Some(t) = v.as_table_mut() {
|
||||
t.remove("feeds");
|
||||
if let Some(g) = t.get_mut("general").and_then(|g| g.as_table_mut()) {
|
||||
for k in STORED_KEYS {
|
||||
g.remove(k);
|
||||
}
|
||||
}
|
||||
}
|
||||
write_private(path, &toml::to_string_pretty(&v)?)
|
||||
}
|
||||
|
||||
/// Whether config.toml still lists feeds or server settings, which the database now holds:
|
||||
/// an edit there would otherwise go unnoticed.
|
||||
pub fn file_holds_stored(path: &Path) -> bool {
|
||||
let Ok(text) = std::fs::read_to_string(path) else { return false };
|
||||
let Ok(v) = text.parse::<toml::Table>() else { return false };
|
||||
v.get("feeds").and_then(|f| f.as_table()).is_some_and(|f| !f.is_empty())
|
||||
|| v.get("general")
|
||||
.and_then(|g| g.as_table())
|
||||
.is_some_and(|g| STORED_KEYS.iter().any(|k| g.contains_key(*k)))
|
||||
}
|
||||
}
|
||||
|
||||
/// Writes a config file readable by its owner alone: feed passwords have lived in it.
|
||||
fn write_private(path: &Path, text: &str) -> Result<()> {
|
||||
if let Some(dir) = path.parent() {
|
||||
std::fs::create_dir_all(dir).with_context(|| format!("creating {}", dir.display()))?;
|
||||
}
|
||||
let text = toml::to_string_pretty(self)?;
|
||||
std::fs::write(path, text).with_context(|| format!("writing {}", path.display()))?;
|
||||
// Passwords may live in here.
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `$IPX_CONFIG`, else `$XDG_CONFIG_HOME/ipx/config.toml`.
|
||||
@@ -370,6 +435,36 @@ fn expand_tilde(p: &Path) -> PathBuf {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn the_file_kept_beside_the_database_has_no_feeds_or_server_settings() {
|
||||
let cfg: Config = toml::from_str(
|
||||
r#"
|
||||
[general]
|
||||
download_dir = "/downloads"
|
||||
schedule = "every 2h"
|
||||
max_new_per_check = 7
|
||||
media_types = ["audio"]
|
||||
[web]
|
||||
bind = "0.0.0.0:8099"
|
||||
token = "t"
|
||||
[feeds.show]
|
||||
url = "http://x/show.xml"
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
let path = std::env::temp_dir().join(format!("ipx-bootstrap-{}.toml", std::process::id()));
|
||||
std::fs::write(&path, toml::to_string(&cfg).unwrap()).unwrap();
|
||||
assert!(Config::file_holds_stored(&path), "a whole config.toml holds them");
|
||||
cfg.save_bootstrap(&path).unwrap();
|
||||
let text = std::fs::read_to_string(&path).unwrap();
|
||||
assert!(!Config::file_holds_stored(&path), "{text}");
|
||||
let back: Config = toml::from_str(&text).unwrap();
|
||||
assert!(back.feeds.is_empty());
|
||||
assert_eq!(back.web.token, "t", "who may sign in stays in the file");
|
||||
assert_eq!(back.general.download_dir, PathBuf::from("/downloads"), "where things are, too");
|
||||
std::fs::remove_file(&path).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_a_config_and_applies_defaults() {
|
||||
let cfg: Config = toml::from_str(
|
||||
|
||||
285
src/entity.rs
Normal file
285
src/entity.rs
Normal file
@@ -0,0 +1,285 @@
|
||||
//! The database's tables as SeaORM entities: the one description of the schema, from which
|
||||
//! `Db::open` creates what a database is missing, on SQLite or Postgres alike (see
|
||||
//! `db::create_missing`). Times are Unix seconds.
|
||||
|
||||
pub mod feeds {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "feeds")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub id: String,
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub url: String,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub title: Option<String>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub image: Option<String>,
|
||||
/// The channel's first <itunes:category>, for the Directory.
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub category: Option<String>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub etag: Option<String>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub last_modified: Option<String>,
|
||||
pub last_checked: Option<i64>,
|
||||
pub ttl_mins: Option<i64>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub last_error: Option<String>,
|
||||
/// When the current run of failures began; NULL while the feed is healthy. Kept through
|
||||
/// repeated failures so the UI can tell a blip (macmanx: failed once, fine an hour
|
||||
/// later) from a feed that has been down for a day.
|
||||
pub error_since: Option<i64>,
|
||||
/// Came from a subscribed OPML that no longer lists it, but has downloads, so kept.
|
||||
#[sea_orm(default_value = false)]
|
||||
pub orphaned: bool,
|
||||
/// The OPML subscription this feed came from.
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub group_id: Option<String>,
|
||||
/// Derived from an OPML and not written to config.toml. Writing 80-odd generated entries
|
||||
/// into a hand-edited file made it unreadable; the OPML is the source of truth, so they
|
||||
/// are re-derived instead. Customising one promotes it to config.
|
||||
#[sea_orm(default_value = false)]
|
||||
pub managed: bool,
|
||||
}
|
||||
|
||||
#[derive(Copy, Clone, Debug, EnumIter, DeriveRelation)]
|
||||
pub enum Relation {}
|
||||
|
||||
impl ActiveModelBehavior for ActiveModel {}
|
||||
}
|
||||
|
||||
pub mod entries {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "entries")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub feed_id: String,
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub guid: String,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub title: Option<String>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub link: Option<String>,
|
||||
pub published: Option<i64>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub description: Option<String>,
|
||||
pub first_seen: i64,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub image: Option<String>,
|
||||
pub duration: Option<i64>,
|
||||
pub episode: Option<i64>,
|
||||
pub season: Option<i64>,
|
||||
}
|
||||
|
||||
#[derive(Copy, Clone, Debug, EnumIter, DeriveRelation)]
|
||||
pub enum Relation {}
|
||||
|
||||
impl ActiveModelBehavior for ActiveModel {}
|
||||
}
|
||||
|
||||
pub mod enclosures {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "enclosures")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key)]
|
||||
pub id: i64,
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub feed_id: String,
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub guid: String,
|
||||
/// The dedupe key, and the reason one file serves every subscriber. A reaped file keeps
|
||||
/// its row with path NULL and state 'reaped', so a purged episode is never fetched again.
|
||||
#[sea_orm(unique, column_type = "Text")]
|
||||
pub url: String,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub mime: Option<String>,
|
||||
pub length: Option<i64>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub path: Option<String>,
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub state: String,
|
||||
#[sea_orm(default_value = 0)]
|
||||
pub bytes_done: i64,
|
||||
pub downloaded_at: Option<i64>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub last_error: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Copy, Clone, Debug, EnumIter, DeriveRelation)]
|
||||
pub enum Relation {}
|
||||
|
||||
impl ActiveModelBehavior for ActiveModel {}
|
||||
}
|
||||
|
||||
pub mod users {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "users")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key)]
|
||||
pub id: i64,
|
||||
/// Unique without regard to case: `db::create_missing` adds the index on lower(name), which
|
||||
/// works the same on both databases where SQLite's COLLATE NOCASE does not.
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub name: String,
|
||||
/// NULL for someone who only ever arrives through the proxy: there is no password to
|
||||
/// check, and leaving it empty is not the same as leaving it unset.
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub pass_hash: Option<String>,
|
||||
#[sea_orm(default_value = false)]
|
||||
pub is_admin: bool,
|
||||
/// For whoever maintains the server. NULL where it is not known.
|
||||
pub created: Option<i64>,
|
||||
pub last_login: Option<i64>,
|
||||
/// The theme chosen in Settings, and light, dark or auto. NULL until one is chosen.
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub theme: Option<String>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub theme_mode: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Copy, Clone, Debug, EnumIter, DeriveRelation)]
|
||||
pub enum Relation {}
|
||||
|
||||
impl ActiveModelBehavior for ActiveModel {}
|
||||
}
|
||||
|
||||
/// A table of one person's rows, gone when they are.
|
||||
macro_rules! owned_by_user {
|
||||
() => {
|
||||
#[derive(Copy, Clone, Debug, EnumIter, DeriveRelation)]
|
||||
pub enum Relation {
|
||||
#[sea_orm(
|
||||
belongs_to = "super::users::Entity",
|
||||
from = "Column::UserId",
|
||||
to = "super::users::Column::Id",
|
||||
on_delete = "Cascade"
|
||||
)]
|
||||
User,
|
||||
}
|
||||
|
||||
impl Related<super::users::Entity> for Entity {
|
||||
fn to() -> RelationDef {
|
||||
Relation::User.def()
|
||||
}
|
||||
}
|
||||
|
||||
impl ActiveModelBehavior for ActiveModel {}
|
||||
};
|
||||
}
|
||||
|
||||
/// What one person wants from a feed. The feed, its items and its files are shared; this is the
|
||||
/// part that is not. NULL in a column means: follow the feed's own setting.
|
||||
pub mod subscriptions {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "subscriptions")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key, auto_increment = false)]
|
||||
pub user_id: i64,
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub feed_id: String,
|
||||
/// JSON array of strings; NULL follows the feed.
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub keywords: Option<String>,
|
||||
pub auto_download: Option<bool>,
|
||||
pub allow_explicit: Option<bool>,
|
||||
pub max_new_per_check: Option<i64>,
|
||||
/// Pinned to the top of this person's feed list, a feed inside a folder included.
|
||||
#[sea_orm(default_value = false)]
|
||||
pub pinned: bool,
|
||||
}
|
||||
|
||||
owned_by_user!();
|
||||
}
|
||||
|
||||
/// Read, kept and how far in. One row per person per item, created on first touch; an item
|
||||
/// nobody has touched has no row at all, which is what unread means.
|
||||
pub mod entry_state {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "entry_state")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key, auto_increment = false)]
|
||||
pub user_id: i64,
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub feed_id: String,
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub guid: String,
|
||||
#[sea_orm(default_value = false)]
|
||||
pub read: bool,
|
||||
#[sea_orm(default_value = false)]
|
||||
pub flagged: bool,
|
||||
#[sea_orm(default_value = 0)]
|
||||
pub position: i64,
|
||||
/// The length this person's player measured, beside the position it is measured against.
|
||||
pub duration: Option<i64>,
|
||||
}
|
||||
|
||||
owned_by_user!();
|
||||
}
|
||||
|
||||
pub mod sessions {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "sessions")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub token: String,
|
||||
pub user_id: i64,
|
||||
pub seen: i64,
|
||||
}
|
||||
|
||||
owned_by_user!();
|
||||
}
|
||||
|
||||
/// The catalogue: every feed configured, with its shared settings as `config::Feed` in JSON, so a
|
||||
/// new setting on a feed needs no new column. It was config.toml's `[feeds]` (issue #18).
|
||||
pub mod catalogue {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "catalogue")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub id: String,
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub spec: String,
|
||||
}
|
||||
|
||||
#[derive(Copy, Clone, Debug, EnumIter, DeriveRelation)]
|
||||
pub enum Relation {}
|
||||
|
||||
impl ActiveModelBehavior for ActiveModel {}
|
||||
}
|
||||
|
||||
/// The server's settings, by name, each a JSON value. `general` is `config::Stored`: what was in
|
||||
/// config.toml's `[general]` and the admin page edits. Its row being there is what says the
|
||||
/// configuration has moved in (issue #18).
|
||||
pub mod settings {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "settings")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key, auto_increment = false, column_type = "Text")]
|
||||
pub name: String,
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub value: String,
|
||||
}
|
||||
|
||||
#[derive(Copy, Clone, Debug, EnumIter, DeriveRelation)]
|
||||
pub enum Relation {}
|
||||
|
||||
impl ActiveModelBehavior for ActiveModel {}
|
||||
}
|
||||
@@ -176,7 +176,9 @@ pub async fn daemon_is_live(path: &Path) -> bool {
|
||||
/// healthcheck left waiting behind a scan or a long download timed out and called a busy daemon
|
||||
/// dead. The answer goes to the client that asked and no one else: broadcast, it ended any
|
||||
/// `ipx fetch` that was watching a scan, since `status` is a terminal event.
|
||||
pub type StatusFn = std::sync::Arc<dyn Fn() -> Event + Send + Sync>;
|
||||
/// A future, since reading the counts is a database query.
|
||||
pub type StatusFn =
|
||||
std::sync::Arc<dyn Fn() -> std::pin::Pin<Box<dyn std::future::Future<Output = Event> + Send>> + Send + Sync>;
|
||||
|
||||
/// Accepts connections, feeding commands to `cmds` and events from `events` back out.
|
||||
pub async fn serve(
|
||||
@@ -249,7 +251,7 @@ async fn handle(
|
||||
// Answered here, not queued behind whatever the worker is on: see StatusFn.
|
||||
Ok(Command::Status) => {
|
||||
tracing::info!(target: "ipx::io", "-> {line}");
|
||||
let ev = status();
|
||||
let ev = status().await;
|
||||
if let Ok(json) = serde_json::to_string(&ev) {
|
||||
tracing::info!(target: "ipx::io", "<- {json}");
|
||||
}
|
||||
@@ -366,7 +368,8 @@ mod tests {
|
||||
// Another client, watching a scan: it must not be handed someone else's answer, which
|
||||
// would end its session.
|
||||
let mut watcher = events.subscribe();
|
||||
let status: StatusFn = std::sync::Arc::new(|| Event::Status { feeds: 1, pending: 2, downloaded: 3 });
|
||||
let status: StatusFn =
|
||||
std::sync::Arc::new(|| Box::pin(async { Event::Status { feeds: 1, pending: 2, downloaded: 3 } }));
|
||||
let (client, server) = UnixStream::pair().unwrap();
|
||||
tokio::spawn(handle(server, events.subscribe(), cmds, status));
|
||||
|
||||
|
||||
424
src/main.rs
424
src/main.rs
File diff suppressed because it is too large
Load Diff
@@ -45,28 +45,28 @@ pub fn aged(candidates: &[Candidate], cutoff: i64) -> Vec<Candidate> {
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub fn run(cfg: &Config, db: &Db, dry_run: bool) -> Result<Report> {
|
||||
pub async fn run(cfg: &Config, db: &Db, dry_run: bool) -> Result<Report> {
|
||||
let mut report = Report::default();
|
||||
|
||||
// Someone may have deleted a file by hand; the row must stop claiming it exists.
|
||||
for (id, path) in db.missing_files()? {
|
||||
for (id, path) in db.missing_files().await? {
|
||||
if !dry_run {
|
||||
db.mark_reaped(id)?;
|
||||
db.mark_reaped(id).await?;
|
||||
}
|
||||
tracing::debug!(path, "file gone, row reaped");
|
||||
report.reconciled += 1;
|
||||
}
|
||||
|
||||
let candidates = db.reap_candidates()?;
|
||||
let candidates = db.reap_candidates().await?;
|
||||
|
||||
if cfg.general.max_age_days > 0 {
|
||||
let cutoff = now() - (cfg.general.max_age_days * 86_400) as i64;
|
||||
report.aged_out = aged(&candidates, cutoff);
|
||||
for c in &report.aged_out {
|
||||
report.bytes_freed += remove(db, c, dry_run)?;
|
||||
report.bytes_freed += remove(db, c, dry_run).await?;
|
||||
}
|
||||
if !dry_run {
|
||||
report.entries_pruned = db.prune_entries(cutoff)?;
|
||||
report.entries_pruned = db.prune_entries(cutoff).await?;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -81,14 +81,14 @@ pub fn run(cfg: &Config, db: &Db, dry_run: bool) -> Result<Report> {
|
||||
let total: u64 = remaining.iter().map(|c| c.bytes.max(0) as u64).sum();
|
||||
report.over_quota = pick(&remaining, total, limit);
|
||||
for c in &report.over_quota {
|
||||
report.bytes_freed += remove(db, c, dry_run)?;
|
||||
report.bytes_freed += remove(db, c, dry_run).await?;
|
||||
}
|
||||
}
|
||||
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
fn remove(db: &Db, c: &Candidate, dry_run: bool) -> Result<u64> {
|
||||
async fn remove(db: &Db, c: &Candidate, dry_run: bool) -> Result<u64> {
|
||||
if dry_run {
|
||||
return Ok(c.bytes.max(0) as u64);
|
||||
}
|
||||
@@ -100,7 +100,7 @@ fn remove(db: &Db, c: &Candidate, dry_run: bool) -> Result<u64> {
|
||||
tracing::warn!(path = c.path, error = %e, "could not delete");
|
||||
return Ok(0);
|
||||
}
|
||||
db.mark_reaped(c.id)?;
|
||||
db.mark_reaped(c.id).await?;
|
||||
Ok(size)
|
||||
}
|
||||
|
||||
@@ -149,12 +149,12 @@ mod tests {
|
||||
// age_key 0 means "never recorded" -- not the same as "infinitely old".
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn query_never_offers_a_file_anyone_starred_and_prefers_ones_everyone_read() {
|
||||
#[tokio::test]
|
||||
async fn query_never_offers_a_file_anyone_starred_and_prefers_ones_everyone_read() {
|
||||
// One file serves both subscribers, so it takes both of them to release it.
|
||||
let db = Db::memory().unwrap();
|
||||
let db = Db::memory().await.unwrap();
|
||||
db.exec_for_test(
|
||||
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1),(2,'sam',0);
|
||||
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',true),(2,'sam',false);
|
||||
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'f'),(2,'f');
|
||||
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
||||
('f', 'keep', 0),
|
||||
@@ -163,20 +163,20 @@ mod tests {
|
||||
('f', 'read', 0);
|
||||
-- Starred by one of the two, so it stays whatever the other thinks.
|
||||
INSERT INTO entry_state (user_id, feed_id, guid, read, flagged) VALUES
|
||||
(1, 'f', 'keep', 1, 1),
|
||||
(2, 'f', 'keep', 1, 0),
|
||||
(1, 'f', 'half', 1, 0),
|
||||
(1, 'f', 'read', 1, 0),
|
||||
(2, 'f', 'read', 1, 0);
|
||||
(1, 'f', 'keep', true, true),
|
||||
(2, 'f', 'keep', true, false),
|
||||
(1, 'f', 'half', true, false),
|
||||
(1, 'f', 'read', true, false),
|
||||
(2, 'f', 'read', true, false);
|
||||
INSERT INTO enclosures (id, feed_id, guid, url, path, bytes_done, state, downloaded_at) VALUES
|
||||
(1, 'f', 'keep', 'u1', '/tmp/keep', 10, 'done', 10),
|
||||
(2, 'f', 'half', 'u2', '/tmp/half', 10, 'done', 20),
|
||||
(3, 'f', 'unread', 'u3', '/tmp/unread', 10, 'done', 30),
|
||||
(4, 'f', 'read', 'u4', '/tmp/read', 10, 'done', 40);",
|
||||
)
|
||||
).await
|
||||
.unwrap();
|
||||
|
||||
let got: Vec<i64> = db.reap_candidates().unwrap().iter().map(|c| c.id).collect();
|
||||
let got: Vec<i64> = db.reap_candidates().await.unwrap().iter().map(|c| c.id).collect();
|
||||
assert_eq!(
|
||||
got,
|
||||
vec![4, 2, 3],
|
||||
@@ -185,12 +185,12 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prune_keeps_entries_that_still_have_a_file() {
|
||||
let db = Db::memory().unwrap();
|
||||
#[tokio::test]
|
||||
async fn prune_keeps_entries_that_still_have_a_file() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
db.exec_for_test(
|
||||
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',1);
|
||||
INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',1);
|
||||
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',true);
|
||||
INSERT INTO entry_state (user_id, feed_id, guid, flagged) VALUES (1,'f','flagged',true);
|
||||
INSERT INTO entries (feed_id, guid, first_seen) VALUES
|
||||
('f', 'has-file', 100),
|
||||
('f', 'no-file', 100),
|
||||
@@ -198,9 +198,9 @@ mod tests {
|
||||
('f', 'recent', 900);
|
||||
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES
|
||||
(1, 'f', 'has-file', 'u1', '/tmp/x', 'done');",
|
||||
)
|
||||
).await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(db.prune_entries(500).unwrap(), 1, "only the old, fileless, unflagged one");
|
||||
assert_eq!(db.prune_entries(500).await.unwrap(), 1, "only the old, fileless, unflagged one");
|
||||
}
|
||||
}
|
||||
|
||||
199
src/web.rs
199
src/web.rs
@@ -16,7 +16,6 @@ use serde::Deserialize;
|
||||
use tower::ServiceExt;
|
||||
use tower_http::services::ServeFile;
|
||||
use serde::Serialize;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
use tokio::sync::{broadcast, mpsc};
|
||||
|
||||
@@ -28,7 +27,6 @@ const COOKIE: &str = "ipx_token";
|
||||
#[derive(Clone)]
|
||||
pub struct WebState {
|
||||
pub ctx: Arc<Ctx>,
|
||||
pub config_path: PathBuf,
|
||||
pub cmds: mpsc::Sender<Command>,
|
||||
pub events: broadcast::Sender<Event>,
|
||||
}
|
||||
@@ -107,16 +105,16 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
||||
let mut user = None;
|
||||
|
||||
if let Some(name) = vouched {
|
||||
user = match state.ctx.db.user_by_name(&name) {
|
||||
user = match state.ctx.db.user_by_name(&name).await {
|
||||
Ok(Some(u)) => Some(u),
|
||||
Ok(None) if cfg.web.auto_create_users => {
|
||||
tracing::info!(user = %name, "creating an account for a name the proxy vouched for");
|
||||
state
|
||||
.ctx
|
||||
.db
|
||||
.create_user(&name, None, state.ctx.db.users().map(|u| u.is_empty()).unwrap_or(false))
|
||||
.ok()
|
||||
.and_then(|id| state.ctx.db.user_by_id(id).ok().flatten())
|
||||
// The first account made is the admin.
|
||||
let first = state.ctx.db.users().await.map(|u| u.is_empty()).unwrap_or(false);
|
||||
match state.ctx.db.create_user(&name, None, first).await {
|
||||
Ok(id) => state.ctx.db.user_by_id(id).await.ok().flatten(),
|
||||
Err(_) => None,
|
||||
}
|
||||
}
|
||||
Ok(None) => {
|
||||
tracing::warn!(user = %name, "proxy vouched for an unknown name and auto_create_users is off");
|
||||
@@ -130,7 +128,7 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
||||
// Every request comes vouched for; signed_in keeps one an hour. Failing to note the time
|
||||
// must not turn anyone away, so its error goes unanswered.
|
||||
if let Some(u) = &user {
|
||||
let _ = state.ctx.db.signed_in(u.id);
|
||||
let _ = state.ctx.db.signed_in(u.id).await;
|
||||
}
|
||||
}
|
||||
let by_proxy = user.is_some();
|
||||
@@ -141,7 +139,7 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
||||
user = state
|
||||
.ctx
|
||||
.db
|
||||
.session_user(&sid, cfg.web.session_days.max(1) * 86_400)
|
||||
.session_user(&sid, cfg.web.session_days.max(1) * 86_400).await
|
||||
.unwrap_or(None);
|
||||
}
|
||||
}
|
||||
@@ -155,11 +153,11 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
||||
if user.is_none() && !token.is_empty() {
|
||||
let supplied = from_query.clone().or_else(|| cookie(&req, COOKIE));
|
||||
if supplied.is_some_and(|t| constant_time_eq(&t, &token)) {
|
||||
user = admin_user(&state);
|
||||
user = admin_user(&state).await;
|
||||
if from_query.is_some() {
|
||||
// The token link is a sign-in; the cookie it leaves behind is not one each time.
|
||||
if let Some(u) = &user {
|
||||
let _ = state.ctx.db.signed_in(u.id);
|
||||
let _ = state.ctx.db.signed_in(u.id).await;
|
||||
}
|
||||
set_cookie = Some(format!(
|
||||
"{COOKIE}={token}; Path=/; HttpOnly; SameSite=Lax; Max-Age=31536000"
|
||||
@@ -247,8 +245,8 @@ fn cookie(req: &Request, name: &str) -> Option<String> {
|
||||
}
|
||||
|
||||
/// The account the shared token stands for: the first admin, or the first user at all.
|
||||
fn admin_user(state: &WebState) -> Option<crate::db::User> {
|
||||
let users = state.ctx.db.users().ok()?;
|
||||
async fn admin_user(state: &WebState) -> Option<crate::db::User> {
|
||||
let users = state.ctx.db.users().await.ok()?;
|
||||
users
|
||||
.iter()
|
||||
.find(|u| u.is_admin)
|
||||
@@ -267,7 +265,7 @@ async fn login(
|
||||
Json(body): Json<Credentials>,
|
||||
) -> Result<Response, ApiError> {
|
||||
let name = body.name.trim().to_ascii_lowercase();
|
||||
let user = state.ctx.db.user_by_name(&name)?;
|
||||
let user = state.ctx.db.user_by_name(&name).await?;
|
||||
// The same answer either way: whether a name exists is not something to leak.
|
||||
let ok = user
|
||||
.as_ref()
|
||||
@@ -280,8 +278,8 @@ async fn login(
|
||||
|
||||
let user = user.expect("verified above");
|
||||
let token = crate::auth::new_session_token();
|
||||
state.ctx.db.create_session(user.id, &token)?;
|
||||
state.ctx.db.signed_in(user.id)?;
|
||||
state.ctx.db.create_session(user.id, &token).await?;
|
||||
state.ctx.db.signed_in(user.id).await?;
|
||||
tracing::info!(user = %user.name, "signed in");
|
||||
|
||||
let days = state.ctx.cfg().web.session_days.max(1);
|
||||
@@ -298,7 +296,7 @@ async fn login(
|
||||
|
||||
async fn logout(State(state): State<WebState>, req: Request) -> Response {
|
||||
if let Some(sid) = cookie(&req, SESSION_COOKIE) {
|
||||
let _ = state.ctx.db.delete_session(&sid);
|
||||
let _ = state.ctx.db.delete_session(&sid).await;
|
||||
}
|
||||
let mut resp = StatusCode::NO_CONTENT.into_response();
|
||||
for c in [
|
||||
@@ -320,7 +318,7 @@ async fn me(
|
||||
) -> Json<serde_json::Value> {
|
||||
let url = state.ctx.cfg().web.sign_out_url.clone();
|
||||
let sign_out = (by_proxy && !url.is_empty()).then_some(url);
|
||||
let (theme, mode) = state.ctx.db.theme(user.id).unwrap_or_default();
|
||||
let (theme, mode) = state.ctx.db.theme(user.id).await.unwrap_or_default();
|
||||
Json(serde_json::json!({
|
||||
"name": user.name, "admin": user.is_admin, "sign_out": sign_out, "theme": theme, "mode": mode,
|
||||
}))
|
||||
@@ -343,7 +341,7 @@ async fn patch_me(
|
||||
if !theme_ok(&body.theme, &body.mode) {
|
||||
return Err(ApiError::bad_request("not a theme"));
|
||||
}
|
||||
state.ctx.db.set_theme(user.id, &body.theme, &body.mode)?;
|
||||
state.ctx.db.set_theme(user.id, &body.theme, &body.mode).await?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
@@ -378,7 +376,7 @@ async fn list_users(
|
||||
let users: Vec<_> = state
|
||||
.ctx
|
||||
.db
|
||||
.users()?
|
||||
.users().await?
|
||||
.iter()
|
||||
.map(|u| {
|
||||
serde_json::json!({
|
||||
@@ -409,7 +407,7 @@ async fn add_user(
|
||||
let name = crate::auth::name_from_header(&body.name).ok_or_else(|| {
|
||||
ApiError::bad_request("a name is required, without commas, semicolons or line breaks")
|
||||
})?;
|
||||
if state.ctx.db.user_by_name(&name)?.is_some() {
|
||||
if state.ctx.db.user_by_name(&name).await?.is_some() {
|
||||
return Err(ApiError::bad_request(format!("{name} already exists")));
|
||||
}
|
||||
// No password is someone the proxy signs in, as with `ipx user add --no-password`.
|
||||
@@ -418,7 +416,7 @@ async fn add_user(
|
||||
} else {
|
||||
Some(crate::auth::hash_password(&body.password).map_err(|e| ApiError::bad_request(format!("{e:#}")))?)
|
||||
};
|
||||
state.ctx.db.create_user(&name, hash.as_deref(), body.admin)?;
|
||||
state.ctx.db.create_user(&name, hash.as_deref(), body.admin).await?;
|
||||
tracing::info!(by = %user.name, user = %name, admin = body.admin, "account added");
|
||||
Ok(StatusCode::CREATED)
|
||||
}
|
||||
@@ -435,7 +433,7 @@ async fn patch_user(
|
||||
Json(body): Json<UserPatch>,
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
require_admin(&user)?;
|
||||
let users = state.ctx.db.users()?;
|
||||
let users = state.ctx.db.users().await?;
|
||||
let target = users
|
||||
.iter()
|
||||
.find(|u| u.id == id)
|
||||
@@ -446,7 +444,7 @@ async fn patch_user(
|
||||
target.name
|
||||
)));
|
||||
}
|
||||
state.ctx.db.set_admin(id, body.admin)?;
|
||||
state.ctx.db.set_admin(id, body.admin).await?;
|
||||
tracing::info!(by = %user.name, user = %target.name, admin = body.admin, "admin changed");
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
@@ -457,7 +455,7 @@ async fn remove_user(
|
||||
Path(id): Path<i64>,
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
require_admin(&user)?;
|
||||
let users = state.ctx.db.users()?;
|
||||
let users = state.ctx.db.users().await?;
|
||||
let target = users
|
||||
.iter()
|
||||
.find(|u| u.id == id)
|
||||
@@ -468,7 +466,7 @@ async fn remove_user(
|
||||
target.name
|
||||
)));
|
||||
}
|
||||
state.ctx.db.delete_user(id)?;
|
||||
state.ctx.db.delete_user(id).await?;
|
||||
tracing::info!(by = %user.name, user = %target.name, "account removed");
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
@@ -510,7 +508,7 @@ async fn admin_page(State(state): State<WebState>, user: crate::db::User) -> Res
|
||||
if !user.is_admin {
|
||||
return Redirect::to("/").into_response();
|
||||
}
|
||||
let theme = state.ctx.db.theme(user.id).unwrap_or_default();
|
||||
let theme = state.ctx.db.theme(user.id).await.unwrap_or_default();
|
||||
let page = with_theme(include_str!(concat!(env!("OUT_DIR"), "/admin.html")), theme);
|
||||
([(header::CACHE_CONTROL, PAGE_CACHE)], Html(page)).into_response()
|
||||
}
|
||||
@@ -584,7 +582,7 @@ const ADMIN_LINK: &str = "<a id=admin ";
|
||||
/// The page, with the log button left out for anyone but an admin. Hiding it from the page's
|
||||
/// script instead showed it for a moment on every load, until /api/me answered.
|
||||
async fn index(State(state): State<WebState>, user: crate::db::User) -> impl IntoResponse {
|
||||
let theme = state.ctx.db.theme(user.id).unwrap_or_default();
|
||||
let theme = state.ctx.db.theme(user.id).await.unwrap_or_default();
|
||||
([(header::CACHE_CONTROL, PAGE_CACHE)], Html(page_for(user.is_admin, theme)))
|
||||
}
|
||||
|
||||
@@ -672,16 +670,16 @@ async fn feeds(
|
||||
let cfg = state.ctx.cfg();
|
||||
// Config entries plus the feeds derived from OPML subscriptions -- the catalogue.
|
||||
// What comes back is only the part of it this person subscribes to.
|
||||
let subs = crate::subscriptions(&state.ctx)?;
|
||||
let subs = crate::subscriptions(&state.ctx).await?;
|
||||
let mine: std::collections::HashMap<String, crate::db::Sub> = state
|
||||
.ctx
|
||||
.db
|
||||
.subscriptions_for(user.id)?
|
||||
.subscriptions_for(user.id).await?
|
||||
.into_iter()
|
||||
.map(|s| (s.feed_id.clone(), s))
|
||||
.collect();
|
||||
let counts = state.ctx.db.subscriber_counts()?;
|
||||
let pinned = state.ctx.db.pinned_feeds(user.id)?;
|
||||
let counts = state.ctx.db.subscriber_counts().await?;
|
||||
let pinned = state.ctx.db.pinned_feeds(user.id).await?;
|
||||
let mut out = Vec::with_capacity(mine.len());
|
||||
for sub in &subs {
|
||||
let (id, feed) = (&sub.id, &sub.cfg);
|
||||
@@ -689,8 +687,8 @@ async fn feeds(
|
||||
// the same fallback the scanner uses (`Db::subscribers`).
|
||||
let up = feed.group.as_deref().and_then(|g| mine.get(g));
|
||||
let Some(mine) = mine.get(id) else { continue };
|
||||
let s = state.ctx.db.feed_summary(id)?;
|
||||
let st = state.ctx.db.http_state(id)?;
|
||||
let s = state.ctx.db.feed_summary(id).await?;
|
||||
let st = state.ctx.db.http_state(id).await?;
|
||||
out.push(FeedRow {
|
||||
id: id.clone(),
|
||||
url: feed.url.clone(),
|
||||
@@ -740,7 +738,7 @@ async fn feeds(
|
||||
last_error: s.last_error,
|
||||
entries: s.entries,
|
||||
downloaded: s.downloaded,
|
||||
unread: state.ctx.db.unread_count(user.id, id)?,
|
||||
unread: state.ctx.db.unread_count(user.id, id).await?,
|
||||
subscribers: counts.get(id).copied().unwrap_or(0),
|
||||
pinned: pinned.contains(id),
|
||||
});
|
||||
@@ -800,13 +798,13 @@ struct PopularRow {
|
||||
/// first. Popular is the top of it, the directory is all of it, and it is all that
|
||||
/// `subscribe_popular` will subscribe you to. An OPML or a Patreon creator is listed as the
|
||||
/// feeds inside it and never itself: both lists are for finding a show.
|
||||
fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||
async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||
let db = &state.ctx.db;
|
||||
let mine: std::collections::HashSet<String> =
|
||||
db.subscriptions_for(user_id)?.into_iter().map(|s| s.feed_id).collect();
|
||||
let counts = db.subscriber_counts()?;
|
||||
let media = db.media_feeds()?;
|
||||
let catalogue = crate::subscriptions(&state.ctx)?;
|
||||
db.subscriptions_for(user_id).await?.into_iter().map(|s| s.feed_id).collect();
|
||||
let counts = db.subscriber_counts().await?;
|
||||
let media = db.media_feeds().await?;
|
||||
let catalogue = crate::subscriptions(&state.ctx).await?;
|
||||
let by_id: std::collections::HashMap<&str, &crate::config::Feed> =
|
||||
catalogue.iter().map(|s| (s.id.as_str(), &s.cfg)).collect();
|
||||
let is_folder: std::collections::HashSet<&str> =
|
||||
@@ -823,7 +821,7 @@ fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let sum = db.feed_summary(&s.id)?;
|
||||
let sum = db.feed_summary(&s.id).await?;
|
||||
let subscribed = mine.contains(&s.id);
|
||||
out.push(PopularRow {
|
||||
id: s.id.clone(),
|
||||
@@ -844,7 +842,7 @@ async fn get_popular(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||
let mut rows = popular(&state, user.id)?;
|
||||
let mut rows = popular(&state, user.id).await?;
|
||||
rows.truncate(10);
|
||||
Ok(Json(rows))
|
||||
}
|
||||
@@ -854,7 +852,7 @@ async fn get_directory(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||
let mut rows = popular(&state, user.id)?;
|
||||
let mut rows = popular(&state, user.id).await?;
|
||||
rows.sort_by_key(sort_name);
|
||||
Ok(Json(rows))
|
||||
}
|
||||
@@ -870,10 +868,10 @@ async fn subscribe_popular(
|
||||
user: crate::db::User,
|
||||
Path(id): Path<String>,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
if !popular(&state, user.id)?.iter().any(|p| p.id == id) {
|
||||
if !popular(&state, user.id).await?.iter().any(|p| p.id == id) {
|
||||
return Err(ApiError::bad_request(format!("{id:?} is not in the directory")));
|
||||
}
|
||||
state.ctx.db.subscribe(user.id, &id)?;
|
||||
state.ctx.db.subscribe(user.id, &id).await?;
|
||||
Ok(Json(serde_json::json!({ "id": id })))
|
||||
}
|
||||
|
||||
@@ -1124,7 +1122,7 @@ async fn entries(
|
||||
user: crate::db::User,
|
||||
Query(page): Query<Page>,
|
||||
) -> Result<Json<EntryPage>, ApiError> {
|
||||
entry_page(&state, user.id, Some(&id), &page)
|
||||
entry_page(&state, user.id, Some(&id), &page).await
|
||||
}
|
||||
|
||||
/// Every subscribed feed's items together, newest first: All Subscriptions.
|
||||
@@ -1133,11 +1131,11 @@ async fn all_entries(
|
||||
user: crate::db::User,
|
||||
Query(page): Query<Page>,
|
||||
) -> Result<Json<EntryPage>, ApiError> {
|
||||
entry_page(&state, user.id, None, &page)
|
||||
entry_page(&state, user.id, None, &page).await
|
||||
}
|
||||
|
||||
/// One feed's page of items, or every subscribed feed's when `feed` is None.
|
||||
fn entry_page(
|
||||
async fn entry_page(
|
||||
state: &WebState,
|
||||
user_id: i64,
|
||||
feed: Option<&str>,
|
||||
@@ -1146,19 +1144,16 @@ fn entry_page(
|
||||
let filter = crate::db::Filter::parse(page.filter.as_deref().unwrap_or("all"));
|
||||
let search = page.q.as_deref().map(str::trim).filter(|q| !q.is_empty());
|
||||
let db = &state.ctx.db;
|
||||
let order = crate::db::order_sql(
|
||||
page.sort.as_deref().unwrap_or("published"),
|
||||
page.dir.as_deref().unwrap_or("desc"),
|
||||
);
|
||||
let order = crate::db::order_sql(page.sort.as_deref().unwrap_or("published"), page.dir.as_deref().unwrap_or("desc"));
|
||||
let mut rows =
|
||||
db.entries_in(user_id, feed, filter, search, page.offset, page.limit.clamp(1, 200), &order)?;
|
||||
db.entries_in(user_id, feed, filter, search, page.offset, page.limit.clamp(1, 200), &order).await?;
|
||||
let mut sanitizer = feed_sanitizer();
|
||||
for row in &mut rows {
|
||||
if let Some(d) = &row.description {
|
||||
row.description = Some(clean_description(&mut sanitizer, d, row.link.as_deref()));
|
||||
}
|
||||
}
|
||||
let total = db.count_in(user_id, feed, filter, search)?;
|
||||
let total = db.count_in(user_id, feed, filter, search).await?;
|
||||
Ok(Json(EntryPage { total, entries: rows }))
|
||||
}
|
||||
|
||||
@@ -1200,10 +1195,10 @@ struct NewFeed {
|
||||
/// The Add feed dialog's explicit box. Like everything on a feed's own dialog it is yours, so it
|
||||
/// goes on your subscription, and before the first scan, which would otherwise skip every
|
||||
/// explicit item.
|
||||
fn explicit_on_add(state: &WebState, user_id: i64, feed_id: &str, allow: bool) -> Result<(), ApiError> {
|
||||
async fn explicit_on_add(state: &WebState, user_id: i64, feed_id: &str, allow: bool) -> Result<(), ApiError> {
|
||||
if allow {
|
||||
let sub = crate::db::Sub { feed_id: feed_id.to_owned(), allow_explicit: Some(true), ..Default::default() };
|
||||
state.ctx.db.set_subscription(user_id, &sub)?;
|
||||
state.ctx.db.set_subscription(user_id, &sub).await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -1217,14 +1212,14 @@ async fn add_feed(
|
||||
let url = crate::feed::expand_input(&body.url);
|
||||
// Someone else may already have it. Then adding costs nothing: no second fetch, no
|
||||
// second copy on disk, just another name against the same feed.
|
||||
if let Some(existing) = crate::subscriptions(&state.ctx)?
|
||||
if let Some(existing) = crate::subscriptions(&state.ctx).await?
|
||||
.into_iter()
|
||||
.find(|s| crate::feed::same_feed(&s.cfg.url, &url))
|
||||
{
|
||||
let already = state.ctx.db.subscription(user.id, &existing.id)?.is_some();
|
||||
state.ctx.db.subscribe(user.id, &existing.id)?;
|
||||
let already = state.ctx.db.subscription(user.id, &existing.id).await?.is_some();
|
||||
state.ctx.db.subscribe(user.id, &existing.id).await?;
|
||||
if !already {
|
||||
explicit_on_add(&state, user.id, &existing.id, body.allow_explicit)?;
|
||||
explicit_on_add(&state, user.id, &existing.id, body.allow_explicit).await?;
|
||||
}
|
||||
scan_soon(&state, Some(existing.id.clone())).await;
|
||||
return Ok(Json(
|
||||
@@ -1232,10 +1227,9 @@ async fn add_feed(
|
||||
));
|
||||
}
|
||||
let id = crate::add_one(&state.ctx, &mut cfg, &url, body.folder, body.keywords).await?;
|
||||
cfg.save(&state.config_path)?;
|
||||
state.ctx.reload_cfg(&state.config_path)?;
|
||||
state.ctx.db.subscribe(user.id, &id)?;
|
||||
explicit_on_add(&state, user.id, &id, body.allow_explicit)?;
|
||||
state.ctx.store_cfg(cfg).await?;
|
||||
state.ctx.db.subscribe(user.id, &id).await?;
|
||||
explicit_on_add(&state, user.id, &id, body.allow_explicit).await?;
|
||||
scan_soon(&state, Some(id.clone())).await;
|
||||
Ok(Json(serde_json::json!({ "id": id, "existing": false })))
|
||||
}
|
||||
@@ -1288,17 +1282,17 @@ async fn patch_feed(
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
// Pinning is yours alone too, and means nothing for a feed you do not subscribe to.
|
||||
if let Some(on) = body.pinned
|
||||
&& !state.ctx.db.set_pinned(user.id, &id, on)?
|
||||
&& !state.ctx.db.set_pinned(user.id, &id, on).await?
|
||||
{
|
||||
return Err(ApiError::not_found("you do not subscribe to that feed"));
|
||||
}
|
||||
// What one person wants -- which items, whether to fetch them, how many at a time --
|
||||
// is theirs. It goes on their subscription and nobody else sees the change.
|
||||
if state.ctx.db.subscription(user.id, &id)?.is_some() {
|
||||
if state.ctx.db.subscription(user.id, &id).await?.is_some() {
|
||||
let mut mine = state
|
||||
.ctx
|
||||
.db
|
||||
.subscription(user.id, &id)?
|
||||
.subscription(user.id, &id).await?
|
||||
.unwrap_or_else(|| crate::db::Sub { feed_id: id.clone(), ..Default::default() });
|
||||
let mut touched = false;
|
||||
if let Some(v) = body.keywords.clone() {
|
||||
@@ -1318,7 +1312,7 @@ async fn patch_feed(
|
||||
touched = true;
|
||||
}
|
||||
if touched {
|
||||
state.ctx.db.set_subscription(user.id, &mine)?;
|
||||
state.ctx.db.set_subscription(user.id, &mine).await?;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1339,13 +1333,13 @@ async fn patch_feed(
|
||||
// Derived feeds have no config entry. Editing one is the moment it earns a real
|
||||
// entry: promote it, so the config holds your decisions and nothing else.
|
||||
if !cfg.feeds.contains_key(&id) {
|
||||
let subs = crate::subscriptions(&state.ctx)?;
|
||||
let subs = crate::subscriptions(&state.ctx).await?;
|
||||
let found = subs
|
||||
.iter()
|
||||
.find(|s| s.id == id)
|
||||
.ok_or_else(|| ApiError::bad_request(format!("no feed with id {id:?}")))?;
|
||||
cfg.feeds.insert(id.clone(), found.cfg.clone());
|
||||
state.ctx.db.unmanage(&id)?;
|
||||
state.ctx.db.unmanage(&id).await?;
|
||||
}
|
||||
|
||||
let checked = match &body.url {
|
||||
@@ -1381,12 +1375,11 @@ async fn patch_feed(
|
||||
if let Some(v) = body.category {
|
||||
feed.category = v.map(|s| s.trim().to_owned()).filter(|s| !s.is_empty());
|
||||
}
|
||||
cfg.save(&state.config_path)?;
|
||||
state.ctx.reload_cfg(&state.config_path)?;
|
||||
state.ctx.store_cfg(cfg).await?;
|
||||
if url_changed {
|
||||
// Refreshing a rotated auth token is the common case; entries and download history
|
||||
// are keyed by feed id, so they survive the change.
|
||||
state.ctx.db.clear_validators(&id)?;
|
||||
state.ctx.db.clear_validators(&id).await?;
|
||||
}
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
@@ -1398,14 +1391,14 @@ async fn remove_feed(
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
// Unsubscribing is personal: it takes the feed off your list and leaves everyone
|
||||
// else's alone.
|
||||
state.ctx.db.unsubscribe(user.id, &id)?;
|
||||
for child in crate::subscriptions(&state.ctx)?
|
||||
state.ctx.db.unsubscribe(user.id, &id).await?;
|
||||
for child in crate::subscriptions(&state.ctx).await?
|
||||
.iter()
|
||||
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
|
||||
{
|
||||
state.ctx.db.unsubscribe(user.id, &child.id)?;
|
||||
state.ctx.db.unsubscribe(user.id, &child.id).await?;
|
||||
}
|
||||
if state.ctx.db.subscriber_counts()?.contains_key(&id) {
|
||||
if state.ctx.db.subscriber_counts().await?.contains_key(&id) {
|
||||
return Ok(StatusCode::NO_CONTENT);
|
||||
}
|
||||
|
||||
@@ -1415,12 +1408,11 @@ async fn remove_feed(
|
||||
if cfg.feeds.remove(&id).is_none() {
|
||||
// A derived feed: forget it here, though the OPML will list it again on the next
|
||||
// read unless you unsubscribe from the OPML itself.
|
||||
state.ctx.db.drop_managed(&id)?;
|
||||
state.ctx.db.drop_managed(&id).await?;
|
||||
return Ok(StatusCode::NO_CONTENT);
|
||||
}
|
||||
cfg.save(&state.config_path)?;
|
||||
state.ctx.reload_cfg(&state.config_path)?;
|
||||
crate::retire_group(&state.ctx, &id)?;
|
||||
state.ctx.store_cfg(cfg).await?;
|
||||
crate::retire_group(&state.ctx, &id).await?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
@@ -1438,10 +1430,10 @@ async fn set_flags(
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
use crate::db::EntryFlag;
|
||||
if let Some(v) = body.read {
|
||||
state.ctx.db.set_entry_flag(user.id, &feed_id, &guid, EntryFlag::Read, v)?;
|
||||
state.ctx.db.set_entry_flag(user.id, &feed_id, &guid, EntryFlag::Read, v).await?;
|
||||
}
|
||||
if let Some(v) = body.flagged {
|
||||
state.ctx.db.set_entry_flag(user.id, &feed_id, &guid, EntryFlag::Flagged, v)?;
|
||||
state.ctx.db.set_entry_flag(user.id, &feed_id, &guid, EntryFlag::Flagged, v).await?;
|
||||
}
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
@@ -1456,12 +1448,12 @@ async fn download_now(
|
||||
let enc = state
|
||||
.ctx
|
||||
.db
|
||||
.enclosure(id)?
|
||||
.enclosure(id).await?
|
||||
.ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
|
||||
if enc.path.is_some() {
|
||||
return Ok(StatusCode::NO_CONTENT); // Already here.
|
||||
}
|
||||
state.ctx.db.requeue(id)?;
|
||||
state.ctx.db.requeue(id).await?;
|
||||
state
|
||||
.cmds
|
||||
.send(Command::Download { enclosure: id })
|
||||
@@ -1485,13 +1477,13 @@ async fn delete_file(
|
||||
let enc = state
|
||||
.ctx
|
||||
.db
|
||||
.enclosure(id)?
|
||||
.enclosure(id).await?
|
||||
.ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
|
||||
|
||||
// There is one copy of the file: deleting it deletes everyone's. Say so before doing
|
||||
// it, once, and let them decide.
|
||||
if !q.force {
|
||||
let (starred, unread) = state.ctx.db.others_wanting(id, user.id)?;
|
||||
let (starred, unread) = state.ctx.db.others_wanting(id, user.id).await?;
|
||||
let people = |n: i64| if n == 1 { "person".to_string() } else { format!("{n} people") };
|
||||
let complaint = match (starred, unread) {
|
||||
(0, 0) => None,
|
||||
@@ -1517,7 +1509,7 @@ async fn delete_file(
|
||||
return Err(e.into());
|
||||
}
|
||||
// The row survives as 'reaped', which is what stops the next scan re-downloading it.
|
||||
state.ctx.db.mark_reaped(id)?;
|
||||
state.ctx.db.mark_reaped(id).await?;
|
||||
state.events.send(Event::Reaped {
|
||||
path: enc.path.unwrap_or_default(),
|
||||
bytes: enc.length.unwrap_or(0).max(0) as u64,
|
||||
@@ -1571,7 +1563,7 @@ async fn media(
|
||||
Path(id): Path<i64>,
|
||||
req: Request,
|
||||
) -> Response {
|
||||
let Ok(Some(enc)) = state.ctx.db.enclosure(id) else {
|
||||
let Ok(Some(enc)) = state.ctx.db.enclosure(id).await else {
|
||||
return (StatusCode::NOT_FOUND, "no such enclosure").into_response();
|
||||
};
|
||||
let Some(path) = enc.path else {
|
||||
@@ -1596,7 +1588,7 @@ async fn set_position(
|
||||
user: crate::db::User,
|
||||
Json(body): Json<Position>,
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
state.ctx.db.set_position(user.id, &feed_id, &guid, body.secs, body.duration)?;
|
||||
state.ctx.db.set_position(user.id, &feed_id, &guid, body.secs, body.duration).await?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
@@ -1608,12 +1600,12 @@ async fn read_all(
|
||||
// A subscription's own row has no entries, so marking it read means everything under it.
|
||||
let mut ids = vec![id.clone()];
|
||||
ids.extend(
|
||||
crate::subscriptions(&state.ctx)?
|
||||
crate::subscriptions(&state.ctx).await?
|
||||
.into_iter()
|
||||
.filter(|s| s.cfg.group.as_deref() == Some(id.as_str()))
|
||||
.map(|s| s.id),
|
||||
);
|
||||
let n = state.ctx.db.mark_all_read(user.id, &ids)?;
|
||||
let n = state.ctx.db.mark_all_read(user.id, &ids).await?;
|
||||
Ok(Json(serde_json::json!({ "marked": n })))
|
||||
}
|
||||
|
||||
@@ -1624,8 +1616,8 @@ async fn read_all_mine(
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
let ids: Vec<String> =
|
||||
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
|
||||
let n = state.ctx.db.mark_all_read(user.id, &ids)?;
|
||||
state.ctx.db.subscriptions_for(user.id).await?.into_iter().map(|s| s.feed_id).collect();
|
||||
let n = state.ctx.db.mark_all_read(user.id, &ids).await?;
|
||||
Ok(Json(serde_json::json!({ "marked": n })))
|
||||
}
|
||||
|
||||
@@ -1646,9 +1638,9 @@ async fn download_latest(
|
||||
Path(id): Path<String>,
|
||||
Json(body): Json<HowMany>,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
let ids = state.ctx.db.undownloaded(&id, body.count.clamp(1, 100))?;
|
||||
let ids = state.ctx.db.undownloaded(&id, body.count.clamp(1, 100)).await?;
|
||||
for enc in &ids {
|
||||
state.ctx.db.requeue(*enc)?;
|
||||
state.ctx.db.requeue(*enc).await?;
|
||||
state
|
||||
.cmds
|
||||
.send(Command::Download { enclosure: *enc })
|
||||
@@ -1666,7 +1658,7 @@ async fn export_opml(
|
||||
// Yours, not the whole catalogue: other people's feeds, and any private URLs in them, are
|
||||
// not yours to download. This used to export config.toml to whoever asked.
|
||||
let mine: std::collections::HashSet<String> =
|
||||
state.ctx.db.subscriptions_for(user.id)?.into_iter().map(|s| s.feed_id).collect();
|
||||
state.ctx.db.subscriptions_for(user.id).await?.into_iter().map(|s| s.feed_id).collect();
|
||||
let mut doc = opml::OPML {
|
||||
head: Some(opml::Head {
|
||||
title: Some("ipx subscriptions".into()),
|
||||
@@ -1674,7 +1666,7 @@ async fn export_opml(
|
||||
}),
|
||||
..Default::default()
|
||||
};
|
||||
for s in crate::subscriptions(&state.ctx)? {
|
||||
for s in crate::subscriptions(&state.ctx).await? {
|
||||
// A feed from an OPML subscription comes back with the OPML itself.
|
||||
if s.managed || !mine.contains(&s.id) {
|
||||
continue;
|
||||
@@ -1682,7 +1674,7 @@ async fn export_opml(
|
||||
let title = state
|
||||
.ctx
|
||||
.db
|
||||
.feed_summary(&s.id)
|
||||
.feed_summary(&s.id).await
|
||||
.ok()
|
||||
.and_then(|sum| sum.title)
|
||||
.unwrap_or_else(|| s.id.clone());
|
||||
@@ -1716,7 +1708,7 @@ async fn import_opml(
|
||||
// file arrives as text, is read here, and is gone when the request ends.
|
||||
let doc = opml::OPML::from_str(&body.xml)
|
||||
.map_err(|e| ApiError::bad_request(format!("that is not an OPML file: {e}")))?;
|
||||
let (added, already) = crate::subscribe_opml(&state.ctx, &state.config_path, &doc, user.id)?;
|
||||
let (added, already) = crate::subscribe_opml(&state.ctx, &doc, user.id).await?;
|
||||
if added > 0 {
|
||||
scan_soon(&state, None).await;
|
||||
}
|
||||
@@ -1790,8 +1782,7 @@ async fn patch_settings(
|
||||
if let Some(v) = body.max_age_days {
|
||||
cfg.general.max_age_days = v;
|
||||
}
|
||||
cfg.save(&state.config_path)?;
|
||||
state.ctx.reload_cfg(&state.config_path)?;
|
||||
state.ctx.store_cfg(cfg).await?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user