Compare commits
29 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 552d20443e | |||
| 367f431e1f | |||
| ef8e2526f2 | |||
| 5058adc93e | |||
| 9e39f2d756 | |||
| b86062b97a | |||
| 95acde3046 | |||
| 0b4f5443b7 | |||
| e699baa22a | |||
| 00bd58ac9d | |||
| cc17b1ddab | |||
| 119336a20d | |||
| 02a46bb500 | |||
| 3e384f34f9 | |||
| d238681ec1 | |||
| 0395717c14 | |||
| 8ce68a881a | |||
| e7c59489ee | |||
| f7f4b466dc | |||
| ec8fd5dd86 | |||
| a387012c69 | |||
| a00516687a | |||
| 38ebc7b02b | |||
| 54d827f655 | |||
| e2593eaa50 | |||
| d52ecfcbd7 | |||
| 142610e8b8 | |||
| 43acc62259 | |||
| 65fdab8b13 |
@@ -32,12 +32,13 @@ lines had no trace id, the access log's time was `ms`, and each event was logged
|
||||
| target | fields | what |
|
||||
|---|---|---|
|
||||
| `ipx::http` | `method`, `path`, `route`, `status`, `duration_ms` | one per web request; `route` is the pattern, empty for an unrouted path |
|
||||
| `ipx::scan` | `ev` and the event's own: `feed`, `new`, `downloaded`, `failed`, `bytes`, `msg`, `url`, `feeds`, `reason`; on a failure `error.type` and, from an HTTP error, `http.response.status_code` | the daemon's events, one line each, in words; warnings are feed and download failures |
|
||||
| `ipx::scan` | `ev` and the event's own: `feed`, `new`, `downloaded`, `failed`, `bytes`, `msg`, `url`, `feeds`, `reason`, `from`, `to`; on a failure `error.type` and, from an HTTP error, `http.response.status_code` | the daemon's events, one line each, in words; warnings are feed and download failures |
|
||||
| `ipx::io` | `ev`, `feeds`, `pending`, `downloaded` on the `status` reply | commands arriving (`-> {...}`) and the healthcheck's answer |
|
||||
| `ipx` | message, sometimes fields | start-up, shutdown, account and config messages |
|
||||
|
||||
Events (`ev`): `feed_start`, `feed_done` (new, downloaded, failed, torrents), `feed_skip` (not due,
|
||||
routine), `feed_error` (msg), `download_done` (bytes), `download_error` (msg, url),
|
||||
routine), `feed_error` (msg), `feed_moved` (from, to: a permanent redirect followed, the address
|
||||
updated), `download_done` (bytes), `download_error` (msg, url),
|
||||
`torrent_deferred`, `reaped`, `scan_done` (feeds checked), `reap_done`, `status` (feeds, pending,
|
||||
downloaded: the healthcheck's, every 30s), `error` (msg).
|
||||
|
||||
@@ -85,6 +86,12 @@ a count says something happened, the lines and traces say why.
|
||||
crash or an OOM kill: check `docker inspect iPX -f '{{.State.OOMKilled}} {{.RestartCount}}'`
|
||||
and the lines just before it. A pending count that only grows means downloads are not
|
||||
keeping up or not running.
|
||||
|
||||
Since 2026-10-02 the daemon sleeps until the next feed is due (at most 10 minutes) instead of
|
||||
scanning every minute (#114), so expect tens of scans in 6 hours, not 360, nearly all with
|
||||
`feeds` above 0; none at all for over 10 minutes means the worker is stuck. And `pending` is
|
||||
the real queue (#113): files a scan will download on its own. Back-catalogue files are
|
||||
`held`, listed but not counted, so it is usually 0 or a handful.
|
||||
5. **Slow and failed traces.**
|
||||
```
|
||||
$Q traces '{resource.deployment.environment.name="production" && duration > 5s}'
|
||||
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -5,3 +5,4 @@
|
||||
/web/dist
|
||||
|
||||
.claude/settings.local.json
|
||||
__pycache__/
|
||||
|
||||
46
CHANGELOG.md
46
CHANGELOG.md
@@ -7,6 +7,49 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.10.0] - 2026-10-05
|
||||
|
||||
### Added
|
||||
|
||||
- API tokens: in Settings, make a named token that lets a script or an agent use iPX as you, sent as `Authorization: Bearer`, and revoke it there. Each shows when it was last used.
|
||||
- In the Directory, a feed you do not subscribe to opens a page of its own: its cover, its category, how many people here subscribe, what the feed says it is, and its latest episodes or posts, with a button to subscribe. Before, only its subscribe button did anything.
|
||||
- Feeds that don't set a category, as most blogs don't, now get one automatically, so they show up under a category in the Directory. iPX picks the closest of Apple's podcast categories from the feed's title and latest headlines. This happens once per feed: the first time a new feed is read, or for a feed you already had, the next time you force a refresh. It needs `TYPESAFE_KEY` set. An admin can change the category in the feed's settings.
|
||||
- Artwork is kept on disk once shown and loads from iPX, not from each publisher's server: fast after the first time, and still there when the publisher's server is not. The admin page sets how much is kept (500 MB by default).
|
||||
- `ipx add --list --category News <url>` puts a feed in the Directory for anyone to subscribe to,
|
||||
and it stays there when its last subscriber leaves. Run for a feed already in the catalogue,
|
||||
it lists it or sets its category.
|
||||
- Feeds nobody subscribes to that are dead for a month or quiet for a year are removed, so the
|
||||
Directory lists feeds worth taking.
|
||||
|
||||
### Changed
|
||||
|
||||
- The Directory is laid out as Apple's is. It opens on the ten most subscribed feeds, ranked, and the categories as tiles showing some of their shows' covers. A category opens a page of its own with its most subscribed feeds, its subcategories (Tech News under News, Video Games under Leisure) and its feeds; See all lists every one. This replaces the grid of every feed under rows of category buttons.
|
||||
- The most subscribed feeds are the Directory's first section instead of a place of their own: Popular is gone from the feed list, and so is its key, g p.
|
||||
- A pinned item in a list wears the same disc as a pinned feed, in the theme's accent, instead of a plain solid pin.
|
||||
- The Directory can be sorted by name, A to Z or Z to A, or by most subscribers. It still opens A to Z.
|
||||
- An item published without a title shows its opening words, in plain text rather than bold, instead of "(untitled)"; one with no text either shows its file's name, or its show and date. Opened, it starts with its text.
|
||||
- A feed that has moved for good (a permanent redirect) is followed to its new address, which iPX then reads from, and says so in the log as `feed_moved`. A temporary redirect changes nothing.
|
||||
- The daemon sleeps until the next feed is due, at most ten minutes, instead of looking every minute; refreshing or adding a feed still wakes it at once.
|
||||
- A pinned feed's pin sits on the corner of its artwork, as a failing feed's mark does, instead of before its name.
|
||||
- The Directory lists feeds nobody subscribes to yet; its most subscribed still lists what people subscribe to.
|
||||
A feed nobody subscribes to is checked once a day, and at once when someone subscribes. A
|
||||
listed feed downloads nothing until someone subscribes.
|
||||
- The Directory loads faster: it asked the database three questions per feed.
|
||||
|
||||
### Fixed
|
||||
|
||||
- A database on SQLite, the default, no longer waits for the disk after nearly every write. On a slow disk a scan of 1,500 feeds took most of an hour and now takes half a minute. Postgres is unaffected.
|
||||
- The search box finds a feed in the Directory by name. It did nothing there.
|
||||
- Feeds that file themselves under Apple's Football or Soccer show under Sports, and those under the old Software How-To under Technology, instead of as categories of their own.
|
||||
- The download queue, in `ipx status`, `/api/status` and the dashboard, counts only what iPX will download on its own: older episodes beyond a show's limit, and files of feeds that do not download automatically, are listed but no longer counted as waiting.
|
||||
- Artwork published on http is stored on https when its host serves it there, so the page loads it directly; the rest still comes through iPX.
|
||||
- An iPhone adding the site to its home screen finds the icon at the first address it tries.
|
||||
- A feed whose server hangs no longer holds up every scan: a feed gets 30 seconds, and connecting anywhere 10.
|
||||
|
||||
### Security
|
||||
|
||||
- A feed's items and files reach only the people who subscribe to it. Anyone signed in could read any feed's items, a paid feed's included, with the addresses of its files, which can carry the subscriber's key, and could play, download or delete its files.
|
||||
|
||||
## [0.9.1] - 2026-09-29
|
||||
|
||||
### Added
|
||||
@@ -692,7 +735,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- Torrent enclosures through librqbit, seeding to a ratio or a time, with a stall timeout.
|
||||
- `ipx import` and `ipx export` for OPML, and systemd units in `contrib/`.
|
||||
|
||||
[unreleased]: https://git.sdf1.net/rays/ipx/compare/v0.9.0...main
|
||||
[unreleased]: https://git.sdf1.net/rays/ipx/compare/v0.10.0...main
|
||||
[0.10.0]: https://git.sdf1.net/rays/ipx/compare/v0.9.1...v0.10.0
|
||||
[0.9.1]: https://git.sdf1.net/rays/ipx/compare/v0.9.0...v0.9.1
|
||||
[0.9.0]: https://git.sdf1.net/rays/ipx/compare/v0.8.4...v0.9.0
|
||||
[0.8.4]: https://git.sdf1.net/rays/ipx/compare/v0.8.3...v0.8.4
|
||||
|
||||
@@ -182,7 +182,8 @@ Non-trivial logic leaves one runnable check behind. Pure functions (`merge_polic
|
||||
command, the healthcheck's `ipx status` included, so it must never write when nothing is
|
||||
missing: SeaORM's experimental schema sync dropped and remade an index on every open, the
|
||||
write lock that took made `ipx status` time out behind a busy daemon, and it was removed for
|
||||
it. A new column on an existing table needs its own `ALTER`; nothing adds one for you.
|
||||
it. A new column on an existing table needs its own `ALTER`; nothing adds one for you. Look for
|
||||
the column first and alter only when it is missing, as `feeds.description` does.
|
||||
* **SQL written by hand in `db.rs` has to run on SQLite and Postgres both** (issue #18): `$1`
|
||||
parameters, bound only if used; `ON CONFLICT`, not `INSERT OR IGNORE`; yes/no columns tested
|
||||
as themselves (`NOT coalesce(s.read, false)`) and written as `true`/`false`, never compared to
|
||||
|
||||
3
Cargo.lock
generated
3
Cargo.lock
generated
@@ -1820,7 +1820,7 @@ checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
|
||||
|
||||
[[package]]
|
||||
name = "ipx"
|
||||
version = "0.9.1"
|
||||
version = "0.10.0"
|
||||
dependencies = [
|
||||
"ammonia",
|
||||
"anyhow",
|
||||
@@ -1843,6 +1843,7 @@ dependencies = [
|
||||
"sea-orm",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2 0.10.9",
|
||||
"tokio",
|
||||
"toml",
|
||||
"tower",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "ipx"
|
||||
version = "0.9.1"
|
||||
version = "0.10.0"
|
||||
edition = "2024"
|
||||
|
||||
[dependencies]
|
||||
@@ -25,6 +25,7 @@ rss = "2.1.1"
|
||||
sea-orm = { version = "2.0.3", default-features = false, features = ["sqlx-sqlite", "sqlx-postgres", "runtime-tokio-rustls", "macros", "with-json", "sqlite-use-returning-for-3_35"] }
|
||||
serde = { version = "1.0.229", features = ["derive"] }
|
||||
serde_json = "1.0.151"
|
||||
sha2 = "0.10.9"
|
||||
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "fs", "io-util", "net", "sync", "time", "signal"] }
|
||||
toml = "1.1.5"
|
||||
tower = { version = "0.5.3", features = ["util"] }
|
||||
|
||||
@@ -9,7 +9,7 @@ behind iPodderX (2004-2008, Ray Slakinski & August Trometer).
|
||||
|
||||
## What it does
|
||||
|
||||
- **The web UI.** It has a toolbar, and a feed list that opens with Directory, Popular and All
|
||||
- **The web UI.** It has a toolbar, and a feed list that opens with the Directory, Currently Listening and All
|
||||
Subscriptions. Items sit in a sortable table with a Files pane, and there is a player bar. It
|
||||
comes in Dark, Light and Classic themes, and works on a phone.
|
||||
- **Several people, one copy.** Each person has their own subscriptions and their own read, pinned
|
||||
|
||||
@@ -51,8 +51,8 @@ build needs node and `npm ci` run once.
|
||||
## Data model
|
||||
|
||||
```
|
||||
feeds id, url, title, image, etag, last_modified, last_checked, ttl_mins,
|
||||
last_error, orphaned, group_id, managed
|
||||
feeds id, url, title, image, category, description, etag, last_modified,
|
||||
last_checked, ttl_mins, last_error, orphaned, group_id, managed
|
||||
entries feed_id, guid, title, link, published, description, first_seen,
|
||||
image, duration, episode, season PK (feed_id, guid)
|
||||
enclosures id, feed_id, guid, url UNIQUE, mime, length, path, state,
|
||||
@@ -71,7 +71,8 @@ before accounts; two bugs came from queries still reading them, and they were dr
|
||||
Schema changes: the tables are the entities in `src/entity.rs`, and `Db::open` creates whatever
|
||||
table or index a database is missing from them (`db::create_missing`), with `IF NOT EXISTS`. It
|
||||
never alters a table that exists, so a new column on one needs its own `ALTER` in
|
||||
`create_missing`, or `sea-orm-migration` once there are several. `Db::memory()` builds its
|
||||
`create_missing`, looked for first and run only when it is missing, as `feeds.description`'s is,
|
||||
or `sea-orm-migration` once there are several. `Db::memory()` builds its
|
||||
database the same way, so the tests run on the schema production gets. A database from before
|
||||
0.7 takes its last columns from the old `migrate()`, so it upgrades through a 0.7 release first.
|
||||
|
||||
@@ -110,7 +111,8 @@ so a long download had to be able to notice the signal itself.
|
||||
## HTTP API
|
||||
|
||||
Everything below `/api` needs a signed-in user; the browser gets a redirect to `/login`, anything
|
||||
else a `401`.
|
||||
else a `401`. A feed's items and files (its entries, `download-latest`, `/api/enclosures/{id}` and
|
||||
`/media/{id}`) answer only someone who subscribes to the feed; anyone else gets a `404`.
|
||||
|
||||
| Route | |
|
||||
|---|---|
|
||||
@@ -126,6 +128,7 @@ else a `401`.
|
||||
| `POST /api/enclosures/{id}/download`, `DELETE /api/enclosures/{id}` | `?force=true` overrides the shared-file warning |
|
||||
| `POST /api/fetch` | |
|
||||
| `GET /api/opml`, `POST /api/opml` | export your subscriptions; subscribe to every feed in an OPML |
|
||||
| `GET /api/directory/{id}` | a listed feed's description and latest twenty items, for its page before you subscribe: title, link, date, length and text, never a file or its address, and only for a feed the directory lists |
|
||||
| `GET /api/popular`, `GET /api/directory`, `POST /api/popular/{id}` | the ten most subscribed feeds, and every listable feed A to Z, with an OPML's feeds in place of the OPML and everyone counted (id, title, art, count, whether it is yours, the feed's iTunes category, whether it carries audio or video; never a URL, never a private feed); subscribe by id |
|
||||
| `GET /api/settings`, `PATCH /api/settings` | admin-only to write |
|
||||
| `GET /api/users`, `POST /api/users`, `PATCH /api/users/{id}`, `DELETE /api/users/{id}` | admin-only; the only admin cannot be demoted or removed |
|
||||
|
||||
@@ -4,7 +4,7 @@ Two places. **config.toml** holds what ipx needs before it reaches its database,
|
||||
who gets in: where things are (`download_dir`, `socket`, `organize`), `[torrent]` and `[web]`.
|
||||
**The database** holds the catalogue of feeds (`[feeds.<id>]` below) and the server settings the
|
||||
admin page edits (`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check`,
|
||||
`media_types`). Change those in the web UI, or with `ipx add`, `ipx rm` and `ipx import`; they
|
||||
`media_types`, `art_cache_mb`). Change those in the web UI, or with `ipx add`, `ipx rm` and `ipx import`; they
|
||||
take effect without a restart.
|
||||
|
||||
The first time ipx meets a database that holds no catalogue, it takes the feeds and those
|
||||
@@ -39,9 +39,10 @@ max_total_gb = 50 # 0 = unlimited
|
||||
max_age_days = 30 # 0 = keep forever
|
||||
max_new_per_check = 3 # per feed, per scan. 0 = unlimited
|
||||
media_types = ["audio", "video"]
|
||||
art_cache_mb = 500 # artwork kept on disk. 0 = none
|
||||
```
|
||||
|
||||
`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check` and `media_types` move into the
|
||||
`schedule`, `max_total_gb`, `max_age_days`, `max_new_per_check`, `media_types` and `art_cache_mb` move into the
|
||||
database as described above; `download_dir`, `socket` and `organize` stay in config.toml.
|
||||
|
||||
* **`schedule`** — how often feeds are re-checked. A feed's own `<ttl>` still wins when it asks to
|
||||
@@ -54,6 +55,10 @@ database as described above; `download_dir`, `socket` and `organize` stay in con
|
||||
read it. `0` disables it entirely.
|
||||
* **`max_age_days`** — items older than this with no file on disk are pruned from the database.
|
||||
Kept ones stay. `0` disables it.
|
||||
* **`art_cache_mb`** — how much show and episode artwork iPX keeps on disk, in `art/` beside the
|
||||
database, so the page loads it from iPX rather than from every publisher's server. Over this,
|
||||
what has gone longest unshown goes first. `0` keeps none: artwork still comes through iPX, fetched
|
||||
each time. 500 by default.
|
||||
* **`max_new_per_check`** — how many of a feed's newest episodes are downloaded; older ones stay
|
||||
listed to download by hand. It stops a new subscription pulling a whole back catalogue.
|
||||
`0` means every episode, for an archive; set it on the feeds you want archived, since on the
|
||||
@@ -124,6 +129,7 @@ folder = "Accidental Tech Podcast" # default: the feed title
|
||||
schedule = "every 6h" # overrides [general] for this feed
|
||||
media_types = ["audio"] # overrides [general] for this feed
|
||||
category = "Technology" # the Directory's, if the feed names none
|
||||
listed = true # in the Directory with no subscribers
|
||||
username = "ray" # HTTP basic auth
|
||||
password_env = "IPX_ATP_PASS" # preferred over a literal `password`
|
||||
```
|
||||
@@ -133,6 +139,12 @@ With more than one account, **`keywords`, `auto_download`, `allow_explicit` and
|
||||
config.toml are the fallback for a feed nobody has claimed. The keys above describe the feed itself
|
||||
and are the same for everyone. See [users.md](users.md).
|
||||
|
||||
**The Directory** lists every feed in the catalogue, whether or not anyone subscribes to it yet.
|
||||
To put one there for others to find, `ipx add --list --category News <url>`: it subscribes no one,
|
||||
and a listed feed stays when its last subscriber leaves, where another is dropped. A feed nobody
|
||||
subscribes to is checked once a day. One that is dead (failing for 30 days) or quiet (nothing
|
||||
new in a year), with nobody subscribed and no file on disk, is removed from the catalogue.
|
||||
|
||||
Feeds derived from a subscribed OPML are **not** in the catalogue: the OPML is the source of truth
|
||||
and they are re-derived on every scan. Editing one in the UI promotes it to a catalogue entry.
|
||||
|
||||
|
||||
14
docs/sso.md
14
docs/sso.md
@@ -233,5 +233,19 @@ Set `auto_create_users = false` once everyone who should have an account has one
|
||||
proxy vouching for an unknown name is logged and refused. Make people ahead of time instead, with
|
||||
the exact name the header will carry.
|
||||
|
||||
## API tokens for scripts and agents
|
||||
|
||||
Anyone can make API tokens for their own account in Settings. A token is sent as
|
||||
`Authorization: Bearer ipx_...` and acts as the person who made it, admin only if they are. Only
|
||||
its SHA-256 is kept; it is shown once, when made, and revoked there too.
|
||||
|
||||
Through the tunnel, Cloudflare Access turns a request with no Access sign-in towards Authentik
|
||||
before ipx sees it, so a token alone does not get in that way. On the LAN, `http://192.168.1.130:8099`
|
||||
takes it directly. From outside, make an Access service token, add a Service Auth policy for it to
|
||||
the `ipodderx` application, and send `CF-Access-Client-Id` and `CF-Access-Client-Secret` beside the
|
||||
`Authorization` header: Access lets the request through, vouches for no name, and ipx takes the
|
||||
API token as who is asking. Do not bypass Access for `/api/*` instead; the token would then be the
|
||||
only thing between the internet and the API.
|
||||
|
||||
See also [users.md](users.md) for what several people share, [configuration.md](configuration.md)
|
||||
for every `[web]` key, and [cli.md](cli.md) for the `ipx user` commands.
|
||||
|
||||
@@ -70,11 +70,15 @@ list with their own read state. Unsubscribing removes it from their list alone;
|
||||
subscriber leaves does the feed stop being scanned, and even then its files and history stay, so
|
||||
re-subscribing does not pull the back catalogue again.
|
||||
|
||||
**Popular** and **Directory** sit at the top of the feed list, above your own feeds. Popular, also
|
||||
shown in the Add feed dialog, lists the ten feeds with the most subscribers on this server, you
|
||||
included. Directory shows every one of them A to Z as a grid of cover art. Above it, a filter
|
||||
picks Podcasts (anything with audio or video) or Blogs (the rest), and chips pick the category
|
||||
each show gives itself in iTunes; the two combine. Your own feeds are marked Subscribed.
|
||||
**Directory** sits at the top of the feed list, above your own feeds. It opens on the ten feeds
|
||||
with the most subscribers on this server, you included, ranked, then the categories, each a tile
|
||||
showing a few of its shows' covers. A category has a page of its own: its most subscribed feeds,
|
||||
then all of them as a grid of cover art, with its subcategories along the top (Tech News under
|
||||
News, Video Games under Leisure) and a choice of order. See all lists every feed, and the search
|
||||
box finds one by name. A feed you do not subscribe to opens a page of its own, with what it says it
|
||||
is and its latest episodes or posts, so you can see what it is first. Every page can be narrowed to Podcasts (anything with audio or video) or
|
||||
Blogs (the rest). A show's category is the one it gives itself in iTunes, or for one that names
|
||||
none, one an admin or iPX picked for it. Your own feeds are marked Subscribed.
|
||||
It shows a title, artwork and a count, never a URL or who reads it. An OPML subscription is listed
|
||||
as the feeds inside it, one by one, and never the OPML itself, so you can take just the shows you
|
||||
want. Anything that looks private is left out: a login configured for the feed, credentials in its URL,
|
||||
|
||||
92
src/art.rs
Normal file
92
src/art.rs
Normal file
@@ -0,0 +1,92 @@
|
||||
//! Artwork kept on disk, so the page loads it from iPX: fast after the first time, nothing asked
|
||||
//! of each publisher's server for every visit, and still there when a publisher is not.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
pub fn dir() -> PathBuf {
|
||||
crate::config::data_dir().join("art")
|
||||
}
|
||||
|
||||
/// Where an image's bytes are kept; its content type is beside it, in `<name>.type`.
|
||||
// ponytail: std's hasher, 64 bits, keyed by the URL. Its algorithm may change with Rust, which
|
||||
// only makes the cache miss once and refill; a collision among a few thousand images is about
|
||||
// one in 10^12. Move to a SHA if either ever matters.
|
||||
pub fn path(url: &str) -> PathBuf {
|
||||
use std::hash::{Hash, Hasher};
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
url.hash(&mut h);
|
||||
dir().join(format!("{:016x}", h.finish()))
|
||||
}
|
||||
|
||||
/// A kept image and its type, marked as just used, which is what keeps it from being trimmed.
|
||||
pub fn read(file: &Path) -> Option<(String, Vec<u8>)> {
|
||||
let kind = std::fs::read_to_string(file.with_extension("type")).ok()?;
|
||||
let body = std::fs::read(file).ok()?;
|
||||
if let Ok(f) = std::fs::File::options().write(true).open(file) {
|
||||
let _ = f.set_modified(std::time::SystemTime::now());
|
||||
}
|
||||
Some((kind, body))
|
||||
}
|
||||
|
||||
/// Keeps an image, written aside and renamed into place, so a page asking for it meanwhile
|
||||
/// never reads half of one.
|
||||
pub fn write(file: &Path, kind: &str, body: &[u8]) -> std::io::Result<()> {
|
||||
std::fs::create_dir_all(file.parent().unwrap_or(Path::new(".")))?;
|
||||
let tmp = file.with_extension("part");
|
||||
std::fs::write(&tmp, body)?;
|
||||
std::fs::write(file.with_extension("type"), kind)?;
|
||||
std::fs::rename(&tmp, file)
|
||||
}
|
||||
|
||||
/// Removes the least recently used images until what is kept fits in `limit` bytes; 0 empties it.
|
||||
/// Returns how many went.
|
||||
pub fn trim(dir: &Path, limit: u64) -> usize {
|
||||
let Ok(entries) = std::fs::read_dir(dir) else { return 0 };
|
||||
let mut kept: Vec<(std::time::SystemTime, u64, PathBuf)> = entries
|
||||
.flatten()
|
||||
.map(|e| e.path())
|
||||
.filter(|p| p.extension().is_none())
|
||||
.filter_map(|p| {
|
||||
let m = p.metadata().ok()?;
|
||||
Some((m.modified().ok()?, m.len(), p))
|
||||
})
|
||||
.collect();
|
||||
let mut total: u64 = kept.iter().map(|(_, n, _)| n).sum();
|
||||
kept.sort();
|
||||
let mut gone = 0;
|
||||
for (_, n, p) in kept {
|
||||
if total <= limit {
|
||||
break;
|
||||
}
|
||||
let _ = std::fs::remove_file(p.with_extension("type"));
|
||||
if std::fs::remove_file(&p).is_ok() {
|
||||
total -= n;
|
||||
gone += 1;
|
||||
}
|
||||
}
|
||||
gone
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn the_least_recently_used_go_first_until_it_fits() {
|
||||
let d = std::env::temp_dir().join(format!("ipx-art-test-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&d);
|
||||
for (name, age) in [("old", 300), ("mid", 200), ("new", 100)] {
|
||||
let f = d.join(name);
|
||||
write(&f, "image/png", &[0; 1000]).unwrap();
|
||||
let t = std::time::SystemTime::now() - std::time::Duration::from_secs(age);
|
||||
std::fs::File::options().write(true).open(&f).unwrap().set_modified(t).unwrap();
|
||||
}
|
||||
// Shown just now: no longer the oldest.
|
||||
assert_eq!(read(&d.join("old")).unwrap().0, "image/png");
|
||||
assert_eq!(trim(&d, 2000), 1);
|
||||
assert!(!d.join("mid").exists() && !d.join("mid.type").exists());
|
||||
assert!(d.join("old").exists() && d.join("new").exists());
|
||||
assert_eq!(trim(&d, 0), 2);
|
||||
let _ = std::fs::remove_dir_all(&d);
|
||||
}
|
||||
}
|
||||
28
src/auth.rs
28
src/auth.rs
@@ -40,6 +40,19 @@ pub fn new_session_token() -> String {
|
||||
bytes.iter().map(|b| format!("{b:02x}")).collect()
|
||||
}
|
||||
|
||||
/// A new API token: `ipx_` and a session's 256 random bits, the prefix so one found in a log
|
||||
/// or a file says what it opens (#123).
|
||||
pub fn new_api_token() -> String {
|
||||
format!("ipx_{}", new_session_token())
|
||||
}
|
||||
|
||||
/// What is kept of an API token. A fast hash is enough: the token is 256 random bits, not a
|
||||
/// password, so there is nothing to guess from a stolen hash.
|
||||
pub fn api_token_hash(token: &str) -> String {
|
||||
use sha2::Digest;
|
||||
sha2::Sha256::digest(token.as_bytes()).iter().map(|b| format!("{b:02x}")).collect()
|
||||
}
|
||||
|
||||
fn getrandom(buf: &mut [u8]) -> std::io::Result<()> {
|
||||
use std::io::Read;
|
||||
std::fs::File::open("/dev/urandom")?.read_exact(buf)
|
||||
@@ -74,6 +87,21 @@ mod tests {
|
||||
assert!(hash_password("short").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn api_tokens_are_prefixed_and_hash_to_hex() {
|
||||
let t = new_api_token();
|
||||
assert!(t.starts_with("ipx_") && t.len() == 68);
|
||||
let h = api_token_hash(&t);
|
||||
assert_eq!(h.len(), 64);
|
||||
assert_eq!(h, api_token_hash(&t), "the same token, the same hash");
|
||||
assert_ne!(h, api_token_hash(&new_api_token()));
|
||||
assert_eq!(
|
||||
api_token_hash("abc"),
|
||||
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
|
||||
"SHA-256"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn session_tokens_are_long_and_distinct() {
|
||||
let a = new_session_token();
|
||||
|
||||
@@ -39,6 +39,9 @@ pub struct General {
|
||||
/// image in an <enclosure>, so taking everything filled the disk with artwork and
|
||||
/// counted it as episodes. Empty means take anything.
|
||||
pub media_types: Vec<String>,
|
||||
/// How much artwork iPX keeps on disk, in MB, so the page loads it from iPX rather than from
|
||||
/// every publisher; the least recently shown goes first. 0 keeps none.
|
||||
pub art_cache_mb: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
|
||||
@@ -163,6 +166,10 @@ pub struct Feed {
|
||||
/// Preferred over `password`: name of an env var holding the password.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub password_env: Option<String>,
|
||||
/// Put in the Directory by an admin (`ipx add --list`): listed with no subscribers, and kept
|
||||
/// in the catalogue when the last one leaves, where anyone else's feed is dropped.
|
||||
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
|
||||
pub listed: bool,
|
||||
}
|
||||
|
||||
fn yes() -> bool {
|
||||
@@ -180,6 +187,7 @@ impl Default for General {
|
||||
max_age_days: 0,
|
||||
max_new_per_check: 3,
|
||||
media_types: vec!["audio".into(), "video".into()],
|
||||
art_cache_mb: 500,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -290,10 +298,18 @@ pub struct Stored {
|
||||
pub max_age_days: u64,
|
||||
pub max_new_per_check: usize,
|
||||
pub media_types: Vec<String>,
|
||||
/// Settings saved before it existed have none: they get the default.
|
||||
#[serde(default = "art_cache_mb")]
|
||||
pub art_cache_mb: u64,
|
||||
}
|
||||
|
||||
fn art_cache_mb() -> u64 {
|
||||
General::default().art_cache_mb
|
||||
}
|
||||
|
||||
/// `[general]` keys that live in the database once it holds the configuration.
|
||||
const STORED_KEYS: [&str; 5] = ["schedule", "max_total_gb", "max_age_days", "max_new_per_check", "media_types"];
|
||||
const STORED_KEYS: [&str; 6] =
|
||||
["schedule", "max_total_gb", "max_age_days", "max_new_per_check", "media_types", "art_cache_mb"];
|
||||
|
||||
impl Stored {
|
||||
pub fn of(cfg: &Config) -> Self {
|
||||
@@ -304,6 +320,7 @@ impl Stored {
|
||||
max_age_days: g.max_age_days,
|
||||
max_new_per_check: g.max_new_per_check,
|
||||
media_types: g.media_types.clone(),
|
||||
art_cache_mb: g.art_cache_mb,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -314,6 +331,7 @@ impl Stored {
|
||||
g.max_age_days = self.max_age_days;
|
||||
g.max_new_per_check = self.max_new_per_check;
|
||||
g.media_types = self.media_types;
|
||||
g.art_cache_mb = self.art_cache_mb;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -571,6 +589,12 @@ mod tests {
|
||||
assert!(wanted_media(Some("image/jpeg"), &["image/jpeg".to_string()]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn settings_saved_before_the_art_cache_keep_the_default() {
|
||||
let old = r#"{"schedule":"every 1h","max_total_gb":0.0,"max_age_days":0,"max_new_per_check":3,"media_types":["audio"]}"#;
|
||||
assert_eq!(serde_json::from_str::<Stored>(old).unwrap().art_cache_mb, 500);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn slugs_are_readable_and_unique() {
|
||||
assert_eq!(slug("Accidental Tech Podcast"), "accidental-tech-podcast");
|
||||
@@ -584,7 +608,7 @@ mod tests {
|
||||
taken.insert("the-daily".to_string(), Feed {
|
||||
url: "u".into(), folder: None, group: None, media_types: None, schedule: None, keywords: vec![], allow_explicit: false,
|
||||
auto_download: true, max_new_per_check: None, username: None,
|
||||
password: None, password_env: None, category: None,
|
||||
password: None, password_env: None, category: None, listed: false,
|
||||
});
|
||||
assert_eq!(unique_slug("The Daily", &taken), "the-daily-2");
|
||||
}
|
||||
@@ -605,6 +629,7 @@ mod tests {
|
||||
password: Some("literal".into()),
|
||||
password_env: None,
|
||||
category: None,
|
||||
listed: false,
|
||||
};
|
||||
assert_eq!(f.password().as_deref(), Some("literal"));
|
||||
|
||||
|
||||
325
src/db.rs
325
src/db.rs
@@ -2,7 +2,7 @@
|
||||
//! per-feed .ipxd plists, history.dat and qmcache.dat.
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use crate::entity::{blocklists, catalogue, enclosures, entries, feeds, hidden, sessions, settings, subscriptions, users};
|
||||
use crate::entity::{api_tokens, blocklists, catalogue, enclosures, entries, feeds, hidden, sessions, settings, subscriptions, users};
|
||||
use sea_orm::sea_query::{Expr, Func};
|
||||
use sea_orm::{
|
||||
ActiveModelTrait, ColumnTrait, ConnectionTrait, EntityTrait, PaginatorTrait, QueryFilter, QueryOrder, Set,
|
||||
@@ -59,6 +59,7 @@ async fn create_missing(orm: &sea_orm::DatabaseConnection) -> Result<()> {
|
||||
schema.create_table_from_entity(settings::Entity),
|
||||
schema.create_table_from_entity(blocklists::Entity),
|
||||
schema.create_table_from_entity(hidden::Entity),
|
||||
schema.create_table_from_entity(api_tokens::Entity),
|
||||
] {
|
||||
orm.execute(table.if_not_exists()).await.context("creating the schema")?;
|
||||
}
|
||||
@@ -68,6 +69,21 @@ async fn create_missing(orm: &sea_orm::DatabaseConnection) -> Result<()> {
|
||||
] {
|
||||
orm.execute_unprepared(sql).await.with_context(|| sql.to_owned())?;
|
||||
}
|
||||
// A column added to a table that already exists, which CREATE TABLE IF NOT EXISTS leaves
|
||||
// as it was. Looked for first, and added only when it is missing: this runs on every open,
|
||||
// the healthcheck's `ipx status` included, and an ALTER takes a lock that waits behind a
|
||||
// busy daemon, which is what made the healthcheck time out once before.
|
||||
if orm.execute_unprepared("SELECT description FROM feeds LIMIT 0").await.is_err() {
|
||||
// A feed is read whole only when it has changed since the last read, so each one's
|
||||
// description would wait for its next item. Forgetting how each was last seen makes
|
||||
// its next read, when it is due anyway, a whole one.
|
||||
for sql in [
|
||||
"ALTER TABLE feeds ADD COLUMN description TEXT",
|
||||
"UPDATE feeds SET etag = NULL, last_modified = NULL",
|
||||
] {
|
||||
orm.execute_unprepared(sql).await.with_context(|| sql.to_owned())?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -98,11 +114,17 @@ fn is_postgres(location: &str) -> bool {
|
||||
location.starts_with("postgres://") || location.starts_with("postgresql://")
|
||||
}
|
||||
|
||||
/// sqlx's defaults for SQLite are what ipx wants: foreign keys on, and a five-second wait for a
|
||||
/// lock, which is what rusqlite was set to.
|
||||
/// sqlx's defaults for SQLite are what ipx wants, foreign keys on and a five-second wait for a
|
||||
/// lock, which is what rusqlite was set to, but for one.
|
||||
async fn connect(location: &str) -> Result<sea_orm::DatabaseConnection> {
|
||||
let mut opts = sea_orm::ConnectOptions::new(url_for(location));
|
||||
opts.sqlx_logging(false);
|
||||
// NORMAL, SQLite's own advice for WAL: synced at checkpoints, not after every commit. FULL,
|
||||
// the default, waited for the disk after nearly every statement, and a feed's items are
|
||||
// written a row at a time: 1.3s a feed on Tower's /tmp, where one synchronous write takes
|
||||
// 52ms, and 39s to subscribe an admin to 1,500 feeds (#135). A crash of ipx loses nothing;
|
||||
// only a power cut can lose the last transactions, and it never corrupts the file.
|
||||
opts.map_sqlx_sqlite_opts(|o| o.synchronous(sea_orm::sqlx::sqlite::SqliteSynchronous::Normal));
|
||||
sea_orm::Database::connect(opts)
|
||||
.await
|
||||
.with_context(|| format!("opening {}", redact(location)))
|
||||
@@ -313,6 +335,11 @@ impl Db {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// What a feed says it is, for its page in the Directory.
|
||||
pub async fn feed_description(&self, feed_id: &str) -> Result<Option<String>> {
|
||||
Ok(feeds::Entity::find_by_id(feed_id.to_owned()).one(&self.orm).await?.and_then(|f| f.description))
|
||||
}
|
||||
|
||||
#[tracing::instrument(skip_all)]
|
||||
pub async fn feed_summary(&self, feed_id: &str) -> Result<FeedSummary> {
|
||||
let mut sum = feeds::Entity::find_by_id(feed_id.to_owned())
|
||||
@@ -438,19 +465,27 @@ pub struct HttpState {
|
||||
pub error_since: Option<i64>,
|
||||
}
|
||||
|
||||
impl From<feeds::Model> for HttpState {
|
||||
fn from(f: feeds::Model) -> Self {
|
||||
HttpState {
|
||||
etag: f.etag,
|
||||
last_modified: f.last_modified,
|
||||
last_checked: f.last_checked,
|
||||
ttl_mins: f.ttl_mins.map(|t| t.max(0) as u64),
|
||||
error_since: f.error_since,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Db {
|
||||
pub async fn http_state(&self, feed_id: &str) -> Result<HttpState> {
|
||||
Ok(feeds::Entity::find_by_id(feed_id.to_owned())
|
||||
.one(&self.orm)
|
||||
.await?
|
||||
.map(|f| HttpState {
|
||||
etag: f.etag,
|
||||
last_modified: f.last_modified,
|
||||
last_checked: f.last_checked,
|
||||
ttl_mins: f.ttl_mins.map(|t| t.max(0) as u64),
|
||||
error_since: f.error_since,
|
||||
})
|
||||
.unwrap_or_default())
|
||||
Ok(feeds::Entity::find_by_id(feed_id.to_owned()).one(&self.orm).await?.map(HttpState::from).unwrap_or_default())
|
||||
}
|
||||
|
||||
/// Every feed's, in one query: a scan, and the daemon working out when the next is due, asked
|
||||
/// feed by feed, 180 queries a minute (#114).
|
||||
pub async fn http_states(&self) -> Result<std::collections::HashMap<String, HttpState>> {
|
||||
Ok(feeds::Entity::find().all(&self.orm).await?.into_iter().map(|f| (f.id.clone(), HttpState::from(f))).collect())
|
||||
}
|
||||
|
||||
/// Upsert after a successful poll. Clears any previous error.
|
||||
@@ -466,13 +501,14 @@ impl Db {
|
||||
ttl_mins: Option<u64>,
|
||||
image: Option<&str>,
|
||||
category: Option<&str>,
|
||||
description: Option<&str>,
|
||||
) -> Result<()> {
|
||||
// category is taken as it comes, unlike title and image: a show that leaves a category
|
||||
// should leave the Directory's chip too.
|
||||
let o = |v: Option<&str>| sea_orm::Value::from(v.map(str::to_owned));
|
||||
self.exec(
|
||||
"INSERT INTO feeds (id, url, title, etag, last_modified, last_checked, ttl_mins, last_error, image, category)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, NULL, $8, $9)
|
||||
"INSERT INTO feeds (id, url, title, etag, last_modified, last_checked, ttl_mins, last_error, image, category, description)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, NULL, $8, $9, $10)
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
url = excluded.url,
|
||||
title = coalesce(excluded.title, feeds.title),
|
||||
@@ -482,6 +518,7 @@ impl Db {
|
||||
ttl_mins = excluded.ttl_mins,
|
||||
image = coalesce(excluded.image, feeds.image),
|
||||
category = excluded.category,
|
||||
description = coalesce(excluded.description, feeds.description),
|
||||
last_error = NULL,
|
||||
error_since = NULL",
|
||||
vec![
|
||||
@@ -494,6 +531,7 @@ impl Db {
|
||||
ttl_mins.map(|t| t as i64).into(),
|
||||
o(image),
|
||||
o(category),
|
||||
o(description),
|
||||
],
|
||||
)
|
||||
.await?;
|
||||
@@ -652,6 +690,35 @@ impl Db {
|
||||
/// through every show's history: 4420 files queued across 12 shows on the default of 3
|
||||
/// (#97). Unlimited, 0 in the settings, is the whole back catalogue, for an archive. An item
|
||||
/// whose files were all skipped by a filter does not hold one of the places.
|
||||
/// Puts a feed's waiting files in the queue or out of it: 'pending' for those among its
|
||||
/// `limit` newest items, which the scan downloads on its own, and 'held' for the rest, listed
|
||||
/// to download by hand. With all of them 'pending', the queue counted 4420 back-catalogue
|
||||
/// files no scan would ever take (#97, #113). 0 holds them all, for a feed nobody downloads
|
||||
/// automatically. A held file still holds its item's place among the newest.
|
||||
#[tracing::instrument(skip_all)]
|
||||
pub async fn hold_back(&self, feed_id: &str, limit: usize) -> Result<()> {
|
||||
let newest = "SELECT n.guid FROM entries n
|
||||
WHERE n.feed_id = $1
|
||||
AND EXISTS (SELECT 1 FROM enclosures y
|
||||
WHERE y.feed_id = n.feed_id AND y.guid = n.guid
|
||||
AND y.state <> 'skipped')
|
||||
ORDER BY coalesce(n.published, n.first_seen) DESC, n.guid DESC
|
||||
LIMIT $2";
|
||||
let limit: sea_orm::Value = (limit.min(i64::MAX as usize) as i64).into();
|
||||
self.exec(
|
||||
&format!("UPDATE enclosures SET state = 'held' WHERE feed_id = $1 AND state = 'pending' AND guid NOT IN ({newest})"),
|
||||
vec![feed_id.into(), limit.clone()],
|
||||
)
|
||||
.await?;
|
||||
// And back, when a limit is raised or a feed downloads again.
|
||||
self.exec(
|
||||
&format!("UPDATE enclosures SET state = 'pending' WHERE feed_id = $1 AND state = 'held' AND guid IN ({newest})"),
|
||||
vec![feed_id.into(), limit],
|
||||
)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[tracing::instrument(skip_all)]
|
||||
pub async fn pending(&self, feed_id: &str, limit: usize) -> Result<Vec<Pending>> {
|
||||
self.rows(
|
||||
@@ -778,13 +845,76 @@ impl Db {
|
||||
.map(|r| Ok(r.try_get("", "id")?))
|
||||
.collect::<Result<_>>()?;
|
||||
for id in &gone {
|
||||
for table in ["enclosures", "entry_state", "hidden", "blocklists", "entries"] {
|
||||
self.exec(&format!("DELETE FROM {table} WHERE feed_id = $1"), vec![id.clone().into()]).await?;
|
||||
}
|
||||
self.forget_rows(id).await?;
|
||||
}
|
||||
Ok(gone)
|
||||
}
|
||||
|
||||
/// Everything stored about a feed but its row: items, file rows, read state, block list.
|
||||
async fn forget_rows(&self, id: &str) -> Result<()> {
|
||||
for table in ["enclosures", "entry_state", "hidden", "blocklists", "entries"] {
|
||||
self.exec(&format!("DELETE FROM {table} WHERE feed_id = $1"), vec![id.into()]).await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A feed's stored artwork still on http, items' and its own: hosts a feed no longer names
|
||||
/// keep their old items' artwork on http otherwise (IGN's 2009 items, #110).
|
||||
pub async fn http_images(&self, feed_id: &str) -> Result<Vec<String>> {
|
||||
self.rows(
|
||||
"SELECT DISTINCT image FROM entries WHERE feed_id = $1 AND image LIKE 'http://%'
|
||||
UNION SELECT image FROM feeds WHERE id = $1 AND image LIKE 'http://%'",
|
||||
vec![feed_id.into()],
|
||||
)
|
||||
.await?
|
||||
.iter()
|
||||
.map(|r| Ok(r.try_get("", "image")?))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Rewrites a feed's stored artwork on `host` from http to https, once the host is known to
|
||||
/// serve it there (`feed::prefer_https`), for the items a scan does not write again.
|
||||
pub async fn secure_images(&self, feed_id: &str, host: &str) -> Result<()> {
|
||||
let like = format!("http://{host}/%");
|
||||
for table in ["entries", "feeds"] {
|
||||
let col = if table == "feeds" { "id" } else { "feed_id" };
|
||||
self.exec(
|
||||
&format!("UPDATE {table} SET image = 'https://' || substr(image, 8) WHERE {col} = $1 AND image LIKE $2"),
|
||||
vec![feed_id.into(), like.clone().into()],
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Forgets a feed and everything stored about it.
|
||||
pub async fn forget_feed(&self, id: &str) -> Result<()> {
|
||||
self.forget_rows(id).await?;
|
||||
self.exec("DELETE FROM feeds WHERE id = $1", vec![id.into()]).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Feeds nobody subscribes to that are dead, failing since before `dead_before`, or quiet,
|
||||
/// their newest item from before `quiet_before`: what keeps the Directory to feeds worth
|
||||
/// taking. One with a file on disk stays, and so does one an OPML subscription lists, which
|
||||
/// comes back with the OPML anyway. A feed with no items yet is not quiet, only new.
|
||||
pub async fn stale_unsubscribed(&self, dead_before: i64, quiet_before: i64) -> Result<Vec<String>> {
|
||||
self.rows(
|
||||
"SELECT f.id FROM feeds f
|
||||
WHERE NOT EXISTS (SELECT 1 FROM subscriptions s WHERE s.feed_id = f.id)
|
||||
AND NOT EXISTS (SELECT 1 FROM enclosures x WHERE x.feed_id = f.id AND x.path IS NOT NULL)
|
||||
AND f.group_id IS NULL AND NOT coalesce(f.managed, false)
|
||||
AND ((f.error_since IS NOT NULL AND f.error_since < $1)
|
||||
OR (SELECT max(coalesce(e.published, e.first_seen)) FROM entries e
|
||||
WHERE e.feed_id = f.id) < $2)",
|
||||
vec![dead_before.into(), quiet_before.into()],
|
||||
)
|
||||
.await?
|
||||
.iter()
|
||||
.map(|r| Ok(r.try_get("", "id")?))
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub async fn prune_entries(&self, older_than: i64) -> Result<usize> {
|
||||
let n = self
|
||||
.exec(
|
||||
@@ -1585,6 +1715,7 @@ impl Db {
|
||||
/// The foreign key would take them anyway; this does not rely on it being switched on.
|
||||
pub async fn delete_user(&self, id: i64) -> Result<()> {
|
||||
sessions::Entity::delete_many().filter(sessions::Column::UserId.eq(id)).exec(&self.orm).await?;
|
||||
api_tokens::Entity::delete_many().filter(api_tokens::Column::UserId.eq(id)).exec(&self.orm).await?;
|
||||
users::Entity::delete_by_id(id).exec(&self.orm).await?;
|
||||
Ok(())
|
||||
}
|
||||
@@ -1637,6 +1768,56 @@ impl Db {
|
||||
Ok(found.map(User::from))
|
||||
}
|
||||
|
||||
/// Keeps a new API token's hash under its owner; the token itself is shown once and not kept.
|
||||
pub async fn create_api_token(&self, user_id: i64, name: &str, hash: &str) -> Result<i64> {
|
||||
let made = api_tokens::ActiveModel {
|
||||
user_id: Set(user_id),
|
||||
name: Set(name.to_owned()),
|
||||
hash: Set(hash.to_owned()),
|
||||
created: Set(now()),
|
||||
last_used: Set(None),
|
||||
..Default::default()
|
||||
}
|
||||
.insert(&self.orm)
|
||||
.await?;
|
||||
Ok(made.id)
|
||||
}
|
||||
|
||||
/// One person's tokens, newest first: names and dates, as the tokens themselves are not kept.
|
||||
pub async fn api_tokens(&self, user_id: i64) -> Result<Vec<api_tokens::Model>> {
|
||||
Ok(api_tokens::Entity::find()
|
||||
.filter(api_tokens::Column::UserId.eq(user_id))
|
||||
.order_by_desc(api_tokens::Column::Id)
|
||||
.all(&self.orm)
|
||||
.await?)
|
||||
}
|
||||
|
||||
/// Revokes one of this person's tokens; someone else's id does nothing. Whether it was theirs.
|
||||
pub async fn delete_api_token(&self, user_id: i64, id: i64) -> Result<bool> {
|
||||
let gone = api_tokens::Entity::delete_many()
|
||||
.filter(api_tokens::Column::Id.eq(id))
|
||||
.filter(api_tokens::Column::UserId.eq(user_id))
|
||||
.exec(&self.orm)
|
||||
.await?;
|
||||
Ok(gone.rows_affected > 0)
|
||||
}
|
||||
|
||||
/// The person an API token acts as, noting when it was used, as a session notes `seen`.
|
||||
pub async fn api_token_user(&self, hash: &str) -> Result<Option<User>> {
|
||||
let found = api_tokens::Entity::find()
|
||||
.filter(api_tokens::Column::Hash.eq(hash))
|
||||
.find_also_related(users::Entity)
|
||||
.one(&self.orm)
|
||||
.await?;
|
||||
let Some((token, Some(user))) = found else { return Ok(None) };
|
||||
api_tokens::Entity::update_many()
|
||||
.col_expr(api_tokens::Column::LastUsed, Expr::val(now()).into())
|
||||
.filter(api_tokens::Column::Id.eq(token.id))
|
||||
.exec(&self.orm)
|
||||
.await?;
|
||||
Ok(Some(User::from(user)))
|
||||
}
|
||||
|
||||
pub async fn delete_session(&self, token: &str) -> Result<()> {
|
||||
sessions::Entity::delete_by_id(token.to_owned()).exec(&self.orm).await?;
|
||||
Ok(())
|
||||
@@ -1924,6 +2105,39 @@ pub fn now() -> i64 {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
async fn sqlite_syncs_at_checkpoints_not_every_commit() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
if db.orm.get_database_backend() != sea_orm::DbBackend::Sqlite {
|
||||
return;
|
||||
}
|
||||
let row = db.orm.query_one_raw(Statement::from_string(sea_orm::DbBackend::Sqlite, "PRAGMA synchronous")).await.unwrap();
|
||||
assert_eq!(row.unwrap().try_get_by_index::<i32>(0).unwrap(), 1, "NORMAL, on every connection in the pool (#135)");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_database_from_before_descriptions_gains_the_column_and_reads_each_feed_whole() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
let record = |etag: &'static str, about: Option<&'static str>| {
|
||||
let db = &db;
|
||||
async move { db.record_feed("f", "https://x/f", Some("F"), Some(etag), Some("then"), None, None, None, about).await.unwrap() }
|
||||
};
|
||||
record("\"a\"", Some("About F")).await;
|
||||
assert_eq!(db.feed_description("f").await.unwrap().as_deref(), Some("About F"));
|
||||
record("\"a\"", None).await;
|
||||
assert_eq!(db.feed_description("f").await.unwrap().as_deref(), Some("About F"), "kept when a read finds none");
|
||||
// A database from before #130.
|
||||
db.exec_for_test("ALTER TABLE feeds DROP COLUMN description").await.unwrap();
|
||||
create_missing(&db.orm).await.unwrap();
|
||||
assert_eq!(db.feed_description("f").await.unwrap(), None);
|
||||
let st = db.http_state("f").await.unwrap();
|
||||
assert!(st.etag.is_none() && st.last_modified.is_none(), "every feed is read whole next time, for its description");
|
||||
// With the column there, opening the database again writes nothing.
|
||||
record("\"b\"", Some("About F")).await;
|
||||
create_missing(&db.orm).await.unwrap();
|
||||
assert_eq!(db.http_state("f").await.unwrap().etag.as_deref(), Some("\"b\""));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn every_sort_column_runs_and_orders_both_ways() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
@@ -1998,6 +2212,52 @@ mod tests {
|
||||
assert_eq!((list["f"].unread, list["g"].unread), (1, 1)); // a read, c hidden; sam's read is sam's
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_hosts_artwork_moves_to_https_for_one_feed() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
db.exec_for_test(
|
||||
"INSERT INTO feeds (id, url, image) VALUES ('f','u','http://a.example/logo.png'),('g','v','http://a.example/g.png');
|
||||
INSERT INTO entries (feed_id, guid, first_seen, image) VALUES
|
||||
('f','1',0,'http://a.example/1.jpg'),('f','2',0,'http://b.example/2.jpg'),('g','3',0,'http://a.example/3.jpg');",
|
||||
).await
|
||||
.unwrap();
|
||||
let mut old = db.http_images("f").await.unwrap();
|
||||
old.sort();
|
||||
assert_eq!(old, ["http://a.example/1.jpg", "http://a.example/logo.png", "http://b.example/2.jpg"]);
|
||||
db.secure_images("f", "a.example").await.unwrap();
|
||||
assert_eq!(db.feed_summary("f").await.unwrap().image.as_deref(), Some("https://a.example/logo.png"));
|
||||
assert!(db.names_image("https://a.example/1.jpg").await.unwrap());
|
||||
assert!(db.names_image("http://b.example/2.jpg").await.unwrap()); // another host
|
||||
assert!(db.names_image("http://a.example/3.jpg").await.unwrap()); // another feed
|
||||
assert_eq!(db.feed_summary("g").await.unwrap().image.as_deref(), Some("http://a.example/g.png"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn stale_feeds_nobody_subscribes_to_are_found_and_the_rest_left() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
db.exec_for_test(
|
||||
"INSERT INTO users (id, name, is_admin) VALUES (1,'ray',true);
|
||||
INSERT INTO feeds (id, url, error_since, managed, group_id) VALUES
|
||||
('dead','a',100,false,null),('quiet','b',null,false,null),('news','c',null,false,null),
|
||||
('new','d',null,false,null),('wanted','e',100,false,null),('kept','f',100,false,null),
|
||||
('child','g',100,true,'opml'),('lately','h',900,false,null);
|
||||
INSERT INTO subscriptions (user_id, feed_id) VALUES (1,'wanted');
|
||||
INSERT INTO entries (feed_id, guid, first_seen, published) VALUES
|
||||
('quiet','q',0,100),('news','n',0,950),('dead','d',0,950),('kept','k',0,100);
|
||||
INSERT INTO enclosures (id, feed_id, guid, url, path, state) VALUES (1,'kept','k','u','/tmp/k','done');",
|
||||
).await
|
||||
.unwrap();
|
||||
// Dead before 500, or nothing newer than 500: dead fails since 100, quiet's newest is 100.
|
||||
// news published lately, new has no items yet, wanted has a subscriber, kept a file,
|
||||
// child comes from an OPML, lately began failing after the cutoff.
|
||||
let mut stale = db.stale_unsubscribed(500, 500).await.unwrap();
|
||||
stale.sort();
|
||||
assert_eq!(stale, ["dead", "quiet"]);
|
||||
db.forget_feed("dead").await.unwrap();
|
||||
assert_eq!(db.feed_summary("dead").await.unwrap().entries, 0);
|
||||
assert!(!db.feed_urls().await.unwrap().iter().any(|(id, _)| id == "dead"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_failing_feed_nobody_subscribes_to_is_forgotten() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
@@ -2021,6 +2281,33 @@ mod tests {
|
||||
assert!(db.prune_abandoned_failures().await.unwrap().is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn only_the_newest_files_stay_queued_and_the_rest_are_held() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
db.exec_for_test(
|
||||
"INSERT INTO entries (feed_id, guid, first_seen, published) VALUES
|
||||
('f','e1',0,100),('f','e2',0,200),('f','e3',0,300),('f','e4',0,400);
|
||||
INSERT INTO enclosures (id, feed_id, guid, url, state, path) VALUES
|
||||
(1,'f','e1','u1','pending',null),(2,'f','e2','u2','pending',null),
|
||||
(3,'f','e3','u3','done','/tmp/3'),(4,'f','e4','u4','pending',null);",
|
||||
).await
|
||||
.unwrap();
|
||||
let states = || async {
|
||||
let mut v: Vec<(i64, String)> = db.rows("SELECT id, state FROM enclosures ORDER BY id", vec![]).await.unwrap()
|
||||
.iter().map(|r| (r.try_get("", "id").unwrap(), r.try_get("", "state").unwrap())).collect();
|
||||
v.sort();
|
||||
v.into_iter().map(|(_, s)| s).collect::<Vec<_>>()
|
||||
};
|
||||
db.hold_back("f", 2).await.unwrap(); // newest two: e4, e3
|
||||
assert_eq!(states().await, ["held", "held", "done", "pending"]);
|
||||
db.hold_back("f", 3).await.unwrap(); // a limit raised brings e2 back
|
||||
assert_eq!(states().await, ["held", "pending", "done", "pending"]);
|
||||
db.hold_back("f", 0).await.unwrap(); // a feed nobody downloads: all held
|
||||
assert_eq!(states().await, ["held", "held", "done", "held"]);
|
||||
db.hold_back("f", usize::MAX).await.unwrap(); // unlimited, an archive: all queued
|
||||
assert_eq!(states().await, ["pending", "pending", "done", "pending"]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_limit_takes_the_newest_episodes_not_the_back_catalogue() {
|
||||
let db = Db::memory().await.unwrap();
|
||||
|
||||
@@ -406,7 +406,7 @@ mod tests {
|
||||
let mut f = crate::config::Feed {
|
||||
url: "u".into(), folder: Some("Subscriptions/Some | Show".into()), group: None, media_types: None,
|
||||
schedule: None, keywords: vec![], allow_explicit: false, auto_download: true,
|
||||
max_new_per_check: None, username: None, password: None, password_env: None, category: None,
|
||||
max_new_per_check: None, username: None, password: None, password_env: None, category: None, listed: false,
|
||||
};
|
||||
assert_eq!(folder_for(&cfg, "id", &f, None), "Subscriptions/Some - Show");
|
||||
|
||||
|
||||
@@ -19,6 +19,10 @@ pub mod feeds {
|
||||
/// The channel's first <itunes:category>, for the Directory.
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub category: Option<String>,
|
||||
/// What the feed says it is, as it says it, for its page in the Directory. Added after
|
||||
/// the table was, so `create_missing` adds it to a database without it.
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub description: Option<String>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub etag: Option<String>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
@@ -102,6 +106,10 @@ pub mod enclosures {
|
||||
pub length: Option<i64>,
|
||||
#[sea_orm(column_type = "Text", nullable)]
|
||||
pub path: Option<String>,
|
||||
/// 'pending' is queued: a scan downloads it on its own. 'held' is listed but outside the
|
||||
/// feed's newest max_new_per_check, or its feed downloads nothing automatically; it can
|
||||
/// be downloaded by hand (`Db::hold_back`). 'skipped' is filtered out ('last_error' says
|
||||
/// why), then 'downloading', 'done', 'error', 'reaped'.
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub state: String,
|
||||
#[sea_orm(default_value = 0)]
|
||||
@@ -283,6 +291,29 @@ pub mod sessions {
|
||||
owned_by_user!();
|
||||
}
|
||||
|
||||
/// A token a script or agent sends as `Authorization: Bearer`, acting as the person who made
|
||||
/// it (#123). Kept as its SHA-256, so the table is no use to anyone who reads it.
|
||||
pub mod api_tokens {
|
||||
use sea_orm::entity::prelude::*;
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
|
||||
#[sea_orm(table_name = "api_tokens")]
|
||||
pub struct Model {
|
||||
#[sea_orm(primary_key)]
|
||||
pub id: i64,
|
||||
pub user_id: i64,
|
||||
/// What its owner called it, to tell one from another when revoking.
|
||||
#[sea_orm(column_type = "Text")]
|
||||
pub name: String,
|
||||
#[sea_orm(unique, column_type = "Text")]
|
||||
pub hash: String,
|
||||
pub created: i64,
|
||||
pub last_used: Option<i64>,
|
||||
}
|
||||
|
||||
owned_by_user!();
|
||||
}
|
||||
|
||||
/// The catalogue: every feed configured, with its shared settings as `config::Feed` in JSON, so a
|
||||
/// new setting on a feed needs no new column. It was config.toml's `[feeds]` (issue #18).
|
||||
pub mod catalogue {
|
||||
|
||||
299
src/feed.rs
299
src/feed.rs
@@ -13,8 +13,11 @@ pub struct ParsedFeed {
|
||||
pub image: Option<String>,
|
||||
/// The site the feed belongs to, where a favicon can stand in for missing artwork.
|
||||
pub site: Option<String>,
|
||||
/// The channel's first `<itunes:category>`, for the Directory's chips.
|
||||
/// The channel's first `<itunes:category>`, for the Directory's categories.
|
||||
pub category: Option<String>,
|
||||
/// What the feed says it is, for its page in the Directory (#130): RSS's `<description>`,
|
||||
/// or iTunes' summary where that is empty, or Atom's subtitle. HTML as often as not.
|
||||
pub description: Option<String>,
|
||||
pub entries: Vec<Entry>,
|
||||
}
|
||||
|
||||
@@ -53,43 +56,72 @@ pub enum Fetched {
|
||||
},
|
||||
}
|
||||
|
||||
/// Conditional GET. reqwest handles gzip and redirects; the original's hand-rolled
|
||||
/// CONNECT/socket.ssl proxy path is gone -- `system-proxy` reads http_proxy/https_proxy.
|
||||
/// The longest a feed may take, connecting to the last byte: a scan handles feeds in order, and
|
||||
/// with no limit one hung server held every scan for as long as it did. Dreamwidth answered 504
|
||||
/// after 60-67 s for a day, and each scan took 70-80 s instead of 15 (#108). A feed is small;
|
||||
/// downloads, which are not, have no such limit.
|
||||
pub const FEED_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
|
||||
|
||||
/// Conditional GET, following redirects itself: `client` must follow none (`feed_client`), so
|
||||
/// each hop is seen. The second value is where the feed now is when every hop said so for good
|
||||
/// (301 or 308): a publisher that moved its feed, which the catalogue should follow rather
|
||||
/// than be redirected on every read. A temporary redirect (302, 307) moves nothing.
|
||||
/// `system-proxy` reads http_proxy/https_proxy.
|
||||
#[tracing::instrument(skip_all, fields(url = %cfg.url))]
|
||||
pub async fn fetch(
|
||||
client: &reqwest::Client,
|
||||
cfg: &FeedCfg,
|
||||
etag: Option<&str>,
|
||||
last_modified: Option<&str>,
|
||||
) -> Result<Fetched> {
|
||||
let mut req = client.get(&cfg.url);
|
||||
if let Some(tag) = etag {
|
||||
req = req.header(IF_NONE_MATCH, tag);
|
||||
}
|
||||
if let Some(lm) = last_modified {
|
||||
req = req.header(IF_MODIFIED_SINCE, lm);
|
||||
}
|
||||
if let Some(user) = &cfg.username {
|
||||
req = req.basic_auth(user, cfg.password());
|
||||
}
|
||||
) -> Result<(Fetched, Option<String>)> {
|
||||
let start = reqwest::Url::parse(&cfg.url).context("the feed's address")?;
|
||||
let mut url = start.clone();
|
||||
let mut permanent = true;
|
||||
for _ in 0..10 {
|
||||
let mut req = client.get(url.clone()).timeout(FEED_TIMEOUT);
|
||||
if let Some(tag) = etag {
|
||||
req = req.header(IF_NONE_MATCH, tag);
|
||||
}
|
||||
if let Some(lm) = last_modified {
|
||||
req = req.header(IF_MODIFIED_SINCE, lm);
|
||||
}
|
||||
// The feed's own host only: a redirect elsewhere must not be handed the password.
|
||||
if let Some(user) = &cfg.username
|
||||
&& url.host_str() == start.host_str()
|
||||
{
|
||||
req = req.basic_auth(user, cfg.password());
|
||||
}
|
||||
|
||||
let resp = req.send().await.context("connecting")?;
|
||||
if resp.status() == StatusCode::NOT_MODIFIED {
|
||||
return Ok(Fetched::NotModified);
|
||||
let resp = req.send().await.context("connecting")?;
|
||||
let status = resp.status();
|
||||
if status.is_redirection() && status != StatusCode::NOT_MODIFIED {
|
||||
let to = resp
|
||||
.headers()
|
||||
.get(reqwest::header::LOCATION)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.ok_or_else(|| anyhow!("HTTP {status} without a Location to go to"))?;
|
||||
url = url.join(to).with_context(|| format!("redirected to {to:?}, which is not an address"))?;
|
||||
permanent &= matches!(status, StatusCode::MOVED_PERMANENTLY | StatusCode::PERMANENT_REDIRECT);
|
||||
continue;
|
||||
}
|
||||
let moved = (permanent && url != start).then(|| url.to_string());
|
||||
if status == StatusCode::NOT_MODIFIED {
|
||||
return Ok((Fetched::NotModified, moved));
|
||||
}
|
||||
if !status.is_success() {
|
||||
// The original surfaced 401/407 specially; the code is enough for a UI to switch on.
|
||||
return Err(anyhow!("HTTP {status}"));
|
||||
}
|
||||
let header = |h: reqwest::header::HeaderName| {
|
||||
resp.headers().get(&h).and_then(|v| v.to_str().ok()).map(str::to_owned)
|
||||
};
|
||||
let etag = header(ETAG);
|
||||
let last_modified = header(LAST_MODIFIED);
|
||||
let bytes = resp.bytes().await.context("reading body")?.to_vec();
|
||||
return Ok((Fetched::Body { bytes, etag, last_modified }, moved));
|
||||
}
|
||||
let status = resp.status();
|
||||
if !status.is_success() {
|
||||
// The original surfaced 401/407 specially; the code is enough for a UI to switch on.
|
||||
return Err(anyhow!("HTTP {status}"));
|
||||
}
|
||||
|
||||
let header = |h: reqwest::header::HeaderName| {
|
||||
resp.headers().get(&h).and_then(|v| v.to_str().ok()).map(str::to_owned)
|
||||
};
|
||||
let etag = header(ETAG);
|
||||
let last_modified = header(LAST_MODIFIED);
|
||||
let bytes = resp.bytes().await.context("reading body")?.to_vec();
|
||||
Ok(Fetched::Body { bytes, etag, last_modified })
|
||||
// Worded as reqwest worded it, which failure_kind reads as a redirect loop.
|
||||
Err(anyhow!("error following redirect for url ({url}): too many redirects"))
|
||||
}
|
||||
|
||||
/// A stored `last_error`, translated into plain words for whoever subscribes: whose problem
|
||||
@@ -253,7 +285,7 @@ pub async fn patreon_shows(
|
||||
) -> Result<(Option<String>, Vec<(String, String)>)> {
|
||||
// The creator feed names its campaign by number in its self link, a few hundred bytes in.
|
||||
// The whole feed runs to megabytes and Patreon ignores Range, so read until it turns up.
|
||||
let mut resp = client.get(url).send().await.context("connecting")?;
|
||||
let mut resp = client.get(url).timeout(FEED_TIMEOUT).send().await.context("connecting")?;
|
||||
if !resp.status().is_success() {
|
||||
return Err(anyhow!("Patreon refused the feed: HTTP {}", resp.status()));
|
||||
}
|
||||
@@ -451,6 +483,30 @@ pub async fn site_icon(client: &reqwest::Client, site: &str) -> Option<String> {
|
||||
is_image(client, ico.as_str()).await.then(|| ico.into())
|
||||
}
|
||||
|
||||
/// Artwork's address on https when its host serves it there, else as it was. The page is https
|
||||
/// and must not load http; the host is asked once per `known` (one feed's read), and an http
|
||||
/// address it does not serve on https stays, for /api/art to fetch (#90). Four of the five
|
||||
/// hosts the catalogue had on http served the same image on https; The Secret Cabal's CDN
|
||||
/// presents another name's certificate (#110).
|
||||
pub async fn prefer_https(
|
||||
client: &reqwest::Client,
|
||||
url: &str,
|
||||
known: &mut std::collections::HashMap<String, bool>,
|
||||
) -> String {
|
||||
let Some(rest) = url.strip_prefix("http://") else { return url.to_owned() };
|
||||
let host = rest.split('/').next().unwrap_or("").to_owned();
|
||||
let secure = format!("https://{rest}");
|
||||
let ok = match known.get(&host) {
|
||||
Some(ok) => *ok,
|
||||
None => {
|
||||
let ok = is_image(client, &secure).await;
|
||||
known.insert(host, ok);
|
||||
ok
|
||||
}
|
||||
};
|
||||
if ok { secure } else { url.to_owned() }
|
||||
}
|
||||
|
||||
/// Whether `url` answers with an image. A site with no favicon often answers 200 with its home
|
||||
/// page, which is not an icon.
|
||||
pub async fn is_image(client: &reqwest::Client, url: &str) -> bool {
|
||||
@@ -681,6 +737,7 @@ fn from_rss(ch: rss::Channel, bytes: &[u8]) -> ParsedFeed {
|
||||
// Only the iTunes one: Apple's list is fixed, while a plain <category> is freeform and
|
||||
// would fill the Directory with one-off tags. The subcategory where there is one: Apple
|
||||
// files every tabletop and gaming show under Leisure, which says little; Games says it.
|
||||
description: non_empty(Some(ch.description())).or_else(|| non_empty(ch.itunes_ext().and_then(|i| i.summary()))),
|
||||
category: ch
|
||||
.itunes_ext()
|
||||
.and_then(|i| i.categories().first())
|
||||
@@ -750,6 +807,7 @@ fn from_atom(feed: atom_syndication::Feed) -> ParsedFeed {
|
||||
.map(|l| l.href().trim().to_owned())
|
||||
.filter(|h| !h.is_empty()),
|
||||
category: None,
|
||||
description: non_empty(feed.subtitle().map(|t| t.as_str())),
|
||||
entries,
|
||||
}
|
||||
}
|
||||
@@ -955,8 +1013,151 @@ fn parse_date(s: &str) -> Option<i64> {
|
||||
.map(|d| d.timestamp())
|
||||
}
|
||||
|
||||
/// Apple's podcast categories, each with its subcategories
|
||||
/// (https://podcasters.apple.com/support/1691-apple-podcasts-categories): the list a podcast's own
|
||||
/// `<itunes:category>` comes from, and a fixed one, so the Directory's categories stay few.
|
||||
const CATEGORIES: &[(&str, &[&str])] = &[
|
||||
("Arts", &["Books", "Design", "Fashion & Beauty", "Food", "Performing Arts", "Visual Arts"]),
|
||||
("Business", &["Careers", "Entrepreneurship", "Investing", "Management", "Marketing", "Non-Profit"]),
|
||||
("Comedy", &["Comedy Interviews", "Improv", "Stand-Up"]),
|
||||
("Education", &["Courses", "How To", "Language Learning", "Self-Improvement"]),
|
||||
("Fiction", &["Comedy Fiction", "Drama", "Science Fiction"]),
|
||||
("Government", &[]),
|
||||
("History", &[]),
|
||||
("Health & Fitness", &["Alternative Health", "Fitness", "Medicine", "Mental Health", "Nutrition", "Sexuality"]),
|
||||
("Kids & Family", &["Education for Kids", "Parenting", "Pets & Animals", "Stories for Kids"]),
|
||||
("Leisure", &["Animation & Manga", "Automotive", "Aviation", "Crafts", "Games", "Hobbies", "Home & Garden", "Video Games"]),
|
||||
("Music", &["Music Commentary", "Music History", "Music Interviews"]),
|
||||
("News", &["Business News", "Daily News", "Entertainment News", "News Commentary", "Politics", "Sports News", "Tech News"]),
|
||||
("Religion & Spirituality", &["Buddhism", "Christianity", "Hinduism", "Islam", "Judaism", "Religion", "Spirituality"]),
|
||||
("Science", &["Astronomy", "Chemistry", "Earth Sciences", "Life Sciences", "Mathematics", "Natural Sciences", "Nature", "Physics", "Social Sciences"]),
|
||||
("Society & Culture", &["Documentary", "Personal Journals", "Philosophy", "Places & Travel", "Relationships"]),
|
||||
("Sports", &["Baseball", "Basketball", "Cricket", "Fantasy Sports", "Football", "Golf", "Hockey", "Rugby", "Running", "Soccer", "Swimming", "Tennis", "Volleyball", "Wilderness", "Wrestling"]),
|
||||
("Technology", &[]),
|
||||
("True Crime", &[]),
|
||||
("TV & Film", &["After Shows", "Film History", "Film Interviews", "Film Reviews", "TV Reviews"]),
|
||||
];
|
||||
|
||||
/// Names that are not in Apple's list but mean one that is (#125). Jev picked from this list when
|
||||
/// it spelt Football and Soccer its own way, and stored those answers; Software How-To is Apple's
|
||||
/// Technology subcategory from before its 2019 list, which older feeds still name.
|
||||
const ALIASES: &[(&str, &str)] =
|
||||
&[("American Football", "Football"), ("Football (Soccer)", "Soccer"), ("Software How-To", "Technology")];
|
||||
|
||||
/// Where a category sits in Apple's list, as (category, subcategory), so the Directory browses a
|
||||
/// category and then its subcategories as Apple's does (#118). A podcast's own is stored as its
|
||||
/// subcategory, and one an admin typed that is not Apple's stands as a category of its own.
|
||||
pub fn category_path(name: &str) -> (String, Option<String>) {
|
||||
// A feed's own text, which can end in a stray comma: "Soccer," was a category of its own.
|
||||
let name = name.trim().trim_end_matches([',', ';', '.']).trim_end();
|
||||
let name = ALIASES.iter().find(|(old, _)| old.eq_ignore_ascii_case(name)).map_or(name, |(_, now)| now);
|
||||
for (category, subs) in CATEGORIES {
|
||||
if category.eq_ignore_ascii_case(name) {
|
||||
return (category.to_string(), None);
|
||||
}
|
||||
if let Some(sub) = subs.iter().find(|s| s.eq_ignore_ascii_case(name)) {
|
||||
return (category.to_string(), Some(sub.to_string()));
|
||||
}
|
||||
}
|
||||
(name.to_string(), None)
|
||||
}
|
||||
|
||||
/// A Directory category for a feed that names none of its own, which most blogs do not (#117):
|
||||
/// TypeSafe's Jev picks one of Apple's categories or subcategories from the feed's title and
|
||||
/// latest items, the subcategory being what a podcast's own category is stored as. The likeliest
|
||||
/// is taken however unsure, since an admin can change it and asking again would cost a call a scan.
|
||||
pub async fn classify(client: &reqwest::Client, key: &str, feed: &ParsedFeed) -> Result<String> {
|
||||
let items: Vec<&str> = feed.entries.iter().filter_map(|e| e.title.as_deref()).take(15).collect();
|
||||
// A subcategory says which category it is in; without that, "Games" or "Drama" is ambiguous.
|
||||
let mut criteria = serde_json::Map::new();
|
||||
for (category, subs) in CATEGORIES {
|
||||
criteria.insert(category.to_string(), serde_json::Value::Null);
|
||||
for sub in *subs {
|
||||
criteria.insert(sub.to_string(), format!("Within {category}").into());
|
||||
}
|
||||
}
|
||||
let body = serde_json::json!({
|
||||
"model": "jev-latest",
|
||||
"state": { "title": feed.title, "recent_items": items },
|
||||
"questions": { "category": {
|
||||
"type": "choice",
|
||||
"instructions": "Which category best describes what this feed publishes?",
|
||||
"criteria": criteria,
|
||||
}},
|
||||
});
|
||||
let answer: serde_json::Value = client
|
||||
.post("https://api.typesafe.ai/v1/systemone")
|
||||
.bearer_auth(key)
|
||||
.json(&body)
|
||||
.timeout(FEED_TIMEOUT)
|
||||
.send()
|
||||
.await?
|
||||
.error_for_status()?
|
||||
.json()
|
||||
.await?;
|
||||
picked(&answer)
|
||||
}
|
||||
|
||||
fn picked(answer: &serde_json::Value) -> Result<String> {
|
||||
let choice = answer["answers"]["category"]["choice"].as_str().unwrap_or_default();
|
||||
CATEGORIES
|
||||
.iter()
|
||||
.flat_map(|(c, subs)| std::iter::once(c).chain(subs.iter()))
|
||||
.find(|c| **c == choice)
|
||||
.map(|c| c.to_string())
|
||||
.ok_or_else(|| anyhow!("no category in the answer: {answer}"))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
/// A server answering by path: /old moves for good to /new, /tmp for now, /chain for good
|
||||
/// to /tmp, /new is the feed.
|
||||
async fn redirecting_server() -> String {
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
let Ok((mut sock, _)) = listener.accept().await else { return };
|
||||
tokio::spawn(async move {
|
||||
let mut buf = [0u8; 2048];
|
||||
let n = sock.read(&mut buf).await.unwrap_or(0);
|
||||
let req = String::from_utf8_lossy(&buf[..n]);
|
||||
let path = req.split_whitespace().nth(1).unwrap_or("/").to_owned();
|
||||
let body = "<?xml version=\"1.0\"?><rss version=\"2.0\"><channel><title>T</title></channel></rss>";
|
||||
let resp = match path.as_str() {
|
||||
"/old" => "HTTP/1.1 301 Moved Permanently\r\nLocation: /new\r\nContent-Length: 0\r\n\r\n".to_owned(),
|
||||
"/tmp" => "HTTP/1.1 302 Found\r\nLocation: /new\r\nContent-Length: 0\r\n\r\n".to_owned(),
|
||||
"/chain" => "HTTP/1.1 308 Permanent Redirect\r\nLocation: /tmp\r\nContent-Length: 0\r\n\r\n".to_owned(),
|
||||
"/loop" => "HTTP/1.1 301 Moved Permanently\r\nLocation: /loop\r\nContent-Length: 0\r\n\r\n".to_owned(),
|
||||
_ => format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n{body}", body.len()),
|
||||
};
|
||||
let _ = sock.write_all(resp.as_bytes()).await;
|
||||
});
|
||||
}
|
||||
});
|
||||
format!("http://{addr}")
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_feed_that_moved_for_good_says_where_and_one_moved_for_now_does_not() {
|
||||
let base = redirecting_server().await;
|
||||
let client = reqwest::Client::builder().redirect(reqwest::redirect::Policy::none()).build().unwrap();
|
||||
let get = |path: &str| {
|
||||
let cfg: crate::config::Feed = serde_json::from_value(serde_json::json!({ "url": format!("{base}{path}") })).unwrap();
|
||||
let client = client.clone();
|
||||
async move { super::fetch(&client, &cfg, None, None).await }
|
||||
};
|
||||
let (got, moved) = get("/old").await.unwrap();
|
||||
assert!(matches!(got, super::Fetched::Body { .. }));
|
||||
assert_eq!(moved, Some(format!("{base}/new")));
|
||||
assert_eq!(get("/tmp").await.unwrap().1, None); // 302: for now
|
||||
assert_eq!(get("/chain").await.unwrap().1, None); // 308 then 302: not for good
|
||||
assert_eq!(get("/new").await.unwrap().1, None); // never moved
|
||||
let looped = get("/loop").await.err().unwrap().to_string();
|
||||
assert_eq!(super::failure_kind(&looped).0, "redirect_loop");
|
||||
}
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
@@ -967,6 +1168,7 @@ mod tests {
|
||||
assert_eq!(feed.title.as_deref(), Some("Test Cast"));
|
||||
assert_eq!(feed.ttl_mins, Some(45));
|
||||
assert_eq!(feed.category.as_deref(), Some("Podcasting"), "the first, by its subcategory");
|
||||
assert_eq!(feed.description.as_deref(), Some("A synthetic feed used by the parser tests."));
|
||||
assert_eq!(feed.entries.len(), 3);
|
||||
|
||||
let ep = &feed.entries[0];
|
||||
@@ -991,6 +1193,17 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_feed_says_what_it_is_in_itunes_summary_or_atom_subtitle() {
|
||||
// An empty <description> gives way to iTunes' summary (#130).
|
||||
let rss = br#"<?xml version="1.0"?><rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
|
||||
<channel><title>R</title><description> </description><itunes:summary>Said here.</itunes:summary></channel></rss>"#;
|
||||
assert_eq!(parse(rss).unwrap().description.as_deref(), Some("Said here."));
|
||||
let atom = br#"<?xml version="1.0"?><feed xmlns="http://www.w3.org/2005/Atom"><title>A</title>
|
||||
<subtitle>Or here.</subtitle><id>urn:a</id><updated>2026-01-01T00:00:00Z</updated></feed>"#;
|
||||
assert_eq!(parse(atom).unwrap().description.as_deref(), Some("Or here."));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_file_wordpress_lists_twice_is_one_enclosure() {
|
||||
let xml = br#"<?xml version="1.0"?><rss version="2.0"><channel><title>R</title>
|
||||
@@ -1390,5 +1603,29 @@ mod tests {
|
||||
assert_eq!(pick_guid(None, None, Some("e"), Some("t")).as_deref(), Some("e"));
|
||||
assert_eq!(pick_guid(None, None, None, None), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn picked_takes_only_an_apple_category() {
|
||||
let answer = |c: &str| serde_json::json!({"answers": {"category": {"type": "choice", "choice": c}}});
|
||||
assert_eq!(super::picked(&answer("Technology")).unwrap(), "Technology");
|
||||
assert_eq!(super::picked(&answer("Tech News")).unwrap(), "Tech News", "a subcategory");
|
||||
assert!(super::picked(&answer("Blogs")).is_err(), "not one of Apple's");
|
||||
assert!(super::picked(&serde_json::json!({"error": "rate limited"})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn category_path_finds_a_subcategory_s_category() {
|
||||
use super::category_path;
|
||||
assert_eq!(category_path("Tech News"), ("News".into(), Some("Tech News".into())));
|
||||
assert_eq!(category_path("video games"), ("Leisure".into(), Some("Video Games".into())), "Apple's spelling");
|
||||
assert_eq!(category_path("Technology"), ("Technology".into(), None));
|
||||
assert_eq!(category_path("Homebrew"), ("Homebrew".into(), None), "not Apple's: its own category");
|
||||
// Apple's spelling, an old answer of Jev's, a stray comma and Apple's old list (#125).
|
||||
assert_eq!(category_path("Soccer"), ("Sports".into(), Some("Soccer".into())));
|
||||
assert_eq!(category_path("Football (Soccer)"), ("Sports".into(), Some("Soccer".into())));
|
||||
assert_eq!(category_path("Soccer,"), ("Sports".into(), Some("Soccer".into())));
|
||||
assert_eq!(category_path("Football"), ("Sports".into(), Some("Football".into())));
|
||||
assert_eq!(category_path("Software How-To"), ("Technology".into(), None));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
16
src/ipc.rs
16
src/ipc.rs
@@ -15,6 +15,9 @@ pub enum Event {
|
||||
FeedSkip { feed: String, reason: String },
|
||||
FeedDone { feed: String, new: usize, downloaded: usize, failed: usize, torrents: usize },
|
||||
FeedError { feed: String, msg: String },
|
||||
/// The feed answered from a new address after redirects that all said it moved for good,
|
||||
/// and the catalogue now has that address (#115).
|
||||
FeedMoved { feed: String, from: String, to: String },
|
||||
Progress {
|
||||
feed: String,
|
||||
/// Which enclosure this is about. Without it a UI cannot tell one download's
|
||||
@@ -50,6 +53,7 @@ impl Event {
|
||||
"{feed}: {new} new entries, {downloaded} downloaded, {failed} failed, {torrents} torrents deferred"
|
||||
),
|
||||
Event::FeedError { feed, msg } => format!("{feed}: error: {msg}"),
|
||||
Event::FeedMoved { feed, from, to } => format!("{feed}: moved for good from {from} to {to}; following it"),
|
||||
Event::Progress { file, done, total, .. } => match total {
|
||||
Some(t) if *t > 0 => format!(
|
||||
" {file}: {:.1}% ({:.1}/{:.1} MB)",
|
||||
@@ -69,7 +73,7 @@ impl Event {
|
||||
*bytes as f64 / 1_048_576.0
|
||||
),
|
||||
Event::Status { feeds, pending, downloaded } => {
|
||||
format!("{feeds} feeds, {pending} pending, {downloaded} downloaded")
|
||||
format!("{feeds} feeds, {pending} queued to download, {downloaded} downloaded")
|
||||
}
|
||||
Event::Error { msg } => format!("error: {msg}"),
|
||||
// Noise in a terminal; a UI still gets them on the socket.
|
||||
@@ -172,7 +176,7 @@ fn log_event(e: &Event, wire: bool) {
|
||||
($level:ident, $target:literal, $text:expr) => {
|
||||
tracing::$level!(
|
||||
target: $target,
|
||||
ev = s("ev"), feed = s("feed"), msg = s("msg"), url = s("url"), reason = s("reason"),
|
||||
ev = s("ev"), feed = s("feed"), msg = s("msg"), url = s("url"), reason = s("reason"), from = s("from"), to = s("to"),
|
||||
new = n("new"), downloaded = n("downloaded"), failed = n("failed"),
|
||||
torrents = n("torrents"), bytes = n("bytes"), feeds = n("feeds"),
|
||||
pending = n("pending"), enclosure = n("enclosure"), files = n("files"),
|
||||
@@ -328,6 +332,14 @@ pub async fn proxy(path: &Path, cmd: &Command) -> Result<()> {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_feed_that_moved_says_so_on_the_wire_and_in_words() {
|
||||
let e = Event::FeedMoved { feed: "x".into(), from: "http://a/f".into(), to: "https://a/f".into() };
|
||||
let wire = serde_json::to_value(&e).unwrap();
|
||||
assert_eq!((wire["ev"].as_str(), wire["from"].as_str(), wire["to"].as_str()), (Some("feed_moved"), Some("http://a/f"), Some("https://a/f")));
|
||||
assert_eq!(e.human().unwrap(), "x: moved for good from http://a/f to https://a/f; following it");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn commands_parse_from_the_wire_form() {
|
||||
let got: Command = serde_json::from_str(r#"{"cmd":"fetch"}"#).unwrap();
|
||||
|
||||
248
src/main.rs
248
src/main.rs
@@ -1,4 +1,5 @@
|
||||
mod access;
|
||||
mod art;
|
||||
mod auth;
|
||||
mod config;
|
||||
mod db;
|
||||
@@ -63,6 +64,12 @@ enum Command {
|
||||
/// Only take enclosures matching these keywords
|
||||
#[arg(long, value_delimiter = ',')]
|
||||
keywords: Vec<String>,
|
||||
/// The Directory's category, for a feed that names none of its own (News, Technology)
|
||||
#[arg(long)]
|
||||
category: Option<String>,
|
||||
/// Put it in the Directory for anyone to subscribe to, and keep it there when nobody does
|
||||
#[arg(long)]
|
||||
list: bool,
|
||||
},
|
||||
/// Unsubscribe. Downloads and history are left alone.
|
||||
Rm { feed: String },
|
||||
@@ -118,6 +125,9 @@ pub struct Ctx {
|
||||
pub cfg: std::sync::RwLock<std::sync::Arc<config::Config>>,
|
||||
pub db: db::Db,
|
||||
pub client: reqwest::Client,
|
||||
/// For feeds only: follows no redirects, so `feed::fetch` sees each hop and can tell a feed
|
||||
/// that moved for good from one sent elsewhere for now.
|
||||
pub feed_client: reqwest::Client,
|
||||
pub out: Emitter,
|
||||
/// Started on first use: a BitTorrent session binds ports and starts a DHT, which is
|
||||
/// rude to do for a config that has never seen a torrent.
|
||||
@@ -248,6 +258,13 @@ async fn main() -> Result<()> {
|
||||
db,
|
||||
client: reqwest::Client::builder()
|
||||
.user_agent(concat!("ipx/", env!("CARGO_PKG_VERSION")))
|
||||
// Connecting only, so it bounds a download's start, not a long download (#108).
|
||||
.connect_timeout(std::time::Duration::from_secs(10))
|
||||
.build()?,
|
||||
feed_client: reqwest::Client::builder()
|
||||
.user_agent(concat!("ipx/", env!("CARGO_PKG_VERSION")))
|
||||
.connect_timeout(std::time::Duration::from_secs(10))
|
||||
.redirect(reqwest::redirect::Policy::none())
|
||||
.build()?,
|
||||
out: if is_daemon { Emitter::socket(events.clone(), false) } else { Emitter::terminal() },
|
||||
torrents: tokio::sync::OnceCell::new(),
|
||||
@@ -259,8 +276,8 @@ async fn main() -> Result<()> {
|
||||
let result = match cli.command {
|
||||
Command::List => list(&ctx).await,
|
||||
Command::Daemon { web } => daemon(ctx, config_path, web, events).await,
|
||||
Command::Add { url, folder, keywords } => {
|
||||
add(&ctx, &url, folder, keywords).await
|
||||
Command::Add { url, folder, keywords, category, list } => {
|
||||
add(&ctx, &url, folder, keywords, category, list).await
|
||||
}
|
||||
Command::Rm { feed } => rm(&ctx, &feed).await,
|
||||
Command::User { cmd } => user_cmd(&ctx, cmd).await,
|
||||
@@ -505,8 +522,6 @@ async fn daemon(
|
||||
let server = tokio::spawn(ipc::serve(socket.clone(), events.clone(), tx_cmd, answer));
|
||||
|
||||
// One command at a time: the queue is what keeps two scans from overlapping.
|
||||
let mut ticker = tokio::time::interval(std::time::Duration::from_secs(60));
|
||||
ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
|
||||
tracing::info!(
|
||||
feeds = subscriptions(&ctx).await.map(|s| s.len()).unwrap_or(0),
|
||||
"daemon started"
|
||||
@@ -543,10 +558,13 @@ async fn daemon(
|
||||
}
|
||||
|
||||
let mut stop = rx_stop.clone();
|
||||
// The first pass at once, as the minute's tick did: what came due while it was down.
|
||||
let mut first = true;
|
||||
loop {
|
||||
if *stop.borrow() {
|
||||
break;
|
||||
}
|
||||
let wait = if std::mem::take(&mut first) { std::time::Duration::ZERO } else { until_next_scan(&ctx).await };
|
||||
tokio::select! {
|
||||
biased;
|
||||
_ = stop.changed() => break,
|
||||
@@ -563,7 +581,7 @@ async fn daemon(
|
||||
break;
|
||||
}
|
||||
}
|
||||
_ = ticker.tick() => {
|
||||
_ = tokio::time::sleep(wait) => {
|
||||
// Per-feed schedule and TTL decide what actually gets polled.
|
||||
let job = run(&ctx, Cmd::Fetch { feed: None, force: false, feeds: vec![] });
|
||||
if !until_stopped(&ctx, &rx_stop, job).await {
|
||||
@@ -673,16 +691,35 @@ async fn add(
|
||||
url: &str,
|
||||
folder: Option<String>,
|
||||
keywords: Vec<String>,
|
||||
category: Option<String>,
|
||||
list: bool,
|
||||
) -> Result<()> {
|
||||
let mut cfg = (*ctx.cfg()).clone();
|
||||
let url = &feed::find_feed(&ctx.client, &feed::expand_input(url)).await?;
|
||||
// Includes feeds derived from an OPML, or the same show could be added twice.
|
||||
if let Some(existing) = subscriptions(ctx).await?.iter().find(|s| feed::same_feed(&s.cfg.url, url)) {
|
||||
// Listing a feed the catalogue already has, or giving it a category, is the point of
|
||||
// asking again: it changes those, and nothing else.
|
||||
if let Some(f) = cfg.feeds.get_mut(&existing.id)
|
||||
&& (list || category.is_some())
|
||||
{
|
||||
f.listed |= list;
|
||||
if category.is_some() {
|
||||
f.category = category;
|
||||
}
|
||||
ctx.store_cfg(cfg).await?;
|
||||
println!("{} is already in the catalogue; updated its Directory listing", existing.id);
|
||||
return Ok(());
|
||||
}
|
||||
anyhow::bail!("already subscribed as {:?}", existing.id);
|
||||
}
|
||||
let id = add_one(ctx, &mut cfg, url, folder, keywords).await?;
|
||||
if let Some(f) = cfg.feeds.get_mut(&id) {
|
||||
f.category = category;
|
||||
f.listed = list;
|
||||
}
|
||||
ctx.store_cfg(cfg).await?;
|
||||
println!("added {id}");
|
||||
println!("added {id}{}", if list { ", listed in the Directory" } else { "" });
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -709,9 +746,10 @@ pub async fn add_one(
|
||||
password: None,
|
||||
password_env: None,
|
||||
category: None,
|
||||
listed: false,
|
||||
};
|
||||
|
||||
let title = match feed::fetch(&ctx.client, &probe, None, None).await {
|
||||
let title = match feed::fetch(&ctx.feed_client, &probe, None, None).await.map(|(got, _)| got) {
|
||||
// An OPML subscription is named from its own <head><title>, not by trying to
|
||||
// parse it as a feed and falling back to the hostname.
|
||||
Ok(feed::Fetched::Body { bytes, .. }) if feed::is_opml(&bytes) => {
|
||||
@@ -835,6 +873,7 @@ pub async fn subscribe_opml(
|
||||
password: None,
|
||||
password_env: None,
|
||||
category: None,
|
||||
listed: false,
|
||||
},
|
||||
);
|
||||
grew = true;
|
||||
@@ -947,12 +986,42 @@ async fn list(ctx: &Ctx) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Removes feeds nobody subscribes to that are dead (failing for 30 days) or quiet (nothing new
|
||||
/// in a year), from the catalogue and the database, so the Directory lists feeds worth taking.
|
||||
/// A feed listed in it with no subscribers stays for as long as it works and publishes.
|
||||
async fn clean_directory(ctx: &Ctx) -> Result<()> {
|
||||
const DEAD: i64 = 30 * 86_400;
|
||||
const QUIET: i64 = 365 * 86_400;
|
||||
let now = db::now();
|
||||
let stale = ctx.db.stale_unsubscribed(now - DEAD, now - QUIET).await?;
|
||||
if stale.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
let mut cfg = (*ctx.cfg()).clone();
|
||||
if stale.iter().fold(false, |any, id| cfg.feeds.remove(id).is_some() || any) {
|
||||
ctx.store_cfg(cfg).await?;
|
||||
}
|
||||
for id in &stale {
|
||||
retire_group(ctx, id).await?;
|
||||
ctx.db.forget_feed(id).await?;
|
||||
tracing::info!(feed = id, "removed a feed nobody subscribes to that is dead or has published nothing in a year");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `standalone` false means this is the sweep that runs before a scan: it reports what it
|
||||
/// deleted, but must not emit the terminal ReapDone, or a client waiting on its `fetch`
|
||||
/// would stop reading before the scan had even started.
|
||||
#[tracing::instrument(name = "reap", skip_all, fields(dry_run = dry_run))]
|
||||
async fn reap(ctx: &Ctx, dry_run: bool, standalone: bool) -> Result<()> {
|
||||
let r = retention::run(&ctx.cfg(), &ctx.db, dry_run).await?;
|
||||
if !dry_run {
|
||||
clean_directory(ctx).await?;
|
||||
let gone = art::trim(&art::dir(), ctx.cfg().general.art_cache_mb * 1_048_576);
|
||||
if gone > 0 {
|
||||
tracing::debug!(gone, "trimmed the artwork kept on disk");
|
||||
}
|
||||
}
|
||||
for c in r.aged_out.iter().chain(r.over_quota.iter()) {
|
||||
ctx.out.emit(Event::Reaped {
|
||||
path: c.path.clone(),
|
||||
@@ -979,15 +1048,17 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
|
||||
anyhow::bail!("no feed with id {id:?}");
|
||||
}
|
||||
|
||||
let subscribed = ctx.db.subscriber_counts().await?;
|
||||
let mut scanned = 0;
|
||||
let mut fresh: Vec<String> = vec![];
|
||||
let in_scope = |s: &Sub| {
|
||||
scope.is_empty() || scope.contains(&s.id) || s.cfg.group.as_ref().is_some_and(|g| scope.contains(g))
|
||||
};
|
||||
let mut states = ctx.db.http_states().await?;
|
||||
let mut due = vec![];
|
||||
for sub in subs.iter().filter(|s| only.is_none_or(|o| o == s.id) && in_scope(s)) {
|
||||
let id = &sub.id;
|
||||
let mut state = ctx.db.http_state(id).await?;
|
||||
let mut state = states.remove(id).unwrap_or_default();
|
||||
// A scan someone asked for reads the feed in full. With the validators it only skipped the
|
||||
// wait: a feed that had not changed answered 304 and nothing was read (#77).
|
||||
if force {
|
||||
@@ -995,8 +1066,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
|
||||
state.last_modified = None;
|
||||
}
|
||||
|
||||
if !force && let Some(last) = state.last_checked {
|
||||
let at = last + due_after(&cfg, &sub.cfg, state.ttl_mins, state.error_since, last) as i64;
|
||||
if !force && let Some(at) = due_at(&cfg, &sub.cfg, &state, subscribed.contains_key(id)) {
|
||||
if at > db::now() {
|
||||
ctx.out.emit(Event::FeedSkip {
|
||||
feed: id.clone(),
|
||||
@@ -1015,7 +1085,7 @@ async fn fetch(ctx: &Arc<Ctx>, only: Option<&str>, force: bool, scope: &[String]
|
||||
const AHEAD: usize = 6;
|
||||
let mut bodies = futures_util::StreamExt::buffered(
|
||||
futures_util::stream::iter(due.iter().map(|(sub, state)| {
|
||||
let (client, feed_cfg) = (ctx.client.clone(), sub.cfg.clone());
|
||||
let (client, feed_cfg) = (ctx.feed_client.clone(), sub.cfg.clone());
|
||||
let (etag, modified) = (state.etag.clone(), state.last_modified.clone());
|
||||
let early = !feed::is_patreon_creator(&feed_cfg.url);
|
||||
tokio::spawn(tracing::Instrument::instrument(
|
||||
@@ -1156,6 +1226,7 @@ pub async fn subscriptions(ctx: &Ctx) -> Result<Vec<Sub>> {
|
||||
password: parent.and_then(|p| p.password.clone()),
|
||||
password_env: parent.and_then(|p| p.password_env.clone()),
|
||||
category: None,
|
||||
listed: false,
|
||||
},
|
||||
managed: true,
|
||||
});
|
||||
@@ -1206,6 +1277,47 @@ async fn retire_stranded(ctx: &Ctx) -> Result<usize> {
|
||||
|
||||
/// Seconds to wait before re-checking a feed.
|
||||
///
|
||||
/// When a feed is next due, or None for one never checked, which is due now. A feed nobody
|
||||
/// subscribes to, one listed in the Directory, is read once a day: enough to keep its entry
|
||||
/// current, without fetching it hourly for no one.
|
||||
pub fn due_at(cfg: &config::Config, feed: &config::Feed, state: &db::HttpState, subscribed: bool) -> Option<i64> {
|
||||
let last = state.last_checked?;
|
||||
let floor = if subscribed { 0 } else { 86_400 };
|
||||
Some(last + due_after(cfg, feed, state.ttl_mins, state.error_since, last).max(floor) as i64)
|
||||
}
|
||||
|
||||
/// How long the daemon may sleep before a feed is due: until the earliest one, at least 30 s and
|
||||
/// at most 10 minutes. It ticked every minute and ran a scan pass each time, 80% of them finding
|
||||
/// nothing due (#114). The floor keeps a feed that never gets a check time from spinning it; the
|
||||
/// ceiling picks up within ten minutes what no command announces, such as `ipx add` or a
|
||||
/// shorter schedule. A command, a refresh or a feed added on the page, wakes it at once anyway.
|
||||
async fn until_next_scan(ctx: &Ctx) -> std::time::Duration {
|
||||
let next = async {
|
||||
let cfg = ctx.cfg();
|
||||
let subscribed = ctx.db.subscriber_counts().await?;
|
||||
let states = ctx.db.http_states().await?;
|
||||
anyhow::Ok(
|
||||
subscriptions(ctx)
|
||||
.await?
|
||||
.iter()
|
||||
// A feed with no row yet has never been checked: due now.
|
||||
.map(|s| {
|
||||
states.get(&s.id).and_then(|st| due_at(&cfg, &s.cfg, st, subscribed.contains_key(&s.id))).unwrap_or(0)
|
||||
})
|
||||
.min(),
|
||||
)
|
||||
};
|
||||
let wait = match next.await {
|
||||
Ok(Some(at)) => (at - db::now()).max(0) as u64,
|
||||
Ok(None) => u64::MAX,
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %format!("{e:#}"), "could not work out when the next feed is due");
|
||||
0
|
||||
}
|
||||
};
|
||||
std::time::Duration::from_secs(wait.clamp(30, 600))
|
||||
}
|
||||
|
||||
/// A per-feed schedule is an explicit instruction and wins outright. Without one, the
|
||||
/// global schedule applies, but the feed's own <ttl> raises it when the publisher asks to
|
||||
/// be polled less often. A feed that is failing backs off (`backoff`), from `error_since`, when
|
||||
@@ -1257,6 +1369,24 @@ enum Outcome {
|
||||
Opml { added: Vec<String>, removed: usize, kept: usize, total: usize },
|
||||
}
|
||||
|
||||
/// A feed that answered from a new address after permanent redirects moves there in the
|
||||
/// catalogue, so it is read from there and no longer redirected every time. One an OPML lists
|
||||
/// is left alone, since the OPML would only put the old address back; so is a move onto an
|
||||
/// address another feed already has.
|
||||
async fn follow_move(ctx: &Ctx, id: &str, to: &str) -> Result<()> {
|
||||
// Said in the event below, not logged here: the event is the log line.
|
||||
if subscriptions(ctx).await?.iter().any(|s| s.id != id && feed::same_feed(&s.cfg.url, to)) {
|
||||
tracing::info!(feed = id, to, "the feed moved to an address another feed already has; leaving it");
|
||||
return Ok(());
|
||||
}
|
||||
let mut cfg = (*ctx.cfg()).clone();
|
||||
let Some(f) = cfg.feeds.get_mut(id) else { return Ok(()) };
|
||||
let from = std::mem::replace(&mut f.url, to.to_owned());
|
||||
ctx.store_cfg(cfg).await?;
|
||||
ctx.out.emit(Event::FeedMoved { feed: id.to_owned(), from, to: to.to_owned() });
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[tracing::instrument(name = "feed", skip_all, fields(feed = id))]
|
||||
async fn scan_one(
|
||||
ctx: &Arc<Ctx>,
|
||||
@@ -1264,9 +1394,13 @@ async fn scan_one(
|
||||
feed_cfg: &config::Feed,
|
||||
state: &db::HttpState,
|
||||
force: bool,
|
||||
// The body the scan already fetched for it, if it did (#104).
|
||||
prefetched: Option<Result<feed::Fetched>>,
|
||||
// The body the scan already fetched for it, if it did (#104), and where it moved, if it did.
|
||||
prefetched: Option<Result<(feed::Fetched, Option<String>)>>,
|
||||
) -> Result<Outcome> {
|
||||
// The queue follows the feed's settings each time it is due, changed or not, so files a
|
||||
// limit or auto-download no longer reaches stop counting as waiting (#113).
|
||||
let policy = policy_for(ctx, id, feed_cfg).await?;
|
||||
ctx.db.hold_back(id, if policy.auto_download { policy.budget } else { 0 }).await?;
|
||||
// A Patreon creator with more than one show is a list of feeds, like an OPML.
|
||||
if feed::is_patreon_creator(&feed_cfg.url) {
|
||||
match feed::patreon_shows(&ctx.client, &feed_cfg.url).await {
|
||||
@@ -1292,10 +1426,13 @@ async fn scan_one(
|
||||
}
|
||||
}
|
||||
|
||||
let mut fetched = match prefetched {
|
||||
let (mut fetched, moved) = match prefetched {
|
||||
Some(got) => got?,
|
||||
None => feed::fetch(&ctx.client, feed_cfg, state.etag.as_deref(), state.last_modified.as_deref()).await?,
|
||||
None => feed::fetch(&ctx.feed_client, feed_cfg, state.etag.as_deref(), state.last_modified.as_deref()).await?,
|
||||
};
|
||||
if let Some(to) = moved {
|
||||
follow_move(ctx, id, &to).await?;
|
||||
}
|
||||
|
||||
// A 304 while nothing is stored means the validator has outlived the data -- a restore
|
||||
// from backup, a manual edit, a cleanup that removed entries. Believe the database over
|
||||
@@ -1306,7 +1443,7 @@ async fn scan_one(
|
||||
if matches!(fetched, feed::Fetched::NotModified) && (stored.entries == 0 || stored.image.is_none()) {
|
||||
tracing::info!(feed = id, "not modified, but nothing stored; refetching without the validator");
|
||||
ctx.db.clear_validators(id).await?;
|
||||
fetched = feed::fetch(&ctx.client, feed_cfg, None, None).await?;
|
||||
fetched = feed::fetch(&ctx.feed_client, feed_cfg, None, None).await?.0;
|
||||
}
|
||||
|
||||
let (bytes, etag, last_modified) = match fetched {
|
||||
@@ -1330,6 +1467,17 @@ async fn scan_one(
|
||||
}
|
||||
|
||||
let mut parsed = feed::parse(&bytes)?;
|
||||
// Artwork on http moves to https where its host serves it (#110), before it is compared
|
||||
// with what is stored, which is then the https address too.
|
||||
let mut secure = std::collections::HashMap::new();
|
||||
if let Some(art) = &parsed.image {
|
||||
parsed.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
|
||||
}
|
||||
for entry in parsed.entries.iter_mut() {
|
||||
if let Some(art) = &entry.image {
|
||||
entry.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
|
||||
}
|
||||
}
|
||||
// Artwork is looked at when it may have changed: the feed names different artwork from what
|
||||
// is stored, or someone asked for a refresh, so an icon the site changes or fixes still
|
||||
// follows it (#80). Looked at on every full read, a feed without validators asked its site
|
||||
@@ -1354,6 +1502,20 @@ async fn scan_one(
|
||||
});
|
||||
}
|
||||
}, art_span).await;
|
||||
// A site's icon comes back on whatever the site is on.
|
||||
if let Some(art) = &parsed.image {
|
||||
parsed.image = Some(feed::prefer_https(&ctx.client, art, &mut secure).await);
|
||||
}
|
||||
// Items stored before, which a scan does not write again, move with their host, and a host
|
||||
// only they still name is asked too.
|
||||
for art in ctx.db.http_images(id).await? {
|
||||
feed::prefer_https(&ctx.client, &art, &mut secure).await;
|
||||
}
|
||||
for (host, ok) in &secure {
|
||||
if *ok {
|
||||
ctx.db.secure_images(id, host).await?;
|
||||
}
|
||||
}
|
||||
ctx.db.record_feed(
|
||||
id,
|
||||
&feed_cfg.url,
|
||||
@@ -1363,9 +1525,31 @@ async fn scan_one(
|
||||
parsed.ttl_mins,
|
||||
parsed.image.as_deref(),
|
||||
parsed.category.as_deref(),
|
||||
parsed.description.as_deref(),
|
||||
).await?;
|
||||
// Asked on a feed's first read, nothing of it stored yet, or when someone forces a refresh:
|
||||
// each call costs, and a feed already here without a category can wait for a forced one.
|
||||
// The answer becomes the catalogue's category, so a feed is not asked twice.
|
||||
// A feed an OPML lists is not in the catalogue and has nowhere to keep one.
|
||||
if parsed.category.is_none()
|
||||
&& (force || stored.entries == 0)
|
||||
&& !parsed.entries.is_empty()
|
||||
&& ctx.cfg().feeds.get(id).is_some_and(|f| f.category.is_none())
|
||||
&& let Ok(key) = std::env::var("TYPESAFE_KEY")
|
||||
{
|
||||
match feed::classify(&ctx.client, &key, &parsed).await {
|
||||
Ok(category) => {
|
||||
let mut cfg = (*ctx.cfg()).clone();
|
||||
if let Some(f) = cfg.feeds.get_mut(id).filter(|f| f.category.is_none()) {
|
||||
tracing::info!(feed = id, category = %category, "categorised");
|
||||
f.category = Some(category);
|
||||
ctx.store_cfg(cfg).await?;
|
||||
}
|
||||
}
|
||||
Err(e) => tracing::warn!(feed = id, "could not categorise: {e:#}"),
|
||||
}
|
||||
}
|
||||
|
||||
let policy = policy_for(ctx, id, feed_cfg).await?;
|
||||
if let Some(parent) = &feed_cfg.group {
|
||||
let listed: Vec<(&str, &str)> = parsed
|
||||
.entries
|
||||
@@ -1412,6 +1596,9 @@ async fn scan_one(
|
||||
if scan.new_entries > 0 {
|
||||
ctx.db.rehide(id).await?; // what is new may hold someone's blocked words
|
||||
}
|
||||
// A new item takes a place among the newest and the oldest of them leaves the queue; a file a
|
||||
// filter lets through again may be outside them.
|
||||
ctx.db.hold_back(id, if policy.auto_download { policy.budget } else { 0 }).await?;
|
||||
|
||||
let budget = policy.budget;
|
||||
if policy.auto_download && budget > 0 {
|
||||
@@ -1666,7 +1853,10 @@ fn merge_policy(subs: &[db::Sub], feed_cfg: &config::Feed, global: usize) -> Pol
|
||||
|
||||
if subs.is_empty() {
|
||||
return Policy {
|
||||
auto_download: feed_cfg.auto_download,
|
||||
// A feed listed in the Directory with nobody subscribed is there to be found, not
|
||||
// downloaded: its files would be for no one. Once someone subscribes, the files
|
||||
// skipped for it are judged again on the next scan, by their settings (#107).
|
||||
auto_download: feed_cfg.auto_download && !feed_cfg.listed,
|
||||
allow_explicit: feed_cfg.allow_explicit,
|
||||
wants: vec![Want { keywords: feed_cfg.keywords.clone(), blocked: vec![] }],
|
||||
budget: cap(feed_cfg.max_new_per_check),
|
||||
@@ -1899,6 +2089,26 @@ fn duration(secs: u64) -> String {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_feed_is_due_after_its_wait_and_daily_with_nobody_subscribed() {
|
||||
let cfg = config::Config::default();
|
||||
let f = feed();
|
||||
let at = |last: Option<i64>, subscribed| {
|
||||
due_at(&cfg, &f, &db::HttpState { last_checked: last, ..Default::default() }, subscribed)
|
||||
};
|
||||
assert_eq!(at(None, true), None); // never checked: due now
|
||||
assert_eq!(at(Some(1000), true), Some(1000 + 3600)); // the hourly default
|
||||
assert_eq!(at(Some(1000), false), Some(1000 + 86_400)); // listed, nobody subscribed
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_listed_feed_nobody_subscribes_to_downloads_nothing() {
|
||||
let mut f = feed();
|
||||
assert!(merge_policy(&[], &f, 3).auto_download);
|
||||
f.listed = true;
|
||||
assert!(!merge_policy(&[], &f, 3).auto_download);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failing_feed_backs_off_doubling_up_to_a_day() {
|
||||
let hour = 3600;
|
||||
@@ -1969,6 +2179,7 @@ mod tests {
|
||||
password: None,
|
||||
password_env: None,
|
||||
category: None,
|
||||
listed: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2028,6 +2239,7 @@ mod tests {
|
||||
cfg: std::sync::RwLock::new(Arc::new(cfg)),
|
||||
db: db::Db::memory().await.unwrap(),
|
||||
client: reqwest::Client::new(),
|
||||
feed_client: reqwest::Client::builder().redirect(reqwest::redirect::Policy::none()).build().unwrap(),
|
||||
out: Emitter::terminal(),
|
||||
torrents: tokio::sync::OnceCell::new(),
|
||||
torrent_slots: Arc::new(tokio::sync::Semaphore::new(2)),
|
||||
|
||||
234
src/web.rs
234
src/web.rs
@@ -38,6 +38,8 @@ pub fn router(state: WebState) -> Router {
|
||||
Router::new()
|
||||
.route("/", get(index))
|
||||
.route("/api/me", get(me).patch(patch_me))
|
||||
.route("/api/tokens", get(list_tokens).post(add_token))
|
||||
.route("/api/tokens/{id}", axum::routing::delete(remove_token))
|
||||
.route("/api/logout", post(logout))
|
||||
.route("/api/feeds", get(feeds).post(add_feed))
|
||||
.route("/api/feeds/{id}", patch(patch_feed).delete(remove_feed))
|
||||
@@ -56,6 +58,7 @@ pub fn router(state: WebState) -> Router {
|
||||
.route("/api/status", get(status))
|
||||
.route("/api/popular", get(get_popular))
|
||||
.route("/api/directory", get(get_directory))
|
||||
.route("/api/directory/{id}", get(get_listed))
|
||||
.route("/api/popular/{id}", post(subscribe_popular))
|
||||
.route("/api/users", get(list_users).post(add_user))
|
||||
.route("/api/users/{id}", patch(patch_user).delete(remove_user))
|
||||
@@ -76,6 +79,8 @@ pub fn router(state: WebState) -> Router {
|
||||
.route("/favicon.png", get(favicon))
|
||||
.route("/favicon-dark.png", get(favicon_dark))
|
||||
.route("/apple-touch-icon.png", get(touch_icon))
|
||||
// iOS asks for this one first when the site is added to a home screen (#109).
|
||||
.route("/apple-touch-icon-precomposed.png", get(touch_icon))
|
||||
.route("/app.js", get(app_js))
|
||||
.route("/app.css", get(app_css))
|
||||
.route("/login.js", get(login_js))
|
||||
@@ -98,8 +103,8 @@ pub async fn serve(state: WebState, bind: &str) -> Result<()> {
|
||||
.context("serving the web ui")
|
||||
}
|
||||
|
||||
/// Who is asking, in order of how specific the claim is: a proxy that vouches for a name,
|
||||
/// a session cookie, then the shared token (which is the admin).
|
||||
/// Who is asking, in order of how specific the claim is: a proxy that vouches for a name, an
|
||||
/// API token, a session cookie, then the shared token (which is the admin).
|
||||
///
|
||||
/// Any of them has to survive being put in a cookie: an `<audio src>` request is issued by
|
||||
/// the browser, and there is no way to attach a header to it.
|
||||
@@ -142,6 +147,26 @@ async fn auth(State(state): State<WebState>, mut req: Request, next: Next) -> Re
|
||||
}
|
||||
let by_proxy = user.is_some();
|
||||
|
||||
// An API token, made by its owner in Settings, for a script or an agent acting as them
|
||||
// (#123). One that is not known is turned away rather than tried as anything else: a
|
||||
// script sent with a revoked token should hear so, not fall through to a cookie.
|
||||
if user.is_none()
|
||||
&& let Some(bearer) = req
|
||||
.headers()
|
||||
.get(header::AUTHORIZATION)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|v| v.strip_prefix("Bearer "))
|
||||
{
|
||||
match state.ctx.db.api_token_user(&crate::auth::api_token_hash(bearer.trim())).await {
|
||||
Ok(Some(u)) => user = Some(u),
|
||||
Ok(None) => return (StatusCode::UNAUTHORIZED, "unknown or revoked API token").into_response(),
|
||||
Err(e) => {
|
||||
tracing::error!(error = %e, "looking up an API token");
|
||||
return StatusCode::INTERNAL_SERVER_ERROR.into_response();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 2. A session cookie from signing in here.
|
||||
if user.is_none() {
|
||||
if let Some(sid) = cookie(&req, SESSION_COOKIE) {
|
||||
@@ -390,6 +415,54 @@ fn last_admin(users: &[crate::db::User], id: i64) -> bool {
|
||||
admins == [id]
|
||||
}
|
||||
|
||||
/// Your API tokens: names and dates, never the tokens, which are not kept.
|
||||
async fn list_tokens(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
let tokens: Vec<_> = state
|
||||
.ctx
|
||||
.db
|
||||
.api_tokens(user.id).await?
|
||||
.iter()
|
||||
.map(|t| serde_json::json!({ "id": t.id, "name": t.name, "created": t.created, "last_used": t.last_used }))
|
||||
.collect();
|
||||
Ok(Json(serde_json::json!(tokens)))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct NewToken {
|
||||
name: String,
|
||||
}
|
||||
|
||||
/// Makes an API token that acts as you, and answers with it: the only time it is shown.
|
||||
async fn add_token(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
Json(body): Json<NewToken>,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
let name = body.name.trim();
|
||||
if name.is_empty() {
|
||||
return Err(ApiError::bad_request("give the token a name, to know it by when revoking it"));
|
||||
}
|
||||
let token = crate::auth::new_api_token();
|
||||
let id = state.ctx.db.create_api_token(user.id, name, &crate::auth::api_token_hash(&token)).await?;
|
||||
tracing::info!(user = %user.name, token = name, "API token made");
|
||||
Ok(Json(serde_json::json!({ "id": id, "name": name, "token": token })))
|
||||
}
|
||||
|
||||
async fn remove_token(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
Path(id): Path<i64>,
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
if !state.ctx.db.delete_api_token(user.id, id).await? {
|
||||
return Err(ApiError::bad_request(format!("you have no API token with id {id}")));
|
||||
}
|
||||
tracing::info!(user = %user.name, id, "API token revoked");
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
async fn list_users(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
@@ -859,8 +932,10 @@ struct PopularRow {
|
||||
subscribers: i64,
|
||||
/// Yours already. Everyone counts, you included, so your own feeds are listed too.
|
||||
subscribed: bool,
|
||||
/// The feed's own iTunes category, if it names one; most blogs do not.
|
||||
/// Apple's category, from the feed's own iTunes category or else the catalogue's, and the
|
||||
/// subcategory within it where there is one: the Directory browses them in that order.
|
||||
category: Option<String>,
|
||||
subcategory: Option<String>,
|
||||
/// Any audio or video enclosure. Unlike category, every feed has an answer, so the
|
||||
/// Directory's Podcasts and Blogs between them hold everything.
|
||||
podcast: bool,
|
||||
@@ -870,8 +945,13 @@ struct PopularRow {
|
||||
/// first. Popular is the top of it, the directory is all of it, and it is all that
|
||||
/// `subscribe_popular` will subscribe you to. An OPML or a Patreon creator is listed as the
|
||||
/// feeds inside it and never itself: both lists are for finding a show.
|
||||
async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||
/// The catalogue as others may see it. `everything` is the Directory: every feed, with or
|
||||
/// without subscribers, so the ones an admin listed show before anyone takes them. Without it,
|
||||
/// Popular: only what people subscribe to.
|
||||
async fn popular(state: &WebState, user_id: i64, everything: bool) -> Result<Vec<PopularRow>> {
|
||||
let db = &state.ctx.db;
|
||||
// One pass for every feed's title, artwork and category, not three queries a feed.
|
||||
let mut listed = db.feed_list(user_id, None).await?;
|
||||
let mine: std::collections::HashSet<String> =
|
||||
db.subscriptions_for(user_id).await?.into_iter().map(|s| s.feed_id).collect();
|
||||
let counts = db.subscriber_counts().await?;
|
||||
@@ -886,23 +966,25 @@ async fn popular(state: &WebState, user_id: i64) -> Result<Vec<PopularRow>> {
|
||||
let n = counts.get(&s.id).copied().unwrap_or(0);
|
||||
// A feed inside an OPML that looks private is as private as the OPML.
|
||||
let folder = s.cfg.group.as_deref().and_then(|g| by_id.get(g));
|
||||
if n == 0
|
||||
if (n == 0 && !everything)
|
||||
|| is_folder.contains(s.id.as_str())
|
||||
|| looks_private(&s.cfg)
|
||||
|| folder.is_some_and(|f| looks_private(f))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let sum = db.feed_summary(&s.id).await?;
|
||||
let sum = listed.remove(&s.id).map(|l| l.summary).unwrap_or_default();
|
||||
let subscribed = mine.contains(&s.id);
|
||||
// The feed's own wins; an admin's is for the feeds, mostly blogs, that name none.
|
||||
let path = sum.category.or_else(|| s.cfg.category.clone()).map(|c| crate::feed::category_path(&c));
|
||||
out.push(PopularRow {
|
||||
id: s.id.clone(),
|
||||
title: sum.title,
|
||||
image: sum.image,
|
||||
subscribers: n,
|
||||
subscribed,
|
||||
// The feed's own wins; an admin's is for the feeds, mostly blogs, that name none.
|
||||
category: sum.category.or_else(|| s.cfg.category.clone()),
|
||||
category: path.as_ref().map(|p| p.0.clone()),
|
||||
subcategory: path.and_then(|p| p.1),
|
||||
podcast: media.contains(&s.id),
|
||||
});
|
||||
}
|
||||
@@ -914,7 +996,7 @@ async fn get_popular(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||
let mut rows = popular(&state, user.id).await?;
|
||||
let mut rows = popular(&state, user.id, false).await?;
|
||||
rows.truncate(10);
|
||||
Ok(Json(rows))
|
||||
}
|
||||
@@ -924,11 +1006,58 @@ async fn get_directory(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
) -> Result<Json<Vec<PopularRow>>, ApiError> {
|
||||
let mut rows = popular(&state, user.id).await?;
|
||||
let mut rows = popular(&state, user.id, true).await?;
|
||||
rows.sort_by_key(sort_name);
|
||||
Ok(Json(rows))
|
||||
}
|
||||
|
||||
/// One of a listed feed's latest items, for its page in the Directory before anyone subscribes
|
||||
/// (#128): what it is, never its files, whose addresses are where a key would be.
|
||||
#[derive(Serialize)]
|
||||
struct ListedItem {
|
||||
guid: String,
|
||||
title: Option<String>,
|
||||
link: Option<String>,
|
||||
published: Option<i64>,
|
||||
duration: Option<i64>,
|
||||
description: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct Listed {
|
||||
/// What the feed says it is (#130), sanitized as its items are.
|
||||
description: Option<String>,
|
||||
items: Vec<ListedItem>,
|
||||
}
|
||||
|
||||
/// A listed feed's description and latest twenty items. Only a feed the Directory lists, so a
|
||||
/// guessed id reaches nothing private, as `subscribe_popular` checks.
|
||||
async fn get_listed(
|
||||
State(state): State<WebState>,
|
||||
user: crate::db::User,
|
||||
Path(id): Path<String>,
|
||||
) -> Result<Json<Listed>, ApiError> {
|
||||
if !popular(&state, user.id, true).await?.iter().any(|p| p.id == id) {
|
||||
return Err(ApiError::not_found(format!("{id:?} is not in the directory")));
|
||||
}
|
||||
let order = crate::db::order_sql("published", "desc");
|
||||
let rows = state.ctx.db.entries_in(user.id, Some(&id), crate::db::Filter::parse("all"), None, 0, 20, &order).await?;
|
||||
let mut sanitizer = feed_sanitizer();
|
||||
let description = state.ctx.db.feed_description(&id).await?.map(|d| clean_description(&mut sanitizer, &d, None));
|
||||
let items = rows
|
||||
.into_iter()
|
||||
.map(|e| ListedItem {
|
||||
description: e.description.as_deref().map(|d| clean_description(&mut sanitizer, d, e.link.as_deref())),
|
||||
guid: e.guid,
|
||||
title: e.title,
|
||||
link: e.link,
|
||||
published: e.published,
|
||||
duration: e.duration,
|
||||
})
|
||||
.collect();
|
||||
Ok(Json(Listed { description, items }))
|
||||
}
|
||||
|
||||
fn sort_name(p: &PopularRow) -> String {
|
||||
p.title.clone().unwrap_or_else(|| p.id.clone()).to_lowercase()
|
||||
}
|
||||
@@ -940,10 +1069,12 @@ async fn subscribe_popular(
|
||||
user: crate::db::User,
|
||||
Path(id): Path<String>,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
if !popular(&state, user.id).await?.iter().any(|p| p.id == id) {
|
||||
if !popular(&state, user.id, true).await?.iter().any(|p| p.id == id) {
|
||||
return Err(ApiError::bad_request(format!("{id:?} is not in the directory")));
|
||||
}
|
||||
state.ctx.db.subscribe(user.id, &id).await?;
|
||||
// A listed feed nobody took was checked once a day at most: read it now, not in an hour.
|
||||
scan_soon(&state, Some(id.clone())).await;
|
||||
Ok(Json(serde_json::json!({ "id": id })))
|
||||
}
|
||||
|
||||
@@ -1073,6 +1204,7 @@ mod tests {
|
||||
password: None,
|
||||
password_env: None,
|
||||
category: None,
|
||||
listed: false,
|
||||
};
|
||||
assert!(!looks_private(&f("https://feeds.twit.tv/twit.xml")));
|
||||
assert!(!looks_private(&f("https://example.com/rss?format=mp3")));
|
||||
@@ -1119,7 +1251,7 @@ mod tests {
|
||||
crate::config::Feed {
|
||||
url: url.into(), folder: None, group: None, media_types: None, schedule: None, keywords: vec![], allow_explicit: false,
|
||||
auto_download: true, max_new_per_check: None, username: None,
|
||||
password: None, password_env: None, category: None,
|
||||
password: None, password_env: None, category: None, listed: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1201,9 +1333,22 @@ async fn entries(
|
||||
user: crate::db::User,
|
||||
Query(page): Query<Page>,
|
||||
) -> Result<Json<EntryPage>, ApiError> {
|
||||
subscribed(&state, &user, &id).await?;
|
||||
entry_page(&state, user.id, Some(&id), &page).await
|
||||
}
|
||||
|
||||
/// A feed's items and files are for the people who subscribe to it (#129). Anyone signed in
|
||||
/// could read any feed's by its id, a paid one's included, with its enclosure addresses, which
|
||||
/// can carry the subscriber's key, and fetch, queue or delete its files by theirs. A feed in
|
||||
/// an OPML has a subscription of its own for everyone subscribed to the OPML, so this holds
|
||||
/// for those too. Not found, as for any other feed that is not yours.
|
||||
async fn subscribed(state: &WebState, user: &crate::db::User, feed_id: &str) -> Result<(), ApiError> {
|
||||
match state.ctx.db.subscription(user.id, feed_id).await? {
|
||||
Some(_) => Ok(()),
|
||||
None => Err(ApiError::not_found("you do not subscribe to that feed")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Every subscribed feed's items together, newest first: All Subscriptions.
|
||||
async fn all_entries(
|
||||
State(state): State<WebState>,
|
||||
@@ -1505,8 +1650,12 @@ async fn remove_feed(
|
||||
}
|
||||
|
||||
// Nobody is left: the feed stops being scanned. Its files and history stay, so if
|
||||
// someone subscribes again they do not pull the back catalogue a second time.
|
||||
// someone subscribes again they do not pull the back catalogue a second time. A feed an
|
||||
// admin listed stays in the Directory, for the next person.
|
||||
let mut cfg = (*state.ctx.cfg()).clone();
|
||||
if cfg.feeds.get(&id).is_some_and(|f| f.listed) {
|
||||
return Ok(StatusCode::NO_CONTENT);
|
||||
}
|
||||
if cfg.feeds.remove(&id).is_none() {
|
||||
// A derived feed: forget it here, though the OPML will list it again on the next
|
||||
// read unless you unsubscribe from the OPML itself.
|
||||
@@ -1548,12 +1697,14 @@ async fn set_flags(
|
||||
async fn download_now(
|
||||
State(state): State<WebState>,
|
||||
Path(id): Path<i64>,
|
||||
user: crate::db::User,
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
let enc = state
|
||||
.ctx
|
||||
.db
|
||||
.enclosure(id).await?
|
||||
.ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
|
||||
subscribed(&state, &user, &enc.feed_id).await?;
|
||||
if enc.path.is_some() {
|
||||
return Ok(StatusCode::NO_CONTENT); // Already here.
|
||||
}
|
||||
@@ -1583,6 +1734,7 @@ async fn delete_file(
|
||||
.db
|
||||
.enclosure(id).await?
|
||||
.ok_or_else(|| anyhow::anyhow!("no enclosure {id}"))?;
|
||||
subscribed(&state, &user, &enc.feed_id).await?;
|
||||
|
||||
// There is one copy of the file: deleting it deletes everyone's. Say so before doing
|
||||
// it, once, and let them decide.
|
||||
@@ -1687,7 +1839,10 @@ async fn events(
|
||||
/// routine skip of a feed not due, dozens a minute that change nothing.
|
||||
fn changed_feed(ev: &Event) -> Option<&str> {
|
||||
match ev {
|
||||
Event::FeedDone { feed, .. } | Event::FeedError { feed, .. } | Event::DownloadDone { feed, .. } => Some(feed),
|
||||
Event::FeedDone { feed, .. }
|
||||
| Event::FeedError { feed, .. }
|
||||
| Event::FeedMoved { feed, .. }
|
||||
| Event::DownloadDone { feed, .. } => Some(feed),
|
||||
Event::FeedSkip { feed, reason } if !reason.starts_with("not due") => Some(feed),
|
||||
_ => None,
|
||||
}
|
||||
@@ -1697,11 +1852,15 @@ fn changed_feed(ev: &Event) -> Option<&str> {
|
||||
async fn media(
|
||||
State(state): State<WebState>,
|
||||
Path(id): Path<i64>,
|
||||
user: crate::db::User,
|
||||
req: Request,
|
||||
) -> Response {
|
||||
let Ok(Some(enc)) = state.ctx.db.enclosure(id).await else {
|
||||
return (StatusCode::NOT_FOUND, "no such enclosure").into_response();
|
||||
};
|
||||
if let Err(e) = subscribed(&state, &user, &enc.feed_id).await {
|
||||
return e.into_response();
|
||||
}
|
||||
let Some(path) = enc.path else {
|
||||
return (StatusCode::NOT_FOUND, "not downloaded").into_response();
|
||||
};
|
||||
@@ -1716,16 +1875,25 @@ struct ArtQuery {
|
||||
u: String,
|
||||
}
|
||||
|
||||
/// Artwork a feed names on plain http, fetched here for the https page. The browser upgrades an
|
||||
/// http image on an https page to https, and a host with no https, such as The Secret Cabal's
|
||||
/// CDN, then answers nothing (issue #90).
|
||||
/// Artwork, from iPX: the page asks here for every image a feed or item names, and the first
|
||||
/// time it is fetched from the publisher and kept on disk (`art`, up to `art_cache_mb`), so
|
||||
/// later it is fast, the publisher is not asked on every visit, and it outlives the publisher's
|
||||
/// server. Only an address some feed or item names as its artwork, and only an image up to
|
||||
/// 5 MB, so the route cannot be pointed at anything else. It began as a way round http-only
|
||||
/// artwork on the https page (#90).
|
||||
async fn art(State(state): State<WebState>, Query(q): Query<ArtQuery>) -> Response {
|
||||
const MAX: usize = 5 << 20;
|
||||
if !q.u.starts_with("http://") || !state.ctx.db.names_image(&q.u).await.unwrap_or(false) {
|
||||
let web = q.u.starts_with("http://") || q.u.starts_with("https://");
|
||||
if !web || !state.ctx.db.names_image(&q.u).await.unwrap_or(false) {
|
||||
return (StatusCode::NOT_FOUND, "no feed names that artwork").into_response();
|
||||
}
|
||||
let keep = state.ctx.cfg().general.art_cache_mb > 0;
|
||||
let file = crate::art::path(&q.u);
|
||||
if keep && let Some((kind, body)) = crate::art::read(&file) {
|
||||
return art_response(kind, body);
|
||||
}
|
||||
let got = async {
|
||||
let mut r = state.ctx.client.get(&q.u).send().await?.error_for_status()?;
|
||||
let mut r = state.ctx.client.get(&q.u).timeout(crate::feed::FEED_TIMEOUT).send().await?.error_for_status()?;
|
||||
let kind = r.headers().get(header::CONTENT_TYPE).and_then(|v| v.to_str().ok()).unwrap_or("").to_owned();
|
||||
anyhow::ensure!(kind.starts_with("image/"), "not an image: {kind}");
|
||||
let mut body = Vec::new();
|
||||
@@ -1737,12 +1905,30 @@ async fn art(State(state): State<WebState>, Query(q): Query<ArtQuery>) -> Respon
|
||||
};
|
||||
match got.await {
|
||||
Ok((kind, body)) => {
|
||||
([(header::CONTENT_TYPE, kind), (header::CACHE_CONTROL, "private, max-age=86400".into())], body).into_response()
|
||||
if keep && let Err(e) = crate::art::write(&file, &kind, &body) {
|
||||
tracing::debug!(error = %e, "could not keep the artwork");
|
||||
}
|
||||
art_response(kind, body)
|
||||
}
|
||||
Err(e) => (StatusCode::BAD_GATEWAY, format!("{e:#}")).into_response(),
|
||||
}
|
||||
}
|
||||
|
||||
/// An image from someone else's server, served from iPX's own address: it must stay an image.
|
||||
/// An SVG opened on its own would otherwise run its script as iPX's page, with its cookies.
|
||||
fn art_response(kind: String, body: Vec<u8>) -> Response {
|
||||
(
|
||||
[
|
||||
(header::CONTENT_TYPE, kind),
|
||||
(header::CACHE_CONTROL, "private, max-age=2592000".into()),
|
||||
(header::X_CONTENT_TYPE_OPTIONS, "nosniff".into()),
|
||||
(header::CONTENT_SECURITY_POLICY, "default-src 'none'; style-src 'unsafe-inline'; sandbox".into()),
|
||||
],
|
||||
body,
|
||||
)
|
||||
.into_response()
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct Position {
|
||||
secs: i64,
|
||||
@@ -1804,8 +1990,10 @@ fn five() -> i64 {
|
||||
async fn download_latest(
|
||||
State(state): State<WebState>,
|
||||
Path(id): Path<String>,
|
||||
user: crate::db::User,
|
||||
Json(body): Json<HowMany>,
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
subscribed(&state, &user, &id).await?;
|
||||
let ids = state.ctx.db.undownloaded(&id, body.count.clamp(1, 100)).await?;
|
||||
for enc in &ids {
|
||||
state.ctx.db.requeue(*enc).await?;
|
||||
@@ -1908,6 +2096,7 @@ struct Settings {
|
||||
download_dir: String,
|
||||
max_total_gb: f64,
|
||||
max_age_days: u64,
|
||||
art_cache_mb: u64,
|
||||
}
|
||||
|
||||
async fn get_settings(State(state): State<WebState>) -> Json<Settings> {
|
||||
@@ -1920,6 +2109,7 @@ async fn get_settings(State(state): State<WebState>) -> Json<Settings> {
|
||||
download_dir: cfg.general.download_dir.display().to_string(),
|
||||
max_total_gb: cfg.general.max_total_gb,
|
||||
max_age_days: cfg.general.max_age_days,
|
||||
art_cache_mb: cfg.general.art_cache_mb,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1930,6 +2120,7 @@ struct SettingsPatch {
|
||||
media_types: Option<Vec<String>>,
|
||||
max_total_gb: Option<f64>,
|
||||
max_age_days: Option<u64>,
|
||||
art_cache_mb: Option<u64>,
|
||||
}
|
||||
|
||||
async fn patch_settings(
|
||||
@@ -1966,6 +2157,9 @@ async fn patch_settings(
|
||||
if let Some(v) = body.max_age_days {
|
||||
cfg.general.max_age_days = v;
|
||||
}
|
||||
if let Some(v) = body.art_cache_mb {
|
||||
cfg.general.art_cache_mb = v;
|
||||
}
|
||||
state.ctx.store_cfg(cfg).await?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
@@ -49,7 +49,6 @@ const drive = PAGE === 'admin.html' ? [
|
||||
['prefsModal', () => ctx.prefsModal()],
|
||||
['opmlModal', () => ctx.opmlModal()],
|
||||
['selectFeed (directory)', () => ctx.selectFeed(':directory')],
|
||||
['selectFeed (popular)', () => ctx.selectFeed(':popular')],
|
||||
['selectFeed (currently listening)', () => ctx.selectFeed(':listening')],
|
||||
['selectFeed (all subscriptions)', () => ctx.selectFeed(':all')],
|
||||
['keysModal', () => ctx.keysModal()],
|
||||
|
||||
@@ -242,7 +242,7 @@ test('an item with several enclosures lists them all', async ({ page }) => {
|
||||
await expect(page.locator('#files .encbox').nth(1).locator('.kind[title^="image"]')).toBeVisible();
|
||||
});
|
||||
|
||||
test('Currently Listening, its own place below Popular, resumes an episode you started or forgets it', async ({ page }) => {
|
||||
test('Currently Listening, its own place below the Directory, resumes an episode you started or forgets it', async ({ page }) => {
|
||||
// Second Episode (900s) is 42 seconds in and unfinished. An earlier test may have opened it,
|
||||
// and opening marks it read; it is listed all the same, because read is not finished. This
|
||||
// test used to set it unread first, which hid exactly the bug in issue #14.
|
||||
@@ -251,12 +251,12 @@ test('Currently Listening, its own place below Popular, resumes an episode you s
|
||||
await page.evaluate(() =>
|
||||
api('/api/entries/test-show/ui-2/position', { method: 'POST', body: JSON.stringify({ secs: 42 }) }));
|
||||
|
||||
// Popular lists feeds and nothing else; the episodes have a place of their own under it.
|
||||
await page.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||
await expect(page.locator('#popular')).toBeVisible();
|
||||
// The Directory lists feeds and nothing else; the episodes have a place of their own under it.
|
||||
await page.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||
await expect(page.locator('#directory')).toBeVisible();
|
||||
await expect(page.locator('#listening')).toHaveCount(0);
|
||||
const places = await page.locator('#feedlist .place b').allTextContents();
|
||||
expect(places.indexOf('Currently Listening')).toBe(places.indexOf('Popular') + 1);
|
||||
expect(places.indexOf('Currently Listening')).toBe(places.indexOf('Directory') + 1);
|
||||
|
||||
await page.locator('#feedlist .place', { hasText: 'Currently Listening' }).click();
|
||||
const row = page.locator('#listening .childrow', { hasText: 'Second Episode' });
|
||||
@@ -812,7 +812,7 @@ test('ipx import subscribes the admin, and ipx export writes the feeds out', asy
|
||||
expect(xml).toContain('http://127.0.0.1:8792/two.xml');
|
||||
});
|
||||
|
||||
test('Popular lists what everyone here reads, but never a private feed', async ({ browser }) => {
|
||||
test('the Directory lists what everyone here reads, the most subscribed first, but never a private feed', async ({ browser, page }) => {
|
||||
const { execFileSync } = require('child_process');
|
||||
const setup = require('./global-setup');
|
||||
const env = {
|
||||
@@ -833,21 +833,23 @@ test('Popular lists what everyone here reads, but never a private feed', async (
|
||||
await piper.locator('button[type=submit]').click();
|
||||
await expect(piper.locator('#feedlist')).toContainText('No feeds.');
|
||||
|
||||
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||
const offered = piper.locator('#popular .childrow');
|
||||
await expect(offered.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||
// The most subscribed are the Directory's first section; Popular, their own place, is gone (#124).
|
||||
await expect(piper.locator('#feedlist .place', { hasText: 'Popular' })).toHaveCount(0);
|
||||
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||
const chart = piper.locator('#dchart .rank');
|
||||
await expect(chart.filter({ hasText: 'Test Show' })).toBeVisible({ timeout: 20_000 });
|
||||
// An OPML is listed as the feeds inside it, and a key in a URL marks someone's paid feed.
|
||||
await expect(offered.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
|
||||
await expect(offered.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||
await expect(chart.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
|
||||
await expect(chart.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||
|
||||
// No URL reaches the page at all, so neither can a key, and the server holds the same line.
|
||||
const listed = await (await piper.request.get('/api/popular')).text();
|
||||
const listed = await (await piper.request.get('/api/directory')).text();
|
||||
expect(listed).not.toContain('secret123');
|
||||
expect(listed).not.toContain('.xml');
|
||||
expect((await piper.request.post('/api/popular/paid-show')).status()).toBe(400);
|
||||
|
||||
// Popular is the top ten of the directory, and the directory is every listed feed A to Z,
|
||||
// with an OPML's feeds in place of the OPML in both.
|
||||
// The chart is /api/popular's top ten, and the directory is every listed feed A to Z, with an
|
||||
// OPML's feeds in place of the OPML in both.
|
||||
const dir = await (await piper.request.get('/api/directory')).json();
|
||||
const top = await (await piper.request.get('/api/popular')).json();
|
||||
const names = dir.map(p => (p.title || p.id).toLowerCase());
|
||||
@@ -855,48 +857,92 @@ test('Popular lists what everyone here reads, but never a private feed', async (
|
||||
const ids = dir.map(p => p.id);
|
||||
expect(ids).not.toContain('test-subscriptions');
|
||||
expect(ids).toEqual(expect.arrayContaining(['grouped-show', 'aardvark-radio']));
|
||||
expect(top.length).toBe(Math.min(10, dir.length));
|
||||
expect(top.every(t => ids.includes(t.id))).toBe(true);
|
||||
expect(ids).not.toContain('paid-show');
|
||||
|
||||
// Subscribe from the directory this time: the same feeds, as a grid of cover art.
|
||||
const tiles = piper.locator('#popular .tile');
|
||||
const pick = (row, name) => piper.locator(`#dirbar .${row} button`, { hasText: new RegExp(`^${name}$`) });
|
||||
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||
await expect(chart).toHaveCount(top.length);
|
||||
await expect(chart.first().locator('.no')).toHaveText('1');
|
||||
await expect(piper.locator('#count')).toContainText(`Directory: ${dir.length} feed`);
|
||||
|
||||
// Then the categories, as tiles, each a page of its own.
|
||||
const cat = name => piper.locator('#directory .cat', { hasText: new RegExp(`^${name}\\d`) });
|
||||
const tiles = piper.locator('#dtiles .tile');
|
||||
expect(dir.find(p => p.id === 'test-show')).toMatchObject({ podcast: true, category: 'Technology' });
|
||||
expect(dir.find(p => p.id === 'picture-blog')).toMatchObject({ podcast: false });
|
||||
await cat('Technology').click();
|
||||
await expect(piper.locator('#directory h2')).toHaveText('Technology');
|
||||
await expect(tiles).toHaveCount(dir.filter(p => p.category === 'Technology').length);
|
||||
await expect(tiles.filter({ hasText: 'Test Show' })).toBeVisible();
|
||||
// Its way back lands on the tile it went in by.
|
||||
await piper.locator('#directory .dback').click();
|
||||
await expect(cat('Technology')).toBeFocused();
|
||||
|
||||
// What a feed is holds on every page. No blog here names Technology, so Blogs has no such tile.
|
||||
const kind = name => piper.locator('#directory .tabs button', { hasText: new RegExp(`^${name}$`) });
|
||||
await kind('Blogs').click();
|
||||
await expect(kind('Blogs')).toHaveAttribute('aria-pressed', 'true');
|
||||
await expect(cat('Technology')).toHaveCount(0);
|
||||
await piper.locator('#directory .dall').click();
|
||||
await expect(tiles).toHaveCount(dir.filter(p => !p.podcast).length);
|
||||
await expect(tiles.filter({ hasText: 'Test Show' })).toHaveCount(0);
|
||||
await kind('All').click();
|
||||
await expect(tiles).toHaveCount(dir.length);
|
||||
await expect(tiles.filter({ hasText: /Grouped Show|grouped-show/ })).toBeVisible();
|
||||
await expect(tiles.filter({ hasText: /Test Subscriptions/ })).toHaveCount(0);
|
||||
await expect(tiles.filter({ hasText: /Paid Show|paid-show/ })).toHaveCount(0);
|
||||
// Sorted by name either way, A to Z as it opens, or by subscribers (#122).
|
||||
const byName = dir.map(p => p.title || p.id).sort((a, b) => a.localeCompare(b, undefined, { sensitivity: 'base', numeric: true }));
|
||||
await expect(tiles.first()).toContainText(byName[0]);
|
||||
await piper.locator('#dirsort').selectOption('za');
|
||||
await expect(tiles.first()).toContainText(byName[byName.length - 1]);
|
||||
await expect(tiles).toHaveCount(dir.length);
|
||||
await piper.locator('#dirsort').selectOption('subscribers');
|
||||
await expect(tiles).toHaveCount(dir.length);
|
||||
await piper.locator('#dirsort').selectOption('az');
|
||||
|
||||
// The search box looks through the Directory by name (#126), and clearing it goes back.
|
||||
await piper.locator('#epSearch').fill('test show');
|
||||
await expect(piper.locator('#directory h2')).toHaveText('Results for “test show”');
|
||||
await expect(tiles).toHaveCount(dir.filter(p => (p.title || p.id).toLowerCase().includes('test show')).length);
|
||||
await piper.locator('#epSearch').fill('');
|
||||
await expect(tiles).toHaveCount(dir.length);
|
||||
|
||||
// Two filters that combine: what a feed is, and what it is about.
|
||||
expect(dir.find(p => p.id === 'test-show')).toMatchObject({ podcast: true, category: 'Technology' });
|
||||
expect(dir.find(p => p.id === 'picture-blog')).toMatchObject({ podcast: false });
|
||||
await pick('tabs', 'Blogs').click();
|
||||
await expect(tiles).toHaveCount(dir.filter(p => !p.podcast).length);
|
||||
await expect(tiles.filter({ hasText: 'Test Show' })).toHaveCount(0);
|
||||
// No empty chips: no blog here names Technology, so Blogs does not offer it.
|
||||
await expect(pick('chips', 'Technology')).toHaveCount(0);
|
||||
await pick('tabs', 'Podcasts').click();
|
||||
await pick('chips', 'Technology').click();
|
||||
await expect(pick('chips', 'Technology')).toHaveAttribute('aria-pressed', 'true');
|
||||
await expect(tiles).toHaveCount(dir.filter(p => p.podcast && p.category === 'Technology').length);
|
||||
// A second press lifts the chip and leaves the kind as it was.
|
||||
await pick('chips', 'Technology').click();
|
||||
await expect(tiles).toHaveCount(dir.filter(p => p.podcast).length);
|
||||
await pick('tabs', 'All').click();
|
||||
// A feed you do not subscribe to opens its page here: what it is and its latest items (#128).
|
||||
await tiles.filter({ hasText: 'Test Show' }).locator('.txt').click();
|
||||
await expect(piper.locator('.show h2')).toHaveText('Test Show');
|
||||
await expect(piper.locator('#dshow .lep', { hasText: 'First Episode' })).toBeVisible();
|
||||
// What the feed says it is, from its own <description> (#130).
|
||||
await expect(piper.locator('#dabout p')).toHaveText('A fixture feed.');
|
||||
await expect(piper.locator('#dabout button')).toBeHidden();
|
||||
// Back, onto the tile it was opened from, the page under it as it was.
|
||||
await piper.locator('#directory .dback').click();
|
||||
await expect(tiles.filter({ hasText: 'Test Show' })).toBeFocused();
|
||||
await expect(tiles).toHaveCount(dir.length);
|
||||
// Only a listed feed has one, and it carries no file or its address.
|
||||
const preview = await (await piper.request.get('/api/directory/test-show')).text();
|
||||
expect(preview).toContain('First Episode');
|
||||
expect(JSON.parse(preview).description).toBe('A fixture feed.');
|
||||
expect(preview).not.toContain('.mp3');
|
||||
expect((await piper.request.get('/api/directory/paid-show')).status()).toBe(404);
|
||||
// Nor does anything else give a feed's items or files to someone who does not subscribe (#129).
|
||||
expect((await piper.request.get('/api/feeds/test-show/entries')).status()).toBe(404);
|
||||
expect((await piper.request.get('/api/feeds/paid-show/entries')).status()).toBe(404);
|
||||
const pics = await page.evaluate(() => api('/api/feeds/picture-blog/entries'));
|
||||
const file = pics.entries.flatMap(e => e.enclosures).find(x => x.path);
|
||||
expect(file).toBeTruthy();
|
||||
expect((await page.request.get(`/media/${file.id}`)).status()).toBe(200);
|
||||
expect((await piper.request.get(`/media/${file.id}`)).status()).toBe(404);
|
||||
expect((await piper.request.post(`/api/enclosures/${file.id}/download`)).status()).toBe(404);
|
||||
expect((await piper.request.delete(`/api/enclosures/${file.id}?force=true`)).status()).toBe(404);
|
||||
expect((await piper.request.post('/api/feeds/picture-blog/download-latest', { data: { count: 1 } })).status()).toBe(404);
|
||||
|
||||
// Add a feed is for an address; Popular and Directory are where you browse (issue #30).
|
||||
// Add a feed is for an address; the Directory is where you browse (issue #30).
|
||||
await piper.locator('#addFeed').click();
|
||||
await expect(piper.locator('#nurl')).toBeVisible();
|
||||
await expect(piper.locator('#modalCard .childrow')).toHaveCount(0);
|
||||
await expect(tiles).toHaveCount(dir.length);
|
||||
await expect(piper.locator('#modalCard .tile, #modalCard .rank')).toHaveCount(0);
|
||||
await piper.locator('#modalCard button[title="Cancel"]').click();
|
||||
|
||||
const row = async () =>
|
||||
(await (await piper.request.get('/api/popular')).json()).find(p => p.id === 'test-show');
|
||||
(await (await piper.request.get('/api/directory')).json()).find(p => p.id === 'test-show');
|
||||
const before = await row();
|
||||
expect(before.subscribed).toBe(false);
|
||||
await tiles.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]').click();
|
||||
@@ -904,9 +950,10 @@ test('Popular lists what everyone here reads, but never a private feed', async (
|
||||
|
||||
// Everyone counts, you included: it stays listed, marked as yours, with one more subscriber.
|
||||
expect(await row()).toMatchObject({ subscribed: true, subscribers: before.subscribers + 1 });
|
||||
await piper.locator('#feedlist .place', { hasText: 'Popular' }).click();
|
||||
await expect(offered.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
|
||||
await expect(offered.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
|
||||
expect((await piper.request.get('/api/feeds/test-show/entries')).status()).toBe(200);
|
||||
await piper.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||
await expect(chart.filter({ hasText: 'Test Show' }).locator('[title^="Subscribed"]')).toBeVisible();
|
||||
await expect(chart.filter({ hasText: 'Test Show' }).locator('button[title="Subscribe"]')).toHaveCount(0);
|
||||
|
||||
// All Subscriptions is every item from piper's feeds and only those: the admin's Picture
|
||||
// Blog is not among them.
|
||||
@@ -956,6 +1003,37 @@ test('reading an item updates its feed\'s count without reloading the list', asy
|
||||
expect(lists, 'the row came back with the read, not by reloading the list').toEqual([]);
|
||||
});
|
||||
|
||||
test('artwork comes from iPX, kept, and only as an image', async ({ page }) => {
|
||||
// Every image a feed names is drawn from iPX's address.
|
||||
expect(await page.evaluate(() => artHTML('https://example.com/a.jpg', 'A'))).toContain('src="/api/art?u=https%3A%2F%2Fexample.com%2Fa.jpg"');
|
||||
// Multi Show's items name art.jpg.
|
||||
await expect(page.locator('.feed', { hasText: 'Multi Show' })).toBeVisible({ timeout: 20_000 });
|
||||
const src = '/api/art?u=' + encodeURIComponent('http://127.0.0.1:8792/art.jpg');
|
||||
for (const _ of [1, 2]) { // fetched, then kept
|
||||
const r = await page.request.get(src);
|
||||
expect(r.status()).toBe(200);
|
||||
expect(r.headers()['content-type']).toMatch(/^image\//);
|
||||
expect(r.headers()['content-security-policy']).toContain('sandbox');
|
||||
}
|
||||
// An address no feed names is not fetched.
|
||||
expect((await page.request.get('/api/art?u=' + encodeURIComponent('http://127.0.0.1:8792/show.xml'))).status()).toBe(404);
|
||||
});
|
||||
|
||||
test('an item without a title is named from its text, then its file, then its show and date', async ({ page }) => {
|
||||
const names = await page.evaluate(() => [
|
||||
entryName({ title: 'A Title' }),
|
||||
entryName({ title: ' ', description: '<p>I like the way <b>AI</b> is evolving.</p><img src="x.png">' }),
|
||||
entryName({ description: '<p>' + 'word '.repeat(40) + '</p>' }).text.length,
|
||||
entryName({ enclosures: [{ url: 'https://k.example/media/KARTAS691.mp3?x=1' }] }),
|
||||
entryName({ feed_id: 'nobody', published: 0 }),
|
||||
]);
|
||||
expect(names[0]).toEqual({ text: 'A Title', derived: false });
|
||||
expect(names[1]).toEqual({ text: 'I like the way AI is evolving.', derived: true });
|
||||
expect(names[2]).toBeLessThanOrEqual(121); // cut at a word, with …
|
||||
expect(names[3]).toEqual({ text: 'KARTAS691', derived: true });
|
||||
expect(names[4]).toEqual({ text: 'nobody', derived: true });
|
||||
});
|
||||
|
||||
test('a deleted file looks as if it was never downloaded', async ({ page }) => {
|
||||
// Other people subscribe to Picture Blog by now, so both prompts come; take them.
|
||||
page.on('dialog', d => d.accept());
|
||||
@@ -1140,13 +1218,49 @@ test('keys move through items and places, after Feedly', async ({ page }) => {
|
||||
await expect(page.locator('#count')).toContainText('All Subscriptions');
|
||||
});
|
||||
|
||||
test('an API token acts as its owner until it is revoked', async ({ page, request }) => {
|
||||
// Made in Settings, shown once.
|
||||
await page.locator('#prefs').click();
|
||||
await page.locator('#tokname').fill('test agent');
|
||||
await page.locator('#tokadd').click();
|
||||
const token = await page.locator('#tokval').inputValue();
|
||||
expect(token).toMatch(/^ipx_[0-9a-f]{64}$/);
|
||||
await expect(page.locator('.tokrow', { hasText: 'test agent' })).toContainText('last used never');
|
||||
|
||||
// Sent as a Bearer header, with no cookie, it is the person who made it.
|
||||
const as = t => ({ headers: { Authorization: `Bearer ${t}` } });
|
||||
const me = await (await request.get('/api/me', as(token))).json();
|
||||
expect(me.name).toBe(await page.evaluate(() => S.me.name));
|
||||
expect((await request.get('/api/feeds', as(token))).status()).toBe(200);
|
||||
// A wrong one is turned away, not let through as anyone.
|
||||
expect((await request.get('/api/me', as('ipx_' + '0'.repeat(64)))).status()).toBe(401);
|
||||
// Only the hash is kept: the list never carries the token.
|
||||
const listed = await page.evaluate(() => api('/api/tokens'));
|
||||
expect(JSON.stringify(listed)).not.toContain(token);
|
||||
expect(listed.find(t => t.name === 'test agent').last_used).toBeTruthy();
|
||||
|
||||
// Revoked, it opens nothing.
|
||||
await page.locator('.tokrow', { hasText: 'test agent' }).locator('[data-tok]').click();
|
||||
await expect(page.locator('.tokrow', { hasText: 'test agent' })).toHaveCount(0);
|
||||
expect((await request.get('/api/me', as(token))).status()).toBe(401);
|
||||
});
|
||||
|
||||
test('an admin can give a blog its Directory category', async ({ page }) => {
|
||||
const patch = (id, category) => page.evaluate(([id, category]) =>
|
||||
api(`/api/feeds/${id}`, { method: 'PATCH', body: JSON.stringify({ category }) }), [id, category]);
|
||||
const listed = async id => (await page.evaluate(() => api('/api/directory'))).find(p => p.id === id);
|
||||
|
||||
await patch('picture-blog', 'Visual Arts');
|
||||
expect(await listed('picture-blog')).toMatchObject({ podcast: false, category: 'Visual Arts' });
|
||||
// Apple's subcategory, under its category, as the Directory browses them (#118).
|
||||
expect(await listed('picture-blog')).toMatchObject({ podcast: false, category: 'Arts', subcategory: 'Visual Arts' });
|
||||
// A category's subcategories are on its own page, as Apple's directory has them.
|
||||
const pill = name => page.locator('#directory .pills button', { hasText: new RegExp(`^${name}\\d`) });
|
||||
await page.locator('#feedlist .place', { hasText: 'Directory' }).click();
|
||||
await expect(page.locator('#directory .pills')).toHaveCount(0);
|
||||
await page.locator('#directory .cat', { hasText: /^Arts\d/ }).click();
|
||||
await pill('Visual Arts').click();
|
||||
await expect(pill('Visual Arts')).toHaveAttribute('aria-pressed', 'true');
|
||||
await expect(page.locator('#dtiles .tile')).toHaveCount(1);
|
||||
// A feed's own iTunes category wins over one given here.
|
||||
await patch('test-show', 'Comedy');
|
||||
expect((await listed('test-show')).category).toBe('Technology');
|
||||
@@ -1233,7 +1347,7 @@ test('the favicon is the logo, square, from both pages', async ({ page }) => {
|
||||
await page.evaluate(() => setTheme('modern', 'dark'));
|
||||
await expect(page.locator('#favicon')).toHaveAttribute('href', /^\/favicon-dark\.png\?v=[0-9a-f]{12}$/);
|
||||
// A browser asks for /favicon.ico on its own, signed in or not.
|
||||
for (const path of ['/favicon.ico', '/favicon.png', '/favicon-dark.png', '/apple-touch-icon.png']) {
|
||||
for (const path of ['/favicon.ico', '/favicon.png', '/favicon-dark.png', '/apple-touch-icon.png', '/apple-touch-icon-precomposed.png']) {
|
||||
const r = await page.request.get(path, { headers: { cookie: '' } });
|
||||
expect(r.status(), path).toBe(200);
|
||||
expect(r.headers()['content-type'], path).toBe('image/png');
|
||||
|
||||
@@ -12,7 +12,10 @@ http.createServer((req, res) => {
|
||||
fs.readFile(file, (err, body) => {
|
||||
if (err) { res.writeHead(404).end('no'); return; }
|
||||
const type = file.endsWith('.mp3') ? 'audio/mpeg'
|
||||
: file.endsWith('.opml') ? 'text/x-opml' : 'application/xml';
|
||||
: file.endsWith('.opml') ? 'text/x-opml'
|
||||
// Artwork as an image, or /api/art refuses it as not one.
|
||||
: file.endsWith('.jpg') ? 'image/jpeg'
|
||||
: file.endsWith('.html') ? 'text/html' : 'application/xml';
|
||||
res.writeHead(200, { 'content-type': type, 'content-length': body.length });
|
||||
res.end(body);
|
||||
});
|
||||
|
||||
141
web/app.css
141
web/app.css
@@ -566,7 +566,14 @@ input:focus,select:focus{outline:0;border-color:var(--accent)}
|
||||
/* A show sits under its folder's title, a size down, so an open folder reads as one. */
|
||||
.feed.child{margin-left:11px}
|
||||
/* Pinned feeds, at the top of the list: a small pin before the name, and a rule under the last. */
|
||||
.feed .fpin .i{width:10px;height:10px;margin-right:5px;vertical-align:-1px;color:var(--accent)}
|
||||
/* Pinned: a disc on the artwork's corner, as a failing feed's mark is, in the accent and the ink
|
||||
the theme pairs with it. A feed both pinned and failing keeps the error below, the pin above. */
|
||||
.fpin{
|
||||
position:absolute;right:-5px;bottom:-5px;width:17px;height:17px;border-radius:50%;
|
||||
display:grid;place-items:center;background:var(--accent);color:var(--ink);border:2px solid var(--bg);
|
||||
}
|
||||
.fpin .i{width:9px;height:9px}
|
||||
.feed.err .fpin{bottom:auto;top:-5px}
|
||||
.feed.lastpin{margin-bottom:9px}
|
||||
.feed.lastpin::after{content:"";position:absolute;left:8px;right:8px;bottom:-5px;border-bottom:1px solid var(--line)}
|
||||
.feed.child .art{width:28px;height:28px;font-size:11px}
|
||||
@@ -630,24 +637,101 @@ body.scan-this .fhead [data-a=scan] .i,body.scan-any .fhead [data-a=scanall] .i,
|
||||
/* Quiet buttons: a filled one on every row outshouted the row that is playing. */
|
||||
#listening [data-a=play]{color:var(--accent)}
|
||||
#listening [data-a=remove]{color:var(--faint)}
|
||||
/* Directory's filters: .tabs for what a feed is, as the item filters are, and chips for what it
|
||||
is about. A picked chip is underlined in --accent2, as the download bar and the now-playing EQ
|
||||
are; a .badge's fill already means unread in the sidebar. */
|
||||
.dirbar{display:flex;flex-wrap:wrap;align-items:center;gap:8px 14px;margin-top:4px}
|
||||
.dirbar .tabs{flex:none}
|
||||
.chips{display:flex;flex-wrap:wrap;gap:2px 6px;flex:1 1 0;min-width:0}
|
||||
.chips button{flex:none;padding:4px 6px;font-size:13px;color:var(--dim);white-space:nowrap;border-bottom:2px solid transparent}
|
||||
.chips button:hover{color:var(--fg)}
|
||||
.chips button[aria-pressed="true"]{color:var(--fg);border-bottom-color:var(--accent2)}
|
||||
/* The Directory, laid out as Apple's is: a large title, then sections. Every page carries what
|
||||
kind of feed (.tabs, as the item filters are) in its title row. */
|
||||
.dhead{margin-bottom:6px}
|
||||
.dback{display:inline-flex;align-items:center;gap:5px;margin:0 0 8px -5px;padding:3px 5px;border-radius:6px;font-size:13px;color:var(--accent)}
|
||||
.dback .i{width:11px;height:11px}
|
||||
.dback:hover{background:var(--raise)}
|
||||
.dtitle{display:flex;flex-wrap:wrap;align-items:center;justify-content:space-between;gap:10px 18px}
|
||||
.dtitle h2{margin:0;font-size:30px;line-height:1.15;font-weight:700;letter-spacing:-.02em;overflow-wrap:anywhere}
|
||||
.dhead .sub{margin:5px 0 0;color:var(--dim);font-size:13px;font-variant-numeric:tabular-nums}
|
||||
.dsec{margin-top:28px}
|
||||
.dsech{display:flex;align-items:baseline;justify-content:space-between;gap:12px;margin-bottom:10px}
|
||||
.dsech h3{margin:0;font-size:19px;line-height:1.25;font-weight:650;letter-spacing:-.01em}
|
||||
.dall{font-size:13px;color:var(--accent);flex:none}
|
||||
.dall:hover{text-decoration:underline}
|
||||
/* The most subscribed, numbered because they are ranked: down one column, then the next. */
|
||||
.chart{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));grid-template-rows:repeat(var(--rows,5),auto);grid-auto-flow:column;column-gap:32px}
|
||||
.rank{display:grid;grid-template-columns:22px 52px minmax(0,1fr) auto;align-items:center;gap:12px;padding:8px 0;border-bottom:1px solid var(--line)}
|
||||
.rank .no{font-size:17px;font-weight:650;color:var(--faint);text-align:right;font-variant-numeric:tabular-nums}
|
||||
.rank .art{width:52px;height:52px;font-size:16px}
|
||||
.rank .txt b{display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;overflow-wrap:anywhere;font-weight:500;font-size:14px;line-height:1.3}
|
||||
.rank .txt small{display:block;margin-top:2px;color:var(--faint);font-size:12px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
|
||||
/* A category: its name, and its three most subscribed shows' covers fanned in the corner, so the
|
||||
shows are what colour it. Not tinted: a tint by the name's hash meant nothing, and the five that
|
||||
drew --good read as yours, which green means everywhere else (#131). The fan opens a little
|
||||
under the pointer, the only thing on the page that moves. */
|
||||
.cats{display:grid;grid-template-columns:repeat(auto-fill,minmax(210px,1fr));gap:12px}
|
||||
.cat{
|
||||
position:relative;overflow:hidden;height:116px;padding:13px 15px;border-radius:12px;text-align:left;
|
||||
display:flex;flex-direction:column;align-items:flex-start;
|
||||
background:var(--panel);border:1px solid var(--line);
|
||||
}
|
||||
.cat:hover{border-color:var(--accent)}
|
||||
/* The whole width: the fan sits below two lines of name, and a narrower name broke Government
|
||||
in two in the wider faces, Classic's and Glass's. */
|
||||
.cat b{font-size:16px;line-height:1.2;font-weight:650;color:var(--fg);overflow-wrap:break-word}
|
||||
.cat small{margin-top:3px;font-size:12px;color:var(--dim);font-variant-numeric:tabular-nums}
|
||||
.cat .fan{position:absolute;right:14px;bottom:-12px;width:112px;height:80px;pointer-events:none}
|
||||
.cat .fan .art{
|
||||
position:absolute;bottom:0;width:62px;height:62px;font-size:16px;border-radius:8px;box-shadow:0 3px 10px rgba(0,0,0,.3);
|
||||
/* Each cover on a layer of its own before the pointer arrives. Given one only as the fan began
|
||||
to open, the browser drew the covers again on the first frames, and again on the way back,
|
||||
and the fan opened in a jerk (#132). All three move, for as long as the eye can follow, as
|
||||
one motion that settles rather than stops. */
|
||||
will-change:transform;
|
||||
transition:rotate .45s cubic-bezier(.2,.7,.2,1),translate .45s cubic-bezier(.2,.7,.2,1);
|
||||
}
|
||||
.cat .fan .art:nth-last-child(1){right:0;rotate:8deg;translate:0 -12px}
|
||||
.cat .fan .art:nth-last-child(2){right:25px;rotate:-3deg;translate:0 -4px}
|
||||
.cat .fan .art:nth-last-child(3){right:50px;rotate:-13deg;translate:0 4px}
|
||||
.cat:hover .fan .art:nth-last-child(1){rotate:13deg;translate:5px -17px}
|
||||
.cat:hover .fan .art:nth-last-child(2){rotate:-2deg;translate:0 -8px}
|
||||
.cat:hover .fan .art:nth-last-child(3){rotate:-19deg;translate:-7px 3px}
|
||||
/* A category's subcategories, each with how many it holds, then the order. A picked one is
|
||||
drawn as a picked tab is. */
|
||||
.dtools{display:flex;align-items:center;gap:10px 14px;margin-bottom:14px}
|
||||
.pills{display:flex;flex-wrap:wrap;gap:6px;flex:1 1 0;min-width:0}
|
||||
.pills button{flex:none;padding:4px 11px;border:1px solid var(--line);border-radius:999px;font-size:13px;color:var(--dim);white-space:nowrap}
|
||||
.pills button small{margin-left:6px;color:var(--faint);font-size:11.5px;font-variant-numeric:tabular-nums}
|
||||
.pills button:hover{border-color:var(--accent);color:var(--fg)}
|
||||
.pills button.on{background:var(--raise);border-color:var(--raise);color:var(--fg);font-weight:500}
|
||||
.dtools .grow{flex:1}
|
||||
#dirsort{flex:none;width:auto;padding:4px 6px;font-size:13px}
|
||||
/* Square cover art, title underneath: podcast art is made to be known at a glance. The subscribe
|
||||
button stays visible, since a button that only shows on hover cannot be reached on a phone. */
|
||||
.tiles{display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:18px 14px;margin-top:14px}
|
||||
.tiles{display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:18px 14px}
|
||||
.tiles>.hint{grid-column:1/-1}
|
||||
.tile{display:grid;grid-template-columns:minmax(0,1fr) auto;gap:6px 2px;align-items:start;cursor:pointer}
|
||||
.tile{display:grid;grid-template-columns:minmax(0,1fr) auto;gap:6px 2px;align-items:start}
|
||||
.tile .art{grid-column:1/-1;width:100%;height:auto;aspect-ratio:1;font-size:34px}
|
||||
.tile .txt b{display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;overflow-wrap:anywhere;font-weight:500;font-size:13px;line-height:1.3}
|
||||
.tile .txt small{display:block;color:var(--faint);font-size:11.5px}
|
||||
.tile:hover .txt b{color:var(--accent)}
|
||||
/* Every one opens: yours its feed, the rest their page here. */
|
||||
.tile,.rank{cursor:pointer}
|
||||
.tile:hover .txt b,.rank:hover .txt b{color:var(--accent)}
|
||||
/* A feed's page before you subscribe, as Apple's show page: the cover large, what it is and who
|
||||
here reads it beside it, then what it has published lately. */
|
||||
.show{display:flex;gap:22px;align-items:flex-end;margin-top:4px}
|
||||
.show>.art{width:180px;height:180px;font-size:48px;border-radius:12px;box-shadow:var(--shadow)}
|
||||
.show .meta{min-width:0;flex:1;display:flex;flex-direction:column;align-items:flex-start;gap:5px}
|
||||
.show h2{margin:0 0 2px;font-size:30px;line-height:1.15;font-weight:700;letter-spacing:-.02em;overflow-wrap:anywhere}
|
||||
.show .sub{margin:0;color:var(--dim);font-size:13px;font-variant-numeric:tabular-nums}
|
||||
.show .acts{margin-top:8px}
|
||||
.dlink{padding:0;font-size:13px;color:var(--accent);text-align:left}
|
||||
.dlink:hover{text-decoration:underline}
|
||||
/* What the feed says it is, three lines of it until More. */
|
||||
.about{max-width:760px;margin-top:18px}
|
||||
.about p{margin:0;color:var(--dim);font-size:14px;line-height:1.55;display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:3;overflow:hidden;overflow-wrap:anywhere}
|
||||
.about.open p{display:block}
|
||||
.about .dlink{margin-top:4px}
|
||||
.latest{max-width:760px}
|
||||
.lep{padding:12px 0;border-bottom:1px solid var(--line)}
|
||||
.lep b{display:block;font-size:15px;font-weight:600;line-height:1.35;overflow-wrap:anywhere}
|
||||
.lep b a{color:var(--fg);text-decoration:none}
|
||||
.lep b a:hover{color:var(--accent)}
|
||||
.lep p{margin:4px 0 0;color:var(--dim);font-size:13px;line-height:1.5;display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:3;overflow:hidden}
|
||||
.lep small{display:flex;align-items:center;gap:7px;margin-top:6px;color:var(--faint);font-size:12px;font-variant-numeric:tabular-nums}
|
||||
/* Outlined, not filled: on --raise the warning and error colours fell short of AA in most light
|
||||
themes, where on the row's own ground they clear it. */
|
||||
.tag{
|
||||
@@ -676,7 +760,7 @@ body.scan-this .fhead [data-a=scan] .i,body.scan-any .fhead [data-a=scanall] .i,
|
||||
/* A selected row is --raise too, and the zero count's pill vanished into it. */
|
||||
.sel .badge.zero{background:var(--bg)}
|
||||
/* Counts, sizes and times that change in place should not jiggle the text around them. */
|
||||
.badge,.feed small,.childrow small,.tile small,.ep,.fhead .sub,.dmeta,#status,#seekrow{font-variant-numeric:tabular-nums}
|
||||
.badge,.feed small,.childrow small,.tile small,.rank small,.ep,.fhead .sub,.dmeta,#status,#seekrow{font-variant-numeric:tabular-nums}
|
||||
|
||||
/* ---------- main ---------- */
|
||||
#main{overflow:hidden;min-height:0;min-width:0;display:flex;flex-direction:column}
|
||||
@@ -824,7 +908,10 @@ body.adminpage #log{max-width:none}
|
||||
icon, so it spilled 3px right of centre and off line with its heading. */
|
||||
.ep .st,.ep .fl{width:22px;height:22px;padding:0;border-radius:5px;display:grid;place-items:center;font-size:11px;color:var(--accent2)}
|
||||
.ep .fl{color:var(--faint);font-size:13px}
|
||||
.ep .fl.on{color:var(--fg)}
|
||||
/* Pinned: the pinned feed's disc (.fpin), the accent and its ink, drawn inside the 22px button so
|
||||
the row does not move when it changes. */
|
||||
.ep .fl.on,.ep .fl.on:hover{color:var(--ink);background:radial-gradient(circle,var(--accent) 8.5px,transparent 9px)}
|
||||
.ep .fl.on .i{width:9px;height:9px}
|
||||
/* The icon's EQ bars mark what is playing: standing still, and moving only while it plays. A
|
||||
paused animation was tried first; the frames it held were a pixel apart and read as dots. */
|
||||
.eq{display:inline-flex;align-items:flex-end;gap:2px;width:13px;height:12px;flex:none}
|
||||
@@ -837,6 +924,8 @@ body.playing .eq i:nth-child(3){animation-delay:-.6s}
|
||||
@keyframes eq{from{transform:scaleY(.35)}}
|
||||
.ep .st:hover,.ep .fl:hover{background:var(--raise)}
|
||||
.ep .t{font-weight:600;font-size:13.5px;display:block;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
|
||||
/* An item with no title shows its opening words: set as text, not as a heading. */
|
||||
.ep .t.notitle{font-weight:400}
|
||||
.ep.read .t{color:var(--dim);font-weight:500}
|
||||
.ep .line{display:flex;gap:9px;align-items:center;flex-wrap:wrap;color:var(--faint);font-size:11.5px}
|
||||
.ep .line:empty{display:none}
|
||||
@@ -937,6 +1026,9 @@ input[type=range]::-moz-range-thumb{width:12px;height:12px;border:0;border-radiu
|
||||
.inline{display:flex;gap:6px;align-items:stretch}
|
||||
.inline input{flex:1;min-width:0}
|
||||
.inline .btn{white-space:nowrap;flex:none}
|
||||
/* An API token in Settings: its name and dates, and the button that revokes it. */
|
||||
.tokrow{display:flex;align-items:center;justify-content:space-between;gap:8px;margin-bottom:6px}
|
||||
.tokrow .hint{display:block}
|
||||
.inline select{flex:none;width:auto}
|
||||
.inline input[type=number]{flex:none;width:90px}
|
||||
.check input{width:16px;height:16px;accent-color:var(--accent)}
|
||||
@@ -1043,9 +1135,22 @@ body:has(#player.on) #status{padding-bottom:3px}
|
||||
#pnow .txt small,#pright #vol{display:none}
|
||||
.wrap.plain{padding:14px}
|
||||
.card{padding:16px}
|
||||
/* The chips get a line of their own and scroll sideways rather than wrap, so they never push
|
||||
the grid down. */
|
||||
.chips{flex:1 1 100%;flex-wrap:nowrap;overflow-x:auto}
|
||||
/* The subcategories get a line of their own and scroll sideways rather than wrap, so they
|
||||
never push the grid down. */
|
||||
.dtitle h2{font-size:25px}
|
||||
.dsec{margin-top:22px}
|
||||
.dsech h3{font-size:17px}
|
||||
.chart{grid-template-columns:minmax(0,1fr);grid-template-rows:none;grid-auto-flow:row}
|
||||
.cats{grid-template-columns:repeat(auto-fill,minmax(150px,1fr));gap:10px}
|
||||
.cat{height:100px;padding:11px 12px}
|
||||
.cat b{font-size:15px}
|
||||
.cat .fan{right:6px;bottom:-16px;transform:scale(.7);transform-origin:bottom right}
|
||||
.pills{flex:1 1 100%;flex-wrap:nowrap;overflow-x:auto}
|
||||
.dtools{flex-wrap:wrap}
|
||||
.show{flex-direction:column;align-items:flex-start;gap:14px}
|
||||
.show>.art{width:150px;height:150px;font-size:40px}
|
||||
.show h2{font-size:24px}
|
||||
#dirsort{margin-left:auto}
|
||||
.tiles{grid-template-columns:repeat(auto-fill,minmax(104px,1fr));gap:14px 10px}
|
||||
|
||||
/* A log line has no room for four columns: keep time and message, wrap as prose. */
|
||||
|
||||
@@ -49,6 +49,10 @@ async function drawServer(){
|
||||
items are never touched.</span></div>
|
||||
<div class="field"><label>Delete items older than (days, 0 = keep)</label>
|
||||
<input type="number" id="gage" min="0" value="${g.max_age_days}"></div>
|
||||
<div class="field"><label>Artwork kept (MB, 0 = none)</label>
|
||||
<input type="number" id="gart" min="0" value="${g.art_cache_mb}">
|
||||
<span class="hint">Show and episode artwork is kept here once shown, so it loads from iPX
|
||||
instead of each publisher. Over this, what has gone longest unshown is dropped first.</span></div>
|
||||
<div class="field"><label>Download folder</label>
|
||||
<span class="hint" style="overflow-wrap:anywhere">${esc(g.download_dir)}</span></div>
|
||||
<div class="cardacts"><button class="btn primary" id="gsave">${ICON.check} Save</button></div>`;
|
||||
@@ -59,7 +63,8 @@ async function drawServer(){
|
||||
max_new_per_check: Math.max(0, Number($('#gmax').value) || 0),
|
||||
media_types: $('#gtypes').value.split(',').map(t => t.trim()).filter(Boolean),
|
||||
max_total_gb: Number($('#gquota').value) || 0,
|
||||
max_age_days: Number($('#gage').value) || 0})});
|
||||
max_age_days: Number($('#gage').value) || 0,
|
||||
art_cache_mb: Math.max(0, Number($('#gart').value) || 0)})});
|
||||
toast('Settings saved');
|
||||
}catch(e){ toast(e.message, true); }
|
||||
};
|
||||
|
||||
@@ -34,97 +34,233 @@ $('#addFeed').onclick=()=>{
|
||||
};
|
||||
};
|
||||
|
||||
// What everyone here reads, you included, as a place to start. The rows carry an id, never a
|
||||
// URL, so a key in someone's feed address never reaches this page.
|
||||
// The rows carry an id, never a URL, so a key in someone's feed address never reaches this page.
|
||||
const NONE_LISTED='<p class="hint">Nothing yet. Feeds people here subscribe to show up here.</p>';
|
||||
async function listFeeds(url,box){
|
||||
let rows=[];
|
||||
try{ rows=await api(url)||[]; }catch{}
|
||||
box.innerHTML=rows.length?'':NONE_LISTED;
|
||||
for(const p of rows) box.appendChild(listedFeed(p,'childrow'));
|
||||
return rows.length;
|
||||
}
|
||||
|
||||
/// One listed feed: a row in Popular and the Add a feed dialog, a tile in Directory's grid. The
|
||||
/// parts are the same either way; the class lays them out.
|
||||
function listedFeed(p,cls){
|
||||
/// One listed feed: a tile in the Directory's grid or, given its place, a row in its chart. The
|
||||
/// parts are the same either way; the class lays them out. Yours opens its feed; any other, its
|
||||
/// page in the Directory, to see what it is before you take it (#128).
|
||||
function listedFeed(p,cls,rank?: number){
|
||||
const el=document.createElement('div');
|
||||
el.className=cls;
|
||||
el.innerHTML=artHTML(p.image,p.title||p.id)+
|
||||
`<div class="txt"><b>${esc(p.title||p.id)}</b>`+
|
||||
`<small class="meta">${p.subscribers} subscriber${p.subscribers===1?'':'s'}</small></div>`+
|
||||
el.className=cls+(p.subscribed?' mine':'');
|
||||
el.tabIndex=0; el.dataset.id=p.id;
|
||||
// As the feed list's rows: Enter or Space opens it, and stops there, or Space would also
|
||||
// play or pause.
|
||||
el.onkeydown=ev=>{
|
||||
if((ev.key==='Enter'||ev.key===' ')&&ev.target===el){ ev.preventDefault(); ev.stopPropagation(); el.click(); }
|
||||
};
|
||||
// A chart says what a show is about, as Apple's does; the grid says who here reads it, when
|
||||
// anyone does. "0 subscribers" under most of the grid said nothing.
|
||||
const meta=rank?(p.subcategory||p.category):p.subscribers?plural(p.subscribers,'subscriber'):'';
|
||||
el.innerHTML=(rank?`<span class="no">${rank}</span>`:'')+artHTML(p.image,NAME(p))+
|
||||
`<div class="txt"><b>${esc(NAME(p))}</b>${meta?`<small class="meta">${esc(meta)}</small>`:''}</div>`+
|
||||
// Green, as a downloaded file is: it is already yours. Plus, beside it, is the way to get one.
|
||||
(p.subscribed?`<span class="subbed" title="Subscribed: click to open it" aria-label="Subscribed">${ICON.subbed}</span>`
|
||||
:`<button class="btn ico" data-a="sub" title="Subscribe" aria-label="Subscribe">${ICON.plus}</button>`);
|
||||
// Yours already: the row opens it instead.
|
||||
if(p.subscribed){ el.onclick=()=>{ closeModal(); selectFeed(p.id); }; return el; }
|
||||
$('[data-a="sub"]',el).onclick=async()=>{
|
||||
try{
|
||||
await api(`/api/popular/${encodeURIComponent(p.id)}`,{method:'POST'});
|
||||
closeModal(); toast(`Subscribed to ${p.title||p.id}`);
|
||||
await loadFeeds(true); selectFeed(p.id);
|
||||
}catch(e){ toast(e.message,true); }
|
||||
};
|
||||
if(p.subscribed){ el.onclick=()=>selectFeed(p.id); return el; }
|
||||
el.onclick=()=>{ dirShow=p.id; $('#content').scrollTop=0; drawDirectory(undefined,'.dback'); };
|
||||
// Its own click, not the tile's as well.
|
||||
$('[data-a="sub"]',el).onclick=ev=>{ ev.stopPropagation(); subscribeListed(p); };
|
||||
return el;
|
||||
}
|
||||
|
||||
// Directory's filters. Kept out here because a finished scan redraws the pane, which would
|
||||
// otherwise clear them.
|
||||
let dirKind='All', dirCat=null;
|
||||
const KINDS={All:()=>true,Podcasts:p=>p.podcast,Blogs:p=>!p.podcast};
|
||||
/// Directory: every listed feed as its cover art, under two filters that combine: what a feed is
|
||||
/// (Podcasts, anything with audio or video, or Blogs, the rest) and what it is about (its iTunes
|
||||
/// category, as chips). Both filter in place, without asking the server again.
|
||||
async function renderDirectory(url,box){
|
||||
let rows=[];
|
||||
try{ rows=await api(url)||[]; }catch{}
|
||||
if(!rows.length){ box.innerHTML=NONE_LISTED; return 0; }
|
||||
const bar=$('#dirbar');
|
||||
// Only a filter when the server has both kinds.
|
||||
const both=rows.some(KINDS.Podcasts)&&rows.some(KINDS.Blogs);
|
||||
const btn=(k,v,on)=>`<button type="button" data-${k}="${esc(v)}" class="${on?'on':''}" aria-pressed="${on}">${esc(v)}</button>`;
|
||||
const draw=()=>{
|
||||
if(!both) dirKind='All';
|
||||
const ofKind=rows.filter(KINDS[dirKind]);
|
||||
// No empty chips: only the categories among the feeds the kind lets through.
|
||||
const cats=[...new Set(ofKind.map(p=>p.category).filter(Boolean))].sort();
|
||||
if(!cats.includes(dirCat)) dirCat=null;
|
||||
bar.innerHTML=
|
||||
(both?`<div class="tabs" role="group" aria-label="Kind">${Object.keys(KINDS).map(k=>btn('kind',k,k===dirKind)).join('')}</div>`:'')+
|
||||
(cats.length?`<div class="chips" role="group" aria-label="Category">${cats.map(c=>btn('cat',c,c===dirCat)).join('')}</div>`:'');
|
||||
// A picked chip lifts on a second press. Everything is redrawn, so the keyboard goes back to
|
||||
// the button just pressed.
|
||||
for(const b of $$('button',bar)) b.onclick=()=>{
|
||||
const k=b.dataset.kind!=null?'kind':'cat', v=b.dataset[k];
|
||||
if(k==='kind') dirKind=v; else dirCat=dirCat===v?null:v;
|
||||
draw(); $(`[data-${k}="${CSS.escape(v)}"]`,bar)?.focus();
|
||||
};
|
||||
box.innerHTML='';
|
||||
for(const p of ofKind.filter(p=>!dirCat||p.category===dirCat)) box.appendChild(listedFeed(p,'tile'));
|
||||
};
|
||||
draw();
|
||||
return rows.length;
|
||||
async function subscribeListed(p){
|
||||
try{
|
||||
await api(`/api/popular/${encodeURIComponent(p.id)}`,{method:'POST'});
|
||||
toast(`Subscribed to ${NAME(p)}`);
|
||||
await loadFeeds(true); selectFeed(p.id);
|
||||
}catch(e){ toast(e.message,true); }
|
||||
}
|
||||
|
||||
/// Directory and Popular open in the main pane, as the original's Directory did.
|
||||
// Where you are in the Directory: what kind of feed, which category and subcategory, in what
|
||||
// order. Out here because a search redraws the page, and the kind and order hold across visits.
|
||||
let dirKind='All', dirCat=null, dirSub=null, dirSort='az', dirRows=[];
|
||||
// The feed whose page is open over all that, by id; going back leaves the rest as it was.
|
||||
let dirShow=null;
|
||||
// The two pages that are not a category: every feed, and the feeds that name none. A category is
|
||||
// Apple's or one an admin typed, neither of which starts with a colon.
|
||||
const EVERY=':every', OTHER=':other';
|
||||
const NAME=p=>(p.title||p.id);
|
||||
const byName=(a,b)=>NAME(a).localeCompare(NAME(b),undefined,{sensitivity:'base',numeric:true});
|
||||
const bySubs=(a,b)=>b.subscribers-a.subscribers||byName(a,b);
|
||||
/// The grid's orders. It opens A to Z, as the server sends it.
|
||||
const DIR_SORTS={
|
||||
subscribers:['Most subscribers',bySubs],
|
||||
az:['Name, A to Z',byName],
|
||||
za:['Name, Z to A',(a,b)=>byName(b,a)],
|
||||
};
|
||||
const KINDS={All:()=>true,Podcasts:p=>p.podcast,Blogs:p=>!p.podcast};
|
||||
const KIND_WORD={All:'feed',Podcasts:'podcast',Blogs:'blog'};
|
||||
|
||||
/// The Directory, as Apple's is laid out: it opens on the most subscribed feeds and the
|
||||
/// categories, and a category has a page of its own, its subcategories along the top of its
|
||||
/// grid. What a feed is (Podcasts, anything with audio or video, or Blogs, the rest) holds across
|
||||
/// every page, and the search box looks through the lot. One fetch draws all of it.
|
||||
async function renderDirectory(url,box){
|
||||
// Each visit starts at the front, as going back to a tab does.
|
||||
dirCat=null; dirSub=null; dirShow=null;
|
||||
try{ dirRows=await api(url)||[]; }catch{ dirRows=[]; }
|
||||
drawDirectory(box);
|
||||
return dirRows.length;
|
||||
}
|
||||
|
||||
/// `focus` is the control to put the keyboard back on, since the whole page is redrawn.
|
||||
function drawDirectory(box=$('#directory'), focus?: string){
|
||||
if(!box) return;
|
||||
if(!dirRows.length){ box.innerHTML=NONE_LISTED; return; }
|
||||
// Only a choice when the server has both kinds.
|
||||
const both=dirRows.some(KINDS.Podcasts)&&dirRows.some(KINDS.Blogs);
|
||||
if(!both) dirKind='All';
|
||||
const rows=dirRows.filter(KINDS[dirKind]), word=KIND_WORD[dirKind];
|
||||
const count=n=>`${n.toLocaleString()} ${word}${n===1?'':'s'}`;
|
||||
const of=c=>p=>c===EVERY||(c===OTHER?!p.category:p.category===c);
|
||||
// A kind can leave a category with nothing in it; that goes back to the front.
|
||||
if(dirCat&&!rows.some(of(dirCat))) dirCat=null;
|
||||
const catName=c=>c===EVERY?`All ${word}s`:c===OTHER?'Other':c;
|
||||
const top=list=>list.filter(p=>p.subscribers).sort(bySubs).slice(0,10);
|
||||
const btn=(k,v,label,on)=>`<button type="button" data-${k}="${esc(v)}" aria-pressed="${on}"${on?' class="on"':''}>${label}</button>`;
|
||||
// back: where its button goes, '' for the front; none on the front itself.
|
||||
const head=(title,sub,back?)=>`<div class="dhead">${back==null?'':
|
||||
`<button type="button" class="dback" data-go="${esc(back)}">${ICON.left}${esc(back?catName(back):'Directory')}</button>`}
|
||||
<div class="dtitle"><h2>${esc(title)}</h2>${both?`<div class="tabs" role="group" aria-label="Kind">${
|
||||
Object.keys(KINDS).map(k=>btn('kind',k,k,k===dirKind)).join('')}</div>`:''}</div>
|
||||
<p class="sub">${esc(sub)}</p></div>`;
|
||||
const sec=(title,body,extra='')=>`<section class="dsec">${title?`<div class="dsech"><h3>${esc(title)}</h3>${extra}</div>`:''}${body}</section>`;
|
||||
const tools=(pills='')=>`<div class="dtools">${pills||'<span class="grow"></span>'}<select id="dirsort" aria-label="Sort">${
|
||||
Object.entries(DIR_SORTS).map(([k,[label]])=>`<option value="${k}"${k===dirSort?' selected':''}>${label}</option>`).join('')}</select></div>`;
|
||||
const q=S.q.trim(), grid='<div class="tiles" id="dtiles"></div>', board='<div class="chart" id="dchart"></div>';
|
||||
let html, chart=[], tiles=null;
|
||||
const show=dirShow&&dirRows.find(p=>p.id===dirShow);
|
||||
if(show){
|
||||
const n=show.subscribers;
|
||||
html=`<div class="dhead"><button type="button" class="dback" data-back>${ICON.left}${esc(q?'Results':dirCat?catName(dirCat):'Directory')}</button></div>
|
||||
<div class="show">${artHTML(show.image,NAME(show))}<div class="meta"><h2>${esc(NAME(show))}</h2>${
|
||||
// Its category's page is one click away from here.
|
||||
show.category?`<button type="button" class="dlink" data-go="${esc(show.category)}">${esc(show.subcategory||show.category)}</button>`:''}
|
||||
<p class="sub">${n?`${n} ${n===1?'person here subscribes':'people here subscribe'}`:'Nobody here subscribes yet'}</p>
|
||||
<div class="acts"><button class="btn ico primary" data-a="sub" title="Subscribe" aria-label="Subscribe">${ICON.plus}</button></div></div></div>
|
||||
<div class="about" id="dabout" hidden><p></p><button type="button" class="dlink" aria-expanded="false" hidden>More</button></div>`
|
||||
+sec(show.podcast?'Latest episodes':'Latest posts','<div class="latest" id="dshow"><p class="hint">Loading…</p></div>');
|
||||
}else if(q){
|
||||
// Names only: a category is a tap away, and "news" matching every News feed buried the
|
||||
// one called that.
|
||||
tiles=rows.filter(p=>NAME(p).toLocaleLowerCase().includes(q.toLocaleLowerCase()));
|
||||
html=head(`Results for “${q}”`,tiles.length?count(tiles.length):`No ${word} here has that in its name.`,dirCat||'')+tools()+grid;
|
||||
}else if(!dirCat){
|
||||
chart=top(rows);
|
||||
const cats=new Map<string,any[]>();
|
||||
for(const p of rows){ const c=p.category||OTHER; if(!cats.has(c)) cats.set(c,[]); cats.get(c).push(p); }
|
||||
// Other, the feeds with no category, goes last.
|
||||
const order=[...cats.keys()].sort((a,b)=>Number(a===OTHER)-Number(b===OTHER)||a.localeCompare(b));
|
||||
html=head('Directory',`${count(rows.length)} on this server. Private feeds are never listed.`)
|
||||
+(chart.length?sec('Most subscribed',board):'')
|
||||
+sec('Categories',`<div class="cats">${order.map(c=>catTile(c,catName(c),count(cats.get(c).length),cats.get(c))).join('')}</div>`,
|
||||
`<button type="button" class="dall" data-go="${EVERY}">See all ${rows.length.toLocaleString()}</button>`);
|
||||
}else{
|
||||
const inCat=rows.filter(of(dirCat));
|
||||
const subs=dirCat===EVERY?[]:[...new Set(inCat.map(p=>p.subcategory).filter(Boolean))].sort();
|
||||
if(!subs.includes(dirSub)) dirSub=null;
|
||||
if(dirCat!==EVERY) chart=top(inCat);
|
||||
tiles=inCat.filter(p=>!dirSub||p.subcategory===dirSub);
|
||||
const pill=(v,label,n)=>btn('sub',v,`${esc(label)}<small>${n}</small>`,(dirSub||'')===v);
|
||||
html=head(catName(dirCat),dirCat===OTHER?`${count(inCat.length)} that name no category`:count(inCat.length),'')
|
||||
+(chart.length?sec('Most subscribed',board):'')
|
||||
// Its own heading only when the chart is above it, to say where one list ends.
|
||||
+sec(chart.length?`${word[0].toUpperCase()+word.slice(1)}s`:'',tools(subs.length?`<div class="pills" role="group" aria-label="Subcategory">${
|
||||
pill('','All',inCat.length)+subs.map(s=>pill(s,s,inCat.filter(p=>p.subcategory===s).length)).join('')}</div>`:'')+grid);
|
||||
}
|
||||
box.innerHTML=html;
|
||||
if(show){ $('.show [data-a="sub"]',box).onclick=()=>subscribeListed(show); fillShow(box,show); }
|
||||
const c=$('#dchart',box);
|
||||
if(c){ c.style.setProperty('--rows',String(Math.ceil(chart.length/2))); chart.forEach((p,i)=>c.appendChild(listedFeed(p,'rank',i+1))); }
|
||||
const t=$('#dtiles',box);
|
||||
if(t) for(const p of tiles.sort(DIR_SORTS[dirSort][1])) t.appendChild(listedFeed(p,'tile'));
|
||||
box.onclick=ev=>{
|
||||
const b=(ev.target as Element).closest<HTMLElement>('[data-go],[data-kind],[data-sub],[data-back]');
|
||||
if(!b) return;
|
||||
// Back from a feed's page, onto the tile or row it was opened from.
|
||||
if(b.dataset.back!=null){ const id=dirShow; dirShow=null; return drawDirectory(box,`[data-id="${CSS.escape(id)}"]`); }
|
||||
if(b.dataset.kind!=null){ dirKind=b.dataset.kind; return drawDirectory(box,`[data-kind="${CSS.escape(dirKind)}"]`); }
|
||||
if(b.dataset.sub!=null){ dirSub=b.dataset.sub||null; return drawDirectory(box,`[data-sub="${CSS.escape(b.dataset.sub)}"]`); }
|
||||
// Going anywhere ends a search.
|
||||
if(S.q){ S.q=''; $('#epSearch').value=''; }
|
||||
const from=dirCat;
|
||||
dirCat=b.dataset.go||null; dirSub=null; dirShow=null;
|
||||
$('#content').scrollTop=0;
|
||||
// Into a page, onto its way back; back out, onto the tile you went in by.
|
||||
drawDirectory(box,dirCat?'.dback':from?`[data-go="${CSS.escape(from)}"]:not(.dback)`:undefined);
|
||||
};
|
||||
const s=$('#dirsort',box);
|
||||
if(s) s.onchange=()=>{ dirSort=s.value; drawDirectory(box,'#dirsort'); };
|
||||
if(focus) $(focus,box)?.focus();
|
||||
}
|
||||
|
||||
/// A listed feed's page, filled in: what it says it is, its first lines unless you ask for the
|
||||
/// rest, and its latest items, each one's title, opening its post on the publisher's site where
|
||||
/// it has one, its first lines, and when and how long.
|
||||
async function fillShow(box,p){
|
||||
const list=$('#dshow',box), about=$('#dabout',box);
|
||||
let got;
|
||||
try{ got=await api(`/api/directory/${encodeURIComponent(p.id)}`); }
|
||||
catch(e){ list.innerHTML=`<p class="hint">${esc(e.message)}</p>`; return; }
|
||||
// Gone back, or on to another, while it loaded.
|
||||
if(dirShow!==p.id||!list.isConnected) return;
|
||||
// Some feeds give their title again as their description, which says nothing new.
|
||||
const said=plainText(got.description);
|
||||
if(said&&said!==NAME(p)){
|
||||
const text=$('p',about), more=$('button',about);
|
||||
text.textContent=said; about.hidden=false;
|
||||
// Only when there is more than the three lines shown.
|
||||
more.hidden=text.scrollHeight<=text.clientHeight+1;
|
||||
more.onclick=()=>{
|
||||
const open=about.classList.toggle('open');
|
||||
more.textContent=open?'Less':'More'; more.setAttribute('aria-expanded',String(open));
|
||||
};
|
||||
}
|
||||
const items=got.items;
|
||||
list.innerHTML=items.length?items.map(e=>{
|
||||
const name=entryName(e), text=name.derived?'':plainText(e.description).slice(0,500);
|
||||
const when=[dateOf(e.published),e.duration&&clock(e.duration)].filter(Boolean);
|
||||
return `<article class="lep"><b>${e.link?`<a href="${esc(e.link)}" target="_blank" rel="noopener">${esc(name.text)}</a>`:esc(name.text)}</b>`
|
||||
+(text?`<p>${esc(text)}</p>`:'')
|
||||
+(when.length?`<small>${when.map(s=>`<span>${s}</span>`).join('<span class="dot"></span>')}</small>`:'')+'</article>';
|
||||
}).join(''):`<p class="hint">Nothing read from it yet. A feed nobody here subscribes to is checked once a day.</p>`;
|
||||
}
|
||||
|
||||
/// A category as a tile: its name, how many feeds, and the covers of its three most subscribed
|
||||
/// fanned in the corner, so each one is coloured by what is in it, as Apple's are.
|
||||
function catTile(c,name,count,rows){
|
||||
// Covers before initials, which are all Other had; the likeliest first, and first is in front:
|
||||
// the fan is drawn back to front.
|
||||
const art=[...rows].sort((a,b)=>Number(!a.image)-Number(!b.image)||bySubs(a,b)).slice(0,3).reverse();
|
||||
return `<button type="button" class="cat" data-go="${esc(c)}"><b>${esc(name)}</b><small>${count}</small>`+
|
||||
`<span class="fan" aria-hidden="true">${art.map(p=>artHTML(p.image,NAME(p))).join('')}</span></button>`;
|
||||
}
|
||||
|
||||
/// The Directory and Currently Listening open in the main pane, as the original's Directory did.
|
||||
async function renderListed(v){
|
||||
const box=$('#content');
|
||||
box.classList.add('plain');
|
||||
$('#tbRemove').disabled=true;
|
||||
syncTools(null);
|
||||
if(v===VIEWS[':directory']){
|
||||
$('#epSearch').placeholder='Find a feed…';
|
||||
box.innerHTML='<div id="directory"><p class="hint">Loading…</p></div>';
|
||||
$('#count').textContent=v.title;
|
||||
const n=await renderDirectory(v.url,$('#directory'));
|
||||
if(VIEWS[S.feed]===v) $('#count').textContent=`${v.title}: ${plural(n,'feed')}`;
|
||||
return;
|
||||
}
|
||||
$('#epSearch').placeholder='Search items…';
|
||||
const listening=v===VIEWS[':listening'], grid=v===VIEWS[':directory'];
|
||||
box.innerHTML=`
|
||||
<div class="fhead slim">
|
||||
<div class="art">${v.icon}</div>
|
||||
<div class="meta"><h2>${v.title}</h2>
|
||||
<div class="sub">${v.blurb}${listening?'':' Everyone counts, you included. Private feeds are never listed.'}</div></div>
|
||||
<div class="meta"><h2>${v.title}</h2><div class="sub">${v.blurb}</div></div>
|
||||
</div>
|
||||
${grid?'<div class="dirbar" id="dirbar"></div>':''}
|
||||
<div class="${grid?'tiles':'childlist'}" id="${listening?'listening':'popular'}"><p class="hint">Loading…</p></div>`;
|
||||
<div class="childlist" id="listening"><p class="hint">Loading…</p></div>`;
|
||||
$('#count').textContent=v.title;
|
||||
const n=await (listening?renderListening:grid?renderDirectory:listFeeds)(v.url,$(listening?'#listening':'#popular',box));
|
||||
if(VIEWS[S.feed]===v) $('#count').textContent=`${v.title}: ${plural(n,listening?'episode':'feed')}`;
|
||||
const n=await renderListening(v.url,$('#listening',box));
|
||||
if(VIEWS[S.feed]===v) $('#count').textContent=`${v.title}: ${plural(n,'episode')}`;
|
||||
}
|
||||
|
||||
/// Currently Listening: episodes you started and have not finished, across every feed you
|
||||
@@ -140,7 +276,7 @@ async function renderListening(url,box){
|
||||
el.className='childrow';
|
||||
el.entry=e;
|
||||
el.innerHTML=artHTML(e.image||feedArt(e.feed_id),e.title||'')+
|
||||
`<div class="txt"><b>${EQ}<span>${esc(e.title||'(untitled)')}</span></b>`+
|
||||
`<div class="txt"><b>${EQ}<span>${esc(entryName(e).text)}</span></b>`+
|
||||
`<small><span class="fd">${esc(feedName(e.feed_id))}</span><span class="left"></span></small></div>`+
|
||||
`<button class="iconbtn" data-a="play"></button>`+
|
||||
`<button class="iconbtn" data-a="remove" title="Remove from Currently Listening" aria-label="Remove from Currently Listening">${ICON.close}</button>`+
|
||||
@@ -193,7 +329,9 @@ $('#tbRead').onclick=()=>{ const e=cur(); if(e) epAction('read',e,null); };
|
||||
$('#tbFlag').onclick=()=>{ const e=cur(); if(e) epAction('flag',e,null); };
|
||||
let searchT;
|
||||
$('#epSearch').oninput=ev=>{ clearTimeout(searchT);
|
||||
searchT=setTimeout(()=>{ S.q=ev.target.value; S.offset=0; loadEntries(); },250); };
|
||||
// The Directory looks through what it already has; the rest ask the server (#126).
|
||||
searchT=setTimeout(()=>{ S.q=ev.target.value; S.offset=0;
|
||||
if(S.feed===':directory'){ dirShow=null; drawDirectory(); } else loadEntries(); },250); };
|
||||
// Crossing the phone breakpoint moves the files between their pane and the text.
|
||||
window.matchMedia?.('(max-width:820px)')?.addEventListener?.('change',()=>{ const e=cur(); if(e) showDetail(e); });
|
||||
|
||||
@@ -239,12 +377,35 @@ async function prefsModal(){
|
||||
<span class="hint">Comma separated words or phrases, in every feed you read. An item with
|
||||
one in its title or text is hidden from you and not downloaded for you. Each feed's
|
||||
settings can add more.</span></div>
|
||||
<div class="field"><label for="tokname">API tokens</label>
|
||||
<div id="tokens"></div>
|
||||
<div class="inline"><input type="text" id="tokname" placeholder="What it is for">
|
||||
<button class="btn" id="tokadd" title="Make an API token" aria-label="Make an API token">${ICON.plus} Make</button></div>
|
||||
<span class="hint">A token lets a script or an agent use iPX as you, adding and removing
|
||||
your feeds and reading your items: it sends <code>Authorization: Bearer</code> and the token.
|
||||
Anyone holding one is you here, so revoke one you no longer use.</span></div>
|
||||
<div class="field"><label>Feeds are checked every</label>
|
||||
<span class="hint">${everyText(g.every_mins)}, for every feed that does not set its own.
|
||||
${admin?'This and the rest of the server\'s settings are on the <a href="/admin">admin page</a>.':'Only an admin changes this.'}</span></div>`);
|
||||
$('#stheme').onchange=e=>setTheme(e.target.value,undefined,true);
|
||||
$('#smode').onchange=e=>setTheme(undefined,e.target.value,true);
|
||||
$('#gopml').onclick=opmlModal;
|
||||
drawTokens();
|
||||
$('#tokadd').onclick=async()=>{
|
||||
const name=$('#tokname').value.trim(); if(!name){ $('#tokname').focus(); return; }
|
||||
try{
|
||||
const t=await api('/api/tokens',{method:'POST',body:JSON.stringify({name})});
|
||||
$('#tokname').value='';
|
||||
await drawTokens();
|
||||
// Shown this once: only its hash is kept, so a lost token is revoked and made again.
|
||||
$('#tokens').insertAdjacentHTML('afterbegin',`<div class="field" id="toknew"><span class="hint">Your new token,
|
||||
${esc(t.name)}. Copy it now: it is not shown again.</span>
|
||||
<div class="inline"><input type="text" id="tokval" readonly value="${esc(t.token)}">
|
||||
<button class="btn ico" id="tokcopy" title="Copy" aria-label="Copy">${ICON.copy}</button></div></div>`);
|
||||
$('#tokcopy').onclick=()=>copyText(t.token,$('#tokcopy'));
|
||||
$('#tokval').select();
|
||||
}catch(err){ toast(err.message,true); }
|
||||
};
|
||||
$('#sblock').onchange=async e=>{
|
||||
const blocked=splitWords(e.target.value);
|
||||
try{
|
||||
@@ -255,6 +416,19 @@ async function prefsModal(){
|
||||
};
|
||||
}
|
||||
|
||||
/// Your API tokens, each with when it was made and last used, and a button to revoke it.
|
||||
async function drawTokens(){
|
||||
const box=$('#tokens'); if(!box) return;
|
||||
const list=await api('/api/tokens').catch(()=>[]);
|
||||
box.innerHTML=list.map(t=>`<div class="tokrow"><span><b>${esc(t.name)}</b>
|
||||
<span class="hint">made ${dateOf(t.created)}, last used ${ago(t.last_used)}</span></span>
|
||||
<button class="btn ico" data-tok="${t.id}" title="Revoke ${esc(t.name)}" aria-label="Revoke ${esc(t.name)}">${ICON.trash}</button></div>`).join('');
|
||||
for(const b of $$('[data-tok]',box)) b.onclick=async()=>{
|
||||
try{ await api(`/api/tokens/${b.dataset.tok}`,{method:'DELETE'}); toast('Revoked'); drawTokens(); }
|
||||
catch(err){ toast(err.message,true); }
|
||||
};
|
||||
}
|
||||
|
||||
const splitWords=(s: string)=>s.split(',').map(w=>w.trim()).filter(Boolean);
|
||||
|
||||
function settingsModal(f, newUrl?: string){
|
||||
@@ -307,7 +481,7 @@ function settingsModal(f, newUrl?: string){
|
||||
// Offer the categories the Directory already shows, so a blog about games joins Games rather
|
||||
// than starting a second chip beside it.
|
||||
if($('#scats')) api('/api/directory').then(rows=>{ $('#scats').innerHTML=[...new Set((rows||[])
|
||||
.map(p=>p.category).filter(Boolean))].sort().map(c=>`<option value="${esc(c)}">`).join(''); }).catch(()=>{});
|
||||
.flatMap(p=>[p.category,p.subcategory]).filter(Boolean))].sort().map(c=>`<option value="${esc(c)}">`).join(''); }).catch(()=>{});
|
||||
$('#ssave').onclick=async()=>{
|
||||
const max=$('#smax').value;
|
||||
try{
|
||||
|
||||
@@ -28,10 +28,8 @@ const unreadFirst=(a,b)=>Number(b.unread>0)-Number(a.unread>0);
|
||||
// The original's source list opened with these, above the feeds. They are places, not feeds:
|
||||
// an id starting with ':' can never be a feed's, since feed ids are slugs.
|
||||
const VIEWS={
|
||||
':directory':{title:'Directory',icon:ICON.directory,url:'/api/directory',
|
||||
blurb:'Every feed anyone on this server subscribes to, A to Z. The feeds inside an OPML are listed one by one, not the OPML.'},
|
||||
':popular':{title:'Popular',icon:ICON.popular,url:'/api/popular',
|
||||
blurb:'The ten feeds with the most subscribers here. The feeds inside an OPML count one by one, not the OPML.'},
|
||||
// The Directory holds the most subscribed feeds too, which had a place of their own, Popular (#124).
|
||||
':directory':{title:'Directory',icon:ICON.directory,url:'/api/directory'},
|
||||
':listening':{title:'Currently Listening',icon:ICON.audio,url:'/api/entries?filter=in_progress&limit=50',
|
||||
blurb:'Episodes you started and have not finished, across every feed you subscribe to. Pick one up where you left off.'},
|
||||
':all':{title:'All Subscriptions',icon:ICON.all},
|
||||
@@ -130,8 +128,9 @@ function renderFeeds(){
|
||||
// The mark sits on the artwork, the thing the eye scans the list by, and the line under
|
||||
// the name says what is wrong in place of the counts.
|
||||
`<span class="fart">${mine.length?folderArt(f,mine):artHTML(f.image,f.title||f.id)}`+
|
||||
(err?`<span class="ferr" role="img" title="${esc(err)}" aria-label="Error: ${esc(err)}">${ICON.alert}</span>`:'')+`</span>`+
|
||||
`<div class="txt"><b>${f.pinned?`<span class="fpin" title="Pinned">${ICON.pinOn}</span>`:''}${esc(f.title||f.id)}</b><small>`+
|
||||
(err?`<span class="ferr" role="img" title="${esc(err)}" aria-label="Error: ${esc(err)}">${ICON.alert}</span>`:'')+
|
||||
(f.pinned?`<span class="fpin" role="img" title="Pinned" aria-label="Pinned">${ICON.pinOn}</span>`:'')+`</span>`+
|
||||
`<div class="txt"><b>${esc(f.title||f.id)}</b><small>`+
|
||||
(nbad?`${plural(nbad,'feed')} not updating`
|
||||
:err?esc(f.failing?.reason||'The last check failed')
|
||||
:`${mine.length?plural(mine.length,'feed'):plural(eps,'item')} · ${saved} downloaded`)+
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
/* ---------------- items ---------------- */
|
||||
let loadSeq=0;
|
||||
async function loadEntries(append?: boolean){
|
||||
// Directory and Popular list feeds, not items.
|
||||
// The Directory lists feeds, and Currently Listening draws its own episodes.
|
||||
if(!S.feed || VIEWS[S.feed]?.url) return;
|
||||
const p=new URLSearchParams({offset:String(S.offset),limit:String(LIMIT),filter:S.filter,sort:S.sort.col,dir:S.sort.dir});
|
||||
if(S.q) p.set('q',S.q);
|
||||
@@ -65,13 +65,14 @@ function epEl(e){
|
||||
el.dataset.guid=e.guid;
|
||||
const num=[e.season?`S${e.season}`:'',e.episode?`E${e.episode}`:''].filter(Boolean).join('');
|
||||
const left = e.position>10 && e.duration ? `${clock(e.duration-e.position)} left` : (e.duration?clock(e.duration):'');
|
||||
const name=entryName(e);
|
||||
el.innerHTML=`
|
||||
<button class="st" data-a="read" title="Mark ${e.read?'unread':'read'}">${
|
||||
player.guid===e.guid?EQ:(e.read?'':'●')}</button>
|
||||
<button class="fl${e.flagged?' on':''}" data-a="flag" aria-pressed="${!!e.flagged}" title="${
|
||||
e.flagged?'Unpin':'Pin, so it is never deleted'}">${e.flagged?ICON.pinOn:ICON.pin}</button>
|
||||
<div class="body">
|
||||
<span class="t">${esc(e.title||'(untitled)')}</span>
|
||||
<span class="t${name.derived?' notitle':''}">${esc(name.text)}</span>
|
||||
<div class="line">${[
|
||||
num&&`<span>${num}</span>`,
|
||||
left&&`<span>${left}</span>`,
|
||||
@@ -136,6 +137,28 @@ function kindIcon(enc){
|
||||
function feedArt(id=S.feed){ const f=S.feeds.find(x=>x.id===id); return f&&f.image; }
|
||||
const feedName=id=>{ const f=S.feeds.find(x=>x.id===id); return f?(f.title||f.id):id; };
|
||||
|
||||
/// What an item is called. Its title, or for one published without (RSS 2.0 makes it optional,
|
||||
/// and Scripting News titles almost none of its posts) the opening of its text, then its file's
|
||||
/// name, then its feed and date: "(untitled)" fifty times down a list said nothing about any of
|
||||
/// them. `derived` marks a name that is not a title, which the list sets as text, not heading.
|
||||
/// Feed HTML as plain words. An inert document: nothing in it loads or runs, where a detached
|
||||
/// element fetches its images. A line break or paragraph's end is a space, or "2010)<br>Recorded"
|
||||
/// read "2010)Recorded".
|
||||
const plainText=(html?: string)=>html
|
||||
? (new DOMParser().parseFromString(html.replace(/<(br|\/p|\/div|\/li|\/h\d)\b[^>]*>/gi,' $&'),'text/html').body.textContent||'')
|
||||
.replace(/\s+/g,' ').trim() : '';
|
||||
function entryName(e): {text: string, derived: boolean}{
|
||||
if(e.title&&e.title.trim()) return {text:e.title, derived:false};
|
||||
const words=plainText(e.description);
|
||||
if(words){
|
||||
const cut=words.length>120 ? words.slice(0,120).replace(/\s+\S*$/,'')+'…' : words;
|
||||
return {text:cut, derived:true};
|
||||
}
|
||||
const file=((e.enclosures||[])[0]?.url||'').split(/[?#]/)[0].split('/').pop().replace(/\.[a-z0-9]{1,5}$/i,'');
|
||||
if(file){ try{ return {text:decodeURIComponent(file), derived:true}; }catch{ return {text:file, derived:true}; } }
|
||||
return {text:[feedName(e.feed_id), dateOf(e.published)].filter(Boolean).join(', '), derived:true};
|
||||
}
|
||||
|
||||
/// Selecting an item shows it in the pane below, rather than expanding the row.
|
||||
/// Replaces one row with a fresh one, leaving the rest of the list and its scroll alone.
|
||||
function swapRow(e){
|
||||
@@ -271,7 +294,8 @@ function detailHtml(e){
|
||||
// description was sanitized server-side with ammonia before it ever reached here
|
||||
return `
|
||||
<button class="btn ico" id="dback" title="Back to the items" aria-label="Back to the items">${ICON.left}</button>
|
||||
<h3 class="dt">${esc(e.title||'(untitled)')}</h3>
|
||||
${/* A post without a title starts with its text: its own first words as a heading above
|
||||
themselves would read as a mistake. */ e.title&&e.title.trim() ? `<h3 class="dt">${esc(e.title)}</h3>` : ''}
|
||||
<div class="dmeta">
|
||||
${/* Joined, so a missing date or number leaves no stray dot behind. */
|
||||
[f&&esc(f.title||f.id), num, dateOf(e.published), e.duration&&clock(e.duration)]
|
||||
@@ -328,7 +352,7 @@ function encBox(x){
|
||||
// Nothing on disk. For an image or a PDF you usually just want to look at it, so link
|
||||
// straight to the publisher's copy in a new tab -- no download, and nothing proxied
|
||||
// through here, which would make ipx a fetch-anything relay.
|
||||
const viewable = !isPlayable(x) && x.state !== 'pending';
|
||||
const viewable = !isPlayable(x) && x.state !== 'pending' && x.state !== 'held';
|
||||
return `<div class="encbox">
|
||||
${kindIcon(x)}
|
||||
<span class="meta" style="flex:1">${size}</span>
|
||||
@@ -346,13 +370,13 @@ async function epAction(a: string, e, el, encId?: number){
|
||||
const path=`/api/entries/${encodeURIComponent(e.feed_id)}/${encodeURIComponent(e.guid)}`;
|
||||
try{
|
||||
if(a==='play') play(e, encId!=null ? enc : undefined);
|
||||
if(a==='share') await share(e.title||'', encId!=null ? enc.url : e.link, el);
|
||||
if(a==='share') await share(entryName(e).text, encId!=null ? enc.url : e.link, el);
|
||||
if(a==='flag'){ e.flagged=!e.flagged; await api(path+'/flags',{method:'POST',body:JSON.stringify({flagged:e.flagged})}); redraw(); }
|
||||
if(a==='read'){ await setRead(e,!e.read); redraw(); }
|
||||
if(a==='get'){
|
||||
if(!enc) return;
|
||||
await api(`/api/enclosures/${enc.id}/download`,{method:'POST'});
|
||||
toast('Queued: '+(e.title||'item'));
|
||||
toast('Queued: '+entryName(e).text);
|
||||
}
|
||||
if(a==='del'){
|
||||
const f=S.feeds.find(x=>x.id===e.feed_id);
|
||||
|
||||
@@ -77,7 +77,7 @@ function installNativePlayback(){
|
||||
realLoad.call(audio);
|
||||
const e = player.entry, f = player.feed;
|
||||
post({t:'load', url:v, enc:player.enc, feedId:f, guid:player.guid,
|
||||
title:(e && e.title) || '', feedTitle:feedName(f),
|
||||
title:e ? entryName(e).text : '', feedTitle:feedName(f),
|
||||
artwork:(e && e.image) || feedArt(f) || null,
|
||||
// Where the host starts is not this: the seek to where you left off is player.ts's, on
|
||||
// loadedmetadata, so one piece of code decides it. This is for the host's now-playing
|
||||
|
||||
@@ -32,7 +32,7 @@ function play(e,enc=e.enclosures.find(isPlayable)){
|
||||
if(e.position>5) audio.addEventListener('loadedmetadata',()=>{audio.currentTime=e.position},{once:true});
|
||||
// Initials, if it comes to that, are the feed's: the episode's read as "SE" beside the feed's art.
|
||||
$('#partwrap').innerHTML=artHTML(e.image||feedArt(e.feed_id),feedName(e.feed_id));
|
||||
$('#ptitle').textContent=e.title||'(untitled)';
|
||||
$('#ptitle').textContent=entryName(e).text;
|
||||
const f=S.feeds.find(x=>x.id===e.feed_id);
|
||||
$('#pfeed').textContent=f?(f.title||f.id):'';
|
||||
$('#player').classList.add('on');
|
||||
@@ -45,8 +45,8 @@ function play(e,enc=e.enclosures.find(isPlayable)){
|
||||
function mediaSession(e,f){
|
||||
if(!('mediaSession' in navigator)) return;
|
||||
navigator.mediaSession.metadata=new MediaMetadata({
|
||||
title:e.title||'', artist:f?(f.title||f.id):'', album:f?(f.title||''):'',
|
||||
artwork:(e.image||(f&&f.image))?[{src:e.image||f.image,sizes:'512x512'}]:[],
|
||||
title:entryName(e).text, artist:f?(f.title||f.id):'', album:f?(f.title||''):'',
|
||||
artwork:(e.image||(f&&f.image))?[{src:artSrc(e.image||f.image),sizes:'512x512'}]:[],
|
||||
});
|
||||
const h={play:()=>audio.play(),pause:()=>audio.pause(),
|
||||
seekbackward:()=>audio.currentTime-=15,seekforward:()=>audio.currentTime+=30};
|
||||
@@ -135,7 +135,7 @@ document.addEventListener('keydown',ev=>{
|
||||
// Feedly's keys, vim's j and k among them: a letter to move through items or feeds, g and a
|
||||
// letter to go somewhere, ? to list them. None fire with Ctrl, Alt or Cmd held, so the browser's
|
||||
// own shortcuts still work, or while a dialog is open.
|
||||
const GO={a:':all',d:':directory',p:':popular',l:':listening'};
|
||||
const GO={a:':all',d:':directory',l:':listening'};
|
||||
let gAt=0;
|
||||
function stepEntry(by){
|
||||
if(VIEWS[S.feed]?.url||!S.entries.length) return;
|
||||
@@ -176,10 +176,10 @@ function keysModal(){
|
||||
const k=s=>`<kbd>${esc(s)}</kbd>`, g=c=>k('g')+' '+k(c);
|
||||
const rows=[
|
||||
['Go to'],
|
||||
[g('a'),'All Subscriptions'],[g('d'),'Directory'],[g('p'),'Popular'],
|
||||
[g('a'),'All Subscriptions'],[g('d'),'Directory'],
|
||||
[g('l'),'Currently Listening'],[g('s'),'Settings'],
|
||||
[k('Shift')+' '+k('J'),'Next feed'],[k('Shift')+' '+k('K'),'Previous feed'],
|
||||
[k('/'),'Search items'],[k('r'),'Refresh'],[k('['),'Show or hide the feed list'],
|
||||
[k('/'),'Search items, or the Directory'],[k('r'),'Refresh'],[k('['),'Show or hide the feed list'],
|
||||
['Items'],
|
||||
[k('j')+' or '+k('n'),'Next item'],[k('k')+' or '+k('p'),'Previous item'],
|
||||
[k('Shift')+' '+k('A'),'Mark all read'],
|
||||
|
||||
@@ -30,7 +30,6 @@ const ICON={
|
||||
close:fa('0 0 384 512','<path fill="currentColor" d="M55.1 73.4c-12.5-12.5-32.8-12.5-45.3 0s-12.5 32.8 0 45.3L147.2 256 9.9 393.4c-12.5 12.5-12.5 32.8 0 45.3s32.8 12.5 45.3 0L192.5 301.3 329.9 438.6c12.5 12.5 32.8 12.5 45.3 0s12.5-32.8 0-45.3L237.8 256 375.1 118.6c12.5-12.5 12.5-32.8 0-45.3s-32.8-12.5-45.3 0L192.5 210.7 55.1 73.4z"/>'), // solid/xmark
|
||||
menu:fa('0 0 448 512','<path fill="currentColor" d="M0 96C0 78.3 14.3 64 32 64l384 0c17.7 0 32 14.3 32 32s-14.3 32-32 32L32 128C14.3 128 0 113.7 0 96zM0 256c0-17.7 14.3-32 32-32l384 0c17.7 0 32 14.3 32 32s-14.3 32-32 32L32 288c-17.7 0-32-14.3-32-32zM448 416c0 17.7-14.3 32-32 32L32 448c-17.7 0-32-14.3-32-32s14.3-32 32-32l384 0c17.7 0 32 14.3 32 32z"/>'), // solid/bars
|
||||
directory:fa('0 0 448 512','<path fill="currentColor" d="M0 96C0 60.7 28.7 32 64 32l320 0c35.3 0 64 28.7 64 64l0 320c0 35.3-28.7 64-64 64L64 480c-35.3 0-64-28.7-64-64L0 96zm64 0l0 64 64 0 0-64-64 0zm320 0l-192 0 0 64 192 0 0-64zM64 224l0 64 64 0 0-64-64 0zm320 0l-192 0 0 64 192 0 0-64zM64 352l0 64 64 0 0-64-64 0zm320 0l-192 0 0 64 192 0 0-64z"/>'), // solid/table-list
|
||||
popular:fa('0 0 576 512','<path fill="currentColor" d="M309.5-18.9c-4.1-8-12.4-13.1-21.4-13.1s-17.3 5.1-21.4 13.1L193.1 125.3 33.2 150.7c-8.9 1.4-16.3 7.7-19.1 16.3s-.5 18 5.8 24.4l114.4 114.5-25.2 159.9c-1.4 8.9 2.3 17.9 9.6 23.2s16.9 6.1 25 2L288.1 417.6 432.4 491c8 4.1 17.7 3.3 25-2s11-14.2 9.6-23.2L441.7 305.9 556.1 191.4c6.4-6.4 8.6-15.8 5.8-24.4s-10.1-14.9-19.1-16.3L383 125.3 309.5-18.9z"/>'), // solid/star
|
||||
all:fa('0 0 512 512','<path fill="currentColor" d="M232.5 5.2c14.9-6.9 32.1-6.9 47 0l218.6 101c8.5 3.9 13.9 12.4 13.9 21.8s-5.4 17.9-13.9 21.8l-218.6 101c-14.9 6.9-32.1 6.9-47 0L13.9 149.8C5.4 145.8 0 137.3 0 128s5.4-17.9 13.9-21.8L232.5 5.2zM48.1 218.4l164.3 75.9c27.7 12.8 59.6 12.8 87.3 0l164.3-75.9 34.1 15.8c8.5 3.9 13.9 12.4 13.9 21.8s-5.4 17.9-13.9 21.8l-218.6 101c-14.9 6.9-32.1 6.9-47 0L13.9 277.8C5.4 273.8 0 265.3 0 256s5.4-17.9 13.9-21.8l34.1-15.8zM13.9 362.2l34.1-15.8 164.3 75.9c27.7 12.8 59.6 12.8 87.3 0l164.3-75.9 34.1 15.8c8.5 3.9 13.9 12.4 13.9 21.8s-5.4 17.9-13.9 21.8l-218.6 101c-14.9 6.9-32.1 6.9-47 0L13.9 405.8C5.4 401.8 0 393.3 0 384s5.4-17.9 13.9-21.8z"/>'), // solid/layer-group
|
||||
unread:fa('0 0 512 512','<path fill="currentColor" d="M48 64c-26.5 0-48 21.5-48 48 0 15.1 7.1 29.3 19.2 38.4l208 156c17.1 12.8 40.5 12.8 57.6 0l208-156c12.1-9.1 19.2-23.3 19.2-38.4 0-26.5-21.5-48-48-48L48 64zM0 196L0 384c0 35.3 28.7 64 64 64l384 0c35.3 0 64-28.7 64-64l0-188-198.4 148.8c-34.1 25.6-81.1 25.6-115.2 0L0 196z"/>'), // solid/envelope: a closed letter, not a record button
|
||||
audio:fa('0 0 448 512','<path fill="currentColor" d="M64 224c0-88.4 71.6-160 160-160s160 71.6 160 160l0 37.5c-10-3.5-20.8-5.5-32-5.5l-16 0c-26.5 0-48 21.5-48 48l0 128c0 26.5 21.5 48 48 48l16 0c53 0 96-43 96-96l0-160C448 100.3 347.7 0 224 0S0 100.3 0 224L0 384c0 53 43 96 96 96l16 0c26.5 0 48-21.5 48-48l0-128c0-26.5-21.5-48-48-48l-16 0c-11.2 0-22 1.9-32 5.5L64 224z"/>'), // solid/headphones
|
||||
@@ -145,10 +144,11 @@ const tint=name=>{ let h=0; for(const c of name||'?') h=(h*31+c.charCodeAt(0))>>
|
||||
function tileHTML(name,cls){
|
||||
return `<div class="art ini ${cls||''}" style="--tint:${tint(name)}">${esc(initials(name))}</div>`;
|
||||
}
|
||||
/// On the https page, the browser upgrades an http:// image to https, and a host that has no
|
||||
/// https shows nothing (issue #90); ipx fetches those itself.
|
||||
/// Every image a feed names comes from iPX, which keeps a copy: fast after the first time, and
|
||||
/// no publisher's server asked on every visit. It began with http-only artwork, which the https
|
||||
/// page could not load (#90).
|
||||
function artSrc(url: string){
|
||||
return location.protocol==='https:'&&/^http:\/\//i.test(url) ? '/api/art?u='+encodeURIComponent(url) : url;
|
||||
return /^https?:\/\//i.test(url) ? '/api/art?u='+encodeURIComponent(url) : url;
|
||||
}
|
||||
function artHTML(url: string | null, name: string, cls?: string){
|
||||
return url
|
||||
|
||||
Reference in New Issue
Block a user