Migrations/bookorbit
1
0
Fork 0
mirror of https://github.com/bookorbit/bookorbit.git synced 2026-11-03 00:11:37 -06:00
Code Issues Packages Projects Releases Wiki Activity

feat: add read-aloud and media playback support

Browse Source 
Add EPUB narration, shared media controls, and native device session integration.

Apply the generated migrations and update all first-party clients alongside the server.

BREAKING CHANGE: introduce incompatible database and API contracts for media playback
This commit is contained in:
neon
2026-09-19 02:02:24 +00:00
parent c292d6ccd2
commit e0d3d42150
844 changed files with 151839 additions and 3415 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/BOOK_REQUESTS.md
-215
View File
@@ -1,215 +0,0 @@
# Book Requests
Book requests let readers ask for a book BookOrbit does not have. An approver decides on the
request, a release is picked from the sources you configured, a download client fetches it, and the
result is filed into a library.
Everything in this guide is operator configuration. Nothing here is on by default, and a fresh
install has no sources, no download clients and no automation.
## What BookOrbit ships
**BookOrbit ships no sources.** There is no bundled tracker, no default indexer, and no list of
sites to pick from. Two protocol adapters are compiled in: `torznab` for torrent indexers and
`newznab` for Usenet indexers. Both need an address and API key you supply. Everything else is an
indexer plugin you install yourself.
Which sources you point BookOrbit at, and what you search for and download through them, is your
decision and your responsibility. BookOrbit does not provide or endorse indexer sources. Its URL,
file, and path safety checks still apply to every configured source.
## Permissions
| Permission | What it allows |
| --------------------------- | ----------------------------------------------------------- |
| `book_request_access` | File a request, see your own, join somebody else's |
| `manage_book_requests` | Approve, reject, and drive fulfilment on every request |
| `book_request_auto_approve` | Your own requests skip the approval queue |
| `book_request_self_fulfill` | Pick the release and download it yourself, with no approver |
The last three each imply `book_request_access`; granting one on its own would produce an account
that can approve requests but has no page to see them on.
Configuring sources, download clients and automation is separate again, and needs
`manage_app_settings`. Installing an indexer plugin needs a superuser: a plugin runs inside the
server process with the server's reach.
## Before anything can be saved: the encryption key
Set `BOOK_REQUEST_ENCRYPTION_KEY` before configuring a download client or an indexer:
```bash
openssl rand -hex 32
```
It must be a 64-character hex string. Unlike `MIGRATION_ENCRYPTION_KEY` and the email key, this one
is not optional in practice: a tracker API key or seedbox password is **refused rather than stored
in the clear** when the key is missing, so the settings form will not save.
Rotating or removing the key leaves stored credentials unreadable. They are not silently dropped:
the affected source refuses to be used and says why, so you can re-enter the credential rather than
discover the loss during a download.
## Sources
Settings > System > Requests > Sources.
Three kinds of row, added two ways, identical once they exist:
- **Torznab**: an address plus an API key. This is how you attach a Prowlarr or Jackett instance,
and one such instance can expose many trackers behind one row per tracker.
- **Newznab**: an address plus an API key. BookOrbit uses generic searches, including an ISBN as
the query when enabled, and fetches the selected NZB without exposing the key to the download
client.
- **Plugins**: a single `.mjs` file installed through the page, loaded at boot from
`<APP_DATA_PATH>/plugins/indexers/<name>/index.mjs`. Plugins live in their own repositories and
are not distributed with BookOrbit.
A plugin may declare a signed update manifest. BookOrbit checks each declared channel at most once
every 12 hours, supports a review-before-install flow, and can install verified updates
automatically when a superuser enables that option for the plugin. The downloaded source must match
both the manifest's SHA-256 and its Ed25519 signature. Installs are atomic, activate without a
restart, and restore the previous plugin if activation fails.
Per source you can set the medium it is searched for, its categories, whether it is handed an ISBN
when the request has one, and a colour that marks its releases in the release picker.
Private network addresses are refused unless you opt the row in with **Allow private address**. A
self-hosted Prowlarr on the LAN is the common reason to; a public tracker resolving to a private
address is not.
Two health facts are shown per row, and they answer different questions:
- **Connected / Not reachable** is the last time you pressed **Test**, which makes a capabilities
call.
- **Searches failing** is how the last real searches went. A source can be reachable and still
refuse every search, and the badge counts consecutive failures so one blip reads differently
from a source that has been broken for a week.
### Per-indexer seed goals
Torrent sources can set a manual seed ratio and total seed time. Each manual field is independent.
Leaving one blank means inheritance; zero and negative values are not special modes. **Apply tracker
seed goals** controls only whether a valid goal reported by Torznab or a torrent plugin fills a
blank manual field. Turning it off does not disable manual values and does not request unlimited
seeding.
For ratio, BookOrbit uses the manual source value, then the tracker value when tracker fallback is
enabled, then omits the value so the download client uses its own default. Seed time follows the
same order: manual source value, tracker value when tracker fallback is enabled, then the download
client's default.
The policy is read when a new grab attempt starts, so a saved edit applies even when BookOrbit is
reusing recently inspected torrent bytes. It applies only to releases selected from that configured
source. A pasted magnet or uploaded torrent file has no indexer policy. Changes do not reconfigure
torrents already in a client.
## Download clients
Settings > System > Requests > Download clients. qBittorrent, Transmission and Deluge accept
torrents; NZBGet accepts NZBs. A Newznab source therefore needs an enabled NZBGet client, while a
Torznab source needs an enabled torrent client. Direct HTTP sources need no external client.
BookOrbit does not connect to an NNTP article server. Configure the article-server account in
NZBGet, then give BookOrbit a separate NZBGet API account. BookOrbit fetches the credentialed NZB,
sends its bytes to NZBGet, and monitors the item by a stable ownership key rather than a transient
NZBGet queue ID.
Each client needs at least one **path mapping**, including when BookOrbit and the client run on the
same host. A mapping translates the download directory the client reports into a path BookOrbit can
open, and it is also the root the import is allowed to read out of. On a single host that mapping
is an identity, `/downloads -> /downloads`; it still has to be stated, because "the same path" and
"no path stated" are different things and only one of them is safe to import from.
Hardlinking from the download directory into the library avoids storing the imported bytes twice
and keeps a torrent seeding after the book is filed. The mapping form can test whether a hardlink
actually works between the two paths, which is worth doing before the first grab: a mapping across
filesystems copies instead, silently doubling the space every book takes.
The same mount is necessary but is not sufficient. On Linux, `fs.protected_hardlinks=1` normally
requires the BookOrbit process user to own the completed source file or be able to write it. The
download client and BookOrbit commonly run with the same UID, or with a shared group that grants
BookOrbit write access to completed files. The mapping test creates its own BookOrbit-owned source
file, so it proves that the paths and mount permit hardlinks but cannot prove that a completed file
owned by a different user will be linkable. If the real file cannot be linked, BookOrbit falls back
to copying it and the book uses space twice while the original keeps seeding.
BookOrbit never stops a seed on its own. Removing a client-managed download is an explicit action
with its own confirmation, and removing one that is still working also fails the request.
qBittorrent accepts a per-torrent ratio and total seeding-time limit through fields documented
since Web API 2.8.1. When both exist, its own stop rules can act when either limit is reached; the
pair is not an AND condition and does not guarantee both tracker minimums. Deluge accepts the ratio
with stop-at-ratio enabled and removal disabled.
Transmission accepts the ratio in a best-effort call after adding the torrent; its idle-time option
is not used as total seed time. A failure in that Transmission policy update leaves the tracked
torrent in place and is logged as a warning.
Client-wide ratio, time, idle, and completion settings still apply. qBittorrent and Deluge can
adopt a duplicate torrent without replacing its existing limits; Transmission attempts to apply
an explicit ratio to an adopted torrent. BookOrbit does not delete and re-add an adopted torrent,
reconcile limits across clients, or run a background seed stopper.
## Automation
Settings > System > Requests > Automation. Everything here is off or unset by default.
- **Auto-grab** picks a release without an approver, but only above a minimum score. Below that the
request waits, and the approver still sees the full ranked list.
- **Auto-retry** falls back to the next-best release when an automatic grab fails, bounded by a
maximum number of attempts per request.
- **Auto-search** re-searches an approved request nothing was found for. Without it a request is
searched exactly once, and a book whose first release is posted next month waits forever. The
interval doubles for each week a request has been waiting, up to a cap, and stops entirely after
a configurable age.
- **Import checking** scores what landed against what was asked for. Below the threshold the file
waits in the Book Dock and the request reads "needs review", where a person either files it
anyway or discards it. Switching this off files whatever was downloaded straight into the target
library.
- **Import formats** decides what to keep when one release carries the same book several times
over: every format, or only the target library's preferred one. A multipart audiobook is never
affected, since its parts are one book rather than competing editions.
- **Default destinations** are per medium, and are the lowest rung of the ladder: a requester's own
pinned destination and an explicit choice at request or approval time both win over them. Without
one, a request that names no library can never be approved.
- **Release profiles** are an ordered tier list per medium describing the edition you want. Empty
is the ordinary state and disengages the tier axis entirely, leaving auto-grab decided by score.
## The lifecycle
```
pending -> approved -> searching -> grabbed -> downloading -> importing -> available
```
- **pending**: waiting on an approver. Auto-approval and self-serve both skip it.
- **needs_review**: the import scored below the threshold and is sitting in the Book Dock. Two ways
out: file it anyway, or discard it, which removes the dock entry and fails the request.
- **failed**: nothing could be downloaded, or a person discarded the import. Still grabbable, so a
different release can be tried.
- **rejected** / **cancelled**: settled by a person. A settled work can be requested again.
One live request per work. A second person asking for the same book is attached to the existing
request as a subscriber rather than opening a second one, and can leave it again from the request's
own menu.
## Troubleshooting
**"No sources are enabled"** on the requests page: either nothing is configured, or every row is
switched off. The notice says which.
**Requests sit at `approved` and nothing happens**: auto-grab is off, which is the default. Either
an approver opens the release picker, or auto-grab is switched on with a minimum score.
**A grab is refused with a path error**: the client has no path mapping, or the download directory
resolved outside the one it has. Add or widen the mapping under Settings > System > Requests.
**Credentials will not save**: `BOOK_REQUEST_ENCRYPTION_KEY` is unset or is not 64 hex characters.
**A source shows "Searches failing" but tests fine**: the address answers a capabilities call and
refuses searches. Usually a rate limit, an expired credential, or categories that do not exist on
that indexer.
## Related
- [Development guide](DEVELOPMENT.md)
- [Testing guide](TESTING.md)
docs/DEVELOPMENT.md
+13 -12
View File
@@ -224,18 +224,19 @@ pnpm --filter @bookorbit/types dev
These are the variables you are most likely to need during development. See `server/.env.example` for the full list with comments.
| Variable | Purpose | Default (dev) |
| --------------------- | ------------------------------------ | --------------------------------------------------------- |
| `DATABASE_URL` | PostgreSQL connection string | `postgres://bookorbit:bookorbit@localhost:5432/bookorbit` |
| `PORT` | Server listen port | `3000` |
| `HOST` | Server listen address (IPv4 or IPv6) | `0.0.0.0` |
| `NODE_ENV` | Runtime mode | `development` |
| `JWT_SECRET` | Signing key for auth tokens | `change-me-in-production` |
| `APP_DATA_PATH` | Storage for covers, avatars, cache | `../local/data` |
| `LIBRARY_BROWSE_ROOT` | Library folder picker root | `/` |
| `APP_URL` | Base URL for email links | `http://localhost:5173` |
| `SWAGGER_ENABLED` | Serve Swagger UI and OpenAPI JSON | `false` |
| `DISABLE_LOCAL_AUTH` | Disable password authentication | `false` |
| Variable | Purpose | Default (dev) |
| ------------------------ | -------------------------------------------------- | --------------------------------------------------------- |
| `DATABASE_URL` | PostgreSQL connection string | `postgres://bookorbit:bookorbit@localhost:5432/bookorbit` |
| `PORT` | Server listen port | `3000` |
| `HOST` | Server listen address (IPv4 or IPv6) | `0.0.0.0` |
| `NODE_ENV` | Runtime mode | `development` |
| `JWT_SECRET` | Signing key for auth tokens | `change-me-in-production` |
| `PODCAST_ENCRYPTION_KEY` | Podcast URL encryption key; required in production | Falls back to `JWT_SECRET` in development |
| `APP_DATA_PATH` | Storage for covers, avatars, cache | `../local/data` |
| `LIBRARY_BROWSE_ROOT` | Library folder picker root | `/` |
| `APP_URL` | Base URL for email links | `http://localhost:5173` |
| `SWAGGER_ENABLED` | Serve Swagger UI and OpenAPI JSON | `false` |
| `DISABLE_LOCAL_AUTH` | Disable password authentication | `false` |
For a native server accessible only on this machine, set `HOST=127.0.0.1` in `server/.env`. `HOST` accepts an IPv4 or IPv6 address without a port or brackets, such as `::1` for IPv6 loopback. An unset or blank value keeps the existing `0.0.0.0` default. `PORT` continues to select the listen port.
docs/whats-new-authoring-guide.md
-128
View File
@@ -1,128 +0,0 @@
# Authoring the "What's New" highlights
This is the end-to-end guide for writing the `## Highlights` section of a GitHub release so it
powers BookOrbit's in-app **What's New** popup and the `/whats-new` archive. It is the single
source of truth for both the workflow and the format.
## How it shows up in the app
- **Popup:** when a user opens the app on a newer version than they last saw, a popup shows the
highlights for the versions they missed (newest expanded, up to 4 versions cumulatively).
- **Archive:** the `/whats-new` page lists every release (highlights + full changelog).
- **Caching:** the server fetches and caches releases for about 10 minutes, so edits appear
within a few minutes (immediately on a server restart). No redeploy is needed.
- **No highlights, no popup:** a release with no `## Highlights` section (or the scaffold left in
place with no bullets) simply shows no popup. Its changelog still appears in the archive. That
is fine for patch releases with nothing user-facing to call out.
## Walkthrough (every release)
1. **Cut the release.** Run the release workflow (Actions -> Release -> Run workflow on `main`).
semantic-release creates the GitHub release as a **draft** and the Docker images
(`:latest` and `:<version>`) are built and pushed. The draft is not visible in the app yet.
2. **Open the draft to edit.** Go to the repo's Releases, click the draft (or open
`https://github.com/bookorbit/bookorbit/releases/edit/<tag>`), and click the edit (pencil) icon.
3. **Review the candidate highlights.** The `## Highlights` scaffold contains a commented list of
this release's `feat` commits as starting candidates. Curate them: keep the user-facing ones,
reword them into friendly benefit-focused lines, and delete the rest.
4. **Write the bullets** above the comment block, one per highlight (see the format below).
5. **Add an icon** to a bullet by ending the line with `<!-- icon: Name -->` (optional).
6. **Add media** by dragging an image or video into the editor (see the media rules below).
7. **Delete the scaffold comment block** once you are done with it.
8. **Validate:** from `server/`, run `pnpm whats-new:check <tag>` (lints, then previews exactly
what the app will render and checks media reachability). Fix any warnings.
9. **Publish.** Click **Publish release**. Within ~10 minutes the popup appears for users on this
version. (Until you publish, the draft stays hidden in the app even though `:latest` is already
live, so do not leave it unpublished.)
## Copy-paste starter
```markdown
## Highlights
- **Two-way Kobo highlight sync** - Your highlights now sync both ways with Kobo, so a note on the device shows up on the web and vice versa. <!-- icon: BookHeart -->
- **Instant library scroll** - The alphabet jump rail is now instant on libraries with thousands of books. <!-- icon: Zap -->
- **Audiobook chapters** - Chapter markers are extracted and shown in the player; jump straight to any chapter. <!-- icon: Headphones -->
<img width="800" alt="chapters" src="https://github.com/user-attachments/assets/REPLACE-WITH-DROPPED-IMAGE" />
- **Smaller fixes** - A batch of polish across the reader and library grid.
```
## Format reference
**Heading**
- The section heading must be exactly `## Highlights`.
- Everything after the next heading (`##` / `###`) or a `---` rule is treated as the technical
changelog, not a highlight.
**Bullets**
- One highlight per line, starting with `- `.
- Shape: `- **Title** - one or two friendly sentences.`
- `**Title**` is the bold headline; the text after the `-` is the description.
**Icons** (optional)
- End the bullet with an HTML comment GitHub renders as nothing: `<!-- icon: Name -->`. It can
also sit on its own indented line right below the bullet.
- `Name` is a PascalCase [lucide](https://lucide.dev/icons) icon name (e.g. `BookHeart`, `Zap`,
`ShieldCheck`, `Headphones`, `Film`). Omit the comment for the default icon (`Sparkles`).
- Unknown names fall back to the default; the lint warns so you can fix a typo.
- The old `- [Icon] **Title** ...` prefix still parses but renders as literal `[Icon]` text on the
GitHub page, so the lint warns against it. Use the comment form.
**Media** (optional)
- Drag an image or video into the GitHub release editor. GitHub uploads it and inserts either an
`<img ... src="https://github.com/user-attachments/assets/...">` tag (images) or a bare
`https://github.com/user-attachments/assets/...` URL (videos). Leave whatever it inserts as-is.
- Only **GitHub-hosted** media renders: `github.com/user-attachments/...` or any
`*.githubusercontent.com` host. Media from other hosts (imgur, etc.) is dropped (the text is
kept) and the lint warns.
- Image vs video is detected by file extension (`.mp4`, `.webm`, `.mov`, `.m4v`, `.ogv`, `.ogg`
are video); GitHub's extension-less attachment URLs are probed for their content type.
- You can attach **several** media to one bullet (put each on its own line under it). They render
as a horizontal gallery; clicking an image opens a carousel. At most **6** media per highlight
render; extras are dropped (the lint warns).
**Soft limits** (the app clamps and the lint warns; nothing breaks)
- Title up to **120** characters, body up to **400** characters (visually line-clamped in the
popup; the full text stays in the archive and search).
- Up to **12** highlights per release.
## Validate before publishing
Run from `server/`:
- `pnpm whats-new:check <tag>` - the one-stop check: lints, then previews exactly what the app
renders (including a media reachability probe). Exits non-zero if the lint fails.
- `pnpm whats-new:lint <file.md>` / `pnpm whats-new:lint --tag <tag>` - static validation only.
Add `--strict` to treat warnings as failures.
- `pnpm whats-new:preview [tag]` - render preview only.
Point these at a different repo for testing with the `GITHUB_RELEASES_REPO` env var (and
`GITHUB_RELEASES_TOKEN` for private repos / rate limits), or pass `--repo owner/name` to preview.
## Pre-publish checklist
- [ ] `pnpm whats-new:check <tag>` is clean (no errors; warnings addressed).
- [ ] Each bullet is `- **Title** - benefit.`, short and user-facing.
- [ ] Icons are valid PascalCase lucide names (no typos), or omitted for the default.
- [ ] Media was dragged in (GitHub-hosted), no more than 6 per highlight.
- [ ] Titles <= 120 chars, bodies <= 400 chars.
- [ ] The scaffold candidate/instruction comment block is deleted.
- [ ] Clicked **Publish release** (the draft is not enough).
## Common mistakes
| Symptom | Cause | Fix |
| ----------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------ |
| Highlight shows the default Sparkles icon | Icon name typo / not a lucide name | Correct the PascalCase name (lint says which) |
| `[Icon]` text shows on the GitHub page | Legacy `[Icon]` prefix | Move the icon into a `<!-- icon: Name -->` comment |
| Image/video does not appear | Media not GitHub-hosted | Drag the file into the editor (do not paste an external URL) |
| Only 6 of N images show | Over the per-highlight media cap | Keep the best 6 |
| Title/body looks cut off in the popup | Over the soft length cap | Shorten it (full text stays in the archive) |
| No popup for the release | No `## Highlights` bullets, or release left as a draft | Add bullets and click Publish |
docs/whats-new-template.md
-8
View File
@@ -1,8 +0,0 @@
# Moved
The "What's New" highlights format and template now live in the consolidated guide:
**[docs/whats-new-authoring-guide.md](whats-new-authoring-guide.md)**
It covers the end-to-end workflow, a copy-paste starter, the full format reference (icons,
media, length caps), validation commands, and a pre-publish checklist.