Normalize legacy float values before migration 0076 so 2.7 databases can upgrade without constraint failures or precision loss. Serialize the compatibility conversion across concurrent startup processes.
Fixes#1189
Add an isPublic flag to collections allowing users to share collections across the instance. Public collections are readable by all authenticated users while write, edit, reorder, and delete operations remain restricted to the owner.
Add fileSize as a filterable rule field resolved against the book's primary file size in bytes. In the filter builder UI, allow operators to input sizes in megabytes with automatic serialization to bytes for queries.
Sanitize match-check batches before strict server validation and classify HTTP failures accurately. Rebind the active catalog client whenever authentication settings change so retries use current credentials.
Fixes#1150
Redesign the bulk rename UI with change-type grouping, inline character diffs, and tree previews. Add two-mode selection for large libraries, virtualize the queue, and move search to the server. Optimize execution by pausing the library watcher instead of tearing it down, and improve SSE progress reporting and run error handling.
Share one path-resolution routine between the preview and the rename so the preview cannot offer a rename the executor refuses: multi-track audiobooks whose parts collapse onto one filename, and renumbering that would overwrite the book's own files.
The series, author and similar endpoints never selected a read status, so
every cover in the detail shelf rendered bare. Attach the requesting user's
status in one bulk lookup per lane, and draw the badge in BookCarousel using
the same icons and placement as the library grid, plus an sr-only label.
BookRepository has no user in scope, so it now returns UnscopedBookRecommendation
and the type system forces the status on where the user exists.
Also adds the untracked scanner overlap repro for issue 1094.
Closes#1163
Saving metadata writes each audio file through a .bookorbit-write-* temp
file, which the watcher classified as content and turned into a folder
rescan of the book mid-write. The rescan read pre-write tags back over the
saved values, and an empty or failed audio read was persisted as nulls.
Closes#1164
Any write to book_metadata bumped updated_at and minted a fresh
CoverImageId, so a device holding that cover locally fell back to the
placeholder offline. Stamp cover_updated_at at each cover write instead;
migration 0080 backfills it from updated_at so no id churns on upgrade.
Also trims the DECIMAL(10,2) scale Booklore hands back for a series
index, which sorted 2.50 after 2.9. Closes#943.
Token values are sanitized before modifiers run, so {authors:sort} rewrote
"J.K. Rowling" into a folder named "Rowling, J.K." that Windows and SMB reject,
serving a mangled 8.3 alias instead. Normalize each segment after assembly rather
than reusing sanitizePathSegment, whose reserved-name guard re-fires on a second
pass and would turn "NUL.txt_" into "NUL.txt__".
Closes#1160
Decode added dates and manual reading dates with an explicit parser instead
of new Date(), which reads a zero-padded year below 100 as a two-digit year:
years 0013 to 0031 became an Invalid Date and made every listing holding the
book answer 500, and years 0001 to 0099 shifted silently into 19xx or 20xx.
The column type is unchanged on the wire, so no migration is needed.
Closes#1143
Imports resolved the naming pattern with the download resolver, which keeps only the last
path segment, so the shipped default's author and series folders were dropped and every file
landed at the library root. Rename, bulk rename and move all kept them, leaving the app
disagreeing with itself about where its own import belonged.
Closes#1157
Both views are toggled from the header and read from the same ladder: one
segment per volume the series should have, coloured read, reading, in library
or missing. Gaps were previously only on the series detail page, so the list
could never say a volume was absent; a Gaps filter now decides that in SQL,
mirroring the existing gap finder branch for branch so the two surfaces can
never disagree.
The card's fanned covers are laid out from cached percentages against an
aspect-ratio box, so nothing is measured at runtime and hover spreads the fan
through one custom property instead of per-cover reactive state.
See docs/COMMIT_GUIDELINES.md for the API additions to SeriesSummary.
The two layouts are the viewMode the app already persists, so search, filters
and sort carry across them. Only 19% of a library's authors have a portrait,
so the gallery leans on monograms and the index treats the name as the content;
a sort name now prints only when it says something the display name does not.
Jump buckets, a cover fallback and two filters come from the server so the A-Z
rail and the chips cover the whole result set rather than the loaded pages.
Reconciles authors.has_photo against the image store on boot: the list resolves
portraits from disk while the filter queried the column, so "no portrait" was
over-reporting by 64.
Both pages now share the stream, entry and detail components, so a highlight
reads the same wherever you meet it. Entries split three ways - date rail,
quote at a real measure, source margin - and the library hub gains an overview
endpoint so its side rail does not recompute six aggregates per scroll.
Notes and anchor-review are server filters now; filtering the loaded window
made the counts and the pager disagree with each other.
Both tabs fetch on mount and were rebuilt on every switch, so each visit
replayed blank, skeleton and content inside 120ms. Keep them alive and
revalidate silently instead, hold the layout until the first load decides
which shape it settles into, and defer skeletons past 220ms.
Also fixes the unhandled rejection failing the server suite in CI: the
Goodreads bot-challenge test rejected before its assertion was attached.
Goodreads caps autocomplete blurbs near 180 characters and flags the cut,
but the mapper kept them, so a detail page it could not reach replaced a
full description with an ellipsised fragment. Drop a flagged snippet, retry
a transient 503 within a search budget, and raise a provider cooldown on a
bot challenge instead of degrading the whole batch.
Replace the paginated card list with a three-column ledger: a marks rail, a
grouped stream that loads on scroll, an index that regroups by chapter, colour
or day, and a chapter position band. Add chapterBreakdown and activity to the
annotation stats so those panels describe the whole book without loading every
row, and move the metadata score colour onto a shared oklch ramp.
Seed the random sort field from a per-view randomSeed on BookQuery rather
than a day counter, so a reload or a scope change draws a new order while
every page of one listing stays coherent. Callers that omit the seed keep
the daily fallback. Adds a shuffle button to the library, smart scope, and
collection toolbars when the active sort is random.
Closes#1153
Vitals, the session ledger and attempts/records fill the pane height, with
the progress band across the row beneath, matching the details tab.
- attempt boundaries and reading stalls are rows inside the ledger, in date
order only, so sorting by length or change still works
- the band toggles between the progress curve and a trace of the stretch of
the book each session covered
- one empty state for a book with no log, in place of four empty panels
- drops the twelve-month heatmap, which was 357 empty cells on a real book
Stats gain longestSessionSeconds, longestSessionAt and backtrackCount, and
sessions gain attemptId; the client cannot derive either from one page.
Extraction reads one file, and each file of a multi-file audiobook embeds
its own chapters starting at zero, so later files had no chapters and the
first file's last chapter ran to the end of the book. Scanning now probes
every audio file in playback order and offsets each file's chapters by the
files before it, rebuilding books whose stored list stops short.
Closes#1146
Current firmware never posts the LeaveContent analytics events the reading
log was fed from, so every Kobo statistic read zero. Derive sessions from the
SpentReadingMinutes counter the device already sends on each state push,
tracked per device in reading_session_sync_cursors so two readers of one book
cannot inflate each other. Analytics sessions still win and retire any
estimate they overlap.
Closes#1058Closes#741
Multiple audiobook editions of one title resolved to the same target path, so
enabling file rename merged or skipped every narrator but the first. Add
{narrators}, plus a :max3 modifier that yields nothing above three values, and
judge an optional group by what its placeholders render so the fallback literal
stays in the user's own pattern. The five duplicate token builders collapse into
one shared helper, which also gives audio uploads the metadata tokens they never
had. Build the shared types package before server lint, since type-aware rules
degrade to unknown on a clean checkout without it.
Refs #1108
<Volume> is xs:int, but it received the ComicVine volume name, which is usually a title. Strict
readers abort deserialization on the type error and discard every field, so metadata that is
present in the archive becomes unreadable. Write the element only when the name parses as an
integer, and emit elements in the schema's xs:sequence so the document validates.
Closes#1064
Choose Off, Problems or All per category instead of a single on/off switch, and
stop announcing scheduled scans that changed nothing. Repeat occurrences now
collapse into one row with a count, so a bulk run no longer emits one
notification per book. Category and severity move into a shared registry that
the client reads instead of keeping its own copies.
Closes#1141
* feat(stats): count KOReader sync reading toward streaks and statistics
- Estimate a reading session from the interval between two advancing kosync pushes by the same device, so a plain KOReader install with no BookOrbit plugin still reaches the reading log, the streak and every daily statistic
- Bound each estimate with a one minute floor and a thirty minute ceiling, so a page turn is not a sitting and an overnight silence is not recorded as a long read
- Derive the estimate id from device, book file and interval start, so two pushes racing for the same file resolve to one row instead of counting the same reading twice
- Measure both the interval and the distance read against the same device's own previous row, so a second device syncing the book in between cannot lend it either one
- Skip estimating for a device that has swept in the last thirty days, since its plugin uploads real page timings for the same reading, and let the exclusion lapse so a removed or broken plugin does not retire the device from every statistic
- Retire estimates once a sweep measures the same reading, matched by an id prefix so the lookup stays bounded and indexed
- Rebuild a reader's daily stats when their timezone is submitted, so a corrected day boundary applies to history rather than only to the next couple of days
- Backfill daily stats once on upgrade for every reader with history, marked done only after a clean pass so a partial repair retries
- Keep a failed estimate or rebuild from taking down the progress push or the settings save that triggered it
Refs #261
* fix(stats): correct estimate logging, lock order, and settings latency
- Read the clock before handing the backfill pass to catch, so an outright failure reports its real elapsed time rather than durationMs=0
- Sort the hourly recompute's groups by user then library, so it and a concurrent per-user rebuild take the shared advisory locks in one order instead of deadlocking a reader who holds two libraries
- Detach the timezone rebuild from the settings write, which never returns its result and would otherwise hold the response open for a full-history walk
- Add durationMs to the koreader.sync_session end and fail logs
- Add durationMs to the sweep cleanup failure log
* fix(kobo): forward x-kobo-userkey and decode the composite sync token in the catch-all proxy
The catch-all proxy in kobo-device.controller.ts forwards device requests to
storeapi.kobo.com through KoboProxyService, which built its outgoing headers from
a static allow-list (FORWARD_HEADERS). The list covered accept, accept-language,
authorization, content-type, user-agent, and a curated set of x-kobo-* headers,
but two of the headers that matter for store-API authentication and cursor
tracking were missing: x-kobo-userkey, the device's per-account secret that
authorizes most Kobo store endpoints, and x-kobo-synctoken, the upstream cursor
the device carries on every store call.
Without x-kobo-userkey, every store interaction proxied through the catch-all
shipped without credentials, so Kobo either rejected the call or treated the
device as unauthenticated. The "Connecting to OverDrive" dialog that issue #738
describes is the borrow endpoint, /v1/library/borrow, hanging because Kobo
cannot reconcile a borrow into a user's library without the user key. The same
gap also left library books a user borrowed on libbyapp.com (which is just
OverDrive pushed to kobo.com) unreachable from the device, because every
proxy-bound sync the device makes lands at Kobo without the credentials it
needs to enumerate the user's entitlements.
The Kobo Plus / Kobo Store entitlement fix in 2.6.0 (closes#789) only patched
the one explicit library/sync call. That call already decodes the BookOrbit
composite sync token via kobo-sync-token.ts and overrides the outgoing header
with the Kobo cursor Kobo understands, so the store page works. Every other
store call still went through the catch-all proxy with the device's
PX.<base64> composite on it, which Kobo treats as a fresh cursor and answers
for. The borrow endpoint in particular was the most visible casualty, but
anything else proxied through @All('*') was affected the same way.
Switch the proxy to a deny-list (DROP_HEADERS) for the headers RFC 7230 names
hop-by-hop or otherwise scoped to a single connection (host, connection,
content-length, content-encoding, transfer-encoding, keep-alive,
proxy-authenticate, proxy-authorization, te, trailers, upgrade), and forward
everything else the device sends. x-kobo-userkey now reaches Kobo on every
store call, so the borrow endpoint can authorize the user's library and the
libby-arrived entitlements surface on the next /v1/library/sync.
Add a forward() path that decodes the device's composite sync token before
relaying, and forwards the Kobo cursor Kobo can use instead. A raw Kobo token
(no PX. prefix) is forwarded verbatim, the way devices that synced with Kobo
directly before BookOrbit was put in the path already do. A composite token
whose cursor is undefined has no Kobo-side value, so omit the header entirely
on the way out rather than leak our composite to Kobo.
The extraHeaders and omitHeaders escape hatches stay in place, so the explicit
library/sync call can still pin the upstream cursor (and the no-cursor path
still drops the header) without going through the new decode-on-forward path.
The librarySync controller in kobo-sync.controller.ts already uses those
exacts knobs for the store page, so this change does not touch it; this
commit is the catch-all side of the same fix.
Closes#738
* fix(kobo): honor RFC 7230 Trailer and Connection fields, and emit sanitized proxy logs
Three small corrections surfaced by code review on the catch-all proxy that
delivers Kobo store entitlements.
The deny-list shipped with the wrong name for one of the RFC 7230 hop-by-hop
fields. `trailers` (plural) does not match the standard `Trailer` header that
arrives in mixed case, so a Trailer field set by an upstream could leak through
on the way to the device. Replace it with `trailer`.
RFC 7230 says the fields listed in `Connection` are also hop-specific, and a
proxy must strip them before forwarding. The two header-copy paths only
checked DROP_HEADERS, so a `Connection: x-kobo-tracking` header from the device
or from upstream could let a nominated field slip through. Parse the
Connection header in both directions, normalize case-insensitively, and add
its fields to the ignore set alongside DROP_HEADERS. The new
readConnectionFieldNames helper does the splitting once and is reused on both
sides of the relay.
Add a structured start/end/fail log around the relay, replacing the previous
single-line warn that interpolated a request-derived targetUrl and a raw
err.message. The repo's logging conventions in AGENTS.md name an external
HTTP call as a non-trivial operation that warrants start/end/fail logs, and
require sanitizeLogValue() for every dynamic value that lands inside a quoted
log field. The fail line now carries durationMs, errorClass, sanitized error,
and sanitized targetUrl. The 502 response with the "Upstream Kobo API
unavailable" body is preserved.
Tests cover the new paths: Trailer is stripped, Connection-nominated fields
are stripped on the request side, and Connection-nominated fields are stripped
on the response side. All 10,486 server tests pass; lint and typecheck clean.
CodeRabbit also asked to instantiate KoboProxyService through
Test.createTestingModule. AGENTS.md says the same thing, but every sibling
test in the kobo module uses direct construction with no testing module, so
changing only this file would invert the local pattern. The convention is
already inconsistent across the module and is not enforced by any lint rule;
that is a separate cleanup better suited to its own PR.
* test(kobo): drop unused makeHeaders helper in proxy service tests
makeHeaders was a placeholder identity function that returned its argument
unchanged. Inline the headers object literals at the 9 call sites.
* fix(kobo): scrub BookOrbit credentials and the composite token in every relay
Move the sync-token substitution into request() so forwardTagDelete, which
relays with no options, stops handing Kobo a PX. token it cannot read. Apply it
before omitHeaders and extraHeaders so store sync keeps both escape hatches.
Strip BookOrbit's session cookies and the reverse proxy forwarding headers:
these routes sit on the BookOrbit origin, so a browser carrying a session
reached them with our auth cookies attached and relayed them to Kobo.
Restore the SSRF regression case that covers the allowed hostname in userinfo.
* test(maintenance): wait for the cover sweep instead of a fixed turn budget
The detached sweep does real filesystem I/O, so 50 setImmediate turns settled it
locally but not on a loaded runner. The drain loop then gave up silently and the
next call failed as "The cover sweep is not complete", pointing at the wrong
place. Poll the store to a deadline and fail loudly when it truly hangs.
---------
Co-authored-by: Daniel Chong <daniel@chong.nz>
Co-authored-by: neon <18559658+neonsolstice@users.noreply.github.com>
Remove database records for books whose files are gone, clear cover
references with no file on disk, and delete cover folders left behind by
deleted books. Each category is reviewed in a paginated list and
re-verified against disk before anything is removed.
Closes#312
Finishing an issue offered no way onward, so the next one had to be opened from the library.
Resolve the next readable book server-side with one keyset query, offer it on the last page, and
let an opt-in setting cross over on a further page turn. The reader route now remounts on a param
change, without which it would keep rendering the book it was mounted with.
Also cast the compared value in compareSeriesIndexSql: as a bare parameter Postgres could not type
it inside the null test and rejected the statement.
Closes#998
Expose each session's originating file so the reader can restore a validated historical progress position without losing exact highlight navigation.
Closes#1045
Preserve curated genres while appending unique values returned by metadata
providers, and expose merge versus replace behavior in the web interface.
Closes#1125
Use Hardcover's default search configuration because custom fields currently return a payload-level error. Surface future search errors in logs so failures remain diagnosable.
Fixes#1123
Use completed reading attempts as the canonical source for timelines, goals, and completion latency. Preserve user and library scoping while excluding deleted, skimmed, and undated attempts.
Fixes#1132
Allow ready Book Dock rows to refresh provider matches without depending on the
failed-only bulk retry path. Preserve bounded queueing, ownership checks, and
stale metadata cleanup.
Fixes#1118
Treat series positions as validated string labels so values such as 5.10 retain their exact representation across storage, metadata providers, file formats, and clients.
Closes#1119
Fetch and cache linked ComicVine volumes so issue candidates expose publisher metadata. Preserve candidates when enrichment fails or is throttled.
Fixes#1102
Each run sized its worker pool to the whole machine, so simultaneous runs
drove the box into swap until no suite finished and tests began failing on
the 5s default timeout. Serialize runs behind a lock, cap worker counts, and
raise testTimeout. See scripts/dev/with-test-lock.mjs.
The catalog manifest and book detail stringified the raw index instead of
going through formatSeriesIndex, so {seriesIndex} resolved to 1 on the device
while every other pattern consumer produced 01. The shipped fixed2 default is
unaffected because it reparses its input numerically.
Closes#1090
Day keys are written per reader but read back using UTC, the database server's CURRENT_DATE, or
the server process's local year, so streaks and day-based achievements were evaluated against the
wrong day. Route every lookup through one resolved timezone on the evaluation context.
Install tzdata so the container honours TZ for date and subprocesses. Node already resolved TZ
through ICU, so no reading data was affected by the missing package.
Closes#1084
Creators and contributors with MARC role nrt now populate narrators, in both
EPUB2 opf:role and EPUB3 refines form. The audio-specific pass also stops being
skipped when a sidecar OPF outranks an audio winner, so chapters and narrators
survive an opfFile-first library, and it runs first so an OPF that names
narrators wins over the composer tag.
Closes#1081
Detect the OPF rendition:layout declaration and announce pre-paginated books
as EPUB3FL instead of KEPUB, which is what selects the device's fixed-layout
renderer. Kepub conversion is left alone so KoboSpan progress sync still works.
The flag is stored per file and backfilled during the sync reconcile pass, so
existing libraries converge without a rescan.
Closes#1044
Rebuilds the users roster, metadata field rules, file naming and metadata
score panels, and replaces the libraries list with an expandable ledger.
Libraries now batch their counts and last scan into one request instead of a
per-library fan-out, and surface scan history from scan_jobs on demand.
Let each user choose which accessible libraries feed dashboard shelves and reading widgets while preserving access checks and Smart Scope filters.
Closes#1113
Read variable-font axes and named instances so readers can select real weight and style variants instead of synthesized faces. Preserve the axis default, accept the full CSS weight range, and keep saved choices stable while font catalogs load.
Require the new settings fields for complete EPUB defaults, keep per-book patches partial, and widen the persisted font-weight constraints to match CSS.
Closes#533
Editing Date Started or Date Finished sent a date-only patch that the attempt layer had nothing to
attach it to. On an unread book no attempt was created, so the request returned 200 with null dates
and the field cleared itself. On a book being read the date landed on the open attempt as an end
date with no outcome, violating reading_attempts_closed_has_outcome_chk and surfacing as a 500.
Treat a date-only patch as a lifecycle statement: a start date opens an attempt and projects
reading, a finish date closes one as completed and projects read, and endedOn is never written
without its outcome. Clearing a finish date keeps the completion, because an outcome with no
completion date is legal and is already what re-read seeding writes. Setting a finish date on an
abandoned book now completes it. Reject a status that contradicts its dates with a stable error
code the client can translate, and derive statusChanged from the resulting status so a date-driven
transition still reaches achievements and the Kobo projection.
Keep unread and want_to_read free of dates they cannot own. The attempt keeps its own, so a book
marked unread no longer projects a start and finish date once a historical attempt is logged, and
marking a finished book unread no longer strands its finish date on the projection.
Stop the Hardcover import sending dates a status cannot carry. A want-to-read row that still holds
a start date would otherwise fail its row and skip the status import entirely.
Closes#1073
Read selection was positional: prefer any open remote read, else a cached book-level id, else the
first read returned, while the primary attempt was re-derived each sync from book dates. The two
could disagree, assigning one Hardcover read to two attempts and violating
reading_attempts_external_uidx, which aborted the sync and persisted the failure.
Treat reading_attempts.external_id as the mapping of record: an attempt keeps the read it owns, an
unlinked attempt adopts only an unclaimed read and claims it locally before the remote write, and a
fresh read is minted when nothing is adoptable. Never merge or delete attempts to resolve a
conflict, and keep soft-deleted ids reserved so importExternalRead tombstones still hold.
Also skip the link write when an attempt already owns its read, and map the aggregated attempt
timestamp to a Date so change detection stops comparing a driver string against lastSyncedAt.
Closes#907
Return a stable error code for private issuer rejections and translate it in settings so administrators can apply the required production override.
Closes#1065
neon
Chris Mansell
Daniel Chong