bookorbit/koreader-plugin

BookOrbit Sync plugin for KOReader

Syncs your KOReader reading life into BookOrbit:

• Native BookOrbit catalog browsing from KOReader: a reading-first dashboard with compact Continue reading rows, goal/streak/highlight summaries, "On device", libraries, collections, SmartScopes, authors, series, all books and scoped or global search. The dashboard can auto-open on startup and falls back to its last cached snapshot when offline. Browse books as a cover mosaic or a detailed list, with read-status, reading-progress bars and on-device indicators, read-status and format filters, a configurable grid, persisted view/sort/grid, recent-search history, loading feedback, download progress and inline retry on failures. Navigation runs off the UI thread (cancellable, no freeze) and the next page's covers are prefetched. The detail page shows genres, tags, page count and an expandable full description, and can set read status and star rating directly (synced back like any other status/rating change). Series open as an ordered read-through view with a "read N/total" header. Authors and series lists are paginated with footer arrows and filtered from the catalog action menu. Downloads are linked to BookOrbit sync after the file is saved, and books already on the device offer a "Read" action that opens the local file. Bulk downloads can run from the current page, all books matching a list, or selected books across pages. The thumbnail cache is bounded and evicts the oldest covers.
• Live reading progress of the open book (pull on open with conflict prompt, optional periodic push every N page turns, push on close and suspend), compatible with the kosync flow BookOrbit already speaks.
• Reading statistics: the raw per-page time events from KOReader's statistics database. BookOrbit turns them into reading sessions and daily stats, so they show up in the existing dashboards.
• Highlights and notes, two ways: device highlights appear in the BookOrbit web reader as native highlights, and highlights, note edits and deletions made in BookOrbit come back to the device.
• Book status (reading / complete / abandoned) and star ratings, with newest-change-wins conflict handling.

The open book is captured from live memory when it is closed, on suspend, and via "Sync this book now", so highlights and status made mid-session sync without reopening the book. Manual book sync checks BookOrbit for newer progress before uploading local progress, so a second device does not overwrite a first device's newer location. The full-library sweep ("Sync all books now") is manual-only.

Only books that BookOrbit already knows are synced. Matching uses the same partial-MD5 file hash KOReader uses internally, which BookOrbit's scanner computes for every file. Books on the device that are not in any BookOrbit library are skipped and re-checked automatically when your library changes.

Install (preconfigured, recommended)

1. In BookOrbit web settings, open Settings, KOReader and create sync credentials (username + password). The KOReader sync permission must be enabled for your account.
2. Click "Download preconfigured plugin". The zip is generated by your server with the server address and your sync credentials already embedded (bookorbit_provision.lua inside the plugin folder).
3. Unzip and copy the bookorbit.koplugin/ folder to the koreader/plugins/ directory on your device.
4. Restart KOReader. The plugin configures itself on startup (a confirmation message appears) and removes the embedded credentials file.

Re-downloading later (for example after changing the sync password or server address) and reinstalling applies the new configuration, even over an existing login. Reinstalling an old zip after a logout does not log back in: every download carries a generation stamp and the plugin only applies a configuration it has not applied before.

Install (manual)

1. Copy the bookorbit.koplugin/ folder from the repository to the koreader/plugins/ directory on your device.
2. Restart KOReader.
3. In BookOrbit web settings, open Settings, KOReader and create sync credentials (username + password). The KOReader sync permission must be enabled for your account.
4. On the device, open the main menu, Tools tab: "BookOrbit sync" sits directly below "Calibre" on the first page.
   • Set the BookOrbit server address (e.g. https://bookorbit.example.com; the /api/v1 suffix is added automatically).
   • Login with the credentials from step 3.
   • Use "Open dashboard" to open the BookOrbit dashboard, browse and download books from the same account.

Setup

1. In the "BookOrbit sync" menu, enable "Auto sync this book" if you want the open book synced automatically. Leave the stock "Progress sync" plugin unconfigured to avoid double syncing.
2. Optional: in "Sync settings", tune "Periodically sync every # pages" (default 10, 0 disables mid-session pushes).
3. Optional: in "Sync settings", set "Open dashboard on startup" to "File manager only" or "Every startup" if you want the BookOrbit dashboard to open automatically after KOReader starts.
4. Optional: assign the "BookOrbit: sync this book" and "BookOrbit: sync all books" actions to gestures.

Menu placement note: the plugin generates settings/reader_menu_order.lua and settings/filemanager_menu_order.lua to pin its entry below Calibre, and keeps them up to date across KOReader updates. Delete those files to reset the menu order. If you maintain your own menu order files, only the single bookorbit_sync entry is inserted (a one-time rewrite normalizes formatting and strips comments).

Two-way highlight sync

Plugin 0.4 exchanges highlights instead of only uploading them (server 0.4+; against an older server the plugin falls back to upload-only automatically):

• Device to web: highlights upload as before. BookOrbit converts their crengine positions (xpointers) to reader positions (CFIs), verifying every conversion against the highlighted text, so they render as native highlights in the web reader where you can edit notes, recolor and delete them.
• Web to device: on book open (with "Auto sync this book" on), after "Sync this book now", and during the full sweep for closed books, the plugin applies web-created highlights, edits and deletions. For the open book they are verified against the rendering engine first; a position that does not match its text is re-anchored by searching the text, and the corrected position is reported back to the server. For closed books they are merged into the sidecar the same way KOHighlights does, and KOReader re-validates them on next open.
• Deletions go both ways: deleting in the web moves nothing until the device confirms; deleting on the device moves the highlight to the BookOrbit trash, where it can be restored (a restore pushes it back to devices).
• Identity is the highlight's creation datetime plus position, so extending a highlight on the device is recognized as a move, never as a delete-and-recreate.
• Styles map across: highlight/underline/strikethrough map 1:1, BookOrbit's squiggly renders as underline on the device (and stays squiggly on the web), KOReader's invert renders inverted on the web. Colors map between KOReader's named palette and hex.
• "Two-way highlight sync" in the plugin settings turns the apply direction off; uploads continue either way.

How syncing works

• With "Auto sync this book" on: progress is pulled when a book opens (with conflict prompts), pushed every N page turns (10 seconds after you stop turning pages, 25 second debounce, silently skipped when offline), and the whole book (progress, highlights, status, rating, reading time) is snapshotted from live memory and uploaded when the book is closed and on suspend.
• "Sync this book now" (menu item or gesture) first checks BookOrbit for newer progress, applies or prompts according to your progress-sync settings, then does the same per-book snapshot on demand, including a forced statistics flush. If another device has newer progress and you decline it, highlights, status, rating and reading time still sync, but local progress is not pushed over the newer server value. If the book is not in any BookOrbit library it tells you so.
• "Sync all books now" runs the full-library sweep: match new books, upload reading time for every matched book, then highlights, statuses and bulk progress. It never runs automatically. If a book is open when it starts, that book's sidecar and statistics are flushed first, so the sweep sees current data.
• Everything is incremental: page-stat events are uploaded past a per-book watermark that only advances after the server acknowledges them, sidecars are only parsed when their modification time changed, and unmatched books are only re-checked when BookOrbit's library actually changed. An interrupted sync resumes where it left off; resends are server-side no-ops.
• Local plugin state lives in settings/bookorbit_sync_state.lua next to your other KOReader settings. Deleting it is safe: the next sync re-uploads everything and the server deduplicates.

First sync on a device with lots of history

Years of statistics upload in batches of 500 events. Roughly 200k events means about 400 requests, a few minutes of background work. The UI stays usable; progress is kept on every acknowledged batch, so you can suspend or lose wifi mid-way without losing anything.

Limitations

• Web-side highlight changes reach a closed book's sidecar only through the manual sweep (and only when that book's sidecar changed since the last sweep) or on the next open of the book.
• Web-created PDF highlights are not supported; device PDF highlights sync up and are listed (and jump to the page) in BookOrbit, but are not drawn over the PDF.
• Books with reading statistics but no sidecar path in the reading history sync reading time only (no highlights/status/progress); the full sweep covers them.
• Position-only bookmarks (no highlighted text) are not synced.
• Device clocks far in the past can delay edit detection for highlights that were last modified from the web (the device edit must carry a newer timestamp than the applied one).
• The full-library sweep is manual-only by design; books you never open again only sync when you press "Sync all books now".
• A cloned device with a different KOReader device_id re-uploads its history under the new device identity, which double-counts reading time. Clones that keep the same device_id deduplicate naturally.

Hand-test checklist

• Menu placement: "BookOrbit sync" appears on the Tools first page directly below Calibre, in both the file manager and the reader; settings/reader_menu_order.lua and settings/filemanager_menu_order.lua exist with the bookorbit marker line; a restart does not rewrite them.
• Menu layout: the first BookOrbit menu page starts with Open dashboard, Installed plugin/update, Last sync, manual sync actions, Auto sync, Two-way highlight sync, Sync settings and Account & setup.
• Catalog browser: "Open dashboard" appears in the Tools menu after login and opens the dashboard root. The dashboard shows compact Continue reading rows, reading goal/streak/highlight summaries, "On device", libraries, collections, SmartScopes, authors, series and all books. Tapping dashboard books/highlights opens the book detail; tapping shortcuts opens the matching existing catalog screen.
• Catalog dashboard cache: open the dashboard successfully, turn wifi off, then open it again through startup auto-open; the last successful dashboard appears with an offline-cache label. Against an older BookOrbit server without the dashboard endpoint, the plugin falls back to the old catalog root and reports that the server needs an update.
• Startup dashboard: with "Open dashboard on startup" set to Off, restart does not open it. With File manager only, restarting into the file manager opens the dashboard once. With Every startup, restarting into the file manager or reader opens the dashboard once. A fresh preconfigured install still shows the provision confirmation without auto-opening over it.
• Catalog search: the title-bar left button opens the catalog action menu everywhere. Dashboard and section pages offer "Search BookOrbit"; book lists offer "Search in this list". Search is no longer a row in the root list. The title bar has no close button; use "Close BookOrbit" from the menu.
• Catalog view and filters (actions menu): toggle mosaic/list view, change the mosaic grid size, filter by read status (all/unread/reading/finished) and by format, sort, and change order (asc/desc). View mode, sort and grid size persist across sessions; the active sort, direction and filters show in the page subtitle.
• Catalog responsiveness: tapping any section, book page or detail shows a dismissable loading message and does not freeze the reader (network runs in a subprocess); tapping the loading message cancels the request. The next page's covers are prefetched a second after a page settles, so paging forward shows covers immediately.
• Catalog detail edits (detail page actions menu): "Set read status" and "Set rating" write straight to BookOrbit and update the page in place; against an older server without these endpoints the action reports it is unsupported instead of erroring. "Description" opens the complete text, and "Genres"/"Tags" list every value (the detail page itself only shows the first few with a "+N more" hint).
• Catalog series view: opening a series shows its books in series order with a "Read N/total" header.
• Catalog authors/series lists: "Filter..." in the catalog action menu narrows the list as you type, and footer arrows move through large libraries.
• Catalog manual refresh: the actions menu "Refresh" re-fetches the current scope and retries any covers that failed.
• Catalog indicators: each card/row shows reading progress, read status and an "On device" badge when a matching file already exists locally. "On device" lists those matched books via a single ?ids= lookup.
• Catalog search history: the search dialog offers a "Recent" button to re-run one of the last searches (stored locally).
• Catalog feedback: section/page/detail fetches show a loading message; failures show an inline "Retry" dialog; downloads show throttled byte/percent progress.
• Catalog download: open a book detail, tap "Download" for the default folder/name, confirm overwrite if needed, watch the download progress, then accept "Open now". Use "Download options" in the detail action menu to change folder/name. The downloaded file is matched by partial MD5 and linked in bookorbit_sync_state.lua; the "On device" badge refreshes after a successful link.
• Catalog long-press actions: hold a dashboard cover, mosaic card or list row. The detail-backed action sheet offers details, read/open when on device, download, download options, read status, rating, description, genres and tags. On normal book lists it also offers Select.
• Catalog selected download: in a normal book list, hold a book and choose "Select", or choose "Select books" from the action menu. Select books across multiple pages, verify the top-right selected badges, selected count and disabled download button while nothing is selected. With books selected, the title-bar search icon becomes a download button and opens the same confirmation as "Download selected". The refresh icon clears selected books; when nothing is selected, it exits selection mode. Returning to the dashboard exits selection mode and restores search/refresh. "Select page", "Clear" and "Exit selection" work from the selection action menu.
• Catalog list bulk download: from a filtered list or series, run "Download this page" and "Download all in this list". The confirmation shows the target folder, format preset, existing-file policy and total count. Broad "All Books" downloads require the same confirmation.
• Catalog bulk settings: in the catalog action menu, change "Bulk download settings" between EPUB/PDF/Comics format presets and Skip/Overwrite existing-file policies. Verify the next queue uses the selected policy.
• Catalog bulk failure handling: simulate a network/server failure or device write failure during a queue. The queue continues where possible, deletes failed partial downloads through the download client, shows failed titles in the completion summary, and "Retry failed" retries only the failed books.
• Catalog bulk cancellation: start a multi-book queue, tap "Cancel after current file", and verify the active transfer finishes or fails before the queue stops with a stopped summary.
• Catalog read vs download: when a book is already on the device, the detail page shows a primary "Read" button that opens the local file. "Download" is shown next to it only when there are formats not yet on the device, starts with the default folder/name, and is hidden once every supported format is downloaded. The detail action menu offers "Download options" for folder/name changes. The file chooser marks already-downloaded formats as "Open" instead of "Download". On-device state is refreshed when the detail opens and after a successful download.
• Custom menu order: remove the marker line and shuffle the tools list; on restart only the bookorbit_sync id is re-inserted, the rest is preserved. A syntactically broken file is left untouched (warning in the log).
• Login: wrong password shows a login error; server URL with and without trailing / and with /api/v1 already appended all work.
• Preconfigured install: download the zip from web settings, install, restart; the "set up for ... as ..." message appears once, the server URL and login are filled in, bookorbit_provision.lua is gone from the plugin folder, and a second restart shows no message.
• Provision after logout: logout in the plugin menu, reinstall the SAME zip, restart; still logged out. Download a FRESH zip, reinstall, restart; logged in again.
• Provision over existing login: with a working login, install a fresh zip with different credentials and restart; the new credentials win and the message appears.
• Open a book that another device read further: pull prompt appears (with "Prompt" strategy); accepting jumps to the newer location.
• Periodic push: with "Auto sync this book" on and the period set to 3, turn 3 pages and stop; about 10 seconds later one progress request hits the server. Turning 3 more pages within 25 seconds gets debounced. Toggling auto sync off cancels a pending push.
• Mid-session highlight, then "Sync this book now": the highlight appears in BookOrbit immediately, no reopen needed.
• Close a book after highlighting and changing status/rating: reading time, highlights, status, rating and progress all land in BookOrbit; reopening and closing again uploads nothing new.
• "Sync this book now" on a book that is not in BookOrbit: shows "not in your BookOrbit library"; closing such a book stays silent.
• Manual "Sync all books now": summary message shows matched books / events / highlights; settings page in BookOrbit shows the sweep. Running it with a freshly annotated book still open includes that book's data.
• Re-running the manual sync immediately: all uploads are no-ops, counts stay stable.
• Add a previously unmatched device book to BookOrbit, rescan the library, run a manual sync: its history backfills.
• Suspend with a book open: the snapshot uploads before wifi drops (where the platform allows); suspend mid-sweep, resume, sync again: no duplicates, sweep completes.
• Airplane mode: manual syncs report the server as unreachable; automatic ones stay silent.
• Gestures: "BookOrbit: sync this book" is assignable from the file manager gesture settings and works inside the reader.
• With the statistics plugin actively recording (book open), a sweep still reads the stats database without errors.

Two-way highlight checks:

• Web highlight on a device-matched book, then open the book on the device: the highlight appears (notification shows the count), with the right color/style and note.
• Edit the note and color of that highlight in the web reader, reopen the book (or "Sync this book now"): the device copy updates.
• Delete a highlight in the web reader; on next device sync it disappears from the book and the bookmark list.
• Delete a highlight on the device; after the book closes (or a sweep), it appears under Annotations, Trash in the web. Restoring it brings it back to the device on the next sync.
• Extend an existing highlight on the device: the web copy moves, no duplicate and nothing lands in trash.
• Create a squiggly highlight on the web: it arrives as underline on the device, and stays squiggly in the web after the device re-syncs it.
• Create an invert highlight on the device: it renders inverted in the web reader.
• Kill KOReader mid-sync and sync again: no duplicates, no lost highlights (acks make re-delivery idempotent).
• Closed-book sweep with pending web highlights: the sidecar gains them with annotations_externally_modified set; opening the book shows them sorted and valid.