fix(reader): reconcile remote progress before manual KOReader book sync

When manually syncing a book, the plugin uploaded progress without checking if another device had written a newer progress state to the server first, causing reading progress to be overwritten and lost.

Fetch the remote progress from the server before starting a manual sync. If a newer progress state is found on the server, prompt the user to apply the remote progress locally first. If the prompt is rejected, proceed with the sync of annotations and reading history while bypassing the progress upload.

Closes #461

koreader-plugin/README.md

@@ -8,7 +8,7 @@ Syncs your KOReader reading life into BookOrbit:
 - Highlights and notes, two ways: device highlights appear in the BookOrbit web reader as native highlights, and highlights, note edits and deletions made in BookOrbit come back to the device.
 - Book status (reading / complete / abandoned) and star ratings, with newest-change-wins conflict handling.
 
-The open book is captured from live memory when it is closed, on suspend, and via "Sync this book now", so highlights and status made mid-session sync without reopening the book. The full-library sweep ("Sync all books now") is manual-only.
+The open book is captured from live memory when it is closed, on suspend, and via "Sync this book now", so highlights and status made mid-session sync without reopening the book. Manual book sync checks BookOrbit for newer progress before uploading local progress, so a second device does not overwrite a first device's newer location. The full-library sweep ("Sync all books now") is manual-only.
 
 Only books that BookOrbit already knows are synced. Matching uses the same partial-MD5 file hash KOReader uses internally, which BookOrbit's scanner computes for every file. Books on the device that are not in any BookOrbit library are skipped and re-checked automatically when your library changes.
 
@@ -54,7 +54,7 @@ Plugin 0.4 exchanges highlights instead of only uploading them (server 0.4+; aga
 ## How syncing works
 
 - With "Auto sync this book" on: progress is pulled when a book opens (with conflict prompts), pushed every N page turns (10 seconds after you stop turning pages, 25 second debounce, silently skipped when offline), and the whole book (progress, highlights, status, rating, reading time) is snapshotted from live memory and uploaded when the book is closed and on suspend.
-- "Sync this book now" (menu item or gesture) does the same per-book snapshot on demand, including a forced statistics flush, so a highlight made seconds ago syncs immediately. If the book is not in any BookOrbit library it tells you so.
+- "Sync this book now" (menu item or gesture) first checks BookOrbit for newer progress, applies or prompts according to your progress-sync settings, then does the same per-book snapshot on demand, including a forced statistics flush. If another device has newer progress and you decline it, highlights, status, rating and reading time still sync, but local progress is not pushed over the newer server value. If the book is not in any BookOrbit library it tells you so.
 - "Sync all books now" runs the full-library sweep: match new books, upload reading time for every matched book, then highlights, statuses and bulk progress. It never runs automatically. If a book is open when it starts, that book's sidecar and statistics are flushed first, so the sweep sees current data.
 - Everything is incremental: page-stat events are uploaded past a per-book watermark that only advances after the server acknowledges them, sidecars are only parsed when their modification time changed, and unmatched books are only re-checked when BookOrbit's library actually changed. An interrupted sync resumes where it left off; resends are server-side no-ops.
 - Local plugin state lives in `settings/bookorbit_sync_state.lua` next to your other KOReader settings. Deleting it is safe: the next sync re-uploads everything and the server deduplicates.

koreader-plugin/bookorbit.koplugin/bookorbit_book_sync.lua

@@ -151,6 +151,9 @@ local function finish(ctx, err)
         if ctx.interactive then
             local text = T(_("BookOrbit: book synced. %1 reading events, %2 highlights up, %3 applied."),
                 ctx.counts.page_stats, ctx.counts.annotations, ctx.counts.ann_applied or 0)
+            if ctx.skip_progress then
+                text = text .. "\n" .. _("Progress was not changed.")
+            end
             if ctx.had_errors then
                 text = text .. "\n" .. _("Some uploads failed and will retry on the next sync.")
             end
@@ -403,6 +406,10 @@ stepProgress = function(ctx)
     local book = ctx.state:getBook(ctx.snap.digest)
     if not book then return finish(ctx, "unmatched") end
 
+    if ctx.skip_progress then
+        return finish(ctx)
+    end
+
     local pct = ctx.snap.percentage
     if type(pct) ~= "number" then
         return finish(ctx)
@@ -463,6 +470,7 @@ function BookOrbitBookSync.run(opts)
         on_finish = opts.on_finish,
         counts = { page_stats = 0, annotations = 0, ann_applied = 0 },
         had_errors = false,
+        skip_progress = opts.skip_progress == true,
     }
 
     ctx.state:rememberFile(snap.file, snap.digest)

koreader-plugin/bookorbit.koplugin/main.lua

@@ -867,6 +867,110 @@ function BookOrbit:syncToProgress(progress)
     end
 end
 
+local function hasUsableRemoteProgress(progress)
+    return progress ~= nil and progress ~= ""
+end
+
+function BookOrbit:remoteProgressIsNewer(body, local_percentage)
+    local local_timestamp = self.last_page_turn_timestamp or 0
+    if body.timestamp ~= nil then
+        if local_timestamp > 0 then
+            return body.timestamp > local_timestamp
+        end
+        return body.percentage > local_percentage
+    end
+    return body.percentage > local_percentage
+end
+
+function BookOrbit:applyRemoteProgress(body, on_done)
+    self:syncToProgress(body.progress)
+    if on_done then
+        UIManager:scheduleIn(0.1, function()
+            on_done(false)
+        end)
+    end
+end
+
+function BookOrbit:reconcileProgressBeforeBookSync(digest, on_done)
+    if not self.ui or not self.ui.document then
+        UIManager:show(InfoMessage:new{ text = _("Open a book to sync it."), timeout = 2 })
+        return
+    end
+    if self:getDocumentDigest() ~= digest then
+        UIManager:show(InfoMessage:new{ text = _("The open book changed. Start the sync again."), timeout = 3 })
+        return
+    end
+
+    local client = self:newClient()
+    local body, err = client:getProgress(digest)
+    self.pull_timestamp = UIManager:getElapsedTimeSinceBoot()
+
+    if not body then
+        logger.dbg("BookOrbit: progress check before book sync failed:", err)
+        UIManager:show(InfoMessage:new{ text = _("Could not check latest progress. Syncing book data without changing progress."), timeout = 4 })
+        on_done(true)
+        return
+    end
+
+    if not body.percentage then
+        on_done(false)
+        return
+    end
+
+    if body.device == Device.model and body.device_id == self.device_id then
+        on_done(false)
+        return
+    end
+
+    body.percentage = Math.roundPercent(body.percentage)
+    local local_progress = self:getLastProgress()
+    local local_percentage = self:getLastPercent()
+
+    if local_percentage == body.percentage or body.progress == local_progress then
+        on_done(false)
+        return
+    end
+
+    local remote_newer = self:remoteProgressIsNewer(body, local_percentage)
+    if not hasUsableRemoteProgress(body.progress) then
+        if remote_newer then
+            UIManager:show(InfoMessage:new{
+                text = _("BookOrbit has newer progress but no exact location. Syncing book data without changing progress."),
+                timeout = 4,
+            })
+            on_done(true)
+        else
+            on_done(false)
+        end
+        return
+    end
+
+    local strategy = remote_newer and self.settings.sync_forward or self.settings.sync_backward
+    if strategy == SYNC_STRATEGY.SILENT then
+        self:applyRemoteProgress(body, on_done)
+    elseif strategy == SYNC_STRATEGY.PROMPT then
+        local template = remote_newer and _("Sync to latest location %1% from device '%2' before uploading this book?")
+            or _("Sync to previous location %1% from device '%2' before uploading this book?")
+        UIManager:show(ConfirmBox:new{
+            text = T(template, Math.round(body.percentage * 100), body.device),
+            ok_callback = function()
+                self:applyRemoteProgress(body, on_done)
+            end,
+            cancel_callback = function()
+                on_done(remote_newer)
+            end,
+        })
+    elseif remote_newer then
+        UIManager:show(InfoMessage:new{
+            text = _("BookOrbit has newer progress. Syncing book data without changing progress."),
+            timeout = 4,
+        })
+        on_done(true)
+    else
+        on_done(false)
+    end
+end
+
 function BookOrbit:updateProgress(ensure_networking, interactive, on_suspend)
     if not self:isLoggedIn() then
         if interactive then promptLogin() end
@@ -1041,9 +1145,21 @@ function BookOrbit:onBookOrbitSyncBook()
     end
 
     local api_opts = self:apiOpts()
+    local run_book_sync = function(skip_progress)
+        local latest_snap = BookOrbitBookSync.capture(self)
+        if not latest_snap then
+            UIManager:show(InfoMessage:new{ text = _("Could not read this book's data."), timeout = 2 })
+            return
+        end
+        if latest_snap.digest ~= snap.digest then
+            UIManager:show(InfoMessage:new{ text = _("The open book changed. Start the sync again."), timeout = 3 })
+            return
+        end
+        BookOrbitBookSync.run{ api = api_opts, snap = latest_snap, reason = "manual", interactive = true, plugin = self,
+            annotation_sync = self.settings.annotation_sync, skip_progress = skip_progress }
+    end
     local run = function()
-        BookOrbitBookSync.run{ api = api_opts, snap = snap, reason = "manual", interactive = true, plugin = self,
-            annotation_sync = self.settings.annotation_sync }
+        self:reconcileProgressBeforeBookSync(snap.digest, run_book_sync)
     end
     if NetworkMgr:willRerunWhenOnline(run) then
         return
@@ -1086,8 +1202,8 @@ function BookOrbit:_onPageUpdate(page)
         self.page_update_counter = self.page_update_counter + 1
         -- A pending periodic push is re-delayed on every page turn so it
         -- only fires once the reader is actually idle.
-        if self.periodic_push_scheduled
-                or (self.settings.pages_before_update or 0) > 0 and self.page_update_counter >= self.settings.pages_before_update then
+        if self.settings.auto_sync and (self.periodic_push_scheduled
+                or (self.settings.pages_before_update or 0) > 0 and self.page_update_counter >= self.settings.pages_before_update) then
             self:schedulePeriodicPush()
         end
     end
@@ -1197,16 +1313,15 @@ function BookOrbit:onBookOrbitToggleAutoSync(toggle, from_menu)
 end
 
 function BookOrbit:registerEvents()
+    self.onPageUpdate = self._onPageUpdate
     if self.settings.auto_sync then
         self.onCloseDocument = self._onCloseDocument
-        self.onPageUpdate = self._onPageUpdate
         self.onResume = self._onResume
         self.onSuspend = self._onSuspend
         self.onNetworkConnected = self._onNetworkConnected
         self.onNetworkDisconnecting = self._onNetworkDisconnecting
     else
         self.onCloseDocument = nil
-        self.onPageUpdate = nil
         self.onResume = nil
         self.onSuspend = nil
         self.onNetworkConnected = nil