Commit Graph

39 Commits

Author SHA1 Message Date
neon db2b3f9696 ci: run E2E only on nightly schedule, skip E2E on PR/push/manual CI, keep manual e2e workflow for on-demand runs, and align matrix/docs accordingly 2026-05-14 16:37:18 +00:00
neon c85379e4ff feat(auth): enforce edit restrictions on demo-restricted accounts 2026-04-25 23:34:57 -06:00
neon cfc3672a6c refactor: rename book-bucket to book-dock and lens to smart-scope 2026-04-20 22:57:27 -06:00
neon ce6a439062 feat(server): add unified entity manager with duplicate detection
- Remove author insights and duplicate suggestions panels, composables, DTOs, and all backend support
- Add genre and tag duplicate detection, merge, and dismiss with server-side paginated scan results
- Consolidate per-entity-type tools into a single entity-manager module backed by the Strategy pattern
- Support browse, merge, rename, delete, split, and dismiss across all entity types (author, genre, tag, narrator, publisher, language, series)
- Replace old tools module with unified entity-manager controller, service, and repository; add 85+ unit tests
2026-04-20 19:10:47 -06:00
neon f2ede06ea2 feat(scanner): add incremental dir-mtime scanning with concurrency fix
Core performance overhaul - new library_dir_scan_state schema table persists per-directory
mtimes so unchanged directories are skipped entirely on subsequent scans, reducing filesystem
stat() calls from O(all-files) to O(changed-dirs) for large libraries.

Incremental scan (walk.ts / scanner.service.ts):
- New library_dir_scan_state table (bigint mtime, FK to library_folders ON DELETE CASCADE)
- findBookCandidates / findLooseFileCandidates now accept knownDirMtimes and return WalkResult
  (candidates + skippedDirs + unchangedDirs + dirMtimes) instead of bare BookCandidate[]
- Per-dir mtime skip with 3 safety rules: never skip disc dirs, never skip dirs with disc
  subdirs, never skip non-root dirs whose parent mtime changed (guards stem-named merging)
- Root directory now also skippable - parent-mtime check conditionally skipped for root since
  root-level files are never stem-merged (explicitly excluded in buildBookCandidates)
- Settings hash invalidation: SHA-256 of {allowedFormats, formatPriority, excludePatterns,
  organizationMode} forces full scan when settings change or on first scan after server restart
- isInUnchangedDir guard prevents false book-missing marks for skipped dirs
- Math.round() on stat().mtimeMs before storing - fixes bigint rejection of macOS fractional
  milliseconds (e.g. 1776138191093.9556) in PostgreSQL

Walk improvements:
- Bounded concurrency: subdirs processed in chunks of 50 to avoid EMFILE on deep trees
- format + role cached on FileStat to avoid re-classifying the same file twice
- statFilesIntoAcc helper extracted to deduplicate parallel stat logic
- skippedDirs set propagated up from permission-denied dirs

ScanJobStore:
- acquireStartLock / releaseStartLock prevent concurrent scan starts for the same library
- markPendingRescan / consumePendingRescan queue a follow-up scan if one arrives while running

waitForStability:
- Accepts optional knownMtimeMs to skip a redundant stat() call when mtime is already known

File event processor (Issue 25, 11, 20):
- Own-book restore: check file's own book before searching all missing books, fixing false
  book-moved results when an editor saves with a new inode (atomic-save pattern)
- Settings cache per library avoids repeated DB lookups during reconcile of many missing books
- Partial move guard: verify old file is gone before reassigning inode (multi-file books)

Live notifications (scanner.service.ts / file-watcher.service.ts):
- scanBookFolder now emits scan:books:added via existing WebSocket infrastructure so the
  frontend grid updates live when the file watcher adds a book (all 3 code paths covered)
- bufferWatcherNotification() with 30s debounce fires one ScanCompleted notification per
  library summarising watcher-driven additions and removals ('Novels updated - 3 added')
- Removals wired in FileWatcherService.process() and reconcile() on book-missing events
- findLibraryName() added to ScannerRepository for notification title lookup
2026-04-17 17:18:14 -06:00
neon 40d829a8a9 feat(notifications): add notification center with real-time updates
- add notifications DB schema, repository, service, gateway, and controller
- add shared notification types, categories, and preference models
- add NotificationAccess permission with authorization matrix probe
- add frontend composable with Socket.IO subscription and REST API
- add NotificationPopover, NotificationItem, and NotificationPreferences components
- integrate notification producers in scanner, metadata fetch, book bucket, author enrichment, email, migration, and file write-back services
- add 30-day auto-cleanup via existing CleanupService cron
- add 29 backend unit tests for notification service and controller
2026-04-15 03:15:05 -06:00
neon 67ec10f703 feat: implement security audit fixes SEC-001 through SEC-031 2026-04-12 16:15:03 -06:00
neon 1f43ea2cb5 feat: enforce non-empty icons for collections, lenses, and libraries 2026-04-12 10:32:31 -06:00
neon b0d6bc1957 chore: setup git hooks, refactor e2e test setup and fix ts build errors 2026-04-10 18:46:41 -06:00
neon 698951a245 refactor(metadata): improve parsing, cover storage, and content type 2026-04-10 00:26:45 -06:00
neon b0ed068fd6 chore: improve download feature 2026-04-08 22:52:52 -06:00
neon 26d1e6ebc8 feat: improve header UI and optimize docker storage configuration 2026-04-08 20:34:38 -06:00
neon 9f6b7d4f2b feat: enhance PDF metadata extraction and refactor book cover storage 2026-04-08 20:32:38 -06:00
neon 54f2894762 feat(scanner): enhance upload durability and add E2E test suites 2026-04-07 12:44:12 -06:00
neon cfc424ea37 feat: move file write settings from global to per-library 2026-04-06 14:54:50 -06:00
neon decd0f1a2e fix(server): fix migration executor bugs and schema corrections 2026-04-05 20:24:08 -06:00
neon 9fd4e162aa feat(server): harden booklore migration with adapters and E2E tests 2026-04-05 16:01:04 -06:00
neon 578cfd584a test(server): harden E2E suites across scanner, auth, and email modules 2026-04-04 23:32:25 -06:00
neon bab3813ff8 feat(metadata): add per-field locking to protect fields from overwrites 2026-04-03 21:31:02 -06:00
neon 0a58a87239 refactor(scanner): rename staging module to book bucket 2026-04-03 20:50:11 -06:00
neon c3fd3f8f2e ci: consolidate E2E into pipeline and gate image builds on test suite 2026-04-02 06:45:00 -06:00
neon 6e999ccaa8 ci: standardize E2E naming across workflows and scripts 2026-04-02 05:25:00 -06:00
neon 00ecdc34cd test(users): add lifecycle E2E suite and harden admin contracts 2026-04-02 04:05:00 -06:00
neon 6115656153 test(email): add email lifecycle E2E suite and harden send behavior 2026-04-02 02:45:00 -06:00
neon 835bc91a09 test(reader): add state isolation E2E suite and fix isolation bugs 2026-04-02 01:25:00 -06:00
neon 64abfa8c41 test(opds): add auth catalog end-to-end tests 2026-04-02 00:05:00 -06:00
neon 914f8a3440 test(auth): add authorization-matrix end-to-end suite 2026-04-01 15:49:30 -06:00
neon 157224be9f test(metadata): expand end-to-end scenarios and assertions 2026-04-01 11:50:59 -06:00
neon a6d84ae20c test(metadata): add initial E2E coverage for metadata write flow 2026-04-01 08:05:00 -06:00
neon 32bbba9aa9 fix(metadata): harden preference concurrency and score recalculation 2026-03-31 04:50:04 -06:00
neon ec44071a91 fix(auth): harden OIDC and stabilize session security hooks 2026-03-31 02:54:53 -06:00
neon 0c195165e6 test(scanner): add staging ingest and finalize E2E test suite 2026-03-30 23:17:27 -06:00
neon 4d58418b93 test(auth): add end-to-end coverage for authentication flows 2026-03-30 03:58:27 -06:00
neon b5cc1ef370 test(scanner): refactor E2E scaffolding and extract shared test harness 2026-03-30 02:11:51 -06:00
neon db30a45782 test(scanner): bootstrap scanner E2E test infrastructure 2026-03-30 01:18:32 -06:00
neon aa2456fbf8 feat(authors): add auto-enrichment queue, worker safety, and live status 2026-03-13 08:14:45 -06:00
neon 891cd19f46 chore: implement local quality foundation 2026-03-06 09:08:15 -06:00
neon 244d38afe8 feat(reader): add settings system with per-book delta and defaults 2026-01-16 02:00:38 -06:00
neon b4ee0cb90c feat(metadata): add cover extraction for epub, mobi, cbz, and pdf 2025-12-22 14:37:28 -06:00