mirror of
https://github.com/bookorbit/bookorbit.git
synced 2026-11-03 00:11:37 -06:00
docs: overhaul development, contributor, testing, and commit guidelines
This commit is contained in:
neon
@@ -0,0 +1,73 @@
|
||||
# AI Usage Policy
|
||||
|
||||
ProjectX is maintained by real people. Every pull request, issue, and discussion is read and reviewed by humans who give their time to this project. Submitting unreviewed, untested AI output shifts the burden of that review onto maintainers - that is the problem this policy addresses. It is not a problem with AI tools themselves.
|
||||
|
||||
## The Rules
|
||||
|
||||
- **All AI use must be disclosed.** If any part of your contribution was written, generated, or significantly shaped by an AI tool, say so in the PR description. There are no exceptions and no minimum threshold. See [Disclosure Format](#disclosure-format) below.
|
||||
|
||||
- **You must understand every line you submit.** If you cannot explain what a piece of code does, why it is structured the way it is, and how it fits into the surrounding system - do not submit it. "The AI wrote it" is not an answer during review.
|
||||
|
||||
- **AI-generated tests must validate real behavior.** Tests that mock away the logic under test, assert on implementation details instead of outcomes, or trivially pass regardless of what the code does will be rejected. Read every assertion before submitting.
|
||||
|
||||
- **No AI-generated media.** Images, icons, audio, video, and other non-text assets must be human-created or properly licensed. Text and code are the only acceptable forms of AI-generated content, subject to the rules above.
|
||||
|
||||
- **Clean up before submitting.** Remove AI boilerplate, placeholder comments, dead code, and empty catch blocks. Submit only what you would be comfortable having reviewed line by line.
|
||||
|
||||
## Disclosure Format
|
||||
|
||||
Include this in your PR description whenever AI tools were used:
|
||||
|
||||
```
|
||||
AI tools used: [tool name, e.g. Claude, Copilot, Cursor]
|
||||
Extent: [what it helped with, e.g. "generated the DTO scaffolding and test boilerplate; all service logic and ownership checks written manually"]
|
||||
```
|
||||
|
||||
Be honest. A brief, accurate note is better than a vague one that obscures how much the AI contributed.
|
||||
|
||||
## Consequences
|
||||
|
||||
PRs that contain undisclosed AI use or that are clearly unreviewed AI output will be closed immediately with a short explanation. There is no detailed feedback given for this - that burden belongs to the contributor, not the reviewer.
|
||||
|
||||
You are always welcome to resubmit once the work is genuinely reviewed, tested, and properly disclosed. The door is not closed permanently.
|
||||
|
||||
## Watch Out for These
|
||||
|
||||
AI tools make specific, recurring mistakes on this codebase. Be especially careful here.
|
||||
|
||||
**Drizzle ORM hallucinations.** LLMs are trained heavily on Prisma and will generate Prisma-style calls - `.findMany()`, `.findUnique()`, `.create()` - that do not exist in Drizzle. Drizzle uses a query builder. Verify every database call against the [Drizzle documentation](https://orm.drizzle.team/docs/overview) and against how the rest of the codebase does it.
|
||||
|
||||
**NestJS decorator and module mistakes.** AI frequently invents decorator options, misuses pipes, omits required module imports, or generates module wiring that looks plausible but is wrong. Check every decorator and every module registration.
|
||||
|
||||
**Ownership and multi-tenancy logic.** ProjectX is a multi-user app. Every query that touches user-owned data must filter by `userId`. AI-generated service methods routinely omit this. A missed ownership check is a data exposure bug, not a style issue. This applies to every module that manages libraries, books, collections, reading sessions, annotations, and preferences.
|
||||
|
||||
**Authorization patterns.** We have specific conventions for permission gates: `@RequirePermission(...)` on the backend, `usePermissions()` on the frontend, and the `LensService` superuser-bypass pattern. AI does not know these patterns and will generate code that bypasses them or reimplements them incorrectly. Write auth and permission logic yourself.
|
||||
|
||||
**Tests that prove nothing.** AI-generated tests often mock the exact thing being tested, assert on shallow output without triggering the real path, or simply confirm that a function returns without throwing. Before submitting a test, ask: would this test fail if the implementation were broken? If the answer is no, the test is not worth having.
|
||||
|
||||
## How to Use AI Well Here
|
||||
|
||||
AI is a genuinely useful tool when applied with judgment. Here is what works well on this project:
|
||||
|
||||
- Use AI to scaffold boilerplate: DTOs, module wiring, Vue component structure, test file setup. Review and adjust everything it produces.
|
||||
- Use AI to explain unfamiliar patterns, library APIs, or architectural concepts. Understanding-driven use is the best use.
|
||||
- Write authentication, authorization, and ownership-critical logic yourself. These are the areas where a mistake has real consequences for real users.
|
||||
- When AI generates a service method, read every line with the question: does this correctly scope to the current user? Does it throw the right exception for unauthorized access?
|
||||
- Run the full stack after every AI-assisted change. `pnpm dev` plus manual verification is not optional.
|
||||
- Use AI to generate a test structure, then replace every assertion with one you wrote and understand yourself.
|
||||
|
||||
## A Note on Maintainers
|
||||
|
||||
These rules apply to outside contributors. Maintainers are exempt. They have demonstrated judgment and accountability over time, and are trusted to apply discretion in how they use AI tools in their own work.
|
||||
|
||||
## AI is Welcome Here
|
||||
|
||||
To be clear: this policy is not an anti-AI stance. Maintainers use AI tools. Many parts of this codebase were written with AI assistance.
|
||||
|
||||
The problem this policy addresses is unqualified, unreviewed output being submitted as a contribution - work that the contributor has not run, does not understand, and cannot stand behind. That is a burden on maintainers regardless of whether it came from a human or a machine.
|
||||
|
||||
Use the tools. Do the work.
|
||||
|
||||
---
|
||||
|
||||
_This policy was written with reference to the [Ghostty AI Policy](https://github.com/ghostty-org/ghostty/blob/main/AI_POLICY.md), which we found thoughtful and worth building from._
|
||||
@@ -0,0 +1,274 @@
|
||||
# Commit Message Guidelines
|
||||
|
||||
Every commit tells a story. A well-written commit message makes it easy to understand why a change was made, trace regressions, and produce a meaningful changelog. These guidelines define the format for all commits to this repository.
|
||||
|
||||
## Format
|
||||
|
||||
```
|
||||
<type>(<scope>): <summary>
|
||||
|
||||
[body]
|
||||
|
||||
[footer]
|
||||
```
|
||||
|
||||
The **header** is required on every commit and must follow the format described below.
|
||||
|
||||
The **body** is required on all commits except `docs`. When present, it must be at least 20 characters and written in imperative mood.
|
||||
|
||||
The **footer** is optional. Use it to reference issues, close PRs, or document breaking changes.
|
||||
|
||||
---
|
||||
|
||||
## Header
|
||||
|
||||
```
|
||||
<type>(<scope>): <summary>
|
||||
│ │ │
|
||||
│ │ └─ imperative mood, lowercase, no period at the end
|
||||
│ │
|
||||
│ └─ Scope (optional): auth|books|library|metadata|kobo|opds|reader|
|
||||
│ collections|annotations|authors|series|cover|
|
||||
│ users|stats|notifications|settings|scanner|
|
||||
│ email|audit|lens|types|docker|deps|server|client
|
||||
│
|
||||
└─ Type: feat|fix|db|perf|refactor|style|docs|test|build|ci|chore|security|revert
|
||||
```
|
||||
|
||||
The `<type>` and `<summary>` fields are required. The `(<scope>)` field is optional.
|
||||
|
||||
### Type
|
||||
|
||||
The type signals the intent of the change at a glance.
|
||||
|
||||
| Type | When to use |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------- |
|
||||
| `feat` | Introduces a new feature or user-visible capability |
|
||||
| `fix` | Patches a bug in existing behavior |
|
||||
| `db` | Adds or modifies a Drizzle schema, generates a migration, or changes seed data |
|
||||
| `perf` | Improves performance without changing observable behavior |
|
||||
| `refactor` | Restructures code without fixing bugs or adding features |
|
||||
| `style` | Visual/CSS-only changes: Tailwind tweaks, spacing, color token adjustments |
|
||||
| `docs` | Documentation changes only |
|
||||
| `test` | Adds or corrects tests without touching production code |
|
||||
| `build` | Changes to the build system, tooling config, or external dependencies (Dockerfile, tsconfig, pnpm) |
|
||||
| `ci` | CI/CD pipeline changes - GitHub Actions workflows and scripts |
|
||||
| `chore` | Routine maintenance that doesn't fit any other category |
|
||||
| `security` | Addresses a security vulnerability; preferred over `fix` so security patches are identifiable in history |
|
||||
| `revert` | Reverts a previous commit - the body must reference the reverted SHA and explain why |
|
||||
|
||||
> If a change genuinely spans multiple types (for example, it adds a feature and fixes a bug), split it into two commits. If that isn't practical, use the type that best represents the dominant intent.
|
||||
|
||||
### Scope
|
||||
|
||||
Scope is optional but strongly encouraged. It narrows the type to the affected area of the codebase, making the log significantly more readable.
|
||||
|
||||
Use one scope from the table below. If a change affects multiple areas, use the most dominant one.
|
||||
|
||||
| Scope | What it covers |
|
||||
| --------------- | ------------------------------------------------------------- |
|
||||
| `auth` | Authentication, authorization, JWT, OIDC, sessions |
|
||||
| `library` | Library management, file system scanning |
|
||||
| `books` | Book data, CRUD, book-level state |
|
||||
| `authors` | Author data and management |
|
||||
| `series` | Book series grouping and management |
|
||||
| `cover` | Book cover image storage, retrieval, and optimization |
|
||||
| `metadata` | Metadata providers, fetch logic, quality scoring, preferences |
|
||||
| `kobo` | Kobo device sync and integration |
|
||||
| `opds` | OPDS catalog endpoints |
|
||||
| `reader` | In-app reader, reading position, reader preferences |
|
||||
| `collections` | Collections and shelf management |
|
||||
| `annotations` | Highlights, bookmarks, notes |
|
||||
| `users` | User management, profiles, roles |
|
||||
| `stats` | Reading sessions, statistics, progress tracking |
|
||||
| `notifications` | In-app and email notifications |
|
||||
| `settings` | Application settings and configuration |
|
||||
| `scanner` | File scanner and upload pipeline |
|
||||
| `email` | Email providers, templates, and send preferences |
|
||||
| `audit` | Audit logging and event tracking |
|
||||
| `lens` | Custom lenses and collection views |
|
||||
| `types` | Shared types in `packages/types/` |
|
||||
| `docker` | Dockerfile and Docker Compose files |
|
||||
| `deps` | Dependency upgrades (pair with `build` or `chore`) |
|
||||
| `server` | Backend-wide changes with no single-module scope |
|
||||
| `client` | Frontend-wide changes with no single-module scope |
|
||||
|
||||
### Summary
|
||||
|
||||
The summary is a short, imperative description of what the commit does.
|
||||
|
||||
Rules:
|
||||
|
||||
- Use the imperative mood: "add", "fix", "remove" (not "added" or "adds")
|
||||
- Start with a lowercase letter
|
||||
- No period at the end
|
||||
- Keep it under 72 characters
|
||||
- Describe **what** the commit does, not **why**; that belongs in the body
|
||||
|
||||
---
|
||||
|
||||
## Body
|
||||
|
||||
Write a body when the **why** behind the change isn't obvious from the header alone.
|
||||
|
||||
When to write a body:
|
||||
|
||||
- The motivation isn't clear from the summary
|
||||
- The change has side effects, tradeoffs, or gotchas worth calling out
|
||||
- You're reverting a commit - reference the SHA and explain the reason
|
||||
- The approach taken deserves a brief explanation
|
||||
|
||||
Format:
|
||||
|
||||
- Separate from the header with a blank line
|
||||
- Use the imperative mood, consistent with the summary
|
||||
- Wrap lines at ~100 characters
|
||||
- Explain **why** this change exists, not **what** changed; the diff already shows that
|
||||
|
||||
---
|
||||
|
||||
## Footer
|
||||
|
||||
Use the footer to link related issues or PRs, or to announce breaking changes.
|
||||
|
||||
### Linking issues and PRs
|
||||
|
||||
```
|
||||
Fixes #123
|
||||
Closes #456
|
||||
Ref #789
|
||||
```
|
||||
|
||||
`Fixes` and `Closes` automatically close the linked issue when the PR is merged. Use `Ref` for a related issue that should stay open.
|
||||
|
||||
### Breaking changes
|
||||
|
||||
If the commit introduces a breaking change, open the footer with `BREAKING CHANGE:` followed by a brief summary, a blank line, and clear migration instructions.
|
||||
|
||||
```
|
||||
BREAKING CHANGE: rename BOOKS_PATH env var to LIBRARY_PATH
|
||||
|
||||
Update your .env or Docker environment to use LIBRARY_PATH instead.
|
||||
The old name is no longer recognized and the app will not start without it.
|
||||
```
|
||||
|
||||
Breaking changes must also be explicitly called out in the PR description, including the upgrade steps a user would need to follow.
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### `feat`
|
||||
|
||||
```
|
||||
feat(kobo): sync reading position on device reconnect
|
||||
```
|
||||
|
||||
```
|
||||
feat(opds): add pagination to catalog feed endpoints
|
||||
|
||||
The OPDS feed now supports ?page= and ?limit= query params.
|
||||
Default page size is 20. Clients that don't send pagination params
|
||||
get the same behavior as before.
|
||||
|
||||
Closes #301
|
||||
```
|
||||
|
||||
### `fix`
|
||||
|
||||
```
|
||||
fix(reader): epub chapter navigation skips last page
|
||||
```
|
||||
|
||||
```
|
||||
fix(stats): reading session duration wrong after timezone change
|
||||
|
||||
Duration was computed using local time offsets rather than UTC,
|
||||
causing sessions that crossed a DST boundary to report incorrect
|
||||
elapsed time.
|
||||
|
||||
Fixes #412
|
||||
```
|
||||
|
||||
### `db`
|
||||
|
||||
```
|
||||
db(books): add language column to books table
|
||||
```
|
||||
|
||||
```
|
||||
db(users): add user_preferences table for per-user app settings
|
||||
|
||||
Generates a migration. Existing users get default values on deploy.
|
||||
```
|
||||
|
||||
### `perf`
|
||||
|
||||
```
|
||||
perf(library): avoid redundant file stat calls during rescan
|
||||
```
|
||||
|
||||
### `refactor`
|
||||
|
||||
```
|
||||
refactor(metadata): extract provider selection into its own service
|
||||
```
|
||||
|
||||
### `style`
|
||||
|
||||
```
|
||||
style(reader): tighten chapter list spacing in sidebar
|
||||
```
|
||||
|
||||
### `docs`
|
||||
|
||||
```
|
||||
docs: add Kobo device setup guide to README
|
||||
```
|
||||
|
||||
### `test`
|
||||
|
||||
```
|
||||
test(collections): cover concurrent shelf update race condition
|
||||
```
|
||||
|
||||
### `build`
|
||||
|
||||
```
|
||||
build(docker): switch to slim node base image in final stage
|
||||
```
|
||||
|
||||
### `ci`
|
||||
|
||||
```
|
||||
ci: add e2e smoke suite to pull request workflow
|
||||
```
|
||||
|
||||
### `chore`
|
||||
|
||||
```
|
||||
chore(deps): upgrade drizzle-orm to 0.38
|
||||
```
|
||||
|
||||
### `security`
|
||||
|
||||
```
|
||||
security(auth): reject tokens signed with legacy algorithm
|
||||
|
||||
Previous tokens signed with HS256 were still accepted after the
|
||||
migration to RS256. This change enforces RS256-only validation.
|
||||
|
||||
BREAKING CHANGE: all existing sessions will be invalidated on deploy.
|
||||
Users will need to log in again.
|
||||
```
|
||||
|
||||
### `revert`
|
||||
|
||||
```
|
||||
revert: revert "feat(kobo): sync reading position on device reconnect"
|
||||
|
||||
Reverts commit a1b2c3d.
|
||||
|
||||
Broke sync for devices running firmware < 4.32. Reverting until
|
||||
a compatibility shim is in place.
|
||||
```
|
||||
+190
-200
@@ -1,76 +1,66 @@
|
||||
# Contributing to ProjectX
|
||||
|
||||
Thanks for your interest in contributing! Whether you're fixing a bug, suggesting a feature, or improving docs, every contribution is appreciated.
|
||||
Thanks for your interest in contributing to ProjectX. Whether you are fixing a typo or building a major feature, this guide walks you through the process from start to finish.
|
||||
|
||||
## What is ProjectX?
|
||||
**Companion docs:**
|
||||
|
||||
ProjectX is a self-hostable digital library for managing and reading books and comics, with first-class Kobo device support.
|
||||
|
||||
**Tech stack:**
|
||||
|
||||
- **Frontend:** Vue 3, TypeScript, Tailwind CSS
|
||||
- **Backend:** NestJS, Fastify, Drizzle ORM
|
||||
- **Database:** PostgreSQL
|
||||
- **Deployment:** Docker
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
projectx/
|
||||
├── client/ # Vue 3 frontend
|
||||
├── server/ # NestJS backend
|
||||
│ └── src/
|
||||
│ ├── modules/ # Feature modules (one per domain)
|
||||
│ ├── db/ # Drizzle schema, migrations
|
||||
│ └── common/ # Guards, filters, decorators
|
||||
└── packages/
|
||||
└── types/ # Shared types between client and server
|
||||
```
|
||||
|
||||
> **Important:** Types shared between the frontend and backend live in `packages/types/` and are imported via the `@projectx/types` alias. Never duplicate shared types in both workspaces. Always add them here.
|
||||
|
||||
---
|
||||
|
||||
## Before You Start
|
||||
|
||||
> **Issue first, PR second.** Every pull request must be linked to an approved issue. If you want to work on something, open an issue and wait for a maintainer to explicitly approve it before writing any code. A thumbs up reaction or a comment saying "go ahead" on the issue counts as approval. PRs submitted without a linked, approved issue will be closed.
|
||||
|
||||
This protects your time and ours: it ensures the work is wanted and you're heading in the right direction before investing effort.
|
||||
|
||||
To set your PR up for success, make sure:
|
||||
|
||||
- It is linked to an approved issue
|
||||
- It includes proof of manual testing (screenshots, screen recording, or test output)
|
||||
- It contains a single focused change, no bundled unrelated work
|
||||
- It is reasonably sized. PRs with 1000+ changed lines are hard to review. Split them up.
|
||||
- No new dependencies are added without prior discussion
|
||||
|
||||
If you're unsure whether something fits the project's direction, open an issue and ask before building it.
|
||||
- [DEVELOPMENT.md](DEVELOPMENT.md) - Local setup, architecture, commands, and technical reference.
|
||||
- [TESTING.md](TESTING.md) - Test architecture, E2E suites, coverage thresholds, and harness details.
|
||||
- [COMMIT_GUIDELINES.md](COMMIT_GUIDELINES.md) - Commit message format, types, scopes, and examples.
|
||||
|
||||
---
|
||||
|
||||
## Where to Start
|
||||
|
||||
Not sure where to begin? Look for issues labeled:
|
||||
Not sure what to work on? Look for issues tagged with:
|
||||
|
||||
- [`good first issue`](../../labels/good%20first%20issue) - small, well-scoped tasks ideal for newcomers
|
||||
- [`help wanted`](../../labels/help%20wanted) - tasks where a contribution would be especially welcome
|
||||
- [`good first issue`](../../labels/good%20first%20issue) - Scoped tasks with clear acceptance criteria. Great for your first contribution.
|
||||
- [`help wanted`](../../labels/help%20wanted) - Larger tasks where maintainer bandwidth is limited.
|
||||
|
||||
If nothing there catches your eye, browse the open issues or propose something new.
|
||||
|
||||
### Documentation Contributions
|
||||
|
||||
Doc improvements are always welcome and have a lighter process:
|
||||
|
||||
- No tests required.
|
||||
- Use the `docs/` branch prefix (e.g. `docs/fix-setup-instructions`).
|
||||
- Focus areas: fixing inaccuracies, improving examples, adding missing guides, clarifying confusing sections.
|
||||
|
||||
The rest of this guide still applies (issue first, PR template, etc.), but expect a faster review cycle.
|
||||
|
||||
---
|
||||
|
||||
## Security Issues
|
||||
|
||||
Please do not open public issues for security vulnerabilities. See [SECURITY.md](.github/SECURITY.md) for how to report them privately.
|
||||
Do **not** open public issues for security vulnerabilities. Use GitHub's [private vulnerability reporting](../../security/advisories/new) instead. See [SECURITY.md](.github/SECURITY.md) for details.
|
||||
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
## Contribution Workflow
|
||||
|
||||
### Fork and Clone
|
||||
This is the end-to-end journey from idea to merged PR. Each phase builds on the previous one.
|
||||
|
||||
[Fork the repository](../../fork) on GitHub, then clone your fork locally:
|
||||
### Phase 1: Find or Create an Issue
|
||||
|
||||
> **Issue first, PR second.** Every pull request must link to an approved issue. PRs without one will be closed.
|
||||
|
||||
- Browse [open issues](../../issues) for something you want to tackle.
|
||||
- If you have a new idea, open one using the [bug report](../../issues/new?template=bug_report.yml) or [feature request](../../issues/new?template=feature_request.yml) template.
|
||||
- Not sure if your idea fits the roadmap? Ask in a new issue before investing time.
|
||||
|
||||
### Phase 2: Get Maintainer Approval
|
||||
|
||||
Wait for explicit approval before writing code. Approval looks like:
|
||||
|
||||
- A :+1: reaction on your issue.
|
||||
- A comment such as "go ahead", "approved", or "let's do it".
|
||||
|
||||
This step prevents wasted effort on changes that conflict with the project direction. Small bug fixes with clear reproduction are usually approved quickly.
|
||||
|
||||
### Phase 3: Fork and Clone
|
||||
|
||||
[Fork the repository](../../fork), then clone your fork:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/<your-username>/projectx.git
|
||||
@@ -78,9 +68,127 @@ cd projectx
|
||||
git remote add upstream https://github.com/neonsolstice/projectx.git
|
||||
```
|
||||
|
||||
### Keep Your Fork in Sync
|
||||
> **Trusted contributors** with direct push access can skip the fork and work on branches in the main repository. The rest of this guide assumes the fork workflow.
|
||||
|
||||
Before starting new work, pull the latest changes from upstream:
|
||||
### Phase 4: Set Up Your Environment
|
||||
|
||||
If this is your first time:
|
||||
|
||||
```bash
|
||||
pnpm setup
|
||||
```
|
||||
|
||||
This handles everything: dependencies, PostgreSQL, migrations, and seed data. See [DEVELOPMENT.md](DEVELOPMENT.md) for prerequisites, manual setup steps, and troubleshooting.
|
||||
|
||||
### Phase 5: Create Your Branch
|
||||
|
||||
Always branch from the latest `main`:
|
||||
|
||||
```bash
|
||||
git fetch upstream
|
||||
git checkout -b <prefix>/<short-description> upstream/main
|
||||
```
|
||||
|
||||
Use these branch prefixes:
|
||||
|
||||
| Prefix | Use for | Example |
|
||||
| ----------- | -------------------------------------- | -------------------------------- |
|
||||
| `feat/` | New features | `feat/reading-progress-sync` |
|
||||
| `fix/` | Bug fixes | `fix/pdf-page-count-off-by-one` |
|
||||
| `refactor/` | Code restructuring | `refactor/library-scan-pipeline` |
|
||||
| `docs/` | Documentation | `docs/opds-setup-guide` |
|
||||
| `chore/` | Maintenance (deps, tooling, CI, build) | `chore/bump-drizzle-v1` |
|
||||
|
||||
### Phase 6: Implement Your Change
|
||||
|
||||
Start the dev server and get to work:
|
||||
|
||||
```bash
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
Refer to [DEVELOPMENT.md](DEVELOPMENT.md) for architecture details, project structure, database workflows, and conventions. A few ground rules:
|
||||
|
||||
- **One logical change per PR.** Do not bundle a bug fix with a refactor or unrelated cleanup.
|
||||
- **No unapproved dependencies.** Propose new dependencies in the linked issue first (see [Adding Dependencies](#adding-dependencies)).
|
||||
- **Keep the scope tight.** If you discover something unrelated that needs fixing, open a separate issue for it.
|
||||
|
||||
### Phase 7: Write and Run Tests
|
||||
|
||||
Testing expectations depend on what you changed:
|
||||
|
||||
| Change type | What is expected |
|
||||
| ----------------------- | ----------------------------------------------- |
|
||||
| Bug fix | Regression test proving the bug is fixed |
|
||||
| New backend feature/API | Server unit tests for the new behavior |
|
||||
| New frontend logic | Client unit tests (composables, utilities) |
|
||||
| UI-only change | Manual verification evidence (screenshot/video) |
|
||||
| Refactor | Existing tests stay green, no new tests needed |
|
||||
|
||||
Run the full test suite:
|
||||
|
||||
```bash
|
||||
pnpm test
|
||||
```
|
||||
|
||||
If your change touches an area covered by an existing E2E suite, run it locally before pushing: `pnpm run e2e:run -- <suite-id>`. Use `pnpm run e2e:list` to see available suites.
|
||||
|
||||
For E2E tests, suite IDs, coverage thresholds, and the test harness, see [TESTING.md](TESTING.md).
|
||||
|
||||
### Phase 8: Verify Code Quality
|
||||
|
||||
The project has two automated gates that catch issues before they reach CI:
|
||||
|
||||
- **Pre-commit hook** runs `lint-staged` on your staged files (ESLint + Prettier).
|
||||
- **Pre-push hook** runs `pnpm verify:fast` (lint + typecheck + tests). Your push is blocked if this fails.
|
||||
|
||||
Before marking your PR ready for review, run the full quality gate yourself:
|
||||
|
||||
```bash
|
||||
pnpm verify
|
||||
```
|
||||
|
||||
See the [Code Quality](DEVELOPMENT.md#code-quality) section in DEVELOPMENT.md for individual commands.
|
||||
|
||||
### Phase 9: Commit Your Work
|
||||
|
||||
Follow the format in [COMMIT_GUIDELINES.md](COMMIT_GUIDELINES.md). The short version:
|
||||
|
||||
```
|
||||
<type>(<scope>): <summary>
|
||||
```
|
||||
|
||||
- **Type:** `feat`, `fix`, `db`, `perf`, `refactor`, `style`, `docs`, `test`, `build`, `ci`, `chore`, `security`, `revert`
|
||||
- **Scope** (optional): `auth`, `books`, `library`, `metadata`, `kobo`, `scanner`, etc.
|
||||
- **Summary:** Imperative mood, lowercase, no period. Describe what the commit does, not what you did.
|
||||
|
||||
Examples: `feat(kobo): add shelf sync endpoint`, `fix(reader): correct page count for multi-volume PDFs`
|
||||
|
||||
### Phase 10: Open a Pull Request
|
||||
|
||||
Push your branch and open a PR against `main`:
|
||||
|
||||
```bash
|
||||
git push origin feat/your-feature-name
|
||||
```
|
||||
|
||||
When creating the PR:
|
||||
|
||||
- **Fill out the PR template.** It asks for a summary, testing evidence, screenshots (if UI changed), and non-obvious decisions.
|
||||
- **Link your issue** with `Closes #123` in the description.
|
||||
- **Open as a draft** if you want early direction feedback before the code is complete.
|
||||
- **Disclose AI usage** if applicable. See [AI_POLICY.md](AI_POLICY.md) for the format and expectations.
|
||||
|
||||
### Phase 11: Respond to Review
|
||||
|
||||
- Expect feedback within a few days, sometimes sooner.
|
||||
- Push additional commits to address review comments. Do not force-push during review unless a reviewer asks you to.
|
||||
- Do not resolve review threads yourself. Let the reviewer confirm the fix.
|
||||
- If feedback is unclear, ask for clarification in the thread.
|
||||
|
||||
### Phase 12: After Merge
|
||||
|
||||
Your changes ship in a future release when maintainers cut one. Keep your fork in sync for your next contribution:
|
||||
|
||||
```bash
|
||||
git fetch upstream
|
||||
@@ -91,167 +199,49 @@ git push origin main
|
||||
|
||||
---
|
||||
|
||||
## Development Setup
|
||||
## Quick-Reference Checklist
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Node.js >= 24
|
||||
- pnpm >= 9
|
||||
- Docker
|
||||
|
||||
### First-time setup
|
||||
|
||||
```bash
|
||||
docker compose up -d # start PostgreSQL
|
||||
cp server/.env.example server/.env
|
||||
pnpm install
|
||||
cd server && pnpm db:migrate
|
||||
```
|
||||
|
||||
### Daily workflow
|
||||
|
||||
```bash
|
||||
pnpm dev # start server + client
|
||||
pnpm verify # run all checks before pushing
|
||||
```
|
||||
|
||||
Other useful commands:
|
||||
|
||||
```bash
|
||||
pnpm verify:fast # lint + typecheck + tests (pre-push gate)
|
||||
pnpm db:reset # wipe and reseed local database
|
||||
pnpm e2e:list # list available e2e suite ids
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Branch Naming
|
||||
|
||||
Create branches from `main` using these prefixes:
|
||||
|
||||
| Prefix | Use for | Example |
|
||||
| ----------- | ------------------ | -------------------------------- |
|
||||
| `feat/` | New features | `feat/reading-progress-sync` |
|
||||
| `fix/` | Bug fixes | `fix/pdf-page-count-off-by-one` |
|
||||
| `refactor/` | Code restructuring | `refactor/library-scan-pipeline` |
|
||||
| `docs/` | Documentation | `docs/opds-setup-guide` |
|
||||
|
||||
---
|
||||
|
||||
## Commit Messages
|
||||
|
||||
We follow [Conventional Commits](https://www.conventionalcommits.org/):
|
||||
|
||||
```
|
||||
<type>(<scope>): <subject>
|
||||
```
|
||||
|
||||
**Types:** `feat`, `fix`, `docs`, `refactor`, `test`, `chore`
|
||||
|
||||
Examples:
|
||||
|
||||
```
|
||||
feat(library): add bulk metadata refresh for selected books
|
||||
fix(cbz): handle archives with no image files gracefully
|
||||
feat(opds): add pagination support to feed endpoints
|
||||
fix(stats): reading session duration calculation off after timezone change
|
||||
chore(deps): upgrade drizzle-orm to 0.31
|
||||
test(shelf): add coverage for concurrent shelf update race condition
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing Expectations
|
||||
|
||||
`pnpm verify` must pass, but that is the minimum, not the goal. Here is what is expected per change type:
|
||||
|
||||
| Change type | Expected tests |
|
||||
| ------------------- | -------------------------------------------------------------------------- |
|
||||
| Bug fix | A regression test that would have caught the bug |
|
||||
| New backend feature | Unit tests for the service, covering the main logic paths |
|
||||
| New API endpoint | At minimum, a test for the happy path and error cases |
|
||||
| New composable | Unit tests covering the reactive logic |
|
||||
| UI-only change | Manual testing with screenshot evidence in the PR |
|
||||
| Refactor | Existing tests must continue to pass. Add none if coverage stays the same. |
|
||||
|
||||
AI-generated tests deserve extra scrutiny. A test that always passes regardless of the implementation is worse than no test at all.
|
||||
|
||||
---
|
||||
|
||||
## Adding Dependencies
|
||||
|
||||
Do not add new dependencies without prior discussion in the linked issue. Every new dependency is a security surface, a bundle size cost, and a long-term maintenance burden.
|
||||
|
||||
If a dependency is needed, explain in the issue: what it does, why an existing dependency or a small custom implementation won't work, and how actively it is maintained.
|
||||
|
||||
---
|
||||
|
||||
## Breaking Changes and Self-Hosted Upgrades
|
||||
|
||||
ProjectX users upgrade by pulling a new Docker image. A change that breaks an existing deployment without a documented upgrade path will not be merged until that path is clearly defined.
|
||||
|
||||
If your PR introduces a breaking change (schema migration, removed API field, changed env var, modified config format):
|
||||
|
||||
- Call it out explicitly in the PR description
|
||||
- Include the upgrade steps a user would need to follow
|
||||
- For schema changes, make sure a Drizzle migration is included. Never hand-write SQL.
|
||||
|
||||
---
|
||||
|
||||
## Submitting a Pull Request
|
||||
|
||||
Use a **draft PR** early if you want feedback on direction before the work is done. It signals the PR is not ready for review and avoids back-and-forth on a finished implementation that needs to be rethought.
|
||||
|
||||
The project has two automatic gates you should be aware of:
|
||||
|
||||
- **pre-commit:** runs `lint-staged` on your staged files. ESLint and Prettier are applied automatically. You do not need to format or lint manually before committing.
|
||||
- **pre-push:** runs `pnpm verify:fast` (lint + typecheck + tests). If this fails, the push is blocked. Fix the errors before pushing.
|
||||
|
||||
`pnpm verify:fast` is a fast check but does not include the `app-smoke` e2e suite. Before opening a PR, run the full gate:
|
||||
|
||||
```bash
|
||||
pnpm verify
|
||||
```
|
||||
|
||||
Before marking your PR ready for review:
|
||||
Before marking your PR ready for review, confirm:
|
||||
|
||||
- [ ] Linked to an approved issue
|
||||
- [ ] `pnpm verify` passes locally (includes `app-smoke`, stricter than pre-push)
|
||||
- [ ] You ran the full stack and manually verified the change works
|
||||
- [ ] UI changes include a screenshot or screen recording
|
||||
- [ ] Tests are included per the expectations above
|
||||
- [ ] No new dependencies added without prior discussion
|
||||
- [ ] Branch is up to date with `main`
|
||||
- [ ] The PR contains a single logical change
|
||||
|
||||
When you open your PR, a template will appear. Fill it out completely.
|
||||
|
||||
**Responding to feedback:** Address all review comments before asking for a re-review. Don't just resolve threads without fixing the underlying issue. PRs with no activity for 30 days after a review request will be closed. You are welcome to reopen when ready.
|
||||
- [ ] One logical change per PR
|
||||
- [ ] `pnpm verify` passes locally
|
||||
- [ ] Tests included per the [testing expectations table](#phase-7-write-and-run-tests)
|
||||
- [ ] Full-stack behavior manually validated
|
||||
- [ ] UI changes include screenshot or recording
|
||||
- [ ] PR template fully completed
|
||||
- [ ] No unintended files (build artifacts, `.env`, personal configs)
|
||||
- [ ] No unapproved new dependencies
|
||||
|
||||
---
|
||||
|
||||
## AI-Assisted Contributions
|
||||
## Policies
|
||||
|
||||
Contributions using AI tools are welcome, but the quality bar is the same as human-written code. **If you ship it, you own it.**
|
||||
### Adding Dependencies
|
||||
|
||||
PRs where the contributor clearly never ran the code, didn't test it, or can't explain what it does make review very difficult and will be closed without detailed feedback.
|
||||
Do not add dependencies without prior discussion in the linked issue. If proposing one, document:
|
||||
|
||||
**If you use AI to help write code, you must still:**
|
||||
- What problem it solves.
|
||||
- Why existing dependencies or a small custom implementation are insufficient.
|
||||
- Maintenance status and community activity of the package.
|
||||
|
||||
- **Run the code yourself.** Start the full stack and manually verify the change works. Trusting AI output without running it is not acceptable.
|
||||
- **Review every line.** You must be able to explain any part of your change during review. "The AI suggested it" is not an answer.
|
||||
- **Keep PRs focused.** One feature or fix per PR. Do not submit a dump of everything the AI generated.
|
||||
- **Scrutinize AI-generated tests.** They often pass trivially without asserting anything meaningful. Tests that don't validate real behavior will be rejected.
|
||||
- **Clean up.** Remove dead code, placeholder comments, empty catch blocks, and unnecessary boilerplate.
|
||||
### Breaking Changes and Self-Hosted Upgrades
|
||||
|
||||
---
|
||||
ProjectX is self-hosted. Any change that can break existing deployments must include a clear upgrade path before merge.
|
||||
|
||||
## What Happens After Merge
|
||||
If your PR introduces a breaking change (schema, API contract, environment variable, config format):
|
||||
|
||||
Merged changes ship with the next release. There is no fixed release cadence. Releases happen when enough meaningful changes have accumulated. If your fix is urgent, mention it in the PR and it can be prioritized.
|
||||
- Call it out explicitly in the PR description.
|
||||
- Document the required upgrade steps.
|
||||
- Include Drizzle migrations for schema changes. Never hand-write migration SQL.
|
||||
|
||||
### AI-Assisted Contributions
|
||||
|
||||
See [AI_POLICY.md](AI_POLICY.md). The short version: disclose all AI usage in your PR description and make sure you understand every line of the diff.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
By contributing, you agree that your contributions will be licensed under the same license as this project.
|
||||
By contributing, you agree that your contributions are licensed under the same license as the project (AGPL-3.0).
|
||||
|
||||
+445
-205
@@ -1,31 +1,171 @@
|
||||
# Development Guide
|
||||
|
||||
## Prerequisites
|
||||
Everything you need to work in this codebase: architecture, setup, workflows, conventions, and commands.
|
||||
|
||||
- Node.js >= 24
|
||||
- pnpm >= 9
|
||||
- Docker
|
||||
For the contribution process (issues, PRs, review), see [CONTRIBUTING.md](CONTRIBUTING.md). For test architecture and E2E suite details, see [TESTING.md](TESTING.md).
|
||||
|
||||
---
|
||||
|
||||
## First-time Setup
|
||||
## Architecture Overview
|
||||
|
||||
ProjectX is a pnpm monorepo with three workspaces: a Vue 3 single-page app, a NestJS API server, and a shared types package.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
Client["Client<br/><small>Vue 3 SPA</small><br/><small>localhost:5173</small>"]
|
||||
Server["Server<br/><small>NestJS + Fastify</small><br/><small>localhost:3000/api</small>"]
|
||||
DB["PostgreSQL<br/><small>Drizzle ORM</small>"]
|
||||
WS["WebSocket<br/><small>Socket.IO</small>"]
|
||||
Types["Shared Types<br/><small>@projectx/types</small>"]
|
||||
|
||||
Client -- "HTTP (fetch)" --> Server
|
||||
Client <-. "real-time events" .-> WS
|
||||
Server --> DB
|
||||
Server --> WS
|
||||
Types -.- Client
|
||||
Types -.- Server
|
||||
```
|
||||
|
||||
| Layer | Tech | Location |
|
||||
| ------------ | ------------------------------- | ----------------- |
|
||||
| Frontend | Vue 3, Tailwind CSS v4, Vite | `client/` |
|
||||
| Backend | NestJS 11, Fastify, Drizzle ORM | `server/` |
|
||||
| Database | PostgreSQL 16, pgvector | Docker container |
|
||||
| Shared types | TypeScript | `packages/types/` |
|
||||
| Real-time | Socket.IO | Server + Client |
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```text
|
||||
projectx/
|
||||
├── client/ # Vue 3 frontend
|
||||
│ └── src/
|
||||
│ ├── features/ # Feature modules (composables + components)
|
||||
│ ├── components/ui/ # Shared UI components
|
||||
│ ├── lib/ # API client, utilities
|
||||
│ ├── router/ # Vue Router config
|
||||
│ └── assets/ # Tailwind theme, static assets
|
||||
├── server/ # NestJS backend
|
||||
│ └── src/
|
||||
│ ├── modules/ # Feature modules (controller + service + dto)
|
||||
│ ├── db/
|
||||
│ │ ├── schema/ # Drizzle table definitions (one file per domain)
|
||||
│ │ └── migrations/ # Generated SQL migrations
|
||||
│ ├── common/ # Guards, filters, decorators, pipes
|
||||
│ └── config/ # Typed config (app, db, auth)
|
||||
│ └── test/
|
||||
│ ├── e2e/ # E2E test helpers and harness
|
||||
│ └── *.e2e-spec.ts # E2E test suites
|
||||
├── packages/
|
||||
│ └── types/ # Shared TypeScript types
|
||||
│ └── src/ # Exported via @projectx/types
|
||||
├── scripts/ # Bootstrap, DB reset, E2E orchestration
|
||||
├── docker/ # Postgres init scripts (extensions)
|
||||
├── .github/ # CI workflows, PR/issue templates
|
||||
└── .husky/ # Git hooks (pre-commit, pre-push)
|
||||
```
|
||||
|
||||
> **Shared types** live in `packages/types/` and are imported as `@projectx/types`. Never duplicate types between client and server.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Node.js** >= 24 - [nodejs.org](https://nodejs.org)
|
||||
- **pnpm** >= 9 - `npm install -g pnpm` (or see [pnpm.io](https://pnpm.io/installation))
|
||||
- **Docker** - [docs.docker.com/get-started/get-docker](https://docs.docker.com/get-started/get-docker/)
|
||||
|
||||
---
|
||||
|
||||
## First-Time Setup
|
||||
|
||||
### Option A: One Command
|
||||
|
||||
```bash
|
||||
pnpm setup
|
||||
```
|
||||
|
||||
This creates `server/.env` from the example, installs all dependencies, starts PostgreSQL, runs migrations, and seeds baseline data.
|
||||
This runs all the steps below automatically. If it succeeds, skip to [Running the App](#running-the-app).
|
||||
|
||||
If you prefer to do it manually:
|
||||
---
|
||||
|
||||
### Option B: Step by Step
|
||||
|
||||
#### Step 1 - Clone the repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/<your-username>/projectx.git
|
||||
cd projectx
|
||||
```
|
||||
|
||||
If you are an external contributor, fork first and clone your fork. See [CONTRIBUTING.md](CONTRIBUTING.md) for details.
|
||||
|
||||
#### Step 2 - Create your environment file
|
||||
|
||||
```bash
|
||||
cp server/.env.example server/.env
|
||||
```
|
||||
|
||||
Only do this once. If `server/.env` already exists, skip this step. Copying again will overwrite any local changes you have made.
|
||||
|
||||
The defaults in `.env.example` work out of the box for local development. You only need to edit this file if you want to change ports, point to a different database, or configure optional features like email.
|
||||
|
||||
See the [Key Environment Variables](#key-environment-variables) section for what each variable does.
|
||||
|
||||
#### Step 3 - Install dependencies
|
||||
|
||||
```bash
|
||||
pnpm run db:up # start PostgreSQL
|
||||
cp server/.env.example server/.env # first time only
|
||||
pnpm install
|
||||
```
|
||||
|
||||
This installs dependencies for all three workspaces (server, client, packages/types) in one shot.
|
||||
|
||||
#### Step 4 - Start PostgreSQL
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.dev.yml up -d --wait
|
||||
```
|
||||
|
||||
This starts a PostgreSQL 16 container (with the `pgvector` extension) on port 5432. The `--wait` flag blocks until the database is healthy. Data is persisted in a named Docker volume across restarts.
|
||||
|
||||
> **Already have PostgreSQL running?** You can point `DATABASE_URL` in `server/.env` at your existing instance, but you will still need the `uuid-ossp`, `pg_trgm`, and `vector` extensions installed. Using Docker is the easiest path.
|
||||
|
||||
#### Step 5 - Apply database migrations
|
||||
|
||||
```bash
|
||||
pnpm run db:migrate
|
||||
```
|
||||
|
||||
This creates all tables and indexes. Migrations are generated by Drizzle Kit and live in `server/src/db/migrations/`.
|
||||
|
||||
#### Step 6 - Seed baseline data
|
||||
|
||||
```bash
|
||||
pnpm run db:seed
|
||||
```
|
||||
|
||||
This populates the database with the data required for the app to function (default roles, permissions, initial settings). You only need to run this once.
|
||||
|
||||
#### Step 7 - Start the app
|
||||
|
||||
```bash
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
If everything worked, you will see the server, client, and types watcher start up. The app is ready when you see output from all three processes.
|
||||
|
||||
---
|
||||
|
||||
### Verifying Your Setup
|
||||
|
||||
| Check | Expected |
|
||||
| ---------------------------------------- | ------------------------------------------- |
|
||||
| Open http://localhost:5173 | ProjectX login page loads |
|
||||
| Open http://localhost:3000/api/v1/health | Returns `{"status":"ok"}` |
|
||||
| `docker ps` | Shows `projectx-postgres` container running |
|
||||
|
||||
---
|
||||
|
||||
## Running the App
|
||||
@@ -34,226 +174,101 @@ pnpm run db:seed
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
Starts the server and client concurrently with hot reload.
|
||||
This starts the server, client, and types watcher concurrently with hot reload.
|
||||
|
||||
| Service | URL |
|
||||
| ------- | ------------------------- |
|
||||
| API | http://localhost:3000/api |
|
||||
| Client | http://localhost:5173 |
|
||||
| API | http://localhost:3000/api |
|
||||
|
||||
If PostgreSQL stopped (e.g. after a machine restart):
|
||||
If PostgreSQL is not running, start it first with `pnpm run db:up`.
|
||||
|
||||
<details>
|
||||
<summary>Running workspaces separately</summary>
|
||||
|
||||
```bash
|
||||
pnpm run db:up # then pnpm dev
|
||||
# Terminal 1: Server
|
||||
cd server && pnpm start:dev
|
||||
|
||||
# Terminal 2: Client
|
||||
cd client && pnpm dev
|
||||
```
|
||||
|
||||
If you are editing shared types (`packages/types/`), also start the types watcher:
|
||||
|
||||
```bash
|
||||
# Terminal 3: Types (only needed when editing packages/types/)
|
||||
pnpm --filter @projectx/types dev
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## Database Workflows
|
||||
## Key Environment Variables
|
||||
|
||||
### Apply pending migrations
|
||||
These are the variables you are most likely to need during development. See `server/.env.example` for the full list with comments.
|
||||
|
||||
After pulling changes that include new migrations:
|
||||
| Variable | Purpose | Default (dev) |
|
||||
| --------------- | ---------------------------------- | ------------------------------------------------------ |
|
||||
| `DATABASE_URL` | PostgreSQL connection string | `postgres://projectx:projectx@localhost:5432/projectx` |
|
||||
| `PORT` | Server listen port | `3000` |
|
||||
| `NODE_ENV` | Runtime mode | `development` |
|
||||
| `JWT_SECRET` | Signing key for auth tokens | `change-me-in-production` |
|
||||
| `APP_DATA_PATH` | Storage for covers, avatars, cache | `../local/data` |
|
||||
| `APP_URL` | Base URL for email links | `http://localhost:5173` |
|
||||
|
||||
---
|
||||
|
||||
## Database
|
||||
|
||||
### Schema Organization
|
||||
|
||||
Database tables are defined as Drizzle `pgTable()` calls in `server/src/db/schema/`, one file per domain (e.g. `books.ts`, `auth.ts`, `kobo.ts`). All files are re-exported from `server/src/db/schema/index.ts`.
|
||||
|
||||
Type inference uses `typeof table.$inferSelect` and `$inferInsert`. Never write manual type aliases for DB rows.
|
||||
|
||||
### Common Commands
|
||||
|
||||
| Command | What it does |
|
||||
| -------------------------------------- | ----------------------------------------------- |
|
||||
| `pnpm run db:up` | Start PostgreSQL via Docker |
|
||||
| `pnpm run db:migrate` | Apply pending migrations |
|
||||
| `cd server && pnpm db:generate <name>` | Generate a migration from schema changes |
|
||||
| `pnpm run db:seed` | Seed baseline data |
|
||||
| `pnpm run db:reset` | Drop everything, re-migrate, re-seed (dev only) |
|
||||
| `cd server && pnpm db:studio` | Open Drizzle Studio (visual DB browser) |
|
||||
|
||||
### Making a Schema Change
|
||||
|
||||
1. Edit the relevant file in `server/src/db/schema/`.
|
||||
2. Generate a migration: `cd server && pnpm db:generate describe-your-change`
|
||||
3. Review the generated SQL in `server/src/db/migrations/`.
|
||||
4. Apply it: `pnpm run db:migrate`
|
||||
|
||||
Never hand-write migration SQL. Always let Drizzle Kit generate it from schema diffs.
|
||||
|
||||
### Migration Strategy
|
||||
|
||||
**Before production launch (current phase):** We maintain a single baseline migration (`0000_*.sql`). When adding new tables or columns, wipe the migrations folder and regenerate:
|
||||
|
||||
```bash
|
||||
pnpm run db:migrate
|
||||
rm -rf server/src/db/migrations
|
||||
docker compose -f docker-compose.dev.yml down -v && docker compose -f docker-compose.dev.yml up -d --wait
|
||||
cd server && pnpm db:generate baseline && pnpm run db:migrate
|
||||
```
|
||||
|
||||
### Add a schema change
|
||||
**After production launch:** Never wipe migrations. Every schema change gets its own incremental migration.
|
||||
|
||||
1. Edit the relevant file in `server/src/db/schema/`
|
||||
2. Generate a migration:
|
||||
```bash
|
||||
cd server && pnpm db:generate add-book-tags
|
||||
```
|
||||
This creates a new SQL file in `server/src/db/migrations/`. Review it before applying.
|
||||
3. Apply it:
|
||||
```bash
|
||||
pnpm run db:migrate
|
||||
```
|
||||
### Full Reset
|
||||
|
||||
> **Pre-launch rule:** Before the app ships, maintain a single baseline migration (`0000_*.sql`) rather than stacking incremental ones. When adding a schema change pre-launch:
|
||||
>
|
||||
> ```bash
|
||||
> rm -rf server/src/db/migrations
|
||||
> docker compose -f docker-compose.dev.yml down -v
|
||||
> pnpm run db:up
|
||||
> cd server && pnpm db:generate baseline
|
||||
> pnpm run db:migrate
|
||||
> ```
|
||||
>
|
||||
> After go-live, never wipe migrations. Every change gets its own incremental migration.
|
||||
|
||||
### Full database reset
|
||||
|
||||
Wipes all data, re-runs migrations, re-seeds, and clears generated files (covers, author images):
|
||||
When local migration state is broken or you need a clean slate:
|
||||
|
||||
```bash
|
||||
pnpm run db:reset
|
||||
```
|
||||
|
||||
Use this when:
|
||||
|
||||
- Migrations are in a broken or inconsistent state
|
||||
- A migration requires starting from scratch (e.g. column type change with no clean upgrade path)
|
||||
- You want a known-clean local state
|
||||
|
||||
### Inspect the database
|
||||
|
||||
```bash
|
||||
cd server && pnpm db:studio
|
||||
```
|
||||
|
||||
Opens Drizzle Studio in your browser for browsing and editing tables directly.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
pnpm run test # all tests (server + client)
|
||||
pnpm run test:server # server unit tests only
|
||||
pnpm run test:client # client unit tests only
|
||||
pnpm run e2e:run -- all # run all e2e suites sequentially
|
||||
pnpm run e2e:run -- app-smoke # app smoke suite (default DB)
|
||||
pnpm run e2e:run -- scanner-scenarios # scanner scenario suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- scanner-file-operations # scanner file operation suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- auth-session-security # auth session security suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- auth-recovery-oidc-logout # auth recovery and OIDC logout suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- book-bucket-ingest-finalize # Book Bucket ingest/finalize suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- metadata-write # metadata write suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- metadata-lock # metadata lock suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- authorization-matrix # authorization matrix suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- opds-auth-catalog # OPDS auth/catalog suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- reader-state-isolation # reader state isolation suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- email-lifecycle # email lifecycle suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:run -- users-admin-lifecycle # users admin lifecycle suite (dedicated projectx_e2e DB)
|
||||
pnpm run e2e:all # shortcut for running all suites
|
||||
pnpm run e2e:list # list supported suite ids
|
||||
```
|
||||
|
||||
Watch mode while working on a specific area:
|
||||
|
||||
```bash
|
||||
cd server && pnpm test:watch
|
||||
```
|
||||
|
||||
Run a focused scanner scenario:
|
||||
|
||||
```bash
|
||||
pnpm run e2e:run -- scanner-scenarios --testNamePattern=book-per-folder-disc-folder-flattening
|
||||
```
|
||||
|
||||
### E2E runner architecture
|
||||
|
||||
- All suite metadata lives in `scripts/e2e/suite-registry.mjs` (suite id, spec file, JUnit output, DB prep mode).
|
||||
- Local and CI use the same command: `pnpm run e2e:run -- <suite-id>`.
|
||||
- Use `all` as a composite suite id to run every configured suite in order, or use `pnpm run e2e:all`.
|
||||
- Dedicated-db suites auto-start local PostgreSQL when `CI != true`, then reset and migrate the e2e database through `pnpm run e2e:db:prepare`.
|
||||
- JUnit outputs are written to `test-results/server/`.
|
||||
|
||||
### E2E suite details
|
||||
|
||||
All suites write JUnit XML to `test-results/server/`. Dedicated-db suites also auto-start PostgreSQL locally (when needed) and use the `projectx_e2e` database.
|
||||
|
||||
| Suite id | DB mode | JUnit output | Extra artifacts |
|
||||
| ----------------------------- | ------------ | ------------------------------------------- | ------------------------------------------------ |
|
||||
| `app-smoke` | default-db | `app-smoke-e2e-junit.xml` | none |
|
||||
| `scanner-scenarios` | dedicated-db | `scanner-scenarios-e2e-junit.xml` | `scanner-scenarios-e2e-scenarios.json` |
|
||||
| `scanner-file-operations` | dedicated-db | `scanner-file-operations-e2e-junit.xml` | `scanner-file-operations-e2e-scenarios.json` |
|
||||
| `auth-session-security` | dedicated-db | `auth-session-security-e2e-junit.xml` | none |
|
||||
| `auth-recovery-oidc-logout` | dedicated-db | `auth-recovery-oidc-logout-e2e-junit.xml` | none |
|
||||
| `book-bucket-ingest-finalize` | dedicated-db | `book-bucket-ingest-finalize-e2e-junit.xml` | `book-bucket-ingest-finalize-e2e-scenarios.json` |
|
||||
| `metadata-write` | dedicated-db | `metadata-write-e2e-junit.xml` | `metadata-write-e2e-scenarios.json` |
|
||||
| `metadata-lock` | dedicated-db | `metadata-lock-e2e-junit.xml` | none |
|
||||
| `authorization-matrix` | dedicated-db | `authorization-matrix-e2e-junit.xml` | none |
|
||||
| `opds-auth-catalog` | dedicated-db | `opds-auth-catalog-e2e-junit.xml` | `opds-auth-catalog-e2e-scenarios.json` |
|
||||
| `reader-state-isolation` | dedicated-db | `reader-state-isolation-e2e-junit.xml` | `reader-state-isolation-e2e-scenarios.json` |
|
||||
| `email-lifecycle` | dedicated-db | `email-lifecycle-e2e-junit.xml` | `email-lifecycle-e2e-scenarios.json` |
|
||||
| `users-admin-lifecycle` | dedicated-db | `users-admin-lifecycle-e2e-junit.xml` | `users-admin-lifecycle-e2e-scenarios.json` |
|
||||
|
||||
### E2E in CI (how to trigger)
|
||||
|
||||
All suite workflows call the reusable `E2E Runner (reusable)` workflow, which runs:
|
||||
|
||||
```bash
|
||||
pnpm run e2e:run -- <suite-id>
|
||||
```
|
||||
|
||||
| Workflow name | Suite id | Triggered by |
|
||||
| ----------------------------------- | ----------------------------- | --------------------------------------------------------------------------- |
|
||||
| `E2E - App Smoke` | `app-smoke` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Scanner Scenarios` | `scanner-scenarios` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Scanner File Operations` | `scanner-file-operations` | `workflow_dispatch`, nightly schedule (`0 4 * * *`), `push`, `pull_request` |
|
||||
| `E2E - Auth Session Security` | `auth-session-security` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Auth Recovery OIDC Logout` | `auth-recovery-oidc-logout` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Book Bucket Ingest Finalize` | `book-bucket-ingest-finalize` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Metadata Write` | `metadata-write` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Metadata Lock` | `metadata-lock` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Authorization Matrix` | `authorization-matrix` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - OPDS Auth Catalog` | `opds-auth-catalog` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Reader State Isolation` | `reader-state-isolation` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Email Lifecycle` | `email-lifecycle` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
| `E2E - Users Admin Lifecycle` | `users-admin-lifecycle` | `workflow_dispatch`, `push`, `pull_request` |
|
||||
|
||||
Manual trigger steps:
|
||||
|
||||
1. Open GitHub Actions.
|
||||
2. Select any `E2E - ...` workflow for the suite you want to run.
|
||||
3. Click **Run workflow**.
|
||||
|
||||
Each run uploads `test-results/server/` as an artifact and publishes JUnit annotations from `test-results/server/*-e2e-junit.xml`.
|
||||
|
||||
---
|
||||
|
||||
## Code Quality
|
||||
|
||||
Run before pushing:
|
||||
|
||||
```bash
|
||||
pnpm run verify:fast # lint + typecheck + tests (same as pre-push hook)
|
||||
pnpm run verify # above + e2e app-smoke suite (run before opening a PR)
|
||||
```
|
||||
|
||||
Individual checks:
|
||||
|
||||
```bash
|
||||
pnpm run lint:check # check for lint errors
|
||||
pnpm run lint:fix # auto-fix lint errors
|
||||
pnpm run typecheck # server + client baseline typecheck
|
||||
pnpm run typecheck:full # server + full client typecheck (slower)
|
||||
```
|
||||
|
||||
Format:
|
||||
|
||||
```bash
|
||||
cd server && npx prettier --write .
|
||||
cd client && npx prettier --write .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Scenarios
|
||||
|
||||
### Pulled changes that include a new migration
|
||||
|
||||
```bash
|
||||
pnpm run db:migrate
|
||||
```
|
||||
|
||||
### Pulled changes that changed the DB schema significantly and your local state is stale
|
||||
|
||||
```bash
|
||||
pnpm run db:reset
|
||||
```
|
||||
|
||||
### Something is wrong and you want a completely clean slate
|
||||
|
||||
```bash
|
||||
pnpm run db:reset
|
||||
```
|
||||
|
||||
If that is not enough (e.g. Docker volume is corrupted):
|
||||
If that is insufficient (e.g. corrupted Docker volumes), stop the container with `-v` to drop volumes, then start fresh:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.dev.yml down -v
|
||||
@@ -261,3 +276,228 @@ pnpm run db:up
|
||||
pnpm run db:migrate
|
||||
pnpm run db:seed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
| Command | What it runs |
|
||||
| -------------------------------- | -------------------------------- |
|
||||
| `pnpm run test` | Server + client unit tests |
|
||||
| `pnpm run test:server` | Server unit tests only |
|
||||
| `pnpm run test:client` | Client unit tests only |
|
||||
| `cd server && pnpm test:watch` | Server tests in watch mode |
|
||||
| `pnpm run e2e:run -- <suite-id>` | Run a single E2E suite |
|
||||
| `pnpm run e2e:all` | Run all E2E suites sequentially |
|
||||
| `pnpm run e2e:list` | List available E2E suite IDs |
|
||||
| `pnpm run coverage` | Unit tests with coverage reports |
|
||||
|
||||
Server unit tests use Vitest with `@nestjs/testing`. Client tests use Vitest with `@vue/test-utils` in a jsdom environment. Most E2E suites boot the full app against a dedicated test database; a few smoke-style suites run against the dev database.
|
||||
|
||||
For test architecture, suite details, coverage thresholds, the E2E harness, and examples, see [TESTING.md](TESTING.md).
|
||||
|
||||
---
|
||||
|
||||
## Code Quality
|
||||
|
||||
### Quality Gates
|
||||
|
||||
| Command | What it checks | When to use |
|
||||
| ------------------------ | -------------------------------------------- | ---------------------------- |
|
||||
| `pnpm run verify:fast` | Lint + typecheck + tests | Quick check while developing |
|
||||
| `pnpm run verify` | Lint + typecheck + tests (full) | Before opening a PR |
|
||||
| `pnpm run verify:strict` | Format check + lint + full typecheck + tests | Strictest gate |
|
||||
|
||||
### Individual Checks
|
||||
|
||||
| Command | What it does |
|
||||
| ----------------------- | -------------------------------------- |
|
||||
| `pnpm run lint:check` | ESLint across server + client |
|
||||
| `pnpm run lint:fix` | ESLint auto-fix across server + client |
|
||||
| `pnpm run typecheck` | TypeScript type checking |
|
||||
| `pnpm run format` | Prettier format across server + client |
|
||||
| `pnpm run format:check` | Prettier check (no write) |
|
||||
|
||||
### Git Hooks
|
||||
|
||||
Two hooks run automatically via [Husky](https://typicode.github.io/husky/):
|
||||
|
||||
- **Pre-commit:** Runs `lint-staged` on staged files. This auto-fixes lint issues and formats code with Prettier. You do not need to format manually before committing.
|
||||
- **Pre-push:** Runs `pnpm verify:fast`. Your push is blocked if lint, typecheck, or tests fail.
|
||||
|
||||
---
|
||||
|
||||
## Must-Know Conventions
|
||||
|
||||
These are the rules that most commonly cause PR rejections. Following them from the start will save you review cycles.
|
||||
|
||||
### Backend
|
||||
|
||||
- **One module per feature.** Each domain lives in `server/src/modules/<feature>/` with its own controller, service, module, and `dto/` subfolder.
|
||||
- **DTOs for all input.** Request bodies are validated via `class-validator` decorators on DTO classes. The global `ValidationPipe` rejects unknown fields.
|
||||
- **Throw NestJS exceptions.** Use `NotFoundException`, `BadRequestException`, `ForbiddenException`, etc. Never throw raw `Error`.
|
||||
- **Multi-user ownership.** User-owned data needs a `userId` foreign key. Services must check ownership and throw `ForbiddenException` for non-owners.
|
||||
|
||||
### Frontend
|
||||
|
||||
- **Composition API only.** Every component uses `<script setup lang="ts">`. Never use the Options API.
|
||||
- **Bare method references in templates.** Write `@click="handleFoo"`, not `@click="handleFoo()"` or `@click="() => handleFoo()"`. This is enforced by ESLint and will block commits.
|
||||
- **Composables over stores.** Feature-local state lives in composables (`features/<name>/composables/use*.ts`). Use Pinia only when state genuinely needs to be shared app-wide.
|
||||
- **Native fetch only.** HTTP calls use the native `fetch` API. No axios or other HTTP clients.
|
||||
|
||||
---
|
||||
|
||||
## CI/CD Pipeline
|
||||
|
||||
### What Runs on Pull Requests
|
||||
|
||||
The CI workflow (`.github/workflows/ci.yml`) triggers on every PR to `main`:
|
||||
|
||||
1. **Dependency review** - Checks for supply chain issues in new dependencies.
|
||||
2. **Change detection** - Determines which workspaces (server/client) have changed files.
|
||||
3. **Lint** - Runs ESLint and format checks on changed workspaces.
|
||||
4. **Type check** - Runs TypeScript checking on changed workspaces.
|
||||
5. **Tests** - Runs unit tests on changed workspaces with coverage.
|
||||
6. **E2E** - Selects and runs relevant E2E suites based on changed file paths.
|
||||
|
||||
All jobs are conditional: if you only changed client code, server lint/typecheck/tests are skipped.
|
||||
|
||||
### What Runs on Main
|
||||
|
||||
The same pipeline plus:
|
||||
|
||||
- **Container image build** (`.github/workflows/container-image.yml`) - Builds a Docker image and pushes to GitHub Container Registry.
|
||||
|
||||
### E2E in CI
|
||||
|
||||
E2E suites are dispatched automatically based on which files changed. On nightly builds and manual dispatches, all suites run. Each suite runs in its own job with a dedicated PostgreSQL service container.
|
||||
|
||||
### Reading CI Failures
|
||||
|
||||
- **Red lint job** - Run `pnpm run lint:check` locally to see the errors.
|
||||
- **Red typecheck job** - Run `pnpm run typecheck` locally.
|
||||
- **Red test job** - Run `pnpm run test` locally. Check the test report artifact for details.
|
||||
- **Red E2E job** - Run `pnpm run e2e:run -- <suite-id>` locally. Ensure your E2E database is prepared with `pnpm run e2e:db:prepare`.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Migrations fail after pulling new changes**
|
||||
|
||||
```bash
|
||||
pnpm run db:migrate
|
||||
```
|
||||
|
||||
If that fails (e.g. conflicting migration state), do a full reset:
|
||||
|
||||
```bash
|
||||
pnpm run db:reset
|
||||
```
|
||||
|
||||
**Type errors after pulling**
|
||||
|
||||
Rebuild dependencies and types:
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
```
|
||||
|
||||
The shared types package rebuilds automatically during install.
|
||||
|
||||
**Pre-push hook is failing**
|
||||
|
||||
Run the same command the hook uses to see the full error output:
|
||||
|
||||
```bash
|
||||
pnpm run verify:fast
|
||||
```
|
||||
|
||||
**E2E tests failing locally**
|
||||
|
||||
Make sure the E2E database is prepared:
|
||||
|
||||
```bash
|
||||
pnpm run e2e:db:prepare
|
||||
pnpm run e2e:run -- <suite-id>
|
||||
```
|
||||
|
||||
**Port already in use**
|
||||
|
||||
The server defaults to port 3000 (configured via `PORT` in `server/.env`) and the client to 5173 (Vite default). If either is occupied, find the process that owns it and stop it, or change the server port in `server/.env`.
|
||||
|
||||
**Hot reload not picking up changes**
|
||||
|
||||
Restart the dev server. If you changed `package.json` or installed new dependencies, run `pnpm install` first.
|
||||
|
||||
**Docker volumes corrupted**
|
||||
|
||||
Nuclear option. This destroys all local data. Stop the container with `-v` to drop volumes, then rebuild:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.dev.yml down -v
|
||||
pnpm run db:up
|
||||
pnpm run db:migrate
|
||||
pnpm run db:seed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Command Reference
|
||||
|
||||
All commands are run from the repository root unless noted otherwise.
|
||||
|
||||
### Setup
|
||||
|
||||
| Command | Description |
|
||||
| -------------- | ---------------------------------------------------- |
|
||||
| `pnpm setup` | One-command bootstrap (env, deps, DB, migrate, seed) |
|
||||
| `pnpm install` | Install all workspace dependencies |
|
||||
|
||||
### Development
|
||||
|
||||
| Command | Description |
|
||||
| --------------------- | --------------------------------------------- |
|
||||
| `pnpm dev` | Start server + client + types with hot reload |
|
||||
| `pnpm run dev:server` | Start server only |
|
||||
| `pnpm run dev:client` | Start client only |
|
||||
|
||||
### Database
|
||||
|
||||
| Command | Description |
|
||||
| -------------------------------------- | ---------------------------------------- |
|
||||
| `pnpm run db:up` | Start PostgreSQL via Docker |
|
||||
| `pnpm run db:migrate` | Apply pending migrations |
|
||||
| `cd server && pnpm db:generate <name>` | Generate migration from schema changes |
|
||||
| `pnpm run db:seed` | Seed baseline data |
|
||||
| `pnpm run db:reset` | Drop, re-migrate, and re-seed (dev only) |
|
||||
| `cd server && pnpm db:studio` | Open Drizzle Studio (DB browser) |
|
||||
|
||||
### Testing
|
||||
|
||||
| Command | Description |
|
||||
| -------------------------------- | -------------------------------- |
|
||||
| `pnpm run test` | Run server + client unit tests |
|
||||
| `pnpm run test:server` | Run server unit tests |
|
||||
| `pnpm run test:client` | Run client unit tests |
|
||||
| `cd server && pnpm test:watch` | Server tests in watch mode |
|
||||
| `pnpm run coverage` | Unit tests with coverage reports |
|
||||
| `pnpm run coverage:server` | Server coverage only |
|
||||
| `pnpm run coverage:client` | Client coverage only |
|
||||
| `pnpm run e2e:run -- <suite-id>` | Run a single E2E suite |
|
||||
| `pnpm run e2e:all` | Run all E2E suites |
|
||||
| `pnpm run e2e:list` | List available suite IDs |
|
||||
| `pnpm run e2e:db:prepare` | Prepare the E2E database |
|
||||
|
||||
### Code Quality
|
||||
|
||||
| Command | Description |
|
||||
| ------------------------ | -------------------------------------------- |
|
||||
| `pnpm run verify` | Full quality gate (lint + typecheck + tests) |
|
||||
| `pnpm run verify:fast` | Quick quality gate |
|
||||
| `pnpm run verify:strict` | Strictest gate (adds format check) |
|
||||
| `pnpm run lint:check` | ESLint check (server + client) |
|
||||
| `pnpm run lint:fix` | ESLint auto-fix (server + client) |
|
||||
| `pnpm run typecheck` | TypeScript type checking |
|
||||
| `pnpm run format` | Prettier format (server + client) |
|
||||
| `pnpm run format:check` | Prettier check without writing |
|
||||
|
||||
+341
@@ -0,0 +1,341 @@
|
||||
# Testing Guide
|
||||
|
||||
ProjectX has three layers of tests:
|
||||
|
||||
- **Server unit tests:** isolated tests for NestJS services, controllers, repositories, and utilities. Dependencies are mocked; no database required.
|
||||
- **Client unit tests:** tests for Vue composables and pure utility functions using a jsdom environment.
|
||||
- **End-to-end (E2E) tests:** full-stack tests that boot a real NestJS application and run HTTP requests against a PostgreSQL database.
|
||||
|
||||
Each layer has a specific job and its own tooling. Don't cover the same thing at multiple layers.
|
||||
|
||||
For testing expectations by change type, see [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
For the development setup, see [DEVELOPMENT.md](DEVELOPMENT.md).
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Command | What it does |
|
||||
| -------------------------------- | ------------------------------------ |
|
||||
| `pnpm run test` | All unit tests (server + client) |
|
||||
| `pnpm run test:server` | Server unit tests only |
|
||||
| `pnpm run test:client` | Client unit tests only |
|
||||
| `cd server && pnpm test:watch` | Server unit tests in watch mode |
|
||||
| `pnpm run coverage` | Coverage reports for server + client |
|
||||
| `pnpm run coverage:server` | Server coverage only |
|
||||
| `pnpm run coverage:client` | Client coverage only |
|
||||
| `pnpm run e2e:run -- <suite-id>` | Run a specific E2E suite |
|
||||
| `pnpm run e2e:all` | Run all E2E suites sequentially |
|
||||
| `pnpm run e2e:list` | List all available E2E suite IDs |
|
||||
| `pnpm run e2e:db:prepare` | Reset and migrate the E2E database |
|
||||
|
||||
---
|
||||
|
||||
## Unit Tests
|
||||
|
||||
| | Server | Client |
|
||||
| ---------- | --------------------------------------------------------------------------- | --------------------------------------------------------------------- |
|
||||
| Config | `server/vitest.config.ts` | `client/vitest.config.ts` |
|
||||
| Test files | `server/src/**/*.test.ts` (e.g. `book.service.ts` → `book.service.test.ts`) | `client/src/**/*.test.ts` (e.g. `useSearch.ts` → `useSearch.test.ts`) |
|
||||
| Env | Node | jsdom |
|
||||
| Globals | Enabled (no import needed) | Disabled (import from `vitest`) |
|
||||
| Coverage | 85% statements/lines/functions, 70% branches (CI enforced) | 1% (not blocking) |
|
||||
|
||||
Coverage excluded on the server: test files, `*.module.ts`, `main.ts`, `src/db/schema/**`,
|
||||
type definitions, interfaces, constants, enums, config files, and scripts.
|
||||
|
||||
---
|
||||
|
||||
## End-to-End Tests
|
||||
|
||||
### Architecture
|
||||
|
||||
```text
|
||||
scripts/e2e/
|
||||
suite-registry.mjs <- source of truth: all suite metadata and changedPaths
|
||||
run-suite.mjs <- orchestrator: preps DB if needed, then runs vitest
|
||||
select-matrix.mjs <- CI smart matrix: picks suites based on changed file paths
|
||||
list-matrix.mjs <- full matrix: all suites (used by manual workflow dispatch)
|
||||
prepare-db.sh <- drops, re-creates, and migrates the projectx_e2e schema
|
||||
|
||||
server/
|
||||
vitest.config.e2e.ts <- separate vitest config (forks pool, sequential files)
|
||||
test/
|
||||
e2e.setup.ts <- global setup: mocks waitForStability
|
||||
e2e/
|
||||
app-harness.ts <- shared NestJS bootstrap and DB helpers
|
||||
scanner/ <- scanner-specific harness helpers
|
||||
*.e2e-spec.ts <- one spec file per suite (18 total)
|
||||
```
|
||||
|
||||
**Key vitest E2E settings** (`server/vitest.config.e2e.ts`):
|
||||
|
||||
- `pool: 'forks'`: each spec file runs in a separate process for full isolation
|
||||
- `fileParallelism: false`: spec files within a run execute sequentially
|
||||
- `setupFiles: ['test/e2e.setup.ts']`: runs before every suite; mocks `waitForStability`
|
||||
- `DATABASE_URL` is always overridden to `E2E_DATABASE_URL` (defaults to `projectx_e2e`), so
|
||||
E2E tests never touch the dev database
|
||||
|
||||
### DB modes
|
||||
|
||||
Every suite runs in one of two modes:
|
||||
|
||||
| Mode | `prepareDedicatedDatabase` | Database used | When to use |
|
||||
| ------------ | -------------------------- | ------------------------------------------------------ | ------------------------------------------------ |
|
||||
| default-db | `false` | `E2E_DATABASE_URL` with no reset before run | Fast smoke tests that don't write data |
|
||||
| dedicated-db | `true` | `projectx_e2e` dropped and re-migrated before each run | Any test that writes data or needs a clean state |
|
||||
|
||||
For dedicated-db suites, `run-suite.mjs` automatically runs `pnpm run e2e:db:prepare` before
|
||||
launching vitest. This drops and re-creates the `projectx_e2e` schema and applies all
|
||||
migrations fresh.
|
||||
|
||||
### Running E2E tests
|
||||
|
||||
```bash
|
||||
# Run a single suite
|
||||
pnpm run e2e:run -- scanner-scenarios
|
||||
|
||||
# Run all suites sequentially
|
||||
pnpm run e2e:all
|
||||
|
||||
# List all registered suite IDs
|
||||
pnpm run e2e:list
|
||||
|
||||
# Filter by test name within a suite
|
||||
pnpm run e2e:run -- scanner-scenarios --testNamePattern=book-per-folder-flat-root-file
|
||||
|
||||
# Use a custom E2E database URL
|
||||
E2E_DATABASE_URL=postgres://... pnpm run e2e:run -- scanner-scenarios
|
||||
```
|
||||
|
||||
The default E2E database URL is `postgres://projectx:projectx@localhost:5432/projectx_e2e`.
|
||||
PostgreSQL must be running locally (start it with `pnpm run db:up` if needed).
|
||||
|
||||
> **Override the database:** Use `E2E_DATABASE_URL`, not `DATABASE_URL`. The vitest config
|
||||
> reads `E2E_DATABASE_URL` and injects it as `DATABASE_URL` into test workers; setting
|
||||
> `DATABASE_URL` directly has no effect.
|
||||
|
||||
E2E tests use the same `server/.env` as development. If tests behave differently locally vs CI,
|
||||
check that your env vars match the values in `server/.env.example`.
|
||||
|
||||
### Suite reference
|
||||
|
||||
| Suite ID | Lane | DB mode | Timeout | Description |
|
||||
| ----------------------------- | ----- | ------------ | ------- | --------------------------------------------------------------------------------------------------------------------- |
|
||||
| `guard-mechanics` | smoke | default-db | 30 min | `@Public` / `@RequirePermission` guard wiring; uses its own minimal NestJS testing module, not the shared app harness |
|
||||
| `scanner-scenarios` | full | dedicated-db | 45 min | File organization scenario matrix for the library scanner |
|
||||
| `scanner-file-operations` | full | dedicated-db | 60 min | File add/remove/rename operations and scanner reconciliation |
|
||||
| `auth-session-security` | full | dedicated-db | 50 min | Cookie flags, token rotation, concurrent sessions |
|
||||
| `auth-recovery-oidc-logout` | full | dedicated-db | 50 min | Password reset flow and OIDC logout |
|
||||
| `book-bucket-ingest-finalize` | full | dedicated-db | 45 min | Book Bucket upload ingest and finalize flow |
|
||||
| `metadata-write` | full | dedicated-db | 60 min | Metadata write operations (field updates, bulk edits) |
|
||||
| `metadata-lock` | full | dedicated-db | 60 min | Metadata lock enforcement across operations |
|
||||
| `migration-booklore` | full | dedicated-db | 70 min | Booklore v1 to ProjectX data migration |
|
||||
| `authorization-matrix` | full | dedicated-db | 60 min | Permission enforcement across all protected routes |
|
||||
| `book-api-contract` | full | dedicated-db | 60 min | Book API response shape and field contract |
|
||||
| `app-settings-oidc-contract` | full | dedicated-db | 60 min | App settings and OIDC configuration contract |
|
||||
| `library-admin-workflows` | full | dedicated-db | 60 min | Library create/update/delete admin workflows |
|
||||
| `reader-format-delivery` | full | dedicated-db | 60 min | Book file serving and format delivery |
|
||||
| `opds-auth-catalog` | full | dedicated-db | 45 min | OPDS feed authentication and catalog structure |
|
||||
| `email-lifecycle` | full | dedicated-db | 45 min | Email provider, template, and send lifecycle |
|
||||
| `reader-state-isolation` | full | dedicated-db | 45 min | Reading progress, bookmarks, and annotations isolated per user |
|
||||
| `users-admin-lifecycle` | full | dedicated-db | 50 min | User create/update/delete admin lifecycle |
|
||||
|
||||
JUnit XML output for each suite is written to `test-results/server/<suite-id>-e2e-junit.xml`.
|
||||
|
||||
> **Timeout column** is the GitHub Actions job timeout, not a per-test vitest timeout.
|
||||
> Individual tests have no configured timeout and use vitest defaults.
|
||||
|
||||
**Lanes:** The smoke lane (`guard-mechanics`) always runs on PRs. Full-lane suites run only
|
||||
when their `changedPaths` list in the registry matches the files changed in the PR. Nightly
|
||||
scheduled runs and manual dispatches always run all suites.
|
||||
|
||||
### App harness
|
||||
|
||||
`server/test/e2e/app-harness.ts` is the shared infrastructure used by every E2E suite.
|
||||
|
||||
**Bootstrapping:**
|
||||
|
||||
```typescript
|
||||
import {
|
||||
createE2EContext,
|
||||
closeE2EContext,
|
||||
type E2EContext,
|
||||
} from "./e2e/app-harness";
|
||||
|
||||
let ctx: E2EContext;
|
||||
|
||||
beforeAll(async () => {
|
||||
ctx = await createE2EContext();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await closeE2EContext(ctx);
|
||||
});
|
||||
```
|
||||
|
||||
`createE2EContext()` boots the full `AppModule` via `Test.createTestingModule`, applies the
|
||||
same global pipes and exception filter as production, registers cookie support, and returns:
|
||||
|
||||
| Property | Type | What it is |
|
||||
| ---------------- | ------------------------ | ------------------------------------ |
|
||||
| `ctx.app` | `NestFastifyApplication` | The running application instance |
|
||||
| `ctx.db` | Drizzle ORM instance | Connected to the E2E database |
|
||||
| `ctx.adminToken` | `string` | A valid JWT for the seeded superuser |
|
||||
|
||||
`MetadataService` is replaced with a no-op mock by default so E2E tests never make external
|
||||
metadata provider calls. To customise the mock for a specific suite, pass the result of
|
||||
`makeMetadataNoopMock()` and configure its `vi.fn()` methods before calling
|
||||
`createE2EContext()`.
|
||||
|
||||
**Making HTTP requests:**
|
||||
|
||||
```typescript
|
||||
const response = await ctx.app.inject({
|
||||
method: "GET",
|
||||
url: "/api/v1/books/99",
|
||||
headers: { authorization: `Bearer ${ctx.adminToken}` },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(200);
|
||||
const body = response.json() as { id: number; title: string };
|
||||
```
|
||||
|
||||
`app.inject()` is Fastify's built-in injection; no network socket or supertest needed.
|
||||
|
||||
**Direct database access:**
|
||||
|
||||
```typescript
|
||||
import { eq } from "drizzle-orm";
|
||||
import { books } from "../../src/db/schema";
|
||||
|
||||
const [book] = await ctx.db.select().from(books).where(eq(books.id, 99));
|
||||
expect(book.status).toBe("present");
|
||||
```
|
||||
|
||||
**Key helper functions:**
|
||||
|
||||
| Function | Returns | Purpose |
|
||||
| --------------------------------------------------- | -------------------------------- | --------------------------------------------------------------------------- |
|
||||
| `seedLibrary(db, input)` | `{ libraryId, libraryFolderId }` | Insert a library and folder row directly into the DB |
|
||||
| `triggerLibraryScan(ctx, libraryId)` | `jobId: number` | POST to the scan endpoint |
|
||||
| `triggerAndWaitForLibraryScan(ctx, libraryId, ms?)` | `ScanJob` | Trigger a scan and poll until complete (default 30s timeout) |
|
||||
| `waitForScanCompletion(db, jobId, ms?, pollMs?)` | `ScanJob` | Poll `scanJobs` until `completed` or `failed` (100ms default poll interval) |
|
||||
| `waitForCondition(check, timeoutMs?, pollMs?)` | `void` | Retry an async assertion until it passes or times out (200ms poll) |
|
||||
| `loadLibraryBookState(db, libraryId)` | `LibraryBookState[]` | Snapshot all books and files for a library, sorted for comparison |
|
||||
| `loadIntegritySnapshot(db)` | `IntegritySnapshot` | Return counts of all orphan rows across relational tables |
|
||||
| `assertNoIntegrityViolations(db)` | `void` | Throw if any orphan rows are found |
|
||||
| `makeMetadataNoopMock()` | `MetadataNoopMock` | Return a mock `MetadataService` with all methods resolved to `undefined` |
|
||||
|
||||
Suite-specific helpers extend or re-export from the app harness. See
|
||||
`server/test/e2e/scanner/scanner-harness.ts` for an example; it re-exports the common
|
||||
functions and adds `startLibraryWatcher` and `stopLibraryWatcher`.
|
||||
|
||||
### Adding tests to an existing suite
|
||||
|
||||
1. Open `server/test/<suite-id>.e2e-spec.ts`.
|
||||
2. Locate the shared `ctx` set up in `beforeAll`/`afterAll`.
|
||||
3. Add `describe`/`it` blocks using `ctx.app.inject()` for HTTP and `ctx.db` for assertions.
|
||||
4. Because `fileParallelism: false`, all tests in a suite share the same app instance. Use
|
||||
`beforeEach`/`afterEach` to clean up any rows your tests insert to avoid cross-test
|
||||
interference.
|
||||
|
||||
### Adding a new suite
|
||||
|
||||
**Step 1 - Create the spec file**
|
||||
|
||||
Create `server/test/<suite-id>.e2e-spec.ts`. Use `createE2EContext`/`closeE2EContext` in
|
||||
`beforeAll`/`afterAll`. Any existing `*.e2e-spec.ts` is a reference for the standard structure.
|
||||
|
||||
**Step 2 - Add suite helpers (optional)**
|
||||
|
||||
For fixture builders or harness utilities specific to your suite:
|
||||
|
||||
```text
|
||||
server/test/e2e/<suite-name>/
|
||||
```
|
||||
|
||||
**Step 3 - Register in the suite registry**
|
||||
|
||||
Add an entry to `scripts/e2e/suite-registry.mjs`:
|
||||
|
||||
```javascript
|
||||
"my-new-suite": {
|
||||
id: "my-new-suite",
|
||||
name: "My New Suite",
|
||||
timeout: 60,
|
||||
lane: "full",
|
||||
description: "Short description of what this suite covers",
|
||||
vitestTarget: "test/my-new-suite.e2e-spec.ts",
|
||||
junitOutput: `${TEST_RESULTS_DIR}/my-new-suite-e2e-junit.xml`,
|
||||
prepareDedicatedDatabase: true, // false only if your suite never writes data
|
||||
useDedicatedDatabase: true,
|
||||
changedPaths: [
|
||||
"server/src/modules/my-feature/**",
|
||||
"server/test/my-new-suite.e2e-spec.ts",
|
||||
"server/test/e2e/my-suite/**",
|
||||
...SHARED_DB_AND_HELPER_PATHS,
|
||||
],
|
||||
},
|
||||
```
|
||||
|
||||
Keep `changedPaths` precise. Overly broad patterns will schedule your suite on unrelated PRs.
|
||||
|
||||
**Step 4 - CI wiring**
|
||||
|
||||
No workflow changes needed. `scripts/e2e/select-matrix.mjs` reads the registry automatically.
|
||||
|
||||
---
|
||||
|
||||
## Coverage
|
||||
|
||||
```bash
|
||||
pnpm run coverage # server + client
|
||||
pnpm run coverage:server # text + lcov output to server/coverage/
|
||||
pnpm run coverage:client # text + HTML output to client/coverage/
|
||||
```
|
||||
|
||||
Server thresholds (85% statements/lines/functions, 70% branches) are enforced and will fail the
|
||||
CI test job if dropped below. Client thresholds are 1% by design.
|
||||
|
||||
---
|
||||
|
||||
## CI Integration
|
||||
|
||||
### Unit tests
|
||||
|
||||
Unit tests run on every push and PR via `.github/workflows/ci.yml`. The workflow is
|
||||
change-aware: if only client files changed, only the client test job runs. Each job has a
|
||||
20-minute timeout. JUnit XML results are published as annotations on the PR or commit, and test
|
||||
report artifacts are retained for 7 days.
|
||||
|
||||
### E2E suites
|
||||
|
||||
E2E suites are selected by `scripts/e2e/select-matrix.mjs`:
|
||||
|
||||
- **Pull request:** The smoke suite (`guard-mechanics`) always runs. Full-lane suites run only
|
||||
when their `changedPaths` match the PR diff.
|
||||
- **Push to main / nightly / manual dispatch:** All 18 suites run.
|
||||
|
||||
Each selected suite runs as its own GitHub Actions job via the reusable
|
||||
`.github/workflows/e2e-runner.yml` workflow, which spins up a PostgreSQL 16 + pgvector service
|
||||
container and calls `pnpm run e2e:run -- <suite-id>`.
|
||||
|
||||
**Manual full run:** Trigger `.github/workflows/e2e.yml` from the Actions tab. It accepts an
|
||||
optional `vitest-args` input for filtering by test name.
|
||||
|
||||
### Reading CI failures
|
||||
|
||||
| Failing job | Local command to reproduce |
|
||||
| ----------- | -------------------------------- |
|
||||
| lint | `pnpm run lint:check` |
|
||||
| typecheck | `pnpm run typecheck` |
|
||||
| tests | `pnpm run test` |
|
||||
| E2E suite | `pnpm run e2e:run -- <suite-id>` |
|
||||
|
||||
For E2E failures, check the JUnit artifact on the run for the full test report. If the suite
|
||||
uses a dedicated DB, prepare it first:
|
||||
|
||||
```bash
|
||||
pnpm run e2e:db:prepare && pnpm run e2e:run -- <suite-id>
|
||||
```
|
||||
Reference in New Issue
Block a user