mirror of
https://github.com/bookorbit/bookorbit.git
synced 2026-11-03 00:11:37 -06:00
chore(translations): add catalog verification script and update localization docs
This commit is contained in:
neon
@@ -80,6 +80,7 @@ jobs:
|
||||
id: translation-progress
|
||||
continue-on-error: true
|
||||
env:
|
||||
CROWDIN_TOKEN: ${{ secrets.CROWDIN_TOKEN }}
|
||||
SVG_PATH: ${{ runner.temp }}/translation-progress.svg
|
||||
run: |
|
||||
node scripts/crowdin-stats/generate.mjs --out "$SVG_PATH"
|
||||
|
||||
@@ -2,14 +2,14 @@
|
||||
|
||||
# BookOrbit
|
||||
|
||||
A self-hosted library management and reading platform for ebooks, PDFs, audiobooks, and comics.
|
||||
A self-hosted library and reading platform for ebooks, PDFs, audiobooks, and comics.
|
||||
|
||||
[](https://github.com/bookorbit/bookorbit/releases)
|
||||
[](https://github.com/bookorbit/bookorbit/stargazers)
|
||||
[](https://github.com/bookorbit/bookorbit/actions/workflows/ci.yml)
|
||||
[](https://github.com/bookorbit/bookorbit/actions/workflows/release.yml)
|
||||
[](https://codecov.io/gh/bookorbit/bookorbit)
|
||||
[](https://crowdin.com/project/bookorbit)
|
||||
[](https://github.com/bookorbit/bookorbit/actions/workflows/ci.yml)
|
||||
[](https://github.com/bookorbit/bookorbit/actions/workflows/release.yml)
|
||||
[](https://codecov.io/gh/bookorbit/bookorbit)
|
||||
[](https://crowdin.com/project/bookorbit)
|
||||
|
||||
[](https://bookorbit.app)
|
||||
[](https://demo.bookorbit.app/magic?token=2d92cb900e184cf0eb8b11f72cffc6011673d1016e1b300d750eb3d76abc1572)
|
||||
@@ -17,64 +17,52 @@ A self-hosted library management and reading platform for ebooks, PDFs, audioboo
|
||||
[](https://github.com/bookorbit/bookorbit/blob/main/docs/CONTRIBUTING.md)
|
||||
[](LICENSE)
|
||||
|
||||
<!--
|
||||
[](https://github.com/bookorbit/bookorbit/pkgs/container/bookorbit)
|
||||
[Website](https://bookorbit.app) · [Demo](https://demo.bookorbit.app) · [Discussions](https://github.com/bookorbit/bookorbit/discussions) · [Contributing](https://github.com/bookorbit/bookorbit/blob/main/docs/CONTRIBUTING.md) · [Changelog](https://github.com/bookorbit/bookorbit/releases)
|
||||
-->
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## What is BookOrbit?
|
||||
|
||||
**[BookOrbit](https://bookorbit.app)** is a self-hosted digital library and reading platform. Organize and read your books, sync progress and annotations seamlessly across Kobo, KOReader, and the web reader, enrich your collection with metadata from multiple providers, and push reading data to Hardcover automatically. Supports multiple users with OIDC/SSO, detailed reading statistics, OPDS, customizable dashboard widgets, Send-to-Kindle delivery, and Smart Scopes - all running on infrastructure you control.
|
||||
**[BookOrbit](https://bookorbit.app)** organizes your books and reads them back to you anywhere: the web reader, a Kobo, or KOReader. Progress, highlights, and reading status move between all three, so you can start a chapter in one place and finish it in another.
|
||||
|
||||
Around that core sit 14 metadata providers, reading statistics and achievements, OPDS and Send-to-Kindle delivery, multi-user accounts with OIDC/SSO, and automatic sync out to Hardcover, Readwise, and StoryGraph. All of it runs on infrastructure you control.
|
||||
|
||||
[](https://bookorbit.app)
|
||||
|
||||
---
|
||||
|
||||
## Live Demo
|
||||
|
||||
Want to try BookOrbit before installing? Explore the live instance instantly: no account required!
|
||||
Try the live instance before you install. No account required.
|
||||
|
||||
[](https://demo.bookorbit.app/magic?token=2d92cb900e184cf0eb8b11f72cffc6011673d1016e1b300d750eb3d76abc1572)
|
||||
|
||||
_Experience the interface, built-in readers, and dashboard first-hand._
|
||||
|
||||
> **Note:** The demo includes a sample library of public domain books. Some administrative features are limited in the public demo. Self-hosting BookOrbit provides the full experience.
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
### Reading Experience & Sync
|
||||
|
||||
- **Built-in Web Readers**: Native support for eBooks (EPUB, MOBI, AZW3), PDFs, Comics (CBZ, CBR), and Audiobooks (M4B, MP3) with no extra plugins required.
|
||||
- **Three-Way Sync (Kobo + KOReader + BookOrbit)**: Progress and annotations flow bidirectionally between Kobo devices, KOReader, and the BookOrbit web reader. Pick up on any surface where you left off on another - including highlights and deletes.
|
||||
- **KOReader Plugin**: Native on-device catalog browser with search, download, and status/rating management, alongside full progress and annotation sync.
|
||||
- **Annotations & Highlights**: Highlights from the web reader, KOReader, and Kobo merge into a unified searchable hub - filterable by color, style, and source, exportable as Markdown, CSV, or JSON.
|
||||
- **Hardcover Sync**: Automatically pushes status, progress, reading dates, and ratings to Hardcover on configurable triggers. Pull read history back from Hardcover to backfill blank BookOrbit entries.
|
||||
- **Readwise Sync**: Automatically pushes new highlights and notes to Readwise as you create them, from both the web reader and synced devices, with book covers matched by ISBN.
|
||||
- **Achievements & Reading Goals**: Yearly goals, daily streaks, monthly challenges, and 50+ achievements across five categories. Reading DNA profiles your reading style from your actual session history.
|
||||
- **Reading Statistics**: Track your daily reading time, view heatmaps, maintain streaks, and monitor library health.
|
||||
- **Built-in Web Readers**: Ebooks (EPUB, KEPUB, MOBI, AZW3, AZW, FB2), PDFs, comics (CBZ, CBR, CB7), and audiobooks (M4B, MP3, M4A, OPUS, OGG, FLAC), with no extra plugins required.
|
||||
- **Three-Way Sync (Kobo + KOReader + BookOrbit)**: Progress and annotations flow bidirectionally between Kobo devices, KOReader, and the BookOrbit web reader. Pick up on any surface where you left off on another, including highlights and deletions.
|
||||
- **KOReader Plugin**: An on-device catalog browser with search, download, and status and rating management, alongside full progress and annotation sync.
|
||||
- **Annotations & Highlights**: Highlights from the web reader, KOReader, and Kobo merge into one searchable hub. Filter by color, style, and source; export as Markdown, CSV, or JSON.
|
||||
- **Hardcover, Readwise & StoryGraph Sync**: Push status, progress, reading dates, and ratings to Hardcover on configurable triggers; status and progress to The StoryGraph; and new highlights and notes to Readwise as you create them, from both the web reader and synced devices. Hardcover read history can be pulled back to backfill blank BookOrbit entries.
|
||||
- **Statistics, Goals & Achievements**: Daily reading time, heatmaps, streaks, and library health, plus yearly goals, monthly challenges, and 50+ achievements across five categories. Reading DNA profiles your reading style from your actual session history.
|
||||
|
||||
### Library Management
|
||||
|
||||
- **Multiple Libraries**: Isolate content with per-library folders, custom scan rules, and format priorities.
|
||||
- **Rich Metadata Providers**: Fetch robust metadata from Google Books, Amazon, Goodreads, Open Library, Audible, ComicVine, and more.
|
||||
- **Smart Scopes & Collections**: Organize your collection using curated lists and dynamic, rule-based saved filters.
|
||||
- **14 Metadata Providers**: Google Books, Open Library, Amazon, Goodreads, Kobo, Hardcover, Audible, Audnexus, Libro.fm, and iTunes, plus ComicVine for comics, RanobeDB for light novels, and Aladin and Lubimyczytać for Korean and Polish catalogs. Cover art is sourced separately from iTunes, DuckDuckGo, and AudiobookCovers.
|
||||
- **Smart Scopes & Collections**: Organize your collection with curated lists and dynamic, rule-based saved filters.
|
||||
|
||||
### Platform & Delivery
|
||||
|
||||
- **Multi-User & SSO**: Granular per-user permissions and isolated reading data, with native support for Authentik, Keycloak, and Authelia via OIDC.
|
||||
- **Multilingual Interface**: English, German, Spanish, French, Italian, Dutch, Polish, Brazilian Portuguese, and Slovenian, translated by the community on [Crowdin](https://crowdin.com/project/bookorbit).
|
||||
- **Content Delivery**: OPDS support for compatible apps, Send-to-Kindle via email, and browser drag-and-drop uploads.
|
||||
- **Automated Ingestion**: Configure a Book Dock drop folder for hands-free importing.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start (Docker)
|
||||
|
||||
```bash
|
||||
@@ -95,6 +83,8 @@ JWT_SECRET= # signs login tokens - openssl rand -hex 32
|
||||
SETUP_BOOTSTRAP_TOKEN= # one-time setup wizard token - openssl rand -hex 16
|
||||
```
|
||||
|
||||
On a NAS, or any host where your book folder is owned by a user other than UID 1000, also set `PUID` and `PGID` to match that owner. Run `id -u` and `id -g` as the owning user to find them. Getting these wrong is the most common cause of permission errors on first scan.
|
||||
|
||||
Optionally set `LIBRARY_BROWSE_ROOT=/books` to start the library folder picker at `/books` instead of `/`.
|
||||
|
||||
Then start:
|
||||
@@ -107,8 +97,6 @@ Open `http://your-server-ip:3000` and complete setup using your `SETUP_BOOTSTRAP
|
||||
|
||||
For the full installation guide including reverse proxy setup, file permissions on NAS, external databases, and environment variable reference, see **[bookorbit.app/installation](https://bookorbit.app/installation)**.
|
||||
|
||||
---
|
||||
|
||||
## KOReader Plugin
|
||||
|
||||
The BookOrbit plugin for KOReader adds progress sync, two-way annotation sync, and a native catalog browser: navigate, search, and download books from your library without leaving the device.
|
||||
@@ -121,38 +109,24 @@ The BookOrbit plugin for KOReader adds progress sync, two-way annotation sync, a
|
||||
4. Restart KOReader and open a book.
|
||||
5. Use **Tools > BookOrbit Sync** to connect.
|
||||
|
||||
The download is pre-configured with your server URL and credentials; no manual entry on the device. For full setup and sync options, see **[bookorbit.app/koreader-plugin](https://bookorbit.app/koreader-plugin)**.
|
||||
|
||||
---
|
||||
The download is pre-configured with your server URL and credentials, so there is no manual entry on the device. For full setup and sync options, see **[bookorbit.app/koreader-plugin](https://bookorbit.app/koreader-plugin)**.
|
||||
|
||||
## Documentation and Contributing
|
||||
|
||||
Full documentation is at **[bookorbit.app](https://bookorbit.app/what-is-bookorbit)** - covering libraries, metadata, readers, Kobo sync, OPDS, users and permissions, OIDC setup, and more.
|
||||
Full documentation is at **[bookorbit.app](https://bookorbit.app/what-is-bookorbit)**, covering libraries, metadata, readers, Kobo sync, OPDS, users and permissions, OIDC setup, and more.
|
||||
|
||||
For local development, see [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md). To contribute, see [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for the full workflow: branch naming, test expectations, PR checklist, and commit format.
|
||||
|
||||
---
|
||||
|
||||
## Translations
|
||||
|
||||
BookOrbit is translated by the community on [Crowdin](https://crowdin.com/project/bookorbit). English is the source language; every other catalog is exported from Crowdin and merged through automated pull requests.
|
||||
Help translate BookOrbit into your language on [Crowdin](https://crowdin.com/project/bookorbit).
|
||||
|
||||
[](https://crowdin.com/project/bookorbit)
|
||||
[](https://crowdin.com/project/bookorbit)
|
||||
|
||||
The chart refreshes daily from Crowdin.
|
||||
|
||||
- **Improve an existing language:** translate or review strings on **[Crowdin](https://crowdin.com/project/bookorbit)**. No local setup or Git knowledge needed.
|
||||
- **Request a new language:** open a [Discussion](https://github.com/bookorbit/bookorbit/discussions) and it will be enabled on Crowdin.
|
||||
- **Found wrong English wording?** That one belongs in the code, so open an [issue](https://github.com/bookorbit/bookorbit/issues/new?template=bug_report.yml) instead of translating around it.
|
||||
|
||||
---
|
||||
|
||||
## Community Growth
|
||||
## Star History
|
||||
|
||||
[](https://github.com/bookorbit/bookorbit/stargazers)
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
- **Questions and discussion:** [GitHub Discussions](https://github.com/bookorbit/bookorbit/discussions)
|
||||
@@ -160,8 +134,6 @@ The chart refreshes daily from Crowdin.
|
||||
- **Feature requests:** [GitHub Issues](https://github.com/bookorbit/bookorbit/issues/new?template=feature_request.yml)
|
||||
- **Security vulnerabilities:** Follow the private reporting process in the [Security Policy](.github/SECURITY.md).
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
BookOrbit is licensed under the **[GNU Affero General Public License v3.0](LICENSE)**.
|
||||
|
||||
+34
-2
@@ -120,9 +120,40 @@ The shared locale list automatically updates the language picker, server prefere
|
||||
|
||||
Create `client/src/locales/<locale>.json` with the exact key structure from `en.json` and English fallback values. Do not add partial catalogs. Crowdin will replace fallback values as translations are completed.
|
||||
|
||||
A catalog may instead arrive already translated, for example from a bulk translation pass. That is allowed, but the translations then exist only in Git and Crowdin will overwrite them unless they are imported first. Follow "Seed existing translations into Crowdin" below before enabling export.
|
||||
|
||||
Review `client/src/stores/locale.ts` and add browser-locale matching coverage to `client/src/stores/__tests__/locale.spec.ts`, especially for regional variants. Add every CLDR plural category reported for the locale to each ICU plural message and extend the exhaustive ICU runtime tests in `client/src/i18n/icu.spec.ts`.
|
||||
|
||||
### 4. Allow Crowdin export and PR delivery
|
||||
### 4. Seed existing translations into Crowdin
|
||||
|
||||
Skip this step when the catalog holds English fallback values. It applies only when the catalog already contains real translations.
|
||||
|
||||
Crowdin does not import repository translations on its own, and untranslated strings export as English source text. A pre-translated catalog that is never imported is therefore silently reverted to English by the first export.
|
||||
|
||||
Order matters, because merging the `crowdin.yml` change enables export for the language:
|
||||
|
||||
1. Pause translation synchronization, or keep the language out of `export_languages` until verification passes.
|
||||
2. Add the target language in Crowdin.
|
||||
3. Import the catalog with Crowdin's translation import, allowing translations that match the source so entries such as product names and identifiers are not dropped.
|
||||
4. Verify, reconcile, and only then enable export.
|
||||
|
||||
**The progress percentage does not verify anything.** Crowdin's bulk translation import is lossy: it silently skips entries that the per-string translation endpoint accepts. Observed causes include translations that add punctuation the English source lacks, and ICU messages carrying a plural category the source does not use, such as Spanish or Portuguese `many`. A language can report 96 to 98 percent while real translations are missing, and those gaps export as English over good translations in Git.
|
||||
|
||||
Verify by building the export and diffing it against the catalog:
|
||||
|
||||
```bash
|
||||
CROWDIN_TOKEN=... node scripts/crowdin-verify/verify.mjs es-ES es.json
|
||||
```
|
||||
|
||||
The script exits non-zero while any key differs. Pass `--reconcile` to push the remaining strings through the per-string endpoint, which accepts what the bulk import rejected:
|
||||
|
||||
```bash
|
||||
CROWDIN_TOKEN=... node scripts/crowdin-verify/verify.mjs es-ES es.json --reconcile
|
||||
```
|
||||
|
||||
A language is safe to export only when the script reports zero differing keys. Run it for every seeded language before resuming synchronization.
|
||||
|
||||
### 5. Allow Crowdin export and PR delivery
|
||||
|
||||
Update `crowdin.yml`:
|
||||
|
||||
@@ -133,7 +164,7 @@ Update `scripts/classify-crowdin-pr.sh` to allow the new target catalog path. Do
|
||||
|
||||
Before merging, add the target language in Crowdin while the current `main` configuration still excludes it from `export_languages`, or pause translation synchronization. Merge the repository support before allowing the first export. After merge, run a manual source and translation sync and confirm that the generated PR changes only explicitly allowed target catalogs.
|
||||
|
||||
### 5. Verify the new language
|
||||
### 6. Verify the new language
|
||||
|
||||
Run:
|
||||
|
||||
@@ -158,6 +189,7 @@ Then verify manually:
|
||||
- Representative desktop and mobile screens handle translated text expansion.
|
||||
- RTL layout, keyboard navigation, focus states, dialogs, menus, and popovers remain usable when applicable.
|
||||
- A manual Crowdin export produces the expected filename and passes all PR checks.
|
||||
- For a seeded catalog, `scripts/crowdin-verify/verify.mjs` reports zero differing keys.
|
||||
|
||||
Only enable scheduled export for the new language after this manual round trip succeeds.
|
||||
|
||||
|
||||
+10
-2
@@ -51,8 +51,16 @@ pagination limit.
|
||||
The translation generator reads public Crowdin project statistics and renders
|
||||
every language the project exposes, so no code change is needed when a language
|
||||
is added. English is pinned first at 100% as the source language and is
|
||||
excluded from the average. Up to eight languages render as horizontal bars;
|
||||
beyond that the chart switches to a vertical bar chart with a fixed height.
|
||||
excluded from the average, though it renders like any other completed row.
|
||||
|
||||
Every language is one horizontal meter row, so the card grows in height as
|
||||
languages are added rather than switching layout. The label column is measured
|
||||
from the widest language name and long names are truncated, which keeps the bar
|
||||
from being squeezed out. Bars carry three progress tiers, listed in a legend on
|
||||
the card: complete at 100%, in progress at 70% or above, and needs work below
|
||||
that. Three is the ceiling; a fourth tier would need an amber that sits at
|
||||
deuteranopia delta-E 1.4 against the orange, which no threshold tuning can
|
||||
separate.
|
||||
|
||||
## Local commands
|
||||
|
||||
|
||||
+119
-107
@@ -4,43 +4,50 @@ import { rename, unlink, writeFile } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
// The badge feed is a cached snapshot and can lag the project by hours, which
|
||||
// silently renders a stale chart. Prefer the Crowdin API when a token is
|
||||
// available and keep the badge feed as the tokenless fallback.
|
||||
const STATS_URL = "https://badges.awesome-crowdin.com/stats-17791545-912891.json";
|
||||
const API_BASE = "https://api.crowdin.com/api/v2";
|
||||
const PROJECT_ID = "912891";
|
||||
const REQUEST_TIMEOUT_MS = 30_000;
|
||||
const MAX_RESPONSE_BYTES = 1_000_000;
|
||||
const SOURCE_LANGUAGE_ID = "en";
|
||||
|
||||
const WIDTH = 680;
|
||||
const RENDERED_WIDTH = 580;
|
||||
const WIDTH = 540;
|
||||
const PADDING_X = 24;
|
||||
const DIVIDER_Y = 48;
|
||||
const BAR_RADIUS = 4;
|
||||
const FLAG_WIDTH = 26;
|
||||
const LABEL_CHAR_WIDTH = 7.1;
|
||||
const AXIS_CHAR_WIDTH = 6.1;
|
||||
const DIVIDER_Y = 51;
|
||||
|
||||
const MAX_HORIZONTAL_ROWS = 8;
|
||||
const ROW_BASELINE = 78;
|
||||
const ROW_HEIGHT = 42;
|
||||
const ROW_BOTTOM_PADDING = 36;
|
||||
const ROW_BAR_HEIGHT = 16;
|
||||
const ROW_LABEL_WIDTH = 190;
|
||||
const ROW_LABEL_GAP = 12;
|
||||
const ROW_PERCENT_WIDTH = 38;
|
||||
const ROW_PERCENT_GAP = 12;
|
||||
const ROW_HEIGHT = 28;
|
||||
const BAR_HEIGHT = 8;
|
||||
const BAR_RADIUS = 4;
|
||||
const BAR_BASELINE_OFFSET = 9;
|
||||
|
||||
const AXIS_WIDTH = 30;
|
||||
const PLOT_TOP = 74;
|
||||
const PLOT_HEIGHT = 150;
|
||||
const AXIS_TICKS = [0, 25, 50, 75, 100];
|
||||
const COLUMN_MAX_WIDTH = 34;
|
||||
const COLUMN_MIN_WIDTH = 6;
|
||||
const COLUMN_WIDTH_RATIO = 0.62;
|
||||
const COLUMN_LABEL_GAP = 16;
|
||||
const COLUMN_LABEL_MAX_WIDTH = 118;
|
||||
const COLUMN_LABEL_LEFT_BOUND = 10;
|
||||
const COLUMN_PERCENT_MIN_SLOT = 32;
|
||||
const COLUMN_BOTTOM_PADDING = 16;
|
||||
const ROTATED_LABEL_SCALE = Math.SQRT1_2;
|
||||
const FLAG_WIDTH = 26;
|
||||
const LABEL_CHAR_WIDTH = 7.6;
|
||||
const LABEL_MAX_WIDTH = 180;
|
||||
const LABEL_GAP = 18;
|
||||
const PERCENT_WIDTH = 42;
|
||||
const PERCENT_GAP = 14;
|
||||
|
||||
const LEGEND_SWATCH = 8;
|
||||
const LEGEND_CHAR_WIDTH = 5.9;
|
||||
const LEGEND_TEXT_GAP = 14;
|
||||
const LEGEND_ITEM_GAP = 22;
|
||||
const LEGEND_RULE_GAP = 20;
|
||||
const LEGEND_BASELINE_GAP = 40;
|
||||
const LEGEND_BOTTOM_PADDING = 20;
|
||||
const EMPTY_HEIGHT = 96;
|
||||
|
||||
// Ordered high to low; the first tier a percentage clears wins. Three tiers is a
|
||||
// hard ceiling, not a style choice: a fourth needs an amber that sits at deuteranopia
|
||||
// ΔE 1.4 against the orange, which no threshold tuning can separate.
|
||||
const TIERS = [
|
||||
{ className: "bar-done", label: "Complete", min: 100 },
|
||||
{ className: "bar-progress", label: "In progress", min: 70 },
|
||||
{ className: "bar-low", label: "Needs work", min: 0 },
|
||||
];
|
||||
|
||||
export class CrowdinStatsError extends Error {
|
||||
constructor(message, { status, url } = {}) {
|
||||
@@ -55,13 +62,14 @@ export function parseArguments(argv) {
|
||||
const options = {
|
||||
outputPath: "translation-progress.svg",
|
||||
statsUrl: STATS_URL,
|
||||
projectId: PROJECT_ID,
|
||||
};
|
||||
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
const flag = argv[index];
|
||||
const value = argv[index + 1];
|
||||
|
||||
if (!["--out", "--url"].includes(flag)) {
|
||||
if (!["--out", "--url", "--project"].includes(flag)) {
|
||||
throw new Error(`Unknown argument: ${flag}`);
|
||||
}
|
||||
if (!value) {
|
||||
@@ -70,6 +78,7 @@ export function parseArguments(argv) {
|
||||
|
||||
if (flag === "--out") options.outputPath = value;
|
||||
if (flag === "--url") options.statsUrl = value;
|
||||
if (flag === "--project") options.projectId = value;
|
||||
index += 1;
|
||||
}
|
||||
|
||||
@@ -128,6 +137,37 @@ export async function fetchCrowdinStats(url = STATS_URL, fetchImpl = fetch) {
|
||||
return json;
|
||||
}
|
||||
|
||||
export async function fetchCrowdinApiProgress(token, { projectId = PROJECT_ID, fetchImpl = fetch } = {}) {
|
||||
const url = `${API_BASE}/projects/${projectId}/languages/progress?limit=500`;
|
||||
const response = await fetchImpl(url, {
|
||||
headers: { Authorization: `Bearer ${token}`, "User-Agent": "bookorbit-crowdin-stats" },
|
||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new CrowdinStatsError(`Failed to fetch Crowdin progress: HTTP ${response.status}`, {
|
||||
status: response.status,
|
||||
url,
|
||||
});
|
||||
}
|
||||
|
||||
const json = await readJsonResponse(response, url);
|
||||
if (!json || !Array.isArray(json.data)) {
|
||||
throw new CrowdinStatsError("Invalid Crowdin progress response format", { url });
|
||||
}
|
||||
|
||||
// Present the API payload in the same shape as the badge feed so the rest of
|
||||
// the generator does not care which source produced it.
|
||||
return { progress: json.data };
|
||||
}
|
||||
|
||||
export async function fetchStats({ token, statsUrl = STATS_URL, projectId = PROJECT_ID, fetchImpl = fetch } = {}) {
|
||||
if (token) {
|
||||
return { stats: await fetchCrowdinApiProgress(token, { projectId, fetchImpl }), source: "Crowdin API" };
|
||||
}
|
||||
return { stats: await fetchCrowdinStats(statsUrl, fetchImpl), source: "cached badge feed" };
|
||||
}
|
||||
|
||||
function escapeXml(value) {
|
||||
return String(value).replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">").replace(/"/g, """);
|
||||
}
|
||||
@@ -217,93 +257,61 @@ function labelFor(language, availableWidth, charWidth) {
|
||||
|
||||
export function computeLayout(languages) {
|
||||
const count = languages.length;
|
||||
|
||||
if (count <= MAX_HORIZONTAL_ROWS) {
|
||||
return {
|
||||
mode: "rows",
|
||||
count,
|
||||
barWidth: WIDTH - PADDING_X * 2 - ROW_LABEL_WIDTH - ROW_PERCENT_GAP - ROW_PERCENT_WIDTH,
|
||||
height: ROW_BASELINE + Math.max(0, count - 1) * ROW_HEIGHT + ROW_BOTTOM_PADDING,
|
||||
};
|
||||
}
|
||||
|
||||
const plotLeft = PADDING_X + AXIS_WIDTH;
|
||||
const slotWidth = (WIDTH - PADDING_X - plotLeft) / count;
|
||||
const columnWidth = Math.max(COLUMN_MIN_WIDTH, Math.min(COLUMN_MAX_WIDTH, Math.round(slotWidth * COLUMN_WIDTH_RATIO)));
|
||||
|
||||
const columns = languages.map((language, index) => {
|
||||
const centerX = Math.round(plotLeft + slotWidth * (index + 0.5));
|
||||
const spaceToLeftEdge = (centerX - COLUMN_LABEL_LEFT_BOUND) / ROTATED_LABEL_SCALE;
|
||||
const label = labelFor(language, Math.min(COLUMN_LABEL_MAX_WIDTH, spaceToLeftEdge), AXIS_CHAR_WIDTH);
|
||||
return { centerX, label };
|
||||
});
|
||||
|
||||
const labelWidth = Math.max(0, ...columns.map((column) => column.label.width));
|
||||
const labels = languages.map((language) => labelFor(language, LABEL_MAX_WIDTH, LABEL_CHAR_WIDTH));
|
||||
// The label column tracks the widest name rather than a fixed slab, so a short
|
||||
// language list does not open a gap between each name and its bar.
|
||||
const labelWidth = count ? Math.max(...labels.map((label) => label.width)) : 0;
|
||||
const barX = Math.round(PADDING_X + labelWidth + LABEL_GAP);
|
||||
const lastRowY = ROW_BASELINE + Math.max(0, count - 1) * ROW_HEIGHT;
|
||||
|
||||
return {
|
||||
mode: "columns",
|
||||
count,
|
||||
plotLeft,
|
||||
slotWidth,
|
||||
columnWidth,
|
||||
columns,
|
||||
baseline: PLOT_TOP + PLOT_HEIGHT,
|
||||
showPercentages: slotWidth >= COLUMN_PERCENT_MIN_SLOT,
|
||||
height: PLOT_TOP + PLOT_HEIGHT + COLUMN_LABEL_GAP + Math.round(labelWidth * ROTATED_LABEL_SCALE) + COLUMN_BOTTOM_PADDING,
|
||||
labels,
|
||||
barX,
|
||||
barWidth: WIDTH - PADDING_X - PERCENT_WIDTH - PERCENT_GAP - barX,
|
||||
lastRowY,
|
||||
legendY: lastRowY + LEGEND_BASELINE_GAP,
|
||||
height: count ? lastRowY + LEGEND_BASELINE_GAP + LEGEND_BOTTOM_PADDING : EMPTY_HEIGHT,
|
||||
};
|
||||
}
|
||||
|
||||
function barClass(language) {
|
||||
if (language.isSource) return "bar-source";
|
||||
if (language.percent >= 90) return "bar-high";
|
||||
if (language.percent >= 60) return "bar-medium";
|
||||
return "bar-low";
|
||||
return TIERS.find((tier) => language.percent >= tier.min).className;
|
||||
}
|
||||
|
||||
function renderRow(language, index, layout) {
|
||||
const baseline = ROW_BASELINE + index * ROW_HEIGHT;
|
||||
const barX = PADDING_X + ROW_LABEL_WIDTH;
|
||||
const barY = baseline - BAR_BASELINE_OFFSET;
|
||||
const fillWidth = Math.round((language.percent / 100) * layout.barWidth);
|
||||
const label = labelFor(language, ROW_LABEL_WIDTH - ROW_LABEL_GAP, LABEL_CHAR_WIDTH);
|
||||
|
||||
return `
|
||||
<text x="${PADDING_X}" y="${baseline}" class="lang-label">${escapeXml(label.text)}</text>
|
||||
<rect x="${barX}" y="${baseline - 12}" width="${layout.barWidth}" height="${ROW_BAR_HEIGHT}" rx="${BAR_RADIUS}" class="bar-bg" />
|
||||
<rect x="${barX}" y="${baseline - 12}" width="${fillWidth}" height="${ROW_BAR_HEIGHT}" rx="${BAR_RADIUS}" class="${barClass(language)}" />
|
||||
<text x="${barX + layout.barWidth + ROW_PERCENT_GAP}" y="${baseline}" class="pct-label">${language.percent}%</text>`;
|
||||
<text x="${PADDING_X}" y="${baseline}" class="lang-label">${escapeXml(layout.labels[index].text)}</text>
|
||||
<rect x="${layout.barX}" y="${barY}" width="${layout.barWidth}" height="${BAR_HEIGHT}" rx="${BAR_RADIUS}" class="bar-bg" />
|
||||
<rect x="${layout.barX}" y="${barY}" width="${fillWidth}" height="${BAR_HEIGHT}" rx="${BAR_RADIUS}" class="${barClass(language)}" />
|
||||
<text x="${WIDTH - PADDING_X}" y="${baseline}" text-anchor="end" class="pct-label">${language.percent}%</text>`;
|
||||
}
|
||||
|
||||
function renderColumn(language, index, layout) {
|
||||
const { centerX, label } = layout.columns[index];
|
||||
const x = Math.round(centerX - layout.columnWidth / 2);
|
||||
const fillHeight = Math.round((language.percent / 100) * PLOT_HEIGHT);
|
||||
const labelY = layout.baseline + COLUMN_LABEL_GAP;
|
||||
const percentLabel = layout.showPercentages
|
||||
? `
|
||||
<text x="${centerX}" y="${layout.baseline - fillHeight - 7}" text-anchor="middle" class="pct-label-small">${language.percent}%</text>`
|
||||
: "";
|
||||
// Every tier is listed even when no language currently falls in it, so the colour
|
||||
// scale stays self-describing and the card height does not shift as progress moves.
|
||||
function renderLegend(layout) {
|
||||
let x = PADDING_X;
|
||||
|
||||
return `
|
||||
<rect x="${x}" y="${PLOT_TOP}" width="${layout.columnWidth}" height="${PLOT_HEIGHT}" rx="${BAR_RADIUS}" class="bar-bg" />
|
||||
<rect x="${x}" y="${layout.baseline - fillHeight}" width="${layout.columnWidth}" height="${fillHeight}" rx="${BAR_RADIUS}" class="${barClass(language)}" />${percentLabel}
|
||||
<text x="${centerX}" y="${labelY}" text-anchor="end" transform="rotate(-45 ${centerX} ${labelY})" class="lang-label-small">${escapeXml(label.text)}</text>`;
|
||||
}
|
||||
|
||||
function renderAxis(layout) {
|
||||
return AXIS_TICKS.map((tick) => {
|
||||
const y = layout.baseline - Math.round((tick / 100) * PLOT_HEIGHT);
|
||||
return `
|
||||
<line x1="${layout.plotLeft - 6}" y1="${y}" x2="${WIDTH - PADDING_X}" y2="${y}" class="grid" />
|
||||
<text x="${layout.plotLeft - 12}" y="${y + 3.5}" text-anchor="end" class="axis-label">${tick}</text>`;
|
||||
const entries = TIERS.map((tier) => {
|
||||
const entry = `
|
||||
<rect x="${x}" y="${layout.legendY - LEGEND_SWATCH}" width="${LEGEND_SWATCH}" height="${LEGEND_SWATCH}" rx="2" class="${tier.className}" />
|
||||
<text x="${x + LEGEND_TEXT_GAP}" y="${layout.legendY}" class="legend-label">${tier.label}</text>`;
|
||||
x += LEGEND_TEXT_GAP + tier.label.length * LEGEND_CHAR_WIDTH + LEGEND_ITEM_GAP;
|
||||
return entry;
|
||||
}).join("");
|
||||
|
||||
return `
|
||||
<line x1="${PADDING_X}" y1="${layout.lastRowY + LEGEND_RULE_GAP}" x2="${WIDTH - PADDING_X}" y2="${layout.lastRowY + LEGEND_RULE_GAP}" class="grid" />${entries}`;
|
||||
}
|
||||
|
||||
function renderBody(languages, layout) {
|
||||
if (layout.mode === "rows") {
|
||||
return languages.map((language, index) => renderRow(language, index, layout)).join("");
|
||||
}
|
||||
if (layout.count === 0) return "";
|
||||
|
||||
return renderAxis(layout) + languages.map((language, index) => renderColumn(language, index, layout)).join("");
|
||||
return languages.map((language, index) => renderRow(language, index, layout)).join("") + renderLegend(layout);
|
||||
}
|
||||
|
||||
export function generateSvg(statsData) {
|
||||
@@ -315,28 +323,25 @@ export function generateSvg(statsData) {
|
||||
? `Crowdin · ${targetCount} target ${targetCount === 1 ? "language" : "languages"} · ${averagePercent(languages)}% average`
|
||||
: "Crowdin · no translation data";
|
||||
|
||||
return `<svg xmlns="http://www.w3.org/2000/svg" width="${RENDERED_WIDTH}" height="${Math.round((layout.height * RENDERED_WIDTH) / WIDTH)}" viewBox="0 0 ${WIDTH} ${layout.height}" fill="none" role="img" aria-label="BookOrbit translation progress">
|
||||
return `<svg xmlns="http://www.w3.org/2000/svg" width="${WIDTH}" height="${layout.height}" viewBox="0 0 ${WIDTH} ${layout.height}" fill="none" role="img" aria-label="BookOrbit translation progress">
|
||||
<title>BookOrbit translation progress</title>
|
||||
<style>
|
||||
.card { fill: #0d1117; stroke: #30363d; stroke-width: 1px; rx: 12px; }
|
||||
.title { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 15px; font-weight: 600; fill: #f0f6fc; }
|
||||
.subtitle { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 12px; font-weight: 400; fill: #8b949e; }
|
||||
.lang-label { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 13.5px; font-weight: 500; fill: #c9d1d9; }
|
||||
.lang-label-small { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 11.5px; font-weight: 500; fill: #c9d1d9; }
|
||||
.pct-label { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 12.5px; font-weight: 600; fill: #58a6ff; }
|
||||
.pct-label-small { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 10.5px; font-weight: 600; fill: #8b949e; }
|
||||
.axis-label { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 10px; font-weight: 400; fill: #6e7681; }
|
||||
.lang-label { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 14px; font-weight: 500; fill: #e6edf3; }
|
||||
.pct-label { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 13.5px; font-weight: 600; fill: #e6edf3; font-variant-numeric: tabular-nums; }
|
||||
.legend-label { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif; font-size: 11px; font-weight: 400; fill: #8b949e; }
|
||||
.grid { stroke: #21262d; stroke-width: 1; }
|
||||
.bar-bg { fill: #21262d; }
|
||||
.bar-source { fill: #58a6ff; }
|
||||
.bar-high { fill: #2ea44f; }
|
||||
.bar-medium { fill: #d29922; }
|
||||
.bar-done { fill: #2ea043; }
|
||||
.bar-progress { fill: #2f81f7; }
|
||||
.bar-low { fill: #db6d28; }
|
||||
</style>
|
||||
<rect width="${WIDTH - 2}" height="${layout.height - 2}" x="1" y="1" class="card" />
|
||||
|
||||
<text x="${PADDING_X}" y="36" class="title">Translation Progress</text>
|
||||
<text x="${WIDTH - PADDING_X}" y="36" text-anchor="end" class="subtitle">${escapeXml(summary)}</text>
|
||||
<text x="${PADDING_X}" y="35" class="title">Translation Progress</text>
|
||||
<text x="${WIDTH - PADDING_X}" y="35" text-anchor="end" class="subtitle">${escapeXml(summary)}</text>
|
||||
|
||||
<line x1="${PADDING_X}" y1="${DIVIDER_Y}" x2="${WIDTH - PADDING_X}" y2="${DIVIDER_Y}" class="grid" />
|
||||
${renderBody(languages, layout)}
|
||||
@@ -357,12 +362,19 @@ async function writeFileAtomically(path, contents) {
|
||||
|
||||
export async function run(argv = process.argv.slice(2)) {
|
||||
const options = parseArguments(argv);
|
||||
const stats = await fetchCrowdinStats(options.statsUrl);
|
||||
const { stats, source } = await fetchStats({
|
||||
token: process.env.CROWDIN_TOKEN,
|
||||
statsUrl: options.statsUrl,
|
||||
projectId: options.projectId,
|
||||
});
|
||||
const svg = generateSvg(stats);
|
||||
const resolvedPath = resolve(process.cwd(), options.outputPath);
|
||||
await writeFileAtomically(resolvedPath, svg);
|
||||
|
||||
console.log(`Wrote ${resolvedPath} with ${extractLanguages(stats).length} languages`);
|
||||
console.log(`Wrote ${resolvedPath} with ${extractLanguages(stats).length} languages from the ${source}`);
|
||||
if (!process.env.CROWDIN_TOKEN) {
|
||||
console.warn("CROWDIN_TOKEN is not set, so these numbers may lag the project. Set it to read live progress.");
|
||||
}
|
||||
}
|
||||
|
||||
const entryPath = process.argv[1] ? resolve(process.argv[1]) : undefined;
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { computeLayout, extractLanguages, generateSvg, parseArguments } from "./generate.mjs";
|
||||
import { computeLayout, extractLanguages, fetchCrowdinApiProgress, fetchStats, generateSvg, parseArguments } from "./generate.mjs";
|
||||
|
||||
function language(id, name, locale, translationProgress, translated = 0, total = 0) {
|
||||
return { data: { languageId: id, language: { name, locale }, translationProgress, words: { translated, total } } };
|
||||
@@ -27,6 +27,50 @@ test("parseArguments rejects unknown flags and missing values", () => {
|
||||
assert.throws(() => parseArguments(["--out"]), /Missing value for --out/);
|
||||
});
|
||||
|
||||
test("parseArguments accepts a project override", () => {
|
||||
assert.equal(parseArguments([]).projectId, "912891");
|
||||
assert.equal(parseArguments(["--project", "42"]).projectId, "42");
|
||||
});
|
||||
|
||||
test("fetchStats reads live progress from the Crowdin API when a token is available", async () => {
|
||||
const calls = [];
|
||||
const fetchImpl = async (url, init) => {
|
||||
calls.push({ url, authorization: init.headers.Authorization });
|
||||
return new Response(JSON.stringify({ data: [language("de", "German", "de-DE", 93, 4921, 5322)] }));
|
||||
};
|
||||
|
||||
const { stats, source } = await fetchStats({ token: "secret-token", fetchImpl });
|
||||
|
||||
assert.equal(source, "Crowdin API");
|
||||
assert.match(calls[0].url, /\/projects\/912891\/languages\/progress/);
|
||||
assert.equal(calls[0].authorization, "Bearer secret-token");
|
||||
// The API payload is remapped into the badge feed shape the generator expects.
|
||||
assert.equal(stats.progress[0].data.languageId, "de");
|
||||
assert.equal(extractLanguages(stats)[0].percent, 93);
|
||||
});
|
||||
|
||||
test("fetchStats falls back to the cached badge feed when no token is set", async () => {
|
||||
let requested;
|
||||
const fetchImpl = async (url) => {
|
||||
requested = url;
|
||||
return new Response(JSON.stringify(sampleStats));
|
||||
};
|
||||
|
||||
const { stats, source } = await fetchStats({ fetchImpl });
|
||||
|
||||
assert.equal(source, "cached badge feed");
|
||||
assert.match(requested, /badges\.awesome-crowdin\.com/);
|
||||
assert.equal(stats.progress.length, sampleStats.progress.length);
|
||||
});
|
||||
|
||||
test("fetchCrowdinApiProgress rejects failed requests and malformed payloads", async () => {
|
||||
await assert.rejects(() => fetchCrowdinApiProgress("token", { fetchImpl: async () => new Response("nope", { status: 500 }) }), /HTTP 500/);
|
||||
await assert.rejects(
|
||||
() => fetchCrowdinApiProgress("token", { fetchImpl: async () => new Response(JSON.stringify({ unexpected: true })) }),
|
||||
/Invalid Crowdin progress response format/,
|
||||
);
|
||||
});
|
||||
|
||||
test("extractLanguages pins the source language at 100% and sorts targets by progress", () => {
|
||||
const languages = extractLanguages(sampleStats);
|
||||
assert.deepEqual(
|
||||
@@ -65,81 +109,98 @@ test("generateSvg renders every target language with its percentage", () => {
|
||||
assert.match(svg, />97%</);
|
||||
assert.match(svg, /4 target languages · 96% average/);
|
||||
assert.match(svg, /English/);
|
||||
assert.match(svg, /class="bar-source"/);
|
||||
assert.doesNotMatch(svg, />100%<.*\n.*bar-source/);
|
||||
});
|
||||
|
||||
test("generateSvg renders the source language as a full blue bar excluded from the average", () => {
|
||||
test("generateSvg renders the source language like any other completed row but keeps it out of the average", () => {
|
||||
const svg = generateSvg(sampleStats);
|
||||
const sourceBar = /<rect x="(\d+)"[^>]*width="(\d+)"[^>]*class="bar-source"/.exec(svg);
|
||||
assert.ok(sourceBar, "source bar must be rendered");
|
||||
assert.equal(Number(sourceBar[2]), computeLayout(extractLanguages(sampleStats)).barWidth);
|
||||
const [source] = extractLanguages(sampleStats);
|
||||
const layout = computeLayout(extractLanguages(sampleStats));
|
||||
|
||||
assert.equal(source.isSource, true);
|
||||
assert.equal(source.percent, 100);
|
||||
assert.doesNotMatch(svg, /bar-source/);
|
||||
// English sits at 100%, so it takes the completed tier and fills the whole bar.
|
||||
assert.match(svg, new RegExp(`width="${layout.barWidth}" height="8" rx="4" class="bar-done"`));
|
||||
assert.match(svg, /96% average/);
|
||||
});
|
||||
|
||||
test("generateSvg renders at its natural size so label type is never scaled down", () => {
|
||||
const { height } = computeLayout(extractLanguages(sampleStats));
|
||||
assert.match(generateSvg(sampleStats), new RegExp(`<svg [^>]*width="540" height="${height}" viewBox="0 0 540 ${height}"`));
|
||||
});
|
||||
|
||||
test("generateSvg grows the canvas instead of overflowing when languages are added", () => {
|
||||
const withExtra = generateSvg({ progress: [...sampleStats.progress, language("es", "Spanish", "es-ES", "40%", 4000, 19358)] });
|
||||
assert.match(withExtra, /viewBox="0 0 680 324"/);
|
||||
assert.match(withExtra, /viewBox="0 0 540 278"/);
|
||||
assert.match(withExtra, /🇪🇸 {2}Spanish/);
|
||||
assert.match(generateSvg(sampleStats), /viewBox="0 0 680 282"/);
|
||||
assert.match(generateSvg(sampleStats), /viewBox="0 0 540 250"/);
|
||||
});
|
||||
|
||||
test("computeLayout keeps horizontal rows up to eight languages", () => {
|
||||
const rows = Array.from({ length: 8 }, (_, index) => ({ name: `Language ${index}`, flag: "", percent: 50 }));
|
||||
assert.equal(computeLayout(rows).mode, "rows");
|
||||
assert.equal(computeLayout([...rows, { name: "One more", flag: "", percent: 50 }]).mode, "columns");
|
||||
test("computeLayout keeps horizontal rows at any language count", () => {
|
||||
const rows = (count) => Array.from({ length: count }, (_, index) => ({ name: `Language ${index}`, flag: "", percent: 50 }));
|
||||
const nine = computeLayout(rows(9));
|
||||
const twenty = computeLayout(rows(20));
|
||||
|
||||
assert.equal(nine.count, 9);
|
||||
assert.equal(twenty.height - nine.height, 11 * 28, "each extra language adds exactly one row");
|
||||
});
|
||||
|
||||
test("generateSvg switches to vertical columns beyond eight languages", () => {
|
||||
test("generateSvg renders many languages as rows without rotating labels", () => {
|
||||
const many = {
|
||||
progress: [
|
||||
language("en", "English", "en-US", "95%", 1, 2),
|
||||
...Array.from({ length: 12 }, (_, index) => language(`l${index}`, `Language ${index}`, "xx-XX", "50%", 1, 2)),
|
||||
],
|
||||
};
|
||||
const languages = extractLanguages(many);
|
||||
const layout = computeLayout(languages);
|
||||
assert.equal(layout.mode, "columns");
|
||||
assert.equal(layout.count, 13);
|
||||
|
||||
const svg = generateSvg(many);
|
||||
assert.match(svg, new RegExp(`viewBox="0 0 680 ${layout.height}"`));
|
||||
assert.equal(svg.match(/class="bar-bg"/g).length, 13);
|
||||
assert.equal(svg.match(/transform="rotate\(-45 /g).length, 13);
|
||||
assert.match(svg, /class="axis-label">100</);
|
||||
assert.ok(layout.columnWidth >= 6, "columns must stay visible");
|
||||
});
|
||||
|
||||
test("generateSvg keeps horizontal bars inside the card", () => {
|
||||
const svg = generateSvg(sampleStats);
|
||||
for (const [, x, width] of svg.matchAll(/<rect x="(\d+)" y="\d+" width="(\d+)"[^>]*class="bar-bg"/g)) {
|
||||
assert.ok(Number(x) + Number(width) <= 680 - 24, "bar must not overflow the card padding");
|
||||
}
|
||||
});
|
||||
|
||||
test("generateSvg keeps vertical columns inside the plot area", () => {
|
||||
const many = {
|
||||
progress: Array.from({ length: 20 }, (_, index) => language(`l${index}`, `Language ${index}`, "xx-XX", `${index * 5}%`, index, 100)),
|
||||
};
|
||||
const layout = computeLayout(extractLanguages(many));
|
||||
const svg = generateSvg(many);
|
||||
|
||||
for (const [, x, y, width, height] of svg.matchAll(
|
||||
/<rect x="(\d+)" y="(\d+)" width="(\d+)" height="(\d+)"[^>]*class="bar-(?:bg|high|medium|low|source)"/g,
|
||||
)) {
|
||||
assert.ok(Number(x) >= 24, "column must not overflow the left padding");
|
||||
assert.ok(Number(x) + Number(width) <= 680 - 24, "column must not overflow the right padding");
|
||||
assert.ok(Number(y) + Number(height) <= layout.baseline, "column must sit on the baseline");
|
||||
assert.equal(layout.count, 13);
|
||||
assert.match(svg, new RegExp(`viewBox="0 0 540 ${layout.height}"`));
|
||||
assert.equal(svg.match(/class="bar-bg"/g).length, 13);
|
||||
assert.doesNotMatch(svg, /rotate\(/);
|
||||
});
|
||||
|
||||
test("computeLayout sizes the label column to the widest name", () => {
|
||||
const short = computeLayout([{ name: "Thai", flag: "", percent: 50 }]);
|
||||
const long = computeLayout([
|
||||
{ name: "Thai", flag: "", percent: 50 },
|
||||
{ name: "Portuguese, Brazilian", flag: "", percent: 50 },
|
||||
]);
|
||||
|
||||
assert.ok(long.barX > short.barX, "a longer name must push the bar right");
|
||||
assert.equal(short.barX + short.barWidth, long.barX + long.barWidth, "bars stay right-aligned against the percentage column");
|
||||
});
|
||||
|
||||
test("computeLayout caps the label column so one long name cannot squeeze out the bar", () => {
|
||||
const layout = computeLayout([{ name: "A".repeat(200), flag: "🇦🇽", percent: 50 }]);
|
||||
assert.ok(layout.barWidth > 200, `bar collapsed to ${layout.barWidth}px`);
|
||||
assert.match(generateSvg({ progress: [language("xx", "A".repeat(200), "ax-AX", "50%")] }), /…/);
|
||||
});
|
||||
|
||||
test("generateSvg keeps bars inside the card", () => {
|
||||
const svg = generateSvg(sampleStats);
|
||||
assert.equal(svg.match(/class="bar-bg"/g).length, 5, "one track per language row");
|
||||
|
||||
for (const [, x, width] of svg.matchAll(/<rect x="(\d+)" y="\d+" width="(\d+)"[^>]*class="bar-(?:bg|done|progress|low)"/g)) {
|
||||
assert.ok(Number(x) >= 24, "bar must not overflow the left padding");
|
||||
assert.ok(Number(x) + Number(width) <= 540 - 24, "bar must not overflow the right padding");
|
||||
}
|
||||
});
|
||||
|
||||
test("generateSvg colours bars by progress", () => {
|
||||
test("generateSvg colours bars by progress and always explains the scale", () => {
|
||||
const svg = generateSvg({
|
||||
progress: [language("de", "German", "de-DE", "96%"), language("nl", "Dutch", "nl-NL", "70%"), language("sl", "Slovenian", "sl-SI", "12%")],
|
||||
progress: [language("de", "German", "de-DE", "100%"), language("nl", "Dutch", "nl-NL", "70%"), language("sl", "Slovenian", "sl-SI", "12%")],
|
||||
});
|
||||
assert.match(svg, /class="bar-high"/);
|
||||
assert.match(svg, /class="bar-medium"/);
|
||||
assert.match(svg, /class="bar-done"/);
|
||||
assert.match(svg, /class="bar-progress"/);
|
||||
assert.match(svg, /class="bar-low"/);
|
||||
|
||||
// Every tier stays in the legend even when no language currently falls in it.
|
||||
const complete = generateSvg({ progress: [language("de", "German", "de-DE", "100%")] });
|
||||
for (const label of ["Complete", "In progress", "Needs work"]) {
|
||||
assert.match(complete, new RegExp(`>${label}<`));
|
||||
}
|
||||
});
|
||||
|
||||
test("generateSvg escapes language names and handles empty data", () => {
|
||||
@@ -148,16 +209,6 @@ test("generateSvg escapes language names and handles empty data", () => {
|
||||
|
||||
const empty = generateSvg({ progress: [] });
|
||||
assert.match(empty, /no translation data/);
|
||||
assert.match(empty, /viewBox="0 0 540 96"/);
|
||||
assert.doesNotMatch(empty, /NaN|undefined/);
|
||||
});
|
||||
|
||||
test("computeLayout keeps rotated labels inside the card", () => {
|
||||
const languages = extractLanguages({
|
||||
progress: Array.from({ length: 21 }, (_, index) => language(`l${index}`, "Portuguese, Brazilian", "pt-BR", "50%", 1, 2)),
|
||||
});
|
||||
const layout = computeLayout(languages);
|
||||
|
||||
for (const column of layout.columns) {
|
||||
assert.ok(column.centerX - column.label.width * Math.SQRT1_2 >= 9, "rotated label must stay inside the card");
|
||||
}
|
||||
});
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
// Verifies that what Crowdin would export for a language matches the catalog in
|
||||
// Git, and optionally reconciles the difference.
|
||||
//
|
||||
// Crowdin's bulk translation import is lossy: it silently skips entries that the
|
||||
// per-string translation endpoint accepts, so a language can report a healthy
|
||||
// percentage while real translations are missing. Because untranslated strings
|
||||
// export as English source text, those gaps come back as a pull request that
|
||||
// overwrites good translations in Git. The progress percentage cannot detect
|
||||
// this; only diffing the built export against the catalog can.
|
||||
//
|
||||
// Usage:
|
||||
// CROWDIN_TOKEN=... node scripts/crowdin-verify/verify.mjs es-ES es.json
|
||||
// CROWDIN_TOKEN=... node scripts/crowdin-verify/verify.mjs es-ES es.json --reconcile
|
||||
//
|
||||
// Exits non-zero while any key still differs, so it can gate enabling export for
|
||||
// a newly seeded language.
|
||||
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import path from "node:path";
|
||||
|
||||
const API = "https://api.crowdin.com/api/v2";
|
||||
const SOURCE_PATH_SUFFIX = "/client/src/locales/en.json";
|
||||
const repoRoot = fileURLToPath(new URL("../..", import.meta.url));
|
||||
const localesDirectory = path.join(repoRoot, "client/src/locales");
|
||||
|
||||
export function flattenMessages(value, prefix = "", output = new Map()) {
|
||||
for (const [key, child] of Object.entries(value)) {
|
||||
const messageKey = prefix ? `${prefix}.${key}` : key;
|
||||
if (typeof child === "string") output.set(messageKey, child);
|
||||
else if (child && typeof child === "object") flattenMessages(child, messageKey, output);
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
export function differingKeys(desired, exported) {
|
||||
return [...desired.keys()].filter((key) => exported.get(key) !== desired.get(key));
|
||||
}
|
||||
|
||||
export function parseArguments(argv) {
|
||||
const positional = argv.filter((argument) => !argument.startsWith("--"));
|
||||
if (positional.length !== 2) {
|
||||
throw new Error("Usage: verify.mjs <crowdinLanguageId> <catalogFile> [--reconcile]");
|
||||
}
|
||||
return {
|
||||
languageId: positional[0],
|
||||
catalogFile: positional[1],
|
||||
reconcile: argv.includes("--reconcile"),
|
||||
};
|
||||
}
|
||||
|
||||
function createClient(token, projectId) {
|
||||
async function request(endpoint, init = {}) {
|
||||
const response = await fetch(`${API}${endpoint}`, {
|
||||
...init,
|
||||
headers: { Authorization: `Bearer ${token}`, ...(init.headers ?? {}) },
|
||||
});
|
||||
const body = await response.text();
|
||||
if (!response.ok) throw new Error(`${response.status} ${endpoint}: ${body.slice(0, 300)}`);
|
||||
return body ? JSON.parse(body) : null;
|
||||
}
|
||||
|
||||
return {
|
||||
request,
|
||||
|
||||
async sourceFileId() {
|
||||
for (let offset = 0; ; offset += 500) {
|
||||
const page = await request(`/projects/${projectId}/files?limit=500&offset=${offset}`);
|
||||
const match = page.data.find((entry) => entry.data.path.endsWith(SOURCE_PATH_SUFFIX));
|
||||
if (match) return match.data.id;
|
||||
if (page.data.length < 500) throw new Error(`No source file ending in ${SOURCE_PATH_SUFFIX}`);
|
||||
}
|
||||
},
|
||||
|
||||
async stringIds(fileId) {
|
||||
const ids = new Map();
|
||||
for (let offset = 0; ; offset += 500) {
|
||||
const page = await request(`/projects/${projectId}/strings?fileId=${fileId}&limit=500&offset=${offset}`);
|
||||
for (const entry of page.data) ids.set(entry.data.identifier, entry.data.id);
|
||||
if (page.data.length < 500) return ids;
|
||||
}
|
||||
},
|
||||
|
||||
async exportedCatalog(fileId, languageId) {
|
||||
const link = await request(`/projects/${projectId}/translations/builds/files/${fileId}`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ targetLanguageId: languageId }),
|
||||
});
|
||||
const download = await fetch(link.data.url);
|
||||
if (!download.ok) throw new Error(`Export download failed: ${download.status}`);
|
||||
return flattenMessages(JSON.parse(await download.text()));
|
||||
},
|
||||
|
||||
async putTranslation(languageId, stringId, text) {
|
||||
try {
|
||||
await request(`/projects/${projectId}/translations`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ stringId, languageId, text }),
|
||||
});
|
||||
return;
|
||||
} catch (error) {
|
||||
if (!/Duplicate translation/i.test(error.message)) throw error;
|
||||
}
|
||||
|
||||
// The wanted text already exists for this string but a newer translation
|
||||
// shadows it, so Crowdin rejects the write as a duplicate. Crowdin exports
|
||||
// the most recent translation, so drop the newer ones to make the wanted
|
||||
// text current again. Older history is left alone.
|
||||
const existing = await request(`/projects/${projectId}/translations?stringId=${stringId}&languageId=${languageId}&limit=100`);
|
||||
const match = existing.data.find((entry) => entry.data.text === text);
|
||||
if (!match) throw new Error(`Duplicate translation reported for string ${stringId} but no stored translation matches the catalog value`);
|
||||
|
||||
const shadowing = existing.data.filter(
|
||||
(entry) => entry.data.id !== match.data.id && Date.parse(entry.data.createdAt) >= Date.parse(match.data.createdAt),
|
||||
);
|
||||
for (const entry of shadowing) {
|
||||
await request(`/projects/${projectId}/translations/${entry.data.id}`, { method: "DELETE" });
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const { languageId, catalogFile, reconcile } = parseArguments(process.argv.slice(2));
|
||||
const token = process.env.CROWDIN_TOKEN;
|
||||
if (!token) throw new Error("CROWDIN_TOKEN is required");
|
||||
const projectId = process.env.CROWDIN_PROJECT_ID ?? "912891";
|
||||
|
||||
const client = createClient(token, projectId);
|
||||
const fileId = await client.sourceFileId();
|
||||
const desired = flattenMessages(JSON.parse(await readFile(path.join(localesDirectory, catalogFile), "utf8")));
|
||||
|
||||
// Crowdin serves a briefly cached export build, so a rebuild taken immediately
|
||||
// after a write can still show the old content. Retry before believing a
|
||||
// non-zero result.
|
||||
async function differencesNow(attempts = 1) {
|
||||
let missing = [];
|
||||
for (let attempt = 1; attempt <= attempts; attempt += 1) {
|
||||
missing = differingKeys(desired, await client.exportedCatalog(fileId, languageId));
|
||||
if (missing.length === 0) return missing;
|
||||
if (attempt < attempts) await new Promise((resolve) => setTimeout(resolve, 5000));
|
||||
}
|
||||
return missing;
|
||||
}
|
||||
|
||||
let missing = await differencesNow();
|
||||
console.log(`${languageId}: ${missing.length} of ${desired.size} keys differ from ${catalogFile}`);
|
||||
|
||||
if (missing.length > 0 && reconcile) {
|
||||
const ids = await client.stringIds(fileId);
|
||||
let pushed = 0;
|
||||
for (const key of missing) {
|
||||
const stringId = ids.get(key);
|
||||
if (!stringId) {
|
||||
console.log(` skipped ${key}: not present in Crowdin`);
|
||||
continue;
|
||||
}
|
||||
await client.putTranslation(languageId, stringId, desired.get(key));
|
||||
pushed += 1;
|
||||
}
|
||||
console.log(` pushed ${pushed} translations through the per-string endpoint`);
|
||||
missing = await differencesNow(4);
|
||||
console.log(`${languageId}: ${missing.length} keys still differ after reconcile`);
|
||||
}
|
||||
|
||||
for (const key of missing.slice(0, 20)) console.log(` ~ ${key}`);
|
||||
if (missing.length > 20) console.log(` ... and ${missing.length - 20} more`);
|
||||
|
||||
if (missing.length > 0) {
|
||||
console.error(`\n${languageId} is NOT safe to export: Crowdin would overwrite ${missing.length} keys in ${catalogFile}.`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
console.log(`\n${languageId} matches ${catalogFile} exactly; exporting cannot change it.`);
|
||||
}
|
||||
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
try {
|
||||
await main();
|
||||
} catch (error) {
|
||||
console.error(error.message);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user