How we measure Image Translator
Every version is tested before release. We measure what you actually read — the translation of the picture — not only the recognized text. All numbers below come from one run on 2026-10-18: MacBook Air M3, Chrome 155.0.8059.82, recognition on the graphics card (WebGPU), build 0.1.0.
Test sets
- Synthetic pictures from FLORES-200. FLORES-200 is the same set of sentences translated by people into 200 languages (Meta, CC BY-SA 4.0). We render them in Chrome, with system fonts, as 8 kinds of pictures — 27 languages × 8 kinds × 3 samples = 648 images. The sentences and layouts are the same in every language, so languages can be compared with each other. Hebrew and Bengali are included on purpose: the extension can't read them yet, and the test shows it; they are not in the averages.
- Real scans from XFUND. 70 scanned forms — 10 each in Chinese, Japanese, German, French, Spanish, Italian and Portuguese — with handwritten fields, tables and stamps.
Measures
- Translation chrF++ — the translation of the picture against the human translation from FLORES-200: character n-grams up to 6 and word pairs, β = 2, case-insensitive, checked against sacrebleu. The main number.
- Ceiling — the same Chrome translator on the error-free text of the picture. Ceiling minus translation (loss) is everything that recognition and layout cost; the ceiling itself is a property of Chrome's translator.
- CER — character error rate of recognition against the text on the picture (for Chinese, Japanese and Thai without spaces).
- Real scans have no human translation, so there we measure how much of the translation is kept (chrF++ against the translation of the error-free text) and words as a bag against the dataset's annotation: F1, recall (share of words found) and precision.
- Time — median per picture, from the image bytes to the translation: decoding, text detection, script probe (in automatic mode), line recognition, translation.
Results by language
Translation into Russian (for Russian pictures — into English), language of the picture given. With automatic language detection the overall result is the same: 46.3 chrF++.
| Language | Translation chrF++ | Ceiling | Loss | CER % | Time, ms |
|---|---|---|---|---|---|
| All readable languages | 46.3 | 49.1 | 2.8 | 1.9 | 286 |
| English | 56.3 | 57.4 | 1.1 | 0.2 | 448 |
| German | 55.1 | 55.2 | 0.0 | 0.1 | 537 |
| French | 51.9 | 52.2 | 0.3 | 0.3 | 279 |
| Spanish | 46.2 | 46.4 | 0.2 | 0.0 | 222 |
| Italian | 45.9 | 46.5 | 0.6 | 0.1 | 267 |
| Portuguese | 50.9 | 54.1 | 3.3 | 0.4 | 254 |
| Polish | 47.8 | 48.7 | 0.9 | 0.7 | 258 |
| Czech | 51.5 | 52.3 | 0.8 | 0.6 | 205 |
| Turkish | 49.5 | 50.0 | 0.5 | 1.8 | 263 |
| Vietnamese | 29.7 | 47.8 | 18.1 | 11.6 | 355 |
| Indonesian | 47.3 | 47.3 | 0.0 | 0.1 | 206 |
| Russian | 60.8 | 61.0 | 0.3 | 0.9 | 323 |
| Ukrainian | 51.5 | 53.8 | 2.3 | 2.6 | 398 |
| Bulgarian | 50.8 | 53.8 | 3.0 | 1.3 | 399 |
| Greek | 45.9 | 46.7 | 0.8 | 0.9 | 488 |
| Chinese (Simplified) | 45.0 | 45.5 | 0.5 | 0.1 | 155 |
| Chinese (Traditional) | 41.4 | 42.3 | 0.9 | 1.0 | 205 |
| Japanese | 43.7 | 44.9 | 1.2 | 0.2 | 230 |
| Korean | 40.3 | 44.1 | 3.8 | 5.3 | 333 |
| Arabic | 38.7 | 51.5 | 12.7 | 6.5 | 250 |
| Hindi | 46.2 | 47.0 | 0.8 | 2.4 | 299 |
| Marathi | 40.2 | 44.4 | 4.2 | 2.7 | 304 |
| Thai | 39.9 | 44.6 | 4.7 | 1.2 | 306 |
| Tamil | 41.1 | 44.1 | 3.1 | 2.6 | 373 |
| Telugu | 40.7 | 46.7 | 5.9 | 3.6 | 315 |
| Hebrew | 2.4 | 51.0 | 48.6 | 90.8 | 886 |
| Bengali | 2.9 | 44.2 | 41.3 | 86.6 | 1786 |
Median time per picture: 286 ms (90th percentile 627 ms), of which recognition 130 ms and translation 81 ms.
Results by kind of picture
| Kind of picture | Translation chrF++ | Ceiling | Loss | CER % | Time, ms |
|---|---|---|---|---|---|
| All kinds | 46.3 | 49.1 | 2.8 | 1.9 | 286 |
| Banner — large bold text on a plate or gradient | 48.0 | 50.3 | 2.3 | 2.5 | 130 |
| Meme — white outlined text over a photo, often in capitals | 50.7 | 54.1 | 3.4 | 2.6 | 170 |
| Screenshot — a 14–16 px paragraph, like a feed | 47.9 | 51.2 | 3.3 | 1.6 | 379 |
| Retina screenshot — the same at 2× | 45.9 | 48.1 | 2.2 | 1.3 | 421 |
| Promo card — text over a shadow or a plate on colored shapes | 40.0 | 41.2 | 1.2 | 1.1 | 123 |
| Photo of a sign — at an angle, in perspective, with grain and blur | 51.7 | 54.0 | 2.3 | 1.6 | 138 |
| Social post — a heading and a paragraph | 45.9 | 47.9 | 2.0 | 1.7 | 529 |
| Low quality — downscaled to 72–85% and recompressed as JPEG | 43.8 | 48.3 | 4.5 | 2.9 | 343 |
Speed of recognition
79 msrecognition per picture on the graphics card (WebGPU), median. 90% of pictures take under 185 ms in total.
275 msrecognition per picture on the processor (WebAssembly), median. 90% of pictures take under 649 ms in total.
Real scanned forms (XFUND)
| Language | Translation kept, chrF++ | Words F1 | Recall | Precision | Time, ms |
|---|---|---|---|---|---|
| All | 69.9 | 95.6 | 95.6 | 95.6 | 2584 |
| Chinese (Simplified) | 63.4 | 98.0 | 98.3 | 97.6 | 2283 |
| Japanese | 64.1 | 92.9 | 91.4 | 94.4 | 2734 |
| German | 69.9 | 97.5 | 96.6 | 98.4 | 1932 |
| French | 73.9 | 96.2 | 95.7 | 96.6 | 1497 |
| Spanish | 77.7 | 98.4 | 98.2 | 98.5 | 3264 |
| Italian | 74.2 | 92.3 | 95.1 | 89.7 | 3815 |
| Portuguese | 67.3 | 95.7 | 94.9 | 96.6 | 3437 |
Median time per form: 2584 ms, of which recognition 955 ms and translation 819 ms.
Known weak spots
- Vietnamese — the recognizer's alphabet lacks some tone-marked letters, so translation loses 18.1 points.
- Arabic loses 12.7 points; Telugu 5.9, Thai 4.7, Marathi 4.2, Korean 3.8.
- Hebrew and Bengali can't be read yet; Kannada is not in the test and can't be read either.
- Handwriting is recognized only partially.
Free · No account · Desktop Chrome 138+ on Windows, macOS, Linux