English language support for AI presents something of a paradox: it’s the default while also being a linguistic outlier.
Here's a message a helpful assistant might send in English:
"Hey! Send me that file when you get a sec."
Friendly, casual, and perfectly normal. Now take the direct Japanese translation:
ねえ、時間あるときにそのファイル送って。
Nee, jikan aru toki ni sono fairu okutte.
Every word is accurate and corresponds to something from the original English source message. However, said to a Japanese-speaking client or a colleague you don't know well, it reads as curt, even rude. A native speaker would expect something closer to:
お手すきの際に、そのファイルをお送りいただけますでしょうか。
Otesuki no sai ni, sono fairu wo o-okuri itadakemasu deshō ka.
(Roughly: "When you have a free moment, might I humbly ask you to send that file?")
Nothing in the first translation is wrong, yet it still fails to communicate the full message.
%402x.png)
That gap is easy to miss because most large language model development, from the tooling and rating rubrics to the general expectations for how a model should behave, was built around English. English is also a linguistic outlier, which means the foundation everyone is extending to other languages is riskier than it looks. It's why multilingual large language model (MLLM) training is harder than it seems: models built on that foundation often perform well in high-resource languages - those with an abundance of digital data - and quietly fail in low-resource ones.
If your per-language evals swing, or inter-rater agreement drops the moment a project leaves English, the cause usually sits upstream of the model: in the rubric and the rater pool.
Compared with most of the world's languages, English is missing a lot of common linguistic features.
None of this makes English better or worse, but it does make English a poor stand-in for everything else when it comes to natural language processing. Because so much AI tooling was trained on it, the assumptions are baked in quietly. A rubric may not ask whether the model used the right honorific, because English never needed one.
The result is that when a project moves into a language that works differently, teams usually find the gap late, only after it has caused a quality problem, a mismatched rater pool, or a model that just sounds off.
Each language brings requirements English never had to handle, so it can't be added as a translation layer on top of an English model.
These gaps tend to show up in three places, and each one is expensive to discover late.
An English-built quality rubric can leave out a feature like honorifics entirely, even in languages where using the right one is obligatory. The subtler problem is what happens next. When a model fails in a way the rubric does not categorize, raters have no consistent way to score it. Some penalize it, some don't, and the resulting noise looks like rater disagreement or even underreporting of model errors. The real problem is a gap in the rubric.
That noise flows straight into training. Rubrics shape the demonstrations written for supervised fine-tuning (SFT), the preference pairs behind reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO), and the criteria used to calibrate LLM-as-a-judge evaluators. An English rubric translated into another language carries English's blind spots into every one of them.
A client asks for "Arabic." By default, that usually means Modern Standard Arabic, the formal written language. But people actually speak eight main supraregional varieties plus dozens of local ones, and speakers of distant varieties can struggle to understand each other. Hire generically and the people producing the data won't match the people the model is meant to serve. Or, worse, they produce unintentionally mixed dialectal data that makes model performance more unstable.
%402x.png)
Output can be grammatically perfect and still land badly. Asked for help getting finances back on track, one model answered in Arabic with advice like "وقف النزيف فوراً" ("stop the bleeding immediately") and "مهما حدث أثناء فترة العلاج" ("no matter what happens during the treatment period"). Every word is correct, but the urgency and medical framing come straight from English idiomatic business phrasing. In Arabic, that reads as exaggerated and a little strange, which hurts the advice's credibility. A native speaker would expect something calmer, like "خلال هذه المرحلة، يُفضَّل تقليل الاستدانة قدر الإمكان" ("during this phase, it's best to reduce borrowing as much as possible"), which doesn't mix business and medical metaphors.
The usual source is translated training data. Many multilingual fine-tuning pipelines build examples by running English through machine translation, back-translation, or cross-lingual instruction tuning. Some route every request through English as a pivot language, so the model reasons in English and translates the answer. The result is that idioms flatten, metaphors carry over, and the register lands wrong.
Voice and writing direction add their own risks. In tonal languages like Mandarin, a pitch error doesn't change emphasis; it changes the word. In right-to-left scripts like Hebrew or top-to-bottom scripts like Mongolian, a single English word or emoji dropped in the middle of a sentence can scramble how the text renders.
The fix is to treat each language, and often each dialect, as its own project with its own requirements, starting before any data is collected. More translation won't get you there.
We source at the dialect level, hiring for the variety a model needs to speak, not just the language name on the request. When a client asks for "Arabic" or "Portuguese," we work with them to pin down which variants matter for their users and goals. Then we source native-speaker raters and voice talent for each one deliberately, instead of hoping a general pool covers it, so SFT demonstrations and RLHF preference data come from the people the model is meant to serve.
We build rubrics for the language from the start, adapting them to each language before rating begins rather than retrofitting an English rubric after scores come back noisy. For languages with more than one accepted standard, we define which one is being graded against, such as Brazilian vs. European Portuguese or Swiss vs. Moroccan French. That way, two qualified raters don't penalize each other's correct forms. For projects focused on cultural alignment, rubrics can go further and target a specific region or local dialect.
And we set model behavior per language. How a model addresses its users is a core part of its personality: formal or informal "you," which honorifics it uses, and how it handles gender when the grammar demands it. We make those choices explicitly for each language instead of letting them default to whatever English implies.
Our teams build multimodal training datasets from the ground up. On one occasion, for a leading AI developer, they vetted multilingual real-world images and wrote natural user requests across ten use cases, from OCR to chart reasoning, with ideal responses grounded in what each image actually shows. The foreign-language work went well beyond word-for-word output, from extracting Arabic text from scanned documents to translating English into Hungarian in the informal tegező register. That's the kind of formality choice an English-built rubric would never think to check.
To make that work repeatable, we built the Language Atlas: a sourced, linguist-verified database and interactive map for multilingual AI development, covering 140+ languages and more than 1,700 dialects and regional varieties. Each one is scored on 11 traits that tend to break multilingual quality, including gender, honorifics, formality, taboo, tonality, writing system and direction, word order, and how standardized the language is.
The data draws on established references like WALS, Glottolog, Ethnologue, and Grambank, plus input from native speakers on which dialects matter most in practice.
Our teams use it in two ways. As a planning tool, it helps clients decide where to focus at the dialect level, providing answers to their questions like "Which Arabic dialects should we hire for?", "Why can't a friendly English tone just be translated into Japanese?", and "Which languages carry meaning-changing pitch risk for text-to-speech?" As a pre-flight check, it shows where a target language departs from English before a project launches, so those differences are built into hiring criteria and rubric design from day one.
Our multilingual teams use the Atlas alongside their own experience. The database flags the risks, and they decide what to do about them.
The most expensive multilingual problems are set early and discovered late: which dialect you hire for, which standard you grade against, and which register the model speaks in. Each is easy to resolve at the planning stage and costly to unwind once data has been collected and scored.
Talk to our multilingual team about where your models need to go next.
Most large language model development was built around English, from the tooling and rating rubrics to the expectations for how a model should behave. But English is a linguistic outlier. It has no grammatical gender, few honorifics, little formality marking and no tones. When projects move into languages that work differently, those English assumptions come with them. That’s why models often perform well in high-resource languages and quietly fail in low-resource ones.
Translated data carries English with it. Many multilingual fine-tuning pipelines build training examples by running English text through machine translation or back-translation. Some route every request through English as a pivot language. The words may all be correct, but idioms flatten, metaphors carry over and the register lands wrong. The result is output that reads as translated English rather than something a native speaker would say.
An English rubric can leave out features like honorifics entirely, even in languages where using the right one is obligatory. When a model makes a mistake the rubric doesn’t cover, raters have no consistent way to score it, so the scores get noisy. That noise flows into SFT demonstrations, RLHF and DPO preference data, and LLM-as-a-judge calibration. Invisible adapts each rubric to its language before rating begins. Where a language has more than one accepted standard, such as Brazilian and European Portuguese, we define which one is being graded against.
The Language Atlas is Invisible’s linguist-verified database and interactive map for multilingual AI development. It covers 140+ languages and more than 1,700 dialects and regional varieties. Each one is scored on 11 traits that tend to break multilingual quality, including gender, honorifics, formality, tonality and writing direction. The data draws on established references like WALS, Glottolog, Ethnologue and Grambank, plus input from native speakers on which dialects matter most in practice.
