Engineering

How Arabizi Transliteration Works (and Why Letter-by-Letter Converters Fail)

A transliteration engineer's look at converting Arabizi to Arabic: the information Latin script leaves out, why letter-by-letter mapping fails, and how a dictionary + lattice + lexicon ranking engine fixes it.

Abdessamad Bettal11 min read
The Arabizi word 7abibi converted to the Arabic حبيبي

In brief: converting Arabizi to Arabic is not a character substitution problem. Arabizi leaves out information Arabic spelling requires — long vs. short vowels, emphatic letters, the shape of word endings and the seat of the hamza — so a good converter must guess it. The Arabizi.io engine does this in three layers: a dictionary of real Arabizi spellings, a weighted "pronunciation lattice" of every plausible Arabic spelling, and a beam search that ranks those spellings against a 40,000-word Arabic frequency lexicon. It runs entirely in the browser in well under a millisecond per word.

This article explains the problem and the design in detail. I'm the developer of the engine, so the examples and numbers below come from building and testing it.

The problem: Arabizi under-specifies Arabic

Arabizi writes how a word sounds. Arabic spelling encodes several things the sound doesn't make obvious to a Latin-script writer. For every word, a converter has to recover:

What's missingArabiziPossible ArabicCorrect
Vowel length — is the vowel written with ا/و/ي or left out?kitabكتب / كتاب / كيتابكتاب
Emphatic vs. plain consonantssabahسباح / صباحصباح
Which hhobهب / حبحب
Word-final a — ة, ا, ى, or nothing?jamila · marhaba · mustashfaجميلة · مرحبا · مستشفى(three different endings)
Hamza seat — أ, إ, ؤ, ئ or ء?mas2ala · ra2is · su2alمسألة · رئيس · سؤال(depends on neighboring vowels)
Initial vowelana · islamأنا · إسلامneeds أ or إ
Doubled consonantssayyaraسييارة / سيارةسيارة (written once)
Attached wordsw7abibi · bel3arabiو + حبيبي · ب + ال + عربيوحبيبي · بالعربي
Assimilated articleassalam · ennasاسلام / السلامالسلام · الناس

On top of that, Arabizi is not standardized and is mostly written in dialect. The same word can be spelled shukran, shokran, choukran or shukren, and 2 can mean hamza (2ana) or, in Cairo and Beirut, ق (2albi = قلبي).

Why letter-by-letter converters fail

The simplest approach maps each Latin letter to one Arabic letter and drops short vowels. Before rewriting our engine in October 2026, ours worked mostly that way, with a small dictionary on top. Here is what it produced on ordinary words — real output from that version:

InputOld outputProblemNew output
jamilaجملَdropped the long ī, ended in a stray fathaجميلة
kalimaكلمَfinal -a should be ةكلمة
sayyaraساييرَdoubled y written twiceسيارة
2anaءنَhamza not seated on alif, stray fathaأنا
l9ahwaلقهوَarticle not recognizedالقهوة
bel3arabiبلعربيclitic + article not splitبالعربي
mas2alaمسءلَhamza seat depends on the vowels; ة missingمسألة
johnجهنsilent h, vowel droppedجون
jenniferجننفرforeign names spell every vowelجينيفر

Each fix in that last column needs knowledge that isn't in the letters: Arabic morphology (ة endings, the article), orthography rules (hamza seats), and the vocabulary itself (that جميلة is a word and جملة is a different one).

Layer 1: a dictionary of real Arabizi spellings

The first thing the engine does is look the word up in a curated dictionary: several hundred common words and expressions across Modern Standard Arabic and the four big dialect groups (Egyptian, Levantine, Gulf, Maghrebi), plus several hundred personal names. Each entry lists the spelling variants people actually type, so shukran, shokran, choukran and chokran all map to شكراً.

Before giving up on the dictionary, the engine peels off clitics — the one-letter words Arabic attaches to the next word — and tries again:

  • w7abibi → و + 7abibi → وحبيبي
  • bel3arabi → ب + ال + 3arabi → بالعربي
  • l9ahwa → ال + 9ahwa → القهوة
  • fil-madrasa → في + ال + madrasa → في المدرسة (two words in Arabic)
  • assalam → sun-letter article → السلام

Dictionary hits always rank first, because for frequent words a human-curated spelling beats any statistical guess.

Layer 2: the pronunciation lattice

For everything else, the engine builds a lattice: a graph where each Latin unit expands into every Arabic reading it could have, each with a cost (lower is more likely). A simplified excerpt:

Latin unitArabic options (cost)
sس (0) · ص (0.5)
hه (0) · ح (0.3)
tت (0) · ط (0.5)
sh, kh, ghش · خ · غ (0)
3, 7, 5, 9ع · ح · خ · ق (0); 9 also ص (0.4)
internal anothing (0) · ا (0.35)
internal inothing (0.05) · ي (0.3)
final a (long word)ة (0) · ا (0.15) · ى (0.5)
final anـاً tanween (0) · ان (0.3)
initial a / iأ / إ (0) · ا (0.3)
2 between vowelsأ · ئ · ؤ — cheapest one depends on the neighboring vowels

A few context rules shape the costs:

  • The last vowel of a word that ends in a consonant is probably long. In Arabic nouns like kitāb, salām and jamīl the final syllable's vowel is long, so writing its mater (ا/و/ي) is made cheaper there.
  • Doubled consonants are written once (ss, yy, 77), because Arabic marks gemination with an optional shadda rather than a second letter.
  • A capital S, D, T, Z or H is a hint for the emphatic letter (ص ض ط ظ ح) — except at the start of a word, where it's usually just capitalization.
  • s+h, k+h, g+h read separately are heavily penalized, so sh is ش and not سه.

Layer 3: ranking with a frequency lexicon

A lattice for an eight-letter word can encode thousands of spellings. To pick the right one, the engine runs a beam search left to right and, once the lexicon is loaded, prunes every path that isn't the beginning of a real Arabic word. The prefix check is a binary search over a sorted list of 40,000 words, so it costs microseconds.

Each complete spelling gets a score:

score = lattice cost + 0.3 × ln(frequency rank + 1), or lattice cost + 4 if the spelling isn't in the lexicon

So a common word with a slightly unusual mapping (جميلة, rank ~350) beats a rare word or a non-word with a cheaper mapping. The lexicon comes from the open FrequencyWords list built from Arabic film subtitles (CC BY-SA 4.0), cleaned of diacritics and noise. Subtitles turn out to be an excellent source for this job: they contain conversational vocabulary and thousands of foreign names already written in Arabic — جون, مايكل, جينيفر — which is exactly what people type.

The top-ranked spelling is inserted as you type; the next candidates appear as chips you can pick with a click or the Tab key.

Names need different rules

Foreign names are spelled differently from native Arabic words: every vowel is written out so the name can be read (Peter → بيتر, Lucas → لوكاس), and a final -a is usually ا, not ة (Maria → ماريا). When the name in Arabic tool calls the engine, it switches on a name mode that makes dropping a vowel expensive and disables tanween. Without it, short common words win — "Theo" would come out as ذو ("possessor of") instead of ثيو.

Arabic back to Arabizi

The reverse direction is easier but has its own trap: unvocalized Arabic doesn't say which short vowels to write. The Arabic to Arabizi converter uses a reverse dictionary for common words (قهوة → 9ahwa, حبيبي → 7abibi) and a letter-level mapping for the rest. When the input carries tashkeel it follows the marks exactly — shadda doubles the consonant (مُحَمَّد → mu7ammad), the article assimilates before sun letters (الشَّمْس → ash-shams), and a final ة takes its pausal form (المَدْرَسَةِ → al-madrasa).

Performance and privacy

  • Size: the lexicon is ~470 KB of text, about 166 KB gzipped, downloaded once after the page has rendered, so it doesn't slow the first paint. Until it arrives, the dictionary and lattice already work.
  • Speed: roughly 0.1–1 ms per word on a laptop, fast enough to re-run on every keystroke.
  • Privacy: everything runs on your device. No keystrokes, words or text are sent to a server.

Known limitations

No Arabizi converter can be perfect, because the input genuinely lacks information:

  • Context-dependent words. 3am can be عام ("year") or Levantine عم (the "-ing" particle); the engine picks one by frequency. A future version will look at neighboring words.
  • Rare dialect words that aren't in the dictionary or lexicon fall back to a phonetic spelling, which is readable but not always the conventional one.
  • Unvocalized Arabic → Arabizi can only guess short vowels for uncommon words.

That's why every choice is visible and reversible: the alternatives are always one click away.

Try it

Keep reading