How many characters your language gets in a WordPress URL

When The 200-byte trap went up, a reader on CoderLegion asked the question I should have answered in the article itself: is this only an Arabic problem?

It isn't. But "it isn't" is not much of an answer, and the one I gave in the comments was arithmetic, not measurement. So I put a typical news headline in fifteen languages through a fresh WordPress 7.1.3 site and looked at what came out the other end. Then I opened the resulting URLs the ways readers actually open them: whole, cut short, and after the slug had changed.

The short answer: English gets 200 characters. Arabic, Hebrew, Greek and Russian get about 33. Hindi, Thai, Chinese and Japanese get 22. The longer answer has a few surprises in it, and one setting that makes most of it stop mattering.

The rule: 200 characters, counted after encoding

WordPress builds a slug from the title in sanitize_title_with_dashes(). Anything outside plain ASCII is stored percent-encoded, the same %XX form it travels in, and the cut is applied to that encoded form, in one line of current core:

$title = utf8_uri_encode( $title, 200 );

Every UTF-8 byte becomes a three-character %XX token, so what a letter costs depends only on how many bytes its script needs:

• a Latin letter, a digit or a hyphen: 1 byte, costs 1
• Arabic, Persian, Urdu, Hebrew, Greek, Cyrillic: 2 bytes, cost 6
• Hindi, Thai, Chinese, Japanese, Korean: 3 bytes, cost 9
• an emoji: usually 4 bytes, costing 12, and some cost far more

Divide 200 by the cost and you have the ceiling. The cut never splits a character in half, because utf8_uri_encode() stops before one that would not fit. It does split words: nothing in it knows where a word ends. Category and tag slugs go through the same function, with the same 200.

The emoji line hides two surprises. ❤️ is two code points at 9 each, and the second is an invisible one that stays in the URL. A family emoji is three people held together by invisible joiners, and WordPress counts the joiners, then strips them, so the URL ends up with three separate people.

Every slug gets the same 200 characters. A Latin letter spends one of them, an Arabic, Hebrew, Greek or Russian letter six, a Hindi, Thai, Chinese, Japanese or Korean character nine, and an emoji twelve. What is left over is too small for one more.

The numbers, measured

Here is what WordPress 7.1.3 kept of a news headline in each language. "Cost per letter" is counted after accents are removed, so é costs what e does. "Characters kept" includes the hyphens that replaced the spaces, and the last word counted is usually a fragment.

Three things in that table are worth a second look.

The two-byte scripts stopped at 32 letters, not 33, and Persian at 30. Hyphens come out of the same 200, so a headline with six spaces keeps 32 letters and six hyphens, and Persian's missing two are explained further down. In practice that is five to nine words, depending on how long the words are, and the last is usually cut: the Arabic slug ends in «ال», the definite article of a word that did not fit.

Twenty-two characters is fewer letters than it sounds in Hindi and Thai. The count is of code points, and their vowel signs and tone marks are code points of their own. The Thai slug ends in «ใหม», which is «ใหม่» without its tone mark.

Thai, Chinese and Japanese get no hyphens at all. They don't put spaces between words, so the whole slug is a single run of 22 characters.

One more cost doesn't show in the table. When a slug is already taken, WordPress cuts it again to fit a suffix, so a second and a third post with the same Urdu headline each lost a letter to make room for -2 and -3. And with only five to nine words to tell headlines apart, non-Latin slugs are likelier to collide in the first place, for the same reason cut-off links go astray below.

Accented Latin gets the full 200, and the site language decides how

French, German and Vietnamese kept as much as English because their accents never reach the encoder. When a slug is saved, sanitize_title() runs remove_accents() first: é becomes e, and the whole Vietnamese headline came through as plain ASCII.

How a letter is transliterated depends on the site language. On a site set to English, «Maßnahmen» became masnahmen, and «Größe» would become grose, which is a different word. With the site set to German, the same function produces massnahmen and groesse. Danish (soester, paa), Catalan (collegi) and Serbian (djordje) get the same care from their own locales. Slugs are generated when a post is saved, so set the site language before the archive grows, not after.

What spends the budget without showing up

The cut happens before WordPress tidies the slug, so some characters are counted first and only then removed or replaced. Arabic and Persian headlines are full of them, and English ones are not free of them either.

• Angle quotes. «» cost 6 each and are stripped once counted. A headline that opened with «عاجل» and quoted one more word kept one letter of «الشركات», where the same headline without the quotes kept five, «الشرك».
• Invisible marks. A right-to-left mark costs 9, and so does the Persian zero-width non-joiner. Both are stripped. The Persian headline shows a second cost too: «کسب‌وکار» came out as «کسبوکا», glued together and then cut.
• Curly punctuation, in any language. A curly apostrophe, as in what’s, costs 9 and is stripped, and so are curly quotes and the ellipsis. En and em dashes cost 9 and become hyphens. Headlines pasted from a word processor tend to be full of them: a long English headline with four apostrophes, a pair of curly quotes and a dash lost 62 characters of its slug to them, and stopped at 133.

What spends the budget and stays in the URL

Worse are the characters that are counted and then kept. WordPress's list of characters to strip was written for Latin text, so marks from other scripts reach the URL at full price.

• Arabic-Indic digits. ٥٠٠٠ costs 24 where 5000 costs 4, and the cut can fall inside a number. A headline about a 5,000-dinar grant came out ending in «بقيمة-٥٠٠», worth 500: a figure that looks right and isn't. Typed with Western digits, the same headline kept the whole number.
• Diacritics. Each tashkeel mark costs 6. A fully vocalised headline fitted three and a half words, «الحُكُومَةُ-تُعْلِنُ-خُطَّةً-جَدِيد».

Punctuation from other scripts is kept too, at 6 or 9 a mark. This is core's own output:

Arabic comma and question mark
عاجل،-الحكومة-تعلن-خطة؟-جديدة

Arabic semicolon
أولاً؛-ثانياً

Urdu full stop
حکومت-کا-اعلان۔-نیا-منصوبہ

Japanese punctuation
「速報」政府が新計画を発表！

Chinese punctuation
政府宣布，新计划。

The worst are invisible. WordPress strips the old bidirectional embedding marks once it has counted them, but it keeps the newer isolates, U+2066 to U+2069, and the Arabic letter mark. The isolates cost 9 each and the mark 6, and they stay in the URL without ever showing in the headline.

What happens when someone opens a cut-off link

Most sites use slug-only permalinks, /%postname%/. I opened the Arabic test post four ways:

The middle rows come from redirect_guess_404_permalink(). When a slug isn't found, WordPress looks for one that starts with what was requested, post_name LIKE 'cut%', and redirects to the first match. The query has no ORDER BY, so when two headlines share their opening words, which one the reader gets is up to the database. On my test site the older article won, but don't count on it: another database, or the same one reading through a different index, can return the other. News headlines share openings all the time («القبض على المتهمين في» …), and the redirect is a 301, which tells browsers and search engines the move is permanent.

If you would rather WordPress matched exactly than guessed, there is a filter for that:

add_filter( 'strict_redirect_guess_404_permalink', '__return_true' );

With it, a cut-off link gets a 404 even when the guess would have been right. Exact matches are still redirected, and that matters further down: after a switch to ID-anchored permalinks, old slug-only links are found by this same guess, and strict matching keeps them working. The filter that turns guessing off altogether, do_redirect_guess_404_permalink, would send every one of them to a 404.

The last row is the 200-byte trap in miniature. A link longer than the stored slug never matches, because the stored slug doesn't start with it. That is what readers meet when a core update shrinks a widened column under links that are already out in the world.

Put the ID in the URL

The setting that makes most of this stop mattering is under Settings → Permalinks: a custom structure of /%post_id%/%postname%/. The words stay in the URL for readers, and the number decides which article it is. The same requests, plus a few harsher ones:

The reason is a few lines in redirect_canonical(). When the slug doesn't match, the request is a 404 for a moment, but the post ID is still in it, and WordPress sends the reader to that post's real permalink:

if ( is_404() ) {
    // Redirect ?page_id, ?p=, ?attachment_id= to their respective URLs.
    $post_id = max( get_query_var( 'p' ), get_query_var( 'page_id' ), get_query_var( 'attachment_id' ) );

    $redirect_post = $post_id ? get_post( $post_id ) : false;
    // … and for a public post: $redirect_url = get_permalink( $redirect_post );

The last row in the table is the price: the number outranks the words, so a mistyped ID lands on a different article. Links are copied far more often than they are typed, and for a large archive that is a trade I would take.

Switching an existing site went smoothly in the test: old slug-only links were redirected to their articles by the same guess as above, with or without strict matching. So after switching, leave guessing on, and watch the 404 log for a while.

Give the slug its budget back

A small must-use plugin recovers the space those characters waste, and keeps Arabic-script punctuation out of the URL. It runs before WordPress counts, and only when a slug is being saved, so existing URLs keep matching exactly as before:

<?php
/**
 * Give slugs their budget back: drop or replace characters that cost slug
 * space, before WordPress counts them. Mostly Arabic, Persian and Urdu ones,
 * plus the curly quotes, dashes and ellipses that English headlines pick up.
 */
add_filter( 'sanitize_title', function ( $title, $raw_title = '', $context = 'display' ) {
    if ( 'save' !== $context ) {
        return $title; // Leave lookups alone, so existing URLs keep matching.
    }
    // Arabic diacritics, superscript alef and tatweel: 6 each, and they clutter the URL.
    $title = preg_replace( '/[\x{064B}-\x{065F}\x{0670}\x{0640}]/u', '', $title );
    // Quotes, the ellipsis and invisible marks, which WordPress strips anyway,
    // but only after counting them.
    $title = preg_replace( '/[\x{00AB}\x{00BB}\x{2039}\x{203A}\x{2018}-\x{201F}\x{2026}\x{200B}-\x{200F}\x{FEFF}]/u', '', $title );
    // Direction controls: WordPress strips the embeddings only after counting
    // them, and keeps the isolates and the Arabic letter mark in the URL.
    $title = preg_replace( '/[\x{061C}\x{202A}-\x{202E}\x{2066}-\x{2069}]/u', '', $title );
    // Punctuation that WordPress keeps in the URL (Arabic comma, semicolon and
    // question mark, Urdu full stop) or turns into a hyphen only after counting
    // it (no-break space, en and em dash): make it a space, which costs 1.
    $title = preg_replace( '/[\x{060C}\x{061B}\x{061F}\x{06D4}\x{00A0}\x{2013}\x{2014}]/u', ' ', $title );
    // Arabic-Indic and Persian digits cost 6 each; 0 to 9 cost 1.
    $title = strtr( $title, array_combine(
        preg_split( '//u', '٠١٢٣٤٥٦٧٨٩۰۱۲۳۴۵۶۷۸۹', -1, PREG_SPLIT_NO_EMPTY ),
        str_split( '01234567890123456789' )
    ) );
    return $title;
}, 9, 3 );

On the test site, the quoted headline's slug came out identical to the unquoted one. A long headline with an Arabic comma and question mark got plain hyphens in their place, and kept two more letters. A headline carrying invisible isolates lost them and kept three more letters, and the Arabic letter marks were dropped too. The grant headline came out with its whole number, 5000, and the vocalised headline went from three and a half words to six, with no diacritics in the URL. English titles come out as core makes them, only no longer cut short: the long English headline went from 133 characters to 195. The plugin leaves Chinese and Japanese punctuation alone, so there it still costs 9 and stays in the URL.

So, is it only Arabic?

No. Every language outside the Latin alphabet pays for the same 200 characters at a higher price: six per letter for the two-byte scripts, nine for the three-byte ones. English pays too, every time a curly apostrophe or a dash is pasted in. Arabic newsrooms are where I met it first, because that is mainly where I work, not because the rule treats Arabic differently.

What to do about it, roughly in order of effort:

1. If your language uses accented Latin, set the site language before you publish.
2. Edit long slugs by hand. Five to seven words is plenty.
3. On a Russian or other Cyrillic site, consider a transliteration plugin such as Cyr-To-Lat, which writes slugs in Latin letters, so they cost what English costs. Arabic rarely goes that way, because Arabic written in Latin letters is hard to read back.
4. On an Arabic, Persian or Urdu site, or any site whose headlines arrive with curly quotes and dashes, add the plugin above.
5. On a large archive, anchor permalinks on the post ID.
6. If you need longer slugs than any of this allows, widen the column, raise the budget, and stop core updates from shrinking it back. That is what wp-arabic-slug-schema-guard does.

Every number here was measured on a stock WordPress 7.1.3 site in WordPress Playground, which runs on SQLite. The slug lengths are worked out in PHP and will be the same on MySQL. Which article a shared opening lands on may not be, which is rather the point.