Vertical Japanese Is the Hardest Case for OCR — Here's Why

By the onecue.ai team · August 11, 2026 · ~10 min read

Photograph a Japanese product card, a shrine notice, or a paperback page and you will hit a class of OCR failure that horizontal text never produces. The words come back, but as fragments in the wrong order. Translations arrive as disconnected phrases. Sometimes a whole column is simply absent.

The reason is that vertical writing (縦書き) breaks assumptions built into every layer of a standard OCR pipeline — assumptions so basic that they are rarely written down. This article goes through each one on a real example: a photographed product card with four panels of vertical Japanese, which we used as the test case while fixing our own pipeline.

Assumption 1: a line of text is wider than it is tall

Text detectors output boxes around lines, and everything downstream assumes those boxes are wide strips. That is true for every horizontal script. For vertical writing it is exactly inverted: each column is a tall, narrow box, and a page of vertical text is a row of them.

The first consequence is ordering. Vertical Japanese reads top to bottom, then right to left. A pipeline that sorts boxes the usual way — top to bottom, then left to right — produces text that is not merely misordered but reversed at the sentence level. On our test card, the first panel's body text came out as five separate fragments in roughly the opposite of reading order.

The second consequence is fragmentation. Because each column is a separate box, a single sentence spanning four columns arrives as four unrelated strings. If you are only extracting text, that is annoying. If you are translating, it is fatal: a machine translator handed "エネルギーを" with no continuation produces a fragment, and the four fragments are then drawn back onto the image in four separate places, overlapping each other.

Assumption 2: a tall crop is rotated horizontal text

This one is subtle and it cost us the most time.

Detectors hand each cropped region to the recognizer, and by convention a crop taller than it is wide gets rotated 90° first. The reasoning is sound for the case it was written for: a tall box usually means horizontal text photographed sideways — a sign on a rotated storefront, a book spine — so standing it up puts the glyphs the right way round.

Vertical Japanese violates this completely. Its characters are upright; only their arrangement is vertical. Rotating that crop does not fix an orientation problem, it creates one: every character ends up lying on its side, and the recognizer — trained on upright glyphs — returns noise or nothing at all.

We confirmed this from the other direction later. When a user marks a region for re-reading and that region is a vertical column, forcing it through the recognizer returns empty. A 179×1036 pixel column produced no text at all. The fix is not to rotate harder; it is to send those regions back through detection, which splits the column into its individual characters and handles them upright.

Assumption 3: the aspect ratio tells you the crop's shape

Our pipeline had a rule allowing vertical boxes to pass at a lower confidence score than horizontal ones — sensible, since vertical text scores structurally lower. The rule checked whether the crop was taller than it was wide.

Because of Assumption 2, that check could never be true for the text it was written for. The detector rotates tall crops before recognition, so by the time the rule ran, a genuine vertical column had already become a wide crop. Measured on the test card: a column whose detection polygon was 61×268 pixels arrived at the check as a 61×268 crop laid on its side. The exception existed, was documented, was tuned — and had never once fired on a vertical column.

The fix was one line: judge orientation by the detection polygon, which is in image coordinates and unrotated, rather than by the crop, which is in recognizer coordinates and may be rotated. On the test image that recovered two correctly-read columns scoring 0.617 and 0.619 — above the vertical floor of 0.55, below the Japanese threshold of 0.65 — that had been discarded on every previous run. Coverage went from 17/38 to 19/38 boxes.

The general lesson is about coordinate systems. Any check that mixes "what the detector saw" with "what the recognizer received" will be wrong for exactly the inputs that make the two differ.

Assumption 4: a character is a character, not a line

Detection on vertical text is unstable in a specific way: sometimes it produces one box per column, and sometimes it shatters a column into one box per character. Both happen in the same image. On our card, 赤 and は each got their own square box while the column beside them stayed intact.

This means reassembly cannot assume a fixed granularity. Our postprocessing runs two passes:

Stacked characters into a column. Square boxes aligned on the same x range, spaced by less than a character width, are joined top to bottom. The spacing test uses the box width as its unit, not height — in vertical writing the character size is the column width, so a gap measured in heights would happily join two paragraphs that are hundreds of pixels apart.
Columns into a block. Adjacent columns are joined right to left into one string, so the translator receives a whole sentence. Two guards keep this from swallowing the page: candidate columns must overlap the seed column's vertical range (otherwise a column from the panel below joins the one above), and the horizontal gap must be under about 1.5 character widths (otherwise the next panel joins too).

A third guard exists because the first two were not enough. Checking only neighbouring pairs lets a group drift diagonally down the page — A overlaps B, B overlaps C, but A and C share no rows at all. Every candidate must also overlap the block's first column, which stops the walk.

Assumption 5: text that was found will be drawn back the same way

For translation there is one more stage, and it has its own version of the problem. Once a merged block is translated, the translation has to be drawn where the original was. A merged multi-column block is roughly square, so wrapped horizontal lines fit it naturally. But a single standing column that never merged is a tall, narrow box.

Fitting horizontal text into a tall box goes wrong in both directions. A long translation shrinks to nothing. A short one — Japanese is compact, Korean and English often expand — gets sized to the box height and renders as two enormous characters covering a panel. We had exactly that: 준다 drawn at panel scale.

So the renderer now treats a still-tall box as a column. For Korean, Japanese and Chinese targets it stacks the translation one character per line, which is how those scripts are set vertically anyway. For Latin-script targets it rotates the text 90°, the way vertical Latin signage is set. Rather than trusting the aspect ratio alone, it fits the text both ways and takes whichever achieves the larger font — that keeps a narrow detection of "5" or "I" from being turned sideways for no reason.

What is still hard

Being honest about the limits: none of the above improves recognition of vertical text. It fixes ordering, assembly, thresholds, and rendering — everything around the model. On the test card, several columns of body text still come back unread, and the reason is mundane: the photo has focus blur, a hand shadow across the lower panels, and a brush-style font. Those columns score 0.0–0.49, far below any threshold worth having.

We could admit them by lowering the floor. We deliberately do not. A vertical-writing card that returns half its text untranslated is visibly incomplete, and a user can see that and reshoot. A card that returns confident nonsense — a stone's name rendered as an unrelated Korean word — reads as a fact. Untranslated is a worse experience than correct; fabricated is worse than either.

The remaining path is better recognition on degraded input: a server-tier recognition model, or a model trained specifically on vertical Japanese. That is a measurable change with a real cost, and the way to decide it is to sweep parameters against a corpus of actual photographs rather than to guess. Which is the same method that produced everything in this article.

Practical advice for photographing vertical text

Fill the frame with one panel. Resolution per glyph is the single strongest predictor of success. A card photographed whole gives each character a fraction of the pixels it gets when you shoot one panel at a time.
Shoot square to the page. Vertical columns photographed at an angle produce skewed polygons, which makes both the merge geometry and the crop rotation less reliable.
Watch for shadows. A hand or phone shadow across part of the page is the most common cause of a column scoring just below threshold in our data.
Use the correction feature. If a column comes back missing, tap it on the result image — the region is re-read directly, and detection's judgement is bypassed.

You can test any of this on Image Translator with a Japanese source, or extract the raw text and positions with Image To Text.