Removing Text From an Image: What Inpainting Gets Right and Wrong

By the onecue.ai team · August 11, 2026 · ~8 min read

"Use AI inpainting" is the standard answer to removing text from a photograph, and for a large class of images it is correct. For another large class — the one most people actually have — it produces a worse result than a much simpler method. Knowing which is which is most of the craft.

This article covers when generative inpainting is the right tool, when it destroys the image, and what to do instead. The examples come from our own translation pipeline, where every erase has to survive having new text drawn on top of it.

What inpainting is actually doing

An inpainting model takes an image and a mask, and generates plausible content for the masked region based on what surrounds it. For text removal the mask is the text's bounding region, and the model's job is to reconstruct whatever was behind the letters.

The word doing the work is plausible. The model has no knowledge of what was actually behind the text. It is producing content that fits the neighbourhood — which is exactly right when the neighbourhood is rich in structure and no one can verify the original. Text over a photograph of a street, a wooden table, a patterned fabric: the model continues the texture and the seam is genuinely hard to find.

The case where it fails badly

Now consider a red band with white lettering, sitting inside a white panel — a completely ordinary sign, and roughly half of what people photograph.

The mask covers the lettering, which sits on the red band. But the band is narrow, and its neighbourhood — the region the model draws context from — is dominated by the white panel around it. The model does what it was built to do: it fills the masked region with content consistent with its surroundings. The red disappears. Then, in our pipeline, the white translated text gets drawn onto the newly white background, and the result is a blank rectangle where a sign used to be.

This is not a bug in the model. It is the model working correctly on an input that violates its assumption. Generative fill assumes the masked region's content can be inferred from context. For a flat colour block that is smaller than its surroundings, the inference is wrong in a way that destroys the very thing you were trying to preserve.

The alternative: don't guess what you can measure

When text sits on a flat fill, the background is not something to be inferred. It is right there, in the same box, in every pixel that is not part of a letter. You can measure it exactly.

Our renderer classifies each text region before erasing it. It samples the region's colours, takes the per-channel median as the background — glyphs are always the minority of pixels in a text box, so the median lands on the fill — and then asks how uniform the non-glyph pixels are. If their standard deviation is below a small threshold, the box sits on a flat fill that can be reproduced exactly, and generative inpainting is skipped entirely.

The classification matters more than the erasing method. A pipeline that always inpaints ruins signage; a pipeline that always fills with a solid colour ruins photographs. Deciding per region is what makes both cases work.

Erasing glyphs instead of rectangles

"Fill the box with the background colour" has its own failure, and it appears as soon as a sign is photographed at an angle — which is to say, almost always.

A detection box is axis-aligned, but a label photographed at an angle is not. The label's edge runs diagonally through the box. Fill the whole box with the label's colour and that colour spills past the label's real edge, leaving a visible red wedge poking into the white panel around it. Shrink the fill to stay inside and you leave slivers of the original text behind.

The way out is to stop thinking in rectangles. Inside the box, every pixel far enough from the background colour is marked, and those pixels are grouped into connected components:

Components that touch the box border are the surroundings leaking into the box — the panel behind the label, the edge of a neighbouring element. They are left completely untouched.
Components fully enclosed by the flat fill are the characters. Only these get painted over with the sampled background colour.
Everything already matching the background keeps its original pixels, so the label's natural grain, gradient and edge survive.

One practical detail: the marked pixels are dilated by one pixel before grouping. Anti-aliased glyph edges are partially transparent, so their outermost pixels sit between ink and background — without the dilation those edge pixels fall outside their own glyph's component and are left behind as a faint outline of the deleted text.

Choosing an engine, and being able to turn it off

Our worker supports three erase engines, selectable by environment variable: a generative model (LaMa), a classical OpenCV inpaint, and none, which returns the image unchanged.

The third one earns its place for reasons that have nothing to do with output quality. It makes the whole pipeline testable without loading a model — integration tests and rendering tests run in seconds on machines with no GPU and no weights downloaded. It also isolates blame: if a result looks wrong with erasing disabled, the problem is in detection, recognition, or the renderer, not in the erase stage. A surprising amount of debugging is just being able to remove one stage.

The classical OpenCV path matters for a different reason: it is fast and predictable. For small masks on simple backgrounds it is often visually indistinguishable from the generative model while costing a fraction of the time.

Erasing is only half the job

If you are removing text to replace it — translation, localization, versioning a design — the erase has to leave something the new text can sit on legibly. That connects two decisions that look independent:

Sample colours before erasing, not after. Once the region is repainted, the original ink colour is gone, and with it any chance of drawing the replacement in the same colour as the text it replaces.
Re-check contrast after inpainting. On generatively filled regions, the model chose the background. The ink colour sampled from the original may no longer read against what the model painted, so it gets compared to the actual result and swapped for black or white if it has become invisible.

The full set of decisions on the drawing side — length changes, wrapping, fonts, vertical text — is covered in the layout-preserving translation article.

What kind of image are you holding?

A short diagnostic, in the order that matters:

Text on a flat colour (signs, labels, packaging, UI screenshots) — the background is measurable. Expect near-perfect removal, and be suspicious of any tool that visibly repaints the whole region.
Text on texture or photograph (a caption over a scene, a watermark over artwork) — the background must be invented. Generative inpainting is the right tool, and the seam will be visible on close inspection if the texture has strong structure.
Text overlapping both (a caption crossing a subject's edge) — the hardest case, because a single region needs both treatments. Expect artefacts at the boundary.
Text with a shadow or outline — the mask has to cover the effect, not just the glyphs, or you erase the letter and leave its ghost.

You can see the erase-and-redraw path end to end on Image Translator, which shows the original and the rendered result side by side.