The prompt that Tesseract cannot read

PaperPoint used to run a vision-language model for document reading: FastVLM, given an image and a prompt, answering in prose. It now runs Tesseract.

The module that replaced it opens with a sentence that is the whole design:

"""
Module OCR utilisant PyTesseract
Compatible avec l'interface FastVLM existante pour une migration transparente
Support multilingue : français, anglais, allemand, italien
"""

Compatible with the existing FastVLM interface, for a transparent migration. Nothing upstream changed. Same functions, same names, same arguments, same return type. The engine underneath is a completely different kind of program.

This is a pattern worth examining closely, because it is genuinely the right move here and it is not free — and the place the cost lands is not where you would look for it.

What the boundary bought

The surface that survived:

def get_available_models() -> list[str]:
    """Retourne la liste des modèles disponibles (compatibilité FastVLM)."""
    if check_tesseract_installation():
        return ["tesseract-ocr-multilang"]
    return []


def get_model_path(model_name: str = None) -> str:
    """Retourne le chemin vers Tesseract (compatibilité FastVLM)."""
    return "/usr/bin/tesseract"  # Chemin standard Linux


def is_model_available(model_name: str = None) -> bool:
    """Vérifie si Tesseract est disponible (compatibilité FastVLM)."""
    return check_tesseract_installation()


def load_model(model_name: str = None, device: str | None = None) -> bool:
    """
    Initialise Tesseract OCR (compatibilité avec l'interface FastVLM).

    Args:
        model_name: Ignoré (compatibilité)
        device: Ignoré (Tesseract utilise toujours le CPU)
    """

Four functions whose parameters are documented as ignored. A reviewer’s instinct is that this is dishonest code — load_model(model_name="fastvlm-1.5b", device="cuda") accepts both arguments and does nothing with either.

I think the instinct is wrong, and the reason is worth stating. The purpose of this module is to be substitutable for the previous one. Every caller, management command, admin action and test that names a model or asks for a device continues to work, and the alternative is a change that touches all of them for no behavioural gain. A model registry that returns one fictional entry called tesseract-ocr-multilang is exactly what an adapter does: it answers the old question in terms the new engine can honour.

What makes it defensible rather than sloppy is that each one says so. (compatibilité FastVLM) on four consecutive docstrings is a marker: this is not the natural interface for Tesseract, it is a shape being maintained deliberately. Without the marker, the next reader deletes device as dead weight and breaks a caller they never opened.

The two functions that do real work are the honest ones:

def check_tesseract_installation() -> bool:
    """Vérifie si Tesseract est installé et accessible."""
    try:
        result = subprocess.run(
            ["tesseract", "--version"], capture_output=True, text=True, timeout=10
        )
        return result.returncode == 0
    except (subprocess.SubprocessError, FileNotFoundError, subprocess.TimeoutExpired):
        return False

A binary probe with a timeout, catching the three things that actually happen — missing binary, subprocess failure, hang. And the language check, which is the one that would otherwise fail in production and nowhere else:

required_langs = ["eng", "fra", "deu", "ita"]
missing_langs = [lang for lang in required_langs if lang not in available_langs]

if missing_langs:
    logger.warning(f"Missing language packs: {missing_langs}")
    logger.warning("Some OCR functionality may be limited")

A warning rather than a failure. That is the right severity: a container built without tesseract-ocr-deu still reads French documents perfectly, and refusing to start would take the whole feature down for a partial capability loss.

The parameter that had nowhere to go

Then there is prompt.

def analyze_image(image: Image.Image, prompt: str, **kwargs) -> str:
    """
    Analyse une image PIL avec PyTesseract OCR (interface compatible FastVLM).

    Args:
        image: Image PIL en mode RGB
        prompt: Prompt pour l'analyse (utilisé pour optimiser l'OCR)
    """

A vision-language model takes a prompt because a prompt is how you ask it a question. Tesseract does not answer questions. It transcribes.

There were three options. Drop the parameter and update every caller — which abandons the whole point of the exercise. Accept it and ignore it — which turns every existing call site into a lie, because callers pass prompts like extract the invoice total and would silently get a full transcription instead. Or: mine it for whatever a transcriber can actually use.

The module takes the third, and for document type it works genuinely well:

def detect_document_type(prompt: str) -> str:
    """Détecte le type de document à partir du prompt pour optimiser la config OCR."""
    prompt_lower = prompt.lower()

    if any(word in prompt_lower for word in ["table", "tableau", "columns", "colonnes"]):
        return "tables"
    elif any(word in prompt_lower for word in ["form", "formulaire", "field", "champ"]):
        return "forms"
    elif any(word in prompt_lower for word in ["invoice", "facture", "bill", "receipt"]):
        return "invoices"
    elif any(word in prompt_lower for word in ["handwritten", "manuscrit", "signature"]):
        return "handwriting"
    else:
        return "documents"

OCR_CONFIGS = {
    "documents": "--oem 3 --psm 6",   # Bloc uniforme de texte
    "forms": "--oem 3 --psm 8",       # Mot unique
    "tables": "--oem 3 --psm 6",      # Structure tabulaire
    "invoices": "--oem 3 --psm 6",    # Factures/documents structurés
    "handwriting": "--oem 3 --psm 8", # Écriture manuscrite
    "default": "--oem 3 --psm 6",
}

This is the part I would keep. --psm — page segmentation mode — is the single most consequential Tesseract flag and the one nobody sets. Mode 6 assumes a uniform block of text; mode 8 treats the image as one word. Running a photo of a single form field through mode 6 produces noise; running a page through mode 8 produces one wrong word.

Nothing in the old interface offered a place to say this is one field, not a page. The prompt did, in prose, and this table turns the prose into the flag. That is an adapter earning its keep: it found the real knob hiding behind the old abstraction.

Worth noting the table has more keys than distinct values — tables, invoices, documents and default are all --psm 6. That is not redundancy to collapse: the names are the vocabulary the callers use, and the day tables need --psm 4 (single column of variable-size text) the change is one line rather than a new branch.

Where prompt-mining stops working

Language detection uses the same technique and it does not hold up as well:

def detect_language_from_prompt(prompt: str) -> str:
    """Détecte la langue préférée à partir du prompt."""
    prompt_lower = prompt.lower()

    french_keywords = ["français", "french", "extraire", "texte", "document"]
    german_keywords = ["deutsch", "german", "extrahieren", "text", "dokument"]
    italian_keywords = ["italiano", "italian", "estrarre", "testo", "documento"]
    english_keywords = ["english", "extract", "text", "document"]

    french_count = sum(1 for word in french_keywords if word in prompt_lower)
    ...
    max_count = max(counts.values())
    if max_count > 0:
        for lang, count in counts.items():
            if count == max_count:
                return lang

    return SUPPORTED_LANGUAGES[DEFAULT_LANGUAGE]  # Multi-langue par défaut

Three properties of this are worth walking through, because they are the generic failure modes of keyword scoring and they all show up in twenty lines.

Substring matching crosses languages. The test is word in prompt_lower, not word equality. "text" is a substring of "texte", so a purely French prompt — extraire le texte du document — scores 3 for French and 2 for English. It still wins, but the margin is an artefact rather than a signal.

Shared words carry no information. "document" appears in both the French and English lists. Any prompt containing it moves both scores equally, which means it contributes exactly nothing to the decision while looking like evidence.

The tie-break is dictionary order. When counts are equal, the loop returns the first key of counts, which is "fra". So a prompt with no linguistic signal at all — or with balanced signal — resolves to French, not to the declared multilingual default on the last line. For this product that outcome is usually right, because the documents are usually French. But it is right by accident, and the # Multi-langue par défaut comment describes a branch that is reached less often than it appears.

The fix is not more keywords. It is that the prompt is the wrong place to look for the document’s language. The application knows its user’s locale, it knows the form’s language, and the caller could pass lang= — which the function already honours:

language = kwargs.get("lang")
if not language:
    language = detect_language_from_prompt(prompt)

The explicit path exists and takes priority. The guessing is the fallback. So the real recommendation is narrow: pass lang from the calling context, and let prompt-mining be what it is — a last resort that is better than nothing and worse than the answer the application already has.

The multilingual default is genuinely good, though:

"auto": "fra+eng+deu+ita",

Tesseract accepts stacked languages and does the sensible thing with them. On a Swiss platform where a single invoice can carry German headings and French line items, that is the correct default and not a cop-out.

Two passes, because empty is a real answer

extracted_text = pytesseract.image_to_string(
    processed_image, lang=language, config=custom_config
).strip()

if not extracted_text:
    # Essayer avec une configuration plus permissive
    logger.warning("No text detected, trying fallback configuration")
    extracted_text = pytesseract.image_to_string(
        processed_image,
        lang=SUPPORTED_LANGUAGES["auto"],
        config="--oem 3 --psm 3",
    ).strip()

The retry drops both guesses at once: all four languages, and --psm 3 (fully automatic page segmentation, no assumptions). Which is exactly right, because an empty result is the signature of a wrong guess. A photograph of a page run through --psm 8 — one word — very plausibly returns nothing at all, and the second pass is the one that recovers it.

Doing this only on empty output, rather than always, is what keeps it cheap: two OCR passes on every image would double the CPU cost of the common case to rescue the rare one.

Preprocessing is similarly modest and correct:

scale_factor = max(300 / width, 300 / height)
new_size = (int(width * scale_factor), int(height * scale_factor))
image = image.resize(new_size, Image.Resampling.LANCZOS)

if enhance_quality:
    enhancer = ImageEnhance.Contrast(image)
    image = enhancer.enhance(1.2)
    enhancer = ImageEnhance.Sharpness(image)
    image = enhancer.enhance(1.1)

Tesseract wants roughly 300 DPI and struggles below it; upscaling with LANCZOS is the standard remedy. And the enhancement factors are deliberately timid — 1.2 and 1.1. Aggressive contrast on a photo of a printed page crushes thin strokes and destroys the very glyphs you are trying to read. Those two numbers look arbitrary and are the result of the only thing that settles them, which is trying.

The whole preprocessing function is wrapped so it cannot break the pipeline:

except Exception as e:
    logger.warning(f"Image preprocessing failed: {e}")
    return image

Return the original and carry on. Preprocessing is an optimisation; failing it should degrade quality, not the request.

What the boundary did not protect

Here is the cost, and it is not in the signatures.

analyze_image returns a string, because that is what a language model returns. So the module builds one:

def format_ocr_response(extracted_text: str, prompt: str, image_size: tuple, language: str) -> str:
    """Formate la réponse OCR selon le contexte du prompt."""
    if not extracted_text:
        return "Aucun texte détecté dans l'image. L'image peut être de mauvaise qualité ou ne contenir aucun texte lisible."

    prompt_lower = prompt.lower()
    response = f"Texte extrait de l'image:\n\n{extracted_text}"

The engine underneath produces something structured — text, per-word confidences, bounding boxes, all available from pytesseract — and the interface can only carry prose. So the structure is flattened into a French sentence with the transcription pasted after a colon, and any caller that wants the text back has to parse it out of a human-readable preamble.

That is the real price of keeping the old interface: not the ignored device parameter, but the return shape. A VLM’s answer is prose because prose was the answer. Tesseract’s answer is data, and it is being dressed as prose to fit a hole cut for something else.

The sharpest consequence is here:

except Exception as e:
    logger.error(f"OCR error: {str(e)}")
    return f"Erreur lors de l'analyse OCR: {str(e)}"

A failure returns a string, through the same channel as a success. A caller cannot distinguish the OCR failed from the document says “Erreur lors de l’analyse OCR” without string-matching a French prefix — and whatever downstream code stores this will happily save the error message as the extracted text of the document.

That one is worth fixing regardless of the interface question, and it is cheap: raise, or return a small object with ok, text and error. load_model already sets the precedent by raising when initialisation fails:

if not _MODEL_LOADED and not load_model():
    raise RuntimeError("Failed to initialize Tesseract OCR")

Two error paths in one function, one raising and one returning a message. The inconsistency is the tell that the string return is inherited rather than chosen.

One more inheritance:

# Variables globales pour simuler l'état FastVLM
_MODEL_LOADED = False
_DEVICE = "cpu"  # Tesseract utilise toujours le CPU

Global variables to simulate FastVLM state. A loaded neural network is real process state worth tracking; a Tesseract binary on PATH is not. _MODEL_LOADED is a cache of “we checked that the binary exists”, and once you name it that, the module-level mutable global becomes what it should have been — a cached health check, ideally with an expiry, so that a container whose language packs are installed after first boot is not stuck reporting a failure forever.

What carries over

• A stable boundary is a real asset — swapping the engine changed no caller. That is not a small thing, and it is why the ignored parameters are the right call rather than the lazy one.
• Mark every compatibility shim as one. (compatibilité FastVLM) on four docstrings is what stops the next reader from “cleaning up” a parameter that callers still pass.
• An adapter can find knobs the old abstraction hid. The prompt → --psm mapping is the best thing in this module, and it exists only because someone asked what a transcriber could actually use from a prompt.
• Keyword scoring fails three ways at once: substring matches cross languages, shared words look like evidence and are not, and ties resolve to whatever your dict order happens to be. Prefer the answer the application already knows.
• Check what the boundary makes you carry. The signature survived the swap; the return shape did not, and a function that returns its errors as content is the bill.
• Retry once on empty, not always. An empty transcription is the signature of a wrong guess, and it is the only case worth paying a second pass for.