An OCR result that knows which field it filled

Every OCR tutorial for Django ends at the same line:

text = pytesseract.image_to_string(image)

That line is correct, it works, and it is roughly two percent of an OCR feature. What follows it — where the text goes, what happens when it fails, how you find it again eighteen months later, and what the number next to it actually means — is the whole job, and almost nobody writes it down.

This is the document pipeline in PaperPoint, where field agents photograph invoices, delivery notes and handwritten forms on a phone with no network, and the text has to be searchable by the office the following week. Including two places where our own code has a seam worth showing you, because you will build the same ones.

The thing tutorials skip: what is the text of?

Start with the model, because the model is the design.

class OCRResult(TimeStampedModel):
    submission = models.ForeignKey(FormSubmission, on_delete=models.CASCADE,
                                   related_name="ocr_results")
    field = models.ForeignKey(FormField, on_delete=models.SET_NULL, null=True,
                              related_name="ocr_results")
    file = models.ForeignKey(FormFile, on_delete=models.CASCADE,
                             related_name="ocr_results")

    extracted_text = models.TextField()
    confidence_score = models.FloatField(null=True, blank=True)

Three foreign keys, and the interesting one is that there are three.

The naive model has one: OCRResult.file. A file was scanned, here is its text. That is the shape every example uses, and it is enough to display the text next to the image — which is the demo, and it is not the product.

The product question is not “what does this file say?” It is:

“Show me every delivery note where the supplier is Hauser, from the March campaign, and tell me which submission each one came from.”

Answering that from OCRResult.file alone means joining file → submission → template → campaign at query time, on every search, for a table that grows by ten thousand rows a month. So the result carries its own coordinates:

ocr_results = OCRResult.objects.filter(
    field=field, status="success", extracted_text__icontains=search_term
).select_related("file", "submission")

field and submission denormalised onto the row turn that search into one indexed filter and one join. But performance is the smaller half of the argument. The larger half is semantic:

field answers “which question was this the answer to?” The same photograph means different things in the supplier invoice field and the damage report field. An OCR row without a field is a string with no question attached, and a string with no question attached cannot be validated, cannot be compared to last month’s, and cannot be autofilled into anything.

Note the deletion policies, which are not uniform and should not be:

• file → CASCADE. The text is derived from the file. Delete the file, the derivation is meaningless.
• submission → CASCADE. A deleted submission takes its extractions with it. This is the GDPR path: erasing a respondent’s data must not leave their scanned passport’s text in a side table.
• field → SET_NULL, null=True. A customer deleting a field from their form must not destroy eighteen months of extracted text. The question is gone; the answers are still evidence. This is the one people get wrong, because CASCADE is the default that feels tidy, and the day it fires is the day someone reorganises a form and silently deletes the archive.

Three foreign keys, three different answers to “what happens when the other end disappears”. If all three of yours are CASCADE, at least one of them has not been thought about.

Configuration is per document type, not per project

Tesseract’s page segmentation mode is the single setting that decides whether you get text or noise, and the default is wrong for most real documents.

OCR_CONFIGS = {
    "documents":   "--oem 3 --psm 6",   # uniform block of text
    "forms":       "--oem 3 --psm 8",   # single word
    "tables":      "--oem 3 --psm 6",   # tabular structure
    "invoices":    "--oem 3 --psm 6",   # structured documents
    "handwriting": "--oem 3 --psm 8",   # handwriting
    "default":     "--oem 3 --psm 6",
}

SUPPORTED_LANGUAGES = {
    "fr": "fra",
    "en": "eng",
    "de": "deu",
    "it": "ita",
    "auto": "fra+eng+deu+ita",
}

--oem 3 is the LSTM engine, and it is not negotiable — the legacy engine is worse at everything except very clean scans of printed serif text.

--psm is the one to think about. PSM 6 assumes a single uniform block of text; PSM 8 assumes a single word. The gap between them is enormous on the two document types where it matters: a form field containing one handwritten word, segmented as a page, comes back empty; the same field as PSM 8 comes back with the word.

fra+eng+deu+ita as the default deserves a warning that is missing from most advice: multi-language is not free. Tesseract runs its language models together and picks per-word; four languages is measurably slower than one and occasionally worse, because a French word that is also a plausible German word gets the wrong one. It is the right default for a Swiss deployment where the same customer files documents in three languages, and it is the wrong default if you know the language. Know it when you can.

The retry is worth copying:

extracted_text = pytesseract.image_to_string(
    processed_image, lang=language, config=custom_config
).strip()

if not extracted_text:
    logger.warning("No text detected, trying fallback configuration")
    extracted_text = pytesseract.image_to_string(
        processed_image,
        lang=SUPPORTED_LANGUAGES["auto"],
        config="--oem 3 --psm 3",
    ).strip()

An empty result is the one failure mode that is cheap to retry and expensive to leave. PSM 3 is fully automatic segmentation with no assumptions — slower, and right when your assumption about the document was wrong. One retry, one widening of assumptions, then stop. Not a loop.

Preprocessing: three operations, not thirty

There is a genre of OCR advice that recommends a dozen OpenCV steps — deskew, denoise, adaptive threshold, morphological opening. Most of it costs accuracy on photographs taken by a phone, because it was written for scans.

What actually moves the number:

def preprocess_image(image, enhance_quality=True):
    try:
        if image.mode != "RGB":
            image = image.convert("RGB")

        width, height = image.size
        if width < 300 or height < 300:
            scale_factor = max(300 / width, 300 / height)
            image = image.resize(
                (int(width * scale_factor), int(height * scale_factor)),
                Image.Resampling.LANCZOS,
            )

        if enhance_quality:
            image = ImageEnhance.Contrast(image).enhance(1.2)
            image = ImageEnhance.Sharpness(image).enhance(1.1)

        return image
    except Exception as e:
        logger.warning(f"Image preprocessing failed: {e}")
        return image

Upscaling small images is the big win. Tesseract’s LSTM wants roughly 30 pixels of x-height. A cropped signature field at 180 pixels wide returns nothing; the same crop at 300 returns the name. LANCZOS rather than the default, because nearest-neighbour upscaling invents the aliasing that the recogniser then reads as character strokes.

The enhancement factors are deliberately timid. 1.2 contrast and 1.1 sharpness. It is tempting to push them, and pushing them makes clean documents worse: aggressive sharpening turns JPEG artefacts into strokes. These numbers help a bad photograph and do not hurt a good scan, which is the correct trade when you do not control the input.

And it returns the original on failure. Preprocessing is an optimisation. An optimisation that can fail the whole operation is not one — degraded OCR beats no OCR, and the log line says which happened.

Failure is a row, not an exception

This is the design decision that separates a feature from a demo.

except Exception as e:
    logger.error(f"OCR error for file {file_id}: {str(e)}")

    OCRResult.create_from_ocr_analysis(
        file_obj=file_obj,
        ocr_result={
            "text": "",
            "success": False,
            "error": str(e),
            "processing_method": "tesseract_error",
        },
    )

    return {"success": False, "error": str(e)}

A file whose OCR failed gets a row, with status="failed", an empty text and the error message stored next to it.

The alternative — log it and move on — produces a system with no memory of its own failures. Six months later someone asks “how many documents did we fail to read?” and the only honest answer is “we do not know, they look exactly like the ones nobody has processed yet”. Absence of a row is ambiguous: it means not attempted and attempted and failed at the same time, and those need completely different follow-up.

With a row, four things become possible that are otherwise impossible: counting the failure rate, retrying only what failed, showing the operator why a document is blank, and noticing that failures cluster on one device, one supplier’s letterhead, one month.

The status vocabulary distinguishes four outcomes, and the distinction earns its keep:

STATUS_CHOICES = [
    ("success", _("Success")),
    ("partial", _("Partial Success")),
    ("failed",  _("Failed")),
    ("error",   _("Error")),
]

failed is the OCR ran and produced nothing usable — a blank page, a photograph of a thumb. error is the pipeline broke — a missing file, a corrupt PDF, a FileNotFoundError because the volume was not mounted. The first is a fact about the document; the second is a fact about your infrastructure. One is a support question, the other is a page. Merging them means your dashboard cannot tell you which one you have.

And the caching is explicit rather than implicit:

if not force_reprocess:
    existing_ocr = OCRResult.objects.filter(file=file_obj).first()
    if existing_ocr:
        return {
            "success": True,
            "text": existing_ocr.extracted_text,
            "from_cache": True,
            "ocr_result_id": existing_ocr.id,
            "date_processed": existing_ocr.date_processed.isoformat(),
        }

from_cache in the response, and force_reprocess as an argument the caller has to pass. OCR on a 30-page PDF at 300 DPI is tens of seconds of CPU; re-running it because someone refreshed a page is how a queue backs up. Exposing the cache flag to the caller — rather than hiding it — means the operator who improved a scan can ask for a re-run, and everyone else gets the stored answer.

A confidence score that is not a confidence score

Here is the first seam I promised, and it is one of the most common mistakes in document pipelines.

def _estimate_confidence(text):
    if not text or len(text.strip()) == 0:
        return 0.0

    confidence = 0.6

    text_length = len(text.strip())
    if text_length > 100:
        confidence += 0.2
    elif text_length > 20:
        confidence += 0.1
    ...
    if re.search(r"\b\d{1,2}[/-]\d{1,2}[/-]\d{2,4}\b", text):
        confidence += 0.05     # dates
    ...
    strange_chars = len(re.findall(r'[^\w\s\-.,!?()@#$%&*+=/<>:;"\'\[\]{}|\\^~`àáâä…]', text))
    if strange_chars > text_length * 0.2:
        confidence -= 0.2

    return max(0.1, min(1.0, confidence))

Read what this measures: length, word count, whether the text contains something shaped like a date, and the ratio of characters outside an expected set. Those are properties of the output string. None of them is a property of the recognition.

It is a plausibility heuristic, and as a plausibility heuristic it is reasonable — a wall of accented Latin characters with a date in it is more likely to be a real extraction than eleven bytes of punctuation. The problem is entirely in the name. Stored in a column called confidence_score, on a 0–1 scale, next to text extracted by an OCR engine, it will be read by every future developer, every dashboard and every threshold as the OCR engine’s confidence. Somebody will eventually write if confidence_score > 0.8: auto_approve(), and that number can reach 0.8 on fluent nonsense, because fluent nonsense is long and full of letters.

Tesseract does report real confidence, per word:

from pytesseract import Output

data = pytesseract.image_to_data(image, lang=lang, config=cfg,
                                 output_type=Output.DICT)
confs = [int(c) for c in data["conf"] if int(c) >= 0]
mean_conf = (sum(confs) / len(confs) / 100) if confs else 0.0

image_to_data returns the same recognition plus per-word boxes and confidences; -1 marks non-text blocks and must be filtered out. It costs one call instead of image_to_string, not two, because image_to_string is a thin wrapper over the same TSV.

The rule this comes down to:

A column named like a measurement must contain a measurement. If it holds a heuristic, name it plausibility_score — or compute the real one. Keeping both is best: the engine’s mean confidence to decide whether a human looks at it, the heuristic to catch the case where the engine is confidently reading noise.

The choices that Django does not enforce

The second seam, and it is a Django fact worth internalising well beyond OCR.

The model declares:

processing_method = models.CharField(
    max_length=50,
    choices=[
        ("fastvlm_image", _("FastVLM Image")),
        ("fastvlm_pdf",   _("FastVLM PDF")),
        ("tesseract",     _("Tesseract OCR")),
        ("google_vision", _("Google Vision API")),
        ("other",         _("Other")),
    ],
    default="fastvlm_image",
)

The pipeline writes:

processing_method = "tesseract_pdf"     # PDF branch
processing_method = "tesseract_image"   # image branch
processing_method = "tesseract_error"   # failure branch
processing_method = "unknown"           # the catch-all in create_from_ocr_analysis

Four values. None of them is in the choices list.

And it works. Every row saves. No exception is raised, nothing is logged, and the data is silently outside its own declared vocabulary — because in Django, choices is validation metadata, not a database constraint. It is enforced by Model.full_clean(), which ModelForm calls and which Model.save() does not. A pipeline that constructs models directly and calls .save() — which is every background job ever written — never validates anything.

The visible damage is small and annoying: get_processing_method_display() falls back to returning the raw value, so a report that groups by method shows tesseract_image next to Tesseract OCR as if they were different engines. The invisible damage is that the choices list is now documentation of an intention rather than a description of the column, and nobody can tell by reading the model what is actually in there.

Three ways out, in increasing order of strength:

1. Call full_clean() before save() in the pipeline. Correct, and it turns a silent drift into an exception in a background job — which you must then be prepared to handle, or you have traded bad data for lost data.
2. Use a TextChoices enum and reference it everywhere, so the writer cannot spell a value the model has not declared. processing_method=Method.TESSERACT_PDF fails at import, not in production.
3. Add a CheckConstraint, so the database refuses. The strongest, and the one that survives the next developer who writes a management command.

The general lesson is bigger than this field: in Django, choices describes the form, not the table. Anything written by a worker, a signal, a management command or bulk_create bypasses it entirely. If the vocabulary matters, put it in a constraint.

What to take away

If you are wiring OCR into a Django application:

1. Attach the result to the field and the submission, not just the file. The text is an answer to a question; a row that does not know the question is a string.
2. Pick each on_delete separately. SET_NULL on the field is what stops a form edit from erasing an archive.
3. Configure --psm per document type, and retry once with --psm 3 when the result is empty.
4. Upscale small crops to at least 300 px and keep enhancement timid. Return the original image if preprocessing fails.
5. Write a row on failure, and distinguish the document was unreadable from the pipeline broke.
6. Store the engine’s real confidence, from image_to_data. If you also keep a heuristic, name it something that cannot be mistaken for a measurement.
7. choices is not a constraint. Use TextChoices, or a CheckConstraint, for any vocabulary a background job writes.