A form that does not exist when Django starts

A Django form is a class. You write it in a module, Django imports the module at startup, DeclarativeFieldsMetaclass collects the class attributes that are Field instances, and from then on the form’s shape is fixed for the lifetime of the process.

A form builder breaks that assumption on purpose. The customer draws a form in a browser — drags in a signature pad, a GPS capture, a photo series, a reference to a row in another form — presses save, and a respondent fills it in ninety seconds later. Nothing restarted. No module was imported. The form did not exist when Django started, and it has to behave, validate and render exactly like one that did.

This is how that works in PaperPoint, the form builder we run for field data collection — twenty-nine field types, conditional logic, offline capture. The interesting parts are not the metaprogramming, which is short. They are the failure modes, which are not.

The shape of the problem

The Django documentation’s answer to dynamic forms is well known: override __init__, mutate self.fields. That is the right primitive and it is where we start.

class DynamicForm(BootstrapFormMixin, forms.Form):
    def __init__(self, *args, **kwargs):
        self.template = kwargs.pop("template")
        super().__init__(*args, **kwargs)

        self.field_groups = {}
        self.field_conditions = {}
        self.grouped_fields = {}
        self.ungrouped_fields = []

        self._build_form_fields()
        self.apply_bootstrap()

self.fields after super().__init__() is a deepcopy of base_fields — here, empty. Everything the respondent sees is added in the four lines that follow.

That much is a tutorial. What a tutorial does not tell you is what happens on the day a customer saves a field whose configuration is incomplete, and eleven other people are mid-way through filling that form.

Two axes, not one

The first design decision is that a field has two independent attributes, not one:

FIELD_TYPES = [
    ("char", _("Short text")),
    ("text", _("Long text")),
    ("email", _("Email")),
    ...
    ("geo_polygon", _("Geographic polygon")),
    ("data_select_form", _("Reference to another form")),
    ("data_select_column", _("Reference to a unique column")),
    ("matrix", _("Matrix")),
]

WIDGET_TYPES = [
    ("text", _("Text input")),
    ("textarea", _("Textarea")),
    ("select", _("Dropdown list")),
    ("radio", _("Radio buttons")),
    ("checkbox", _("Checkboxes")),
    ...
]

field_type decides what the value is — how it is cleaned, coerced, validated, stored. widget_type decides how it is asked for. A single choice among four options is one field_type and three legitimate widgets: a dropdown, a radio group, or — on a phone, in the field, with gloves on — a set of large tappable cards.

Collapsing the two into one enum is the mistake that is cheap on day one and expensive on day two hundred, because every new rendering of an existing data type then requires a new type, a new migration, and a new branch in every piece of code that switches on type. Twenty-nine field types map onto twenty-two widget types through a DEFAULT_WIDGET_MAPPING, and the customer overrides the default when they want to.

The dispatch, and the dict that used to be rebuilt

Mapping type to class is a dictionary. The only thing worth saying about it is where it lives:

# FormField.field_type -> the form field class that renders it.
#
# Module level on purpose: this used to be a dict literal rebuilt inside
# _create_field_instance, i.e. once per field per form render.
FIELD_TYPE_TO_FORM_FIELD = {
    "char": forms.CharField,
    "text": forms.CharField,
    "email": forms.EmailField,
    "number": forms.IntegerField,
    "date": forms.DateField,
    "boolean": BooleanToggleField,
    "choice": CustomRadioSelectField,
    "multiple_choice": CustomCheckboxSelectField,
    "signature": SignatureField,
    "gps_coordinates": GPSCoordinateField,
    "geo_polygon": GeoPolygonField,
    "matrix": MatrixField,
    ...
}

A dict literal inside a method is rebuilt on every call. Inside _create_field_instance, that is once per field, per form render. Twenty-nine entries, forty fields, and every respondent who loads the page builds eleven hundred and sixty dictionary entries to look up forty of them.

Nobody profiles this, because it never shows up as the slow thing. It shows up as a form page that is inexplicably eighty milliseconds slower than it has any right to be, and it stays that way for a year. Constant data belongs at module level. The comment is in the file so the next person does not helpfully move it back inside for locality.

Not every field is a class call

Half the types are satisfied by field_class(**kwargs). The other half need configuration read out of the model — and those are exactly the types that make the product worth paying for.

FIELD_BUILDERS = {
    "boolean": _build_boolean,
    "multi_image_camera": _build_multi_image_camera,
    "data_select": _build_data_select,
    "data_select_form": _build_data_select_form,
    "data_select_column": _build_data_select_column,
    "matrix": _build_matrix,
}


def _create_field_instance(self, field_model: FormField, **kwargs):
    field_type = field_model.field_type
    field_class = FIELD_TYPE_TO_FORM_FIELD.get(field_type, forms.CharField)
    builder = FIELD_BUILDERS.get(field_type)
    if builder is not None:
        return builder(field_model, field_class, **kwargs)
    return field_class(**kwargs)

Two dictionaries and four lines. The builders are plain functions, not methods, and that is deliberate: none of them touches form state, so each one can be tested with a FormField instance and nothing else — no request, no template, no form. A test for the matrix builder does not need a form to exist.

Read one:

def _build_data_select_form(field_model, field_class, **kwargs):
    cfg = getattr(field_model, "cross_form_config", {}) or {}
    slug = cfg.get("target_form_slug")
    if not slug:
        logger.error("data_select_form %s: missing target_form_slug", field_model.name)
        return forms.CharField(**kwargs)
    try:
        return field_class(
            target_form_slug=slug, display_column=cfg.get("display_column"), **kwargs
        )
    except Exception as exc:
        logger.error("data_select_form %s: %s", field_model.name, exc)
        return forms.CharField(**kwargs)

The rule that holds the whole thing up

Every builder ends the same way, and it is the single most important line of design in the module:

A misconfigured field degrades to a CharField. It does not break the form.

This is not defensive programming for its own sake. It follows from who is doing the configuring.

In a normal Django project, the person who writes forms.py is the person who runs the tests, and a broken field is caught before deployment by construction. In a form builder, the person configuring the field is the customer, working in a browser, at four in the afternoon, on a form that is already collecting answers. They set a data_select_form field to point at another form and have not yet picked which column to display. They are not going to run your test suite.

There are two possible behaviours at that moment:

• The form raises. Every respondent gets a 500. The eleven people mid-way through a submission lose it. The customer does not know they caused it, because the thing they did — opening a configuration panel and not finishing — did not feel like a deployment.
• The field falls back to a text input. One field looks wrong. Everything else works. Every submission still completes. A line lands in the log with the field’s name in it, so support can answer the question in twenty seconds instead of reading a traceback.

The second is obviously right, and it is worth being explicit about why it is right rather than treating it as generic robustness: the blast radius of a configuration mistake must match the size of the mistake. One bad field should cost one field.

The rule is applied at three depths, which is what makes it hold.

In the builder, for a configuration that is missing or malformed:

if not (slug and source_col):
    logger.error("data_select_column %s: need target_form_slug + source_column",
                 field_model.name)
    return forms.CharField(**kwargs)

Around each field, for anything the builders did not anticipate — a database row that violates an assumption, a widget whose import broke, a regex that compiles but explodes:

for field in template_fields:
    try:
        self._create_form_field(field)
        ...
    except Exception as e:
        logger.error(f"Erreur création champ {field.name}: {str(e)}")
        self.fields[field.name] = forms.CharField(
            label=f"{field.label} (Error)",
            required=False,
            help_text=f"Erreur: {str(e)}",
        )

Note required=False. A field that failed to build must not also block submission — otherwise the degradation has just moved the outage from render to submit, which is worse, because the respondent discovers it after typing for ten minutes.

In the lookups, for a type or widget name that no longer exists — a value that survived in a database row after the code that understood it was deleted:

field_class = FIELD_TYPE_TO_FORM_FIELD.get(field_type, forms.CharField)
...
return widget_mapping.get(field.widget_type, forms.TextInput(attrs=attrs))

.get() with a default rather than [...]. This is the case people forget, and it is the one that bites during a migration: the code is deployed, the rows are not yet rewritten, and for four minutes the database contains a type the running code has never heard of.

Also worth stating plainly: a bare except Exception is normally a smell, and here it is load-bearing. The distinction is that this one catches, names the field, logs, and substitutes a working object — it does not swallow. The smell is a bare except that returns None and lets the caller discover the problem later.

The N+1 a dict literal hides

_get_widget is where the module still pays for its own convenience, and it is worth walking through because the pattern is extremely common and almost invisible.

def _get_widget(self, field: FormField):
    attrs = {
        "placeholder": field.placeholder,
        "class": "form-control",
        "data-field-name": field.name,
    }
    ...
    widget_mapping = {
        "text": forms.TextInput(attrs=attrs),
        "textarea": forms.Textarea(attrs={**attrs, "rows": 3}),
        "select": forms.Select(attrs={**attrs, "class": "form-select"}),
        "radio": CustomRadioSelectInput(
            attrs={"class": "form-check-input"},
            choices=_resolve_choices(field),
        ),
        "checkbox": CustomCheckboxSelectInput(
            attrs={"class": "form-check-input"},
            choices=_resolve_choices(field),
        ),
        "signature": SignatureInput(attrs=attrs),
        "gps_coordinates": GPSCoordinateInput(attrs=attrs),
        ...
    }
    return widget_mapping.get(field.widget_type, forms.TextInput(attrs=attrs))

Why can this one not simply move to module level, the way the field-class map did? Because the values are not classes, they are instances, and each carries attrs built from this field — its placeholder, its name. The map is per-field by construction.

That is the real cost, and it is two distinct costs:

Twenty-odd widgets are instantiated to use one. Cheap individually, multiplied by every field on every render.

_resolve_choices(field) runs twice, unconditionally, for every field — including fields that are not choices at all, because a dict literal evaluates all of its values before .get() ever runs:

def _resolve_choices(field):
    liste = getattr(field, "referentiel_liste", None)
    if liste is not None:
        return [(entry.code, entry.name) for entry in liste.active_entries()]
    return [(c.value, c.label) for c in field.choices.all()]

field.choices.all() is safe — _build_form_fields prefetches it:

template_fields = list(
    self.template.fields.all()
    .prefetch_related("choices", "conditions__source_field")
    .order_by("position")
)

referentiel_liste is not in that prefetch. It is a nullable ForeignKey to a shared reference list, so for a text field the id is NULL and Django answers None without a query. For a choice field bound to a reference list, Django caches the related object on the instance after the first access — but active_entries() is a method call on that object, and it is evaluated three times per field: once for the choices kwarg, once for the radio widget, once for the checkbox widget.

The correct shape is a dispatch, not a lift: a mapping from widget_type to a callable that builds only the widget actually needed — the same move already made for field classes, applied to the second half of the module. Writing it down matters more than the microseconds; a dict literal in a hot method is one of the few performance bugs that survives code review indefinitely, because it looks like a table.

Groups, and returning BoundFields

The last piece is the part that touches the template. A generated form still has to render as sections, in order, some collapsible.

def get_fields_by_group(self):
    result = []

    if self.ungrouped_fields:
        ungrouped = [(name, self[name]) for name in self.ungrouped_fields
                     if name in self.fields]
        if ungrouped:
            result.append((None, ungrouped))

    for _group_id, group_info in self.field_groups.items():
        group_fields = [(name, self[name]) for name in group_info["fields"]
                        if name in self.fields]
        if group_fields:
            result.append((group_info, group_fields))

    return result

Three details carry weight.

self[name], not self.fields[name]. self.fields[name] is a Field — it has no value, no errors, no id, and rendering it in a template produces an empty input that silently discards what the respondent typed on a form with errors. self[name] is a BoundField. This is the single most common bug in hand-rolled dynamic-form rendering, and it only appears once validation fails, which is exactly when the respondent is least willing to retype everything.

if name in self.fields. The group’s field list is built during construction — and construction is allowed to fail. A field that raised gets a fallback entry, but a name that never made it into self.fields at all must not become a KeyError at render time, three layers deep in a template where the traceback points at the template and not at the cause.

Ungrouped fields come first, and None is the group key. The template branches on it. A customer who never opened the grouping panel gets a flat form without a phantom “Default” heading they did not create — the absence of configuration renders as absence, not as a placeholder.

What to take away

If you are building forms out of database rows:

1. Separate field_type from widget_type. One is the value, one is the question. Merging them costs a migration every time someone wants an existing data type to look different.
2. Keep the type→class table at module level, and the per-field builders as plain functions. Constant data does not belong in a method; functions that do not touch state do not belong in a class.
3. Decide your degradation rule before you write the first field type, and apply it at every depth. Ours: a misconfigured field becomes a required=False CharField and logs its own name. One bad field costs one field.
4. Use .get() with a default on every lookup keyed by a database value. Rows outlive the code that wrote them.
5. Return BoundFields to the template, and guard every name against a field that failed to build.

The metaprogramming is twenty lines. The other four points are the product.