Dataclasses and type hints

The problem we are solving

Written out, the last chapter's Book class comes to this:

class Book:
    def __init__(self, title, author, pages):
        self.title = title
        self.author = author
        self.pages = pages

    def __repr__(self):
        return f"Book({self.title!r}, {self.author!r}, {self.pages!r})"

    def __eq__(self, other):
        return (self.title, self.author, self.pages) == (other.title, other.author, other.pages)


a = Book("one", "rafi", 300)
print(a)
print(a == Book("one", "rafi", 300))

Book('one', 'rafi', 300)
True

Thirteen lines containing not one decision. Each field's name is written three times — as a parameter of __init__, in the self. line, and in __repr__. Adding a fourth field means adding it in three places, and forgetting one raises nothing — the __repr__ is simply incomplete, in silence.

from dataclasses import dataclass


@dataclass
class Book:
    title: str
    author: str
    pages: int


a = Book("one", "rafi", 300)
print(a)
print(a == Book("one", "rafi", 300))

Book(title='one', author='rafi', pages=300)
True

Seven lines, and each field named once.

By the end of this chapter you can

• Write a @dataclass, and say what it generates
• Use field(default_factory=...), and say why it is needed
• Say what frozen=True gives you
• Write type hints on functions — list[str], dict[str, float], int | None
• Say that hints do nothing at runtime and everything under a checker
• Send a dataclass to JSON with asdict

Prerequisites: Classes.

What @dataclass writes for you

In exchange for those seven lines:

• An __init__ whose parameters follow the fields, in order
• A __repr__ showing every field with its name
• An __eq__ calling two objects equal when all their fields match

Three of the last chapter's problems — <object at 0x...>, a == b being false, and saying the same thing three times — end together.

Methods are written exactly as before:

from dataclasses import dataclass


@dataclass
class OrderLine:
    name: str
    price: float
    quantity: int

    def total(self) -> float:
        return round(self.price * self.quantity * 1.15, 2)


line = OrderLine("pen", 15.0, 3)
print(line)
print(line.total())

OrderLine(name='pen', price=15.0, quantity=3)
51.75

Defaults too, with chapter twenty's rule unchanged: fields with defaults come last.

from dataclasses import dataclass


@dataclass
class Book:
    title: str
    pages: int = 1


print(Book("one"))
print(Book("one", 300))

Book(title='one', pages=1)
Book(title='one', pages=300)

Hints do nothing at runtime

This has to be clear early, because the name suggests otherwise.

from dataclasses import dataclass


@dataclass
class Book:
    title: str
    pages: int


b = Book("one", "three hundred")
print(b)
print(b.pages + 1)

Book(title='one', pages='three hundred')
TypeError: can only concatenate str (not "int") to str

pages: int is written and a piece of text went straight in. Python does not check hints — it records them, and leaves using them to somebody else.

The error arrived much later, when something tried to calculate with that value. In between it was printed, logged, and possibly written to a file.

This is where a checker earns its place. mypy is a separate program that reads the code without running it:

main.py:10: error: Argument 2 to "Book" has incompatible type "str"; expected "int"  [arg-type]

The line number, which argument, what was given and what was expected — and none of it required running the program.

The whole thing in two sentences: hints do nothing at runtime and everything under a checker. Writing a hint is a promise to the checker and to the reader, not to the interpreter.

The recorded hints can be looked at:

def shout(text: str) -> str:
    return text.upper()


print(shout("pen"))
print(shout.__annotations__)

PEN
{'text': <class 'str'>, 'return': <class 'str'>}

The mutable default, a third time

In chapter twenty-one it was a function's default argument. In chapter twenty-six a class attribute. Now a dataclass field — and this time Python stops it itself:

from dataclasses import dataclass


@dataclass
class Shelf:
    books: list = []

ValueError: mutable default <class 'list'> for field books is not allowed: use default_factory

Notice that no object was created at all — the error came as the class was being defined. Python has seen this mistake often enough that here it simply refuses.

The right way is to hand over a factory — saying what to build, rather than the built thing:

from dataclasses import dataclass, field


@dataclass
class Shelf:
    name: str
    books: list[str] = field(default_factory=list)


a = Shelf("a")
b = Shelf("b")
a.books.append("one")

print(a)
print(b)

Shelf(name='a', books=['one'])
Shelf(name='b', books=[])

field(default_factory=list) means "call list() each time". It does chapter twenty-one's if basket is None: basket = [] in one line, with no room to get it wrong.

frozen=True

from dataclasses import dataclass


@dataclass(frozen=True)
class Point:
    x: int
    y: int


p = Point(1, 2)
print(p)
print({p, Point(1, 2)})
p.x = 5

Point(x=1, y=2)
{Point(x=1, y=2)}
dataclasses.FrozenInstanceError: cannot assign to field 'x'

Two things happened together, and they are related.

The object can no longer be changed. p.x = 5 was stopped — as with a tuple.

And it now goes in a set. Remember the last chapter's final problem: writing __eq__ takes the hash away. Here two "equal" points became one in the set, with no error at all.

Without frozen:

from dataclasses import dataclass


@dataclass
class Point:
    x: int
    y: int


print({Point(1, 2)})

TypeError: unhashable type: 'Point'

The reason comes from chapter sixteen: what can change is not safe to hash — change it after putting it in a set and it could never be found again. frozen=True says "this will not change", so Python is willing to supply the hash.

The rule is simple: freeze whatever is not supposed to change after it is made. That buys a promise, and with it the right to be a set member or a dictionary key.

Hints on functions

def totals(rows: list[tuple[str, float]]) -> dict[str, float]:
    result: dict[str, float] = {}
    for name, amount in rows:
        result[name] = result.get(name, 0.0) + amount
    return result

list[str], dict[str, float], tuple[str, float] — what is inside can be stated too, and that is what makes hints useful. list says only "a list"; list[tuple[str, float]] says what the loop will produce.

Give it the wrong thing:

main.py:9: error: Argument 1 to "totals" has incompatible type "str"; expected "list[tuple[str, float]]"  [arg-type]

For "there may or may not be one", use |:

def find(names: list[str], wanted: str) -> str | None:
    for name in names:
        if name == wanted:
            return name
    return None


print(find(["pen"], "pen"))
print(find(["pen"], "bag"))

pen
None

And here is where hints pay best. Use the result without checking it:

main.py:8: error: Item "None" of "str | None" has no attribute "upper"  [union-attr]

That is AttributeError: 'NoneType' object has no attribute 'upper' — the message seen more than any other in practice — except before the program ran, and without ever meeting the particular input that would cause it.

From dataclass to JSON

import json
from dataclasses import dataclass, asdict


@dataclass
class Book:
    title: str
    pages: int


print(asdict(Book("one", 300)))
print(json.dumps(asdict(Book("one", 300))))
print(json.dumps(Book("one", 300)))

{'title': 'one', 'pages': 300}
{"title": "one", "pages": 300}
TypeError: Object of type Book is not JSON serializable

Chapter twenty-five's list said objects of your own do not go into JSON. asdict is the bridge, and it converts nested dataclasses too.

A complete example

The last chapter's order, rebuilt with dataclasses and hints.

"""The order of chapter twenty-six, rebuilt with dataclasses and hints."""

import json
from dataclasses import dataclass, field, asdict

TAX_RATE = 0.15


@dataclass(frozen=True)
class OrderLine:
    """Frozen: a line never changes once made, so it can live in a set."""

    name: str
    price: float
    quantity: int

    def __post_init__(self) -> None:
        if self.quantity < 1:
            raise ValueError(f"quantity must be at least 1: {self.quantity}")

    def total(self) -> float:
        return round(self.price * self.quantity * (1 + TAX_RATE), 2)


@dataclass
class Order:
    """Not frozen: lines are added over time."""

    customer: str
    lines: list[OrderLine] = field(default_factory=list)

    def add(self, name: str, price: float, quantity: int) -> "Order":
        self.lines.append(OrderLine(name, price, quantity))
        return self

    def total(self) -> float:
        return round(sum(line.total() for line in self.lines), 2)

    def largest(self) -> OrderLine | None:
        if not self.lines:
            return None
        return max(self.lines, key=lambda line: line.total())


def main() -> None:
    order = Order("rafi")
    order.add("pen", 15.0, 3)
    order.add("bag", 850.0, 1)
    order.add("ink", 120.0, 2)

    for line in order.lines:
        print(f"{line.name:<6} {line.total():>9.2f}")

    print(f"{'total':<6} {order.total():>9.2f}")
    print()
    print(order.largest())
    print(json.dumps(asdict(order)))

    print(len({OrderLine("pen", 15.0, 3), OrderLine("pen", 15.0, 3)}))

    try:
        order.add("clip", 5.0, 0)
    except ValueError as err:
        print("rejected:", err)


if __name__ == "__main__":
    main()

pen        51.75
bag       977.50
ink       276.00
total    1305.25

OrderLine(name='bag', price=850.0, quantity=1)
{"customer": "rafi", "lines": [{"name": "pen", "price": 15.0, "quantity": 3}, {"name": "bag", "price": 850.0, "quantity": 1}, {"name": "ink", "price": 120.0, "quantity": 2}]}
1
rejected: quantity must be at least 1: 0

Five things worth looking at.

__post_init__ is where validation goes. @dataclass writes __init__ itself, so there is nowhere to put a check — __post_init__ fills that gap, running just after the fields are set. The last chapter's promise therefore survives: having an OrderLine means its quantity is at least one.

One is frozen and the other is not, deliberately. A line is not supposed to change once made, while an order keeps gaining lines. The question is the same each time: does this thing change after it is created?

The 1 is the proof that freezing worked. Two identical OrderLines became one in the set — in the last chapter that line would have raised a TypeError.

list[OrderLine] is not decoration. It is how the checker knows line.total() is valid, and how it would catch line.totl() — without running the code.

"Order" is in quotes. Inside add, where Order is mentioned, the class is not finished being defined — the name does not exist yet. Quoting it lets Python resolve it later. Referring to a class inside itself always needs this.

When it breaks

ValueError: mutable default ... use default_factory = [] or = {} was written. Take field(default_factory=list).

TypeError: non-default argument follows default argument A field with a default is placed before a required one. Chapter twenty's rule again: required first.

dataclasses.FrozenInstanceError Something was assigned on a frozen=True object. When a change is needed, dataclasses.replace(obj, x=5) gives a new object.

TypeError: unhashable type A dataclass without frozen=True cannot go in a set.

A wrong type went in and nothing happened As expected — hints do nothing at runtime. Run mypy.

mypy is not looking at my file Give it the path — mypy main.py. And if the first run buries you, start with mypy --ignore-missing-imports.

TypeError: Object of type X is not JSON serializable Send asdict(obj).

NameError for my own class's name inside itself Put the name in quotes: -> "Order".

Your turn

Rewrite chapter twenty-six's Book and Shelf as dataclasses.

Book is frozen=True with title: str, author: str, pages: int, and a __post_init__ raising ValueError when pages < 1. Shelf has name: str and books: list[Book] = field(default_factory=list), plus add, total_pages() -> int, longest() -> Book | None and by_author(name: str) -> list[Book].

Then pip install mypy, run mypy library.py, and fix until it says nothing.

Then six experiments:

• Pass text for pages — Book("one", "rafi", "300"). Does the program run? And what does mypy say?
• Replace books's field(default_factory=list) with = []. When does the error arrive — when an object is made, or before that?
• Make two identical Books and put them in a set. How many are there? Then remove frozen=True and try again.
• Use longest()'s result directly — shelf.longest().title — on an empty shelf. What does the program say, and what had mypy said beforehand?
• asdict(shelf), write it with json.dumps, and read it back with json.loads. Is what comes back a Shelf?
• Call a misspelled attribute — book.pagse. Does mypy catch it?

Those last two show this chapter's limit and its power together. What comes back from JSON is a dictionary, not a dataclass — the translation runs one way, and the return journey is yours to write. But the gap chapter twenty-six could not close — a misspelling caught only at runtime — the checker closes before the code runs at all.