<style> .labels { position: absolute; display: flex; gap: 0.5rem; top: 1rem; } .labels .beginner, .labels .intermediate, .labels .advanced { font-size: 0.5rem; } .beginner { color:rgb(0, 191, 255); } .beginner::after { content: "beginner"; } .intermediate { color: #90EE90; } .intermediate::after { content: "intermediate"; } .advanced { color: #D70040; } .advanced::after { content: "advanced"; } </style>

What (not) to do when type hinting Python?

16.01.2025 PyWaw #117

<style scoped> .to-right { text-align: right; } </style>

by Bartosz Sławecki (@bswck) Junior Python Engineer @ Printbox

Check-in

Special thanks

Shoutout to:

• Carl Meyer, for helping me understand the type theory
• a user named decorator-factory, for helping me understand the true purpose of this presentation
• Jelle Zijlstra, for participating in the discussion about my talk
• everyone else involved who encouraged me in this endeavor

Who is this talk for?

Everyone! And especially for you, if you:

• are interested in using typing and have no prior practice ()
• occasionally use typing, but not a lot ()
• already use typing extensively and maybe like it ()

What is typing in Python all about?

It's about describing what sets of runtime values can reside in particular variables.

What is type hinting in Python all about?

Type hinting is as simple as turning

<style scoped> .flex { display: flex; gap: 1rem; width: 40rem; justify-content: space-between; align-items: center; } .flex pre { width: 30rem; } </style>

def cube_area(e):
    return f"Cube area: {6 * e ** 2}."

into

def cube_area(e: float) -> str:
    return f"Cube area: {6 * e ** 2}."

If you're starting out

Let's learn about two main kinds of types really quickly.

Don't forget about these useful go-tos

• Python type system specification
• typing standard library docs
• relevant PEPs
• typeshed
• docs of particular type checkers (mypy, pyright, et alia)
• Python docs
• YouTube videos from Anthony Sottile, James Murphy, me et alia

Check out various different type checkers

• mypy (esp. recommended for s)
• pyright (esp. recommended for s)
• pyre
• pytype

When would you use mypy?

• You want to stick with the most popular option
• You want to compile your code with mypyc to C extensions (~2.5x speedup)

Docs: https://mypy.readthedocs.io/en/stable/

When would you use pyright?

• You use Pylance in VS Code
• You like pyright's approach, design decisions and behaviors that differ from mypy's

Docs: https://microsoft.github.io/pyright/, comparison with mypy

When would you use Pyre?

• You want to check out the type checker used for linting Instagram
• You've heard about Pysa and want to test it too

Docs: https://github.com/facebook/pyre-check Some background: https://news.ycombinator.com/item?id=17048682

When would you use pytype?

• You prefer lenient type checking
• You want to rely more on type inference than on explicit annotations (no gradual typing)

Docs: https://google.github.io/pytype/, comparison with mypy

It's not everything...

Coming soon: red-knot

From Astral, the team behind Ruff and uv

<style scoped> img { width: 30rem; } .small { font-size: 0.7rem; } </style>

...and if you like rabbit holes,

check out those: basedmypy, basedpyright, pyanalyze

To conclude,

<style scoped> img { width: 25rem; padding-top: 1rem; } </style>

Typing: a strategy that works

Typing: a strategy that works*

Typing: a strategy that works*

*on my machine

Typing: a strategy that works

For every typing feature, do the following:

0. Learn about it
1. Gradual introduction (remember about chunking and aliasing)
2. Troubleshooting (optionally, trouble-shouting) / Getting it right
3. Staying up to date (but not up late)

Learn about it

Example: From a linter

Gradual introduction

Example: Adding types to my code from 4 years ago

Troubleshooting / Getting it right

• Work through the errors reported by your type checker
• Don't be afraid to google things
• Suggest improvements to type checkers / File bug reports
• Ask questions in Python Discord's #type-hinting

Staying up to date

• Follow the changelogs (or videos about them)
• Subscribe to Codezarre (https://codezarre.com—try to print this website)

Want to dabble even more?

• Read the new typing PEPs
• Participate in typing discussions!
• Contribute to typing-related projects or create them!
  • https://github.com/bswck/class_singledispatch
  • https://github.com/bswck/runtime_generics
  • https://github.com/alexmojaki/eval_type_backport

To avoid common pitfalls...

Mentally separate the typing world from the runtime world

Typing and runtime have their own, sometimes discrepant rules. In the end, it's the runtime that matters.

<style scoped> .examples { display: flex; padding-top: 1rem; justify-content: space-evenly; gap: 1rem; } pre { width: 100%; } </style>

# passes type checking
x: complex = True

# fails at runtime
assert isinstance(x, complex)

# passes type checking
message: str = NotImplemented

# fails at runtime
assert isinstance(message, str)

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    # analyzed by type checkers
    # never executed at runtime
    from circular import something
    from costly import just_for_types
else:
    # always executed at runtime
    # ignored by type checkers
    something very hacky!

Think about types, not classes

In Python, classes are object factories defined by the class statement, and returned by the type(obj) built-in function. Class is a dynamic, runtime concept.

Classes are commonly used to create:

• Nominal types (e.g. str)
• Structural types (TypedDict constructs, Protocols)

Besides that, there are...

• Special forms (e.g. Never, Literal, Generic, TypedDict)
• Weird types (e.g. None, NotImplemented, NewType)

Understand the subtyping relation

The Liskov substitution principle.

We have a function compute_salary(e: Employee). It accepts an argument of type Employee. Does it also accept an argument of type Manager, given that Manager is a subtype of Employee?

Practice the S from SOLID

Don't make others narrow down your types. Have an async function? Create a separate coroutine function. Have a sync function? Create a separate function.

Not both at the same time ;)

What I like about strict static typing in Python

Opinionated section.

Autocompletions are very helpful

• Easier learning of available functionality
• Thinking about developer experience => Faster development in the long run

Explicit type hinting feels PEP 20-ish

import this

def foo() -> int:
    return 5

writing the -> int ensures you always return int, and not a supertype or other incompatible type.

Hacking and golfing are costly

• You are encouraged to rely on single-purpose and statically known constructs
• The code structure can be fairly simpler to be understood by the type checker

You need a good reason to lie

...otherwise don't.

Good reasons:

• the impossibility of expressing a type in the current type system
• lack of ergonomicity to specifying the correct type
• DX—see Werkzeug proxies! (flask.g, flask.request)

Some great projects do lie, so just be sure to have reasons if you need to.

Things to study

That's your homework assignment! https://github.com/bswck (pinned repo)

New features

• Use typing.Self (3.11+) or typing_extensions.Self (3.9+) for methods returning cls/self.
• Leverage PEP 696 (Generator[int, None, None] -> Generator[int]) to reduce redundancy.
• Type hint using generic built-in types (3.9+, PEP 585). E.g. use list[str], not typing.List[str].
• Use X | Y instead of typing.Union[X, Y] in 3.10+ codebases (PEP 604). This applies to typing.Optional, too!
• Don't use dict[str, Any] for annotating fixed-structure data (use TypedDict, dataclasses, or other models instead).

New features

• Review your TypeGuards that could be TypeIss. See (TypeIs vs TypeGuard and PEP 742).
• Don't bother using TYPE_CHECKING in a module where any of your types are evaluated at runtime (e.g. in Pydantic models).
• Be pragmatic about TYPE_CHECKING. Use it to optimize import times, avoid circular imports and import symbols from stubs.

General

• Don't confuse Any with object (check this).
• Don't use Any as an easy way to type a hard-to-annotate interface. Read how to move away from Any.
• Don't use the deprecated aliases from typing.
• Try not to have to use overloads, but use them to logically associate call conventions, especially when unions are involved.
• Use stub files for annotating extension modules.

Opinionated

• DON'T use bare # type: ignore (this applies also to # noqa).
• DON'T skip annotating return values (here's a writeup).
• DO allow yourself to use PEP 563 despite future deprecation.
• DO prefer typing.Never to typing.NoReturn (no difference).
• Avoid T | Awaitable[T] (T | Coroutine[T, None, None]). Single responsibility and interface segregation.

Opinionated (for libraries)

• DO type-check at tail (minimum supported version) or all supported versions (to cover if sys.version_info branches).
• DO use __all__ to control re-exports.
• DO minimize runtime overhead if using inlined types. E.g. this PR

Wrapping up

https://github.com/bswck (pinned repo)

Share your feedback

<style scoped> img { width: 40%; } </style>