Code Typing

With the popularity of Python exploding over the past few years, especially in the machine learning front, Python developers have implemented type hints. With Python type hints, we can better express data expectations within our code. Consider the following function definition:

from datetime import date

def deliveries_by_day(logistics, zips, day=date.today()):
	pass

Without additional context, how do we know what format the function expects for each of the parameters? What is to be returned from the function? Compare this to the same function definition with type hints:

from datetime import date

def deliveries_by_day(logistics: dict, zips: set, day: date = date.today()) -> float:
	pass

This gives the developer a lot more information, while not affecting the runtime in any way (see note below). Additionally, static type checkers such as mypy can be used to help identify potential bugs in your code; often providing feedback directly within your IDE of choice:

deliveries_by_day("foo", "bar")
# Argument 1 to "deliveries_by_day" has incompatible type "str"; expected "Dict[Any, Any]"
# Argument 2 to "deliveries_by_day" has incompatible type "str"; expected "Set[Any]"

Note: This will not prevent code from executing in the Python runtime. The following code will fail static typing, but run without errors:

t: list[str] = [1, 2, 3]

Consider the assumption from a previous post where data was expected in one format, but delivered in another. We knew that the data was grouped, or keyed, by zip code; but discovered that the keys did not contain leading zeroes. Thus data expected for zip code 06101 was actually keyed to 6101. Additionally, the daily deliveries element within the zip code grouping was discovered to sometimes be missing when assumed required.

print(data)
"""
{
  "12345": [...],
	...,
	"6101": [  # 06101
    {"day": "2021-12-31", "deliveries": 3.14, ...},
    {"day": "2022-01-01", ...},  # no deliveries key
    ...
  ],
}
"""

This is an all-too common experience with data sources, and many of those will be outside of your control, i.e. consumed over API or from third-party providers. Pydantic is a tool that uses Python type annotations to easily validate data definitions, using syntax nearly identical to native dataclasses. This allows us to convert the generic alias for `logistics` above as such:

from datetime import date
from pydantic import BaseModel
from typing import Optional

# logistics = dict[str, Union[str, float]]
class Logistics(BaseModel):
  day: date
  deliveries: Optional[float] = None

  class Config:
    extra = "allow"

data = {"day": "2022-01-01", "deliveries": 3.14}
logistics = Logistics(**data)
#> Logistics(day=datetime.date(2021, 12, 31), deliveries=3.14)
logistics.day.isoweekday()
#> 5

try:
  logistics = Logistics(deliveries="foo")
except ValidationError as e:
  print(e)
"""
2 validation errors for Logistics
day
  field required (type=value_error.missing)
deliveries
  value is not a valid float (type=type_error.float)
"""

Data elements will be validated on instantiation: i.e. invalid data will throw exceptions; day will be cast to a date object; deliveries, when present, will be cast to a float; and extra elements (all optional) will be allowed without additional validation.

Json object typing:

from pydantic import BaseModel, Json

class SourceData(BaseModel):
    obj: Json[dict[int, list[Logistics]]]

source_data = SourceData(obj=source)
"""
SourceData(
  obj={
    12345: [...],
    ...,
    6101: [Logistics(day=datetime.date(2021, 12, 31), deliveries=3.14), ...],
    ...,
  }
)
"""

If you would like to look into testing your code, check out our next blog Code Testing in Python: Red, Green and Refactor.