This package implements custom Kedro DataSet types for not only "pure" (JSON-serializable)
Pydantic models, but also models with
Keep reading for a basic tutorial, or check out the API Reference for auto-generated docs.
To simplify the documentation, we will refer to JSON-serializable Pydantic models as "pure" models, while all others will be "arbitrary" models.
Usage with Kedro¶
1 2 3 4
Then use it as usual within your Kedro pipelines:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
If you are using Kedro for the pipelines or data catalog, that should be enough.
If you want to use these datasets stand-alone, keep on reading.
"Pure" Pydantic Models¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Note: YAML support is enabled by
Note that specifying custom JSON encoders will work as usual, even for YAML models.
However, if your custom type is difficult or impossible to encode/decode via JSON, read on to Arbitrary Types.
Automatic Saving of Pydantic Models¶
The easiest way to use
v0.2.0) is through the
You can use it in the place of any other dataset for reading or writing.
When reading, it will figure out what the actual dataset type is. When writing, it will try to save it as a pure model, or fallback to an arbitrary model, depending on the options set. Below you can see the default options: