configzen – explanation without code

The goal of the configzen project is to facilitate the creation and maintenance of structured configurations in a wide range of applications.

Features

So far, configzen allows you to:

1. Load your configuration into pydantic models from various formats supported by anyconfig (JSON, INI, XML, .properties, shellvars (see Augeas docs on shellvars), YAML, TOML, Amazon Ion, BSON, CBOR, ConfigObj, MessagePack and pickle).
2. Load and save your configuration partially, without touching things out of the desired scope of change.
3. Load and save your configuration both synchronously (using standard builtins.open or urllib.request.urlopen for external URLs, similarly to pandas) and asynchronously (using aiofiles for local files).

It is planned that configzen will someday allow you to:

1. Load and save configuration files from URLs asynchronously, using aiohttp framework under the hood) and a dedicated network protocol supporting version control.
2. Transclude configuration files (configuration files could internally import another configuration files, dynamically merge them and not only) – see details.
3. Automatically generate code describing your configuration based on ready-to-use, raw configuration files, so you do not have to write them manually.

Since configzen configuration models are based on pydantic models, you can leverage all features it brings, from advanced validation and parsing to its rich-compatible representations.

Goals

• Providing with an intuitive interface that does not require much learning for managing complex, structured configurations.
• Bringing the ability to constrain the scope of changes to the configuration at runtime within an intuitive API.
• Using the same API for a variety of configuration file formats.
• Getting the most out of everything Python OOP has to offer to solve particular problems (this seems not as precise as the preceding goals…).

Workflow steps

1. The developer creates code that defines the structure and behavior of the configuration at runtime. The code specifies, inter alia:

• what specific variables are defined in it, what types they have,
• whether to accept unspecified variables or not,
• how to validate such variables,
• can such variables be changed (and which ones are not?) while the program is running.

2. The developer prepares the configuration in one of the common configuration file formats and specifies its source. The configuration may have been created before, by hand or not, and has to be accessible either via file path, URL or a data stream.
3. The developer launches the application and their configuration is loaded into primitive Python types (dictionaries, lists, strings, lists etc.) and then processed as specified in step 1. This may include creation of classes built on top of these primitive types, primarily the aforementioned pydantic models, however that might be anything, including dataclasses, named tuples, custom arbitrary classes and such.

Meanwhile, the developer:

• benefits from specifying the structure in such a way that when writing a program in the IDE, they get accurate hints about the configuration variables and it is guaranteed that they have the type that was specified when creating the structure,
• does not need to learn additional terminology for this purpose nor be constrained to a specific file format they do not necessarily like,
• can change their configuration at runtime and save it in constrained scopes of the whole configuration, without touching any configuration variables out of scope of desire.