Python  

Validating YAML and TOML Configurations in Python with Pydantic

Introduction

Modern Python applications often rely on configuration files to manage settings like database connections, application features, and environment-specific parameters. Popular formats include YAML and TOML because they are human-readable and easy to maintain. However, these files are prone to errors, like missing fields or invalid types, which can cause runtime crashes.

This article demonstrates how to use Pydantic, a Python data validation library, to validate YAML and TOML configurations — from basic usage to advanced production-ready techniques.

Understanding the Problem

Suppose we have a configuration file for the application:

app:
  name: MyApp
  version: 1.0
database:
  host: localhost
  port: 5432
  enabled: true

Without validation:

  • A typo in port ("5432a") would crash the app.

  • Missing keys like database would cause runtime errors.

  • Invalid types (enabled: "yes") could lead to unexpected behavior.

Goal: Automatically verify that all required fields exist, are of the correct type, and meet any constraints.

Introduction to Pydantic

Pydantic provides:

  • Type enforcement: Ensures the right type for each field

  • Field validation: Enforces constraints like value ranges

  • Nested models: Supports structured, hierarchical configs

  • Fail-fast validation: Errors are raised immediately if something is wrong

Basic Pydantic example:

from pydantic import BaseModel

class AppConfig(BaseModel):
    name: str
    version: float

app = AppConfig(name="MyApp", version=1.0)

Loading YAML and TOML in Python

Python libraries:

  • YAML: Use PyYAML (pip install pyyaml)

  • TOML: Python 3.11+ has built-in tomllib

import yaml
import tomllib

Defining Pydantic Models for Configuration

Suppose your config has two sections: add and database

from pydantic import BaseModel, Field

class AppConfig(BaseModel):
    name: str
    version: float

class DatabaseConfig(BaseModel):
    host: str
    port: int = Field(gt=0, lt=65536)  # port must be between 1-65535
    enabled: bool

class Settings(BaseModel):
    app: AppConfig
    database: DatabaseConfig

  • Field (gt=0, lt=65536) ensures the port is valid.

  • Nested models enforce structure ( app and database).

Loading and Validating YAML

def load_yaml(file_path: str) -> Settings:
    with open(file_path, "r") as f:
        data = yaml.safe_load(f)
    return Settings.model_validate(data)  # Pydantic v2

Example usage:

settings = load_yaml("config.yaml")
print(settings.app.name)         # MyApp
print(settings.database.port)    # 5432

Loading and Validating TOML

def load_toml(file_path: str) -> Settings:
    with open(file_path, "rb") as f:
        data = tomllib.load(f)
    return Settings.model_validate(data)

TOML example (config.toml):

[app]
name = "MyApp"
version = 1.0

[database]
host = "localhost"
port = 5432
enabled = true

Unified Loader for YAML and TOML

To simplify usage:

import os
import sys

def load_config(file_path: str) -> Settings:
    if not os.path.exists(file_path):
        raise FileNotFoundError(f"Config file not found: {file_path}")
    try:
        if file_path.endswith((".yaml", ".yml")):
            return load_yaml(file_path)
        elif file_path.endswith(".toml"):
            return load_toml(file_path)
        else:
            raise ValueError("Unsupported config format")
    except ValidationError as e:
        print("Config validation failed!")
        print(e)
        sys.exit(1)

Basic Usage in Your Application

from config import load_config
import os

def main():
    config_file = os.path.join(os.path.dirname(__file__), "config.yaml")
    settings = load_config(config_file)
    print("App Name:", settings.app.name)
    print("Database Host:", settings.database.host)
if __name__ == "__main__":
    main()

  • Loads and validates config on application startup.

  • Fails immediately if config is invalid.

Advanced Features

Environment Variable Overrides

With Pydantic v2, you can use BaseSettings to automatically override fields from environment variables:

from pydantic import BaseSettings

class Settings(BaseSettings):
    app_name: str
    database_port: int

    class Config:
        env_prefix = "MYAPP_"

  • MYAPP_APP_NAME in environment overrides app_name.

Custom Validators

from pydantic import field_validator

class DatabaseConfig(BaseModel):
    host: str
    port: int
    enabled: bool

    @field_validator("host")
    @classmethod
    def host_not_empty(cls, v):
        if not v.strip():
            raise ValueError("Host cannot be empty")
        return v

Default Values and Optional Fields

class DatabaseConfig(BaseModel):
    host: str = "localhost"
    port: int = 5432
    enabled: bool = True

  • Makes some fields optional while still validating others.

Fail-Fast and User-Friendly Errors

  • Always wrap loading in a try/except ValidationError.

  • Print field-specific errors so developers know exactly what’s wrong.

try:
    settings = load_config("config.yaml")
except ValidationError as e:
    print("Invalid config:", e)
    sys.exit(1)

Benefits of Using Pydantic

  • Type safety: ensures correct types at startup

  • Structured configs: nested sections like app and database

  • Fail-fast: invalid configs stop the app immediately

  • Easy to extend: environment variables, defaults, and validators

  • Works for YAML, TOML, JSON, and Python dictionaries

Full working example is available at my github Jayant0516 (Jayant Kumar)

Membership not found