Validating YAML and TOML Configurations in Python with Pydantic

Introduction

Modern Python applications often rely on configuration files to manage settings like database connections, application features, and environment-specific parameters. Popular formats include YAML and TOML because they are human-readable and easy to maintain. However, these files are prone to errors, like missing fields or invalid types, which can cause runtime crashes.

This article demonstrates how to use Pydantic, a Python data validation library, to validate YAML and TOML configurations — from basic usage to advanced production-ready techniques.

Understanding the Problem

Suppose we have a configuration file for the application:

app:
  name: MyApp
  version: 1.0
database:
  host: localhost
  port: 5432
  enabled: true

Without validation:

A typo in port ("5432a") would crash the app.
Missing keys like database would cause runtime errors.
Invalid types (enabled: "yes") could lead to unexpected behavior.

Goal: Automatically verify that all required fields exist, are of the correct type, and meet any constraints.

Introduction to Pydantic

Pydantic provides:

Type enforcement: Ensures the right type for each field
Field validation: Enforces constraints like value ranges
Nested models: Supports structured, hierarchical configs
Fail-fast validation: Errors are raised immediately if something is wrong

Basic Pydantic example:

from pydantic import BaseModel

class AppConfig(BaseModel):
    name: str
    version: float

app = AppConfig(name="MyApp", version=1.0)

Loading YAML and TOML in Python

Python libraries:

YAML: Use PyYAML (pip install pyyaml)
TOML: Python 3.11+ has built-in tomllib

import yaml
import tomllib

Defining Pydantic Models for Configuration

Suppose your config has two sections: add and database

from pydantic import BaseModel, Field

class AppConfig(BaseModel):
    name: str
    version: float

class DatabaseConfig(BaseModel):
    host: str
    port: int = Field(gt=0, lt=65536)  # port must be between 1-65535
    enabled: bool

class Settings(BaseModel):
    app: AppConfig
    database: DatabaseConfig

Field (gt=0, lt=65536) ensures the port is valid.
Nested models enforce structure ( app and database).

Loading and Validating YAML

def load_yaml(file_path: str) -> Settings:
    with open(file_path, "r") as f:
        data = yaml.safe_load(f)
    return Settings.model_validate(data)  # Pydantic v2

Example usage:

settings = load_yaml("config.yaml")
print(settings.app.name)         # MyApp
print(settings.database.port)    # 5432

Loading and Validating TOML

def load_toml(file_path: str) -> Settings:
    with open(file_path, "rb") as f:
        data = tomllib.load(f)
    return Settings.model_validate(data)

TOML example (config.toml):

[app]
name = "MyApp"
version = 1.0

[database]
host = "localhost"
port = 5432
enabled = true

Unified Loader for YAML and TOML

To simplify usage:

import os
import sys

def load_config(file_path: str) -> Settings:
    if not os.path.exists(file_path):
        raise FileNotFoundError(f"Config file not found: {file_path}")
    try:
        if file_path.endswith((".yaml", ".yml")):
            return load_yaml(file_path)
        elif file_path.endswith(".toml"):
            return load_toml(file_path)
        else:
            raise ValueError("Unsupported config format")
    except ValidationError as e:
        print("Config validation failed!")
        print(e)
        sys.exit(1)

Basic Usage in Your Application

from config import load_config
import os

def main():
    config_file = os.path.join(os.path.dirname(__file__), "config.yaml")
    settings = load_config(config_file)
    print("App Name:", settings.app.name)
    print("Database Host:", settings.database.host)
if __name__ == "__main__":
    main()

Loads and validates config on application startup.
Fails immediately if config is invalid.

Advanced Features

Environment Variable Overrides

With Pydantic v2, you can use BaseSettings to automatically override fields from environment variables:

from pydantic import BaseSettings

class Settings(BaseSettings):
    app_name: str
    database_port: int

    class Config:
        env_prefix = "MYAPP_"

MYAPP_APP_NAME in environment overrides app_name.

Custom Validators

from pydantic import field_validator

class DatabaseConfig(BaseModel):
    host: str
    port: int
    enabled: bool

    @field_validator("host")
    @classmethod
    def host_not_empty(cls, v):
        if not v.strip():
            raise ValueError("Host cannot be empty")
        return v

Default Values and Optional Fields

class DatabaseConfig(BaseModel):
    host: str = "localhost"
    port: int = 5432
    enabled: bool = True

Makes some fields optional while still validating others.

Fail-Fast and User-Friendly Errors

Always wrap loading in a try/except ValidationError.
Print field-specific errors so developers know exactly what’s wrong.

try:
    settings = load_config("config.yaml")
except ValidationError as e:
    print("Invalid config:", e)
    sys.exit(1)

Benefits of Using Pydantic

Type safety: ensures correct types at startup
Structured configs: nested sections like app and database
Fail-fast: invalid configs stop the app immediately
Easy to extend: environment variables, defaults, and validators
Works for YAML, TOML, JSON, and Python dictionaries

Full working example is available at my github Jayant0516 (Jayant Kumar)