Pydantic Settings: How Smarter Validates and Manages Its Configuration

Smarter reads all of its configuration through a Pydantic Settings class, smarter.common.conf.Settings, whose one instance, smarter_settings, is built before Django’s settings module runs, and supplies it. Every environment variable that Smarter reads is typed, validated against business rules, and turned into derived values such as the environment’s URLs, all before Django sees any of it. The result is immutable, and there is exactly one of it. This post walks through what that Pydantic Settings layer checks, why those checks would be cumbersome to write in Django settings, and how it fits with GitHub Actions, Docker and Helm.

Smarter is a Django platform for building and running LLM applications. It runs on a developer’s laptop, in four Kubernetes environments (alpha, beta, next and production), and in other people’s AWS accounts. Each of those needs dozens of settings, such as domains, credentials for AWS, SMTP and LLM providers, encryption keys, cache lifetimes, rate limits, feature switches. Some of them are secrets. Some depend on others. A few, if wrong, quietly break things in ways that show up days later. Configuration is where a platform like this is most likely to fail in production, and least likely to have tests.

What Django settings give you

Django’s settings are a Python module. Whatever its global names in upper case hold when it’s imported, that’s the configuration. It’s a simple, flexible design, and for a small project it’s all you need. But it leaves everything beyond “assign a value to a name” to you.

• Every environment variable is a string. os.environ.get("DEBUG") returns "False", which is truthy. Each setting needs its own parsing, and each project writes the same == "True" idioms again.
• Nothing is validated. A port of 70000, a malformed email, an AWS region that doesn’t exist, or an encryption key of the wrong length is accepted at startup, and fails later, in the first request that happens to need it.
• Rules between settings have nowhere to live. “The region must be one of the allowed regions” or “a profile overrides access keys” become if statements scattered through a long module, in whatever order the module happens to be written.
• Secrets are plain strings. They print in tracebacks, debug pages and print(settings.__dict__).
• Everything is mutable. Any code can assign settings.FOO = ... at run time, and tests often do.
• Derived values are duplicated. A hostname built from three variables is rebuilt, slightly differently, in every place that needs it.

None of this is a flaw in Django. Its settings module was never meant to be a validation framework. Smarter puts one in front of it.

smarter_settings comes first

Smarter’s configuration flows in one direction.

The settings module is written to stay upstream of Django. Its module docstring says so plainly.

DO NOT import Django or any Django modules in this module. This module
sits upstream of Django and is intended to be used independently of Django.

And Django’s settings then read from it.

ALLOWED_HOSTS = smarter_settings.allowed_hosts
CSRF_COOKIE_DOMAIN = smarter_settings.environment_platform_domain
SESSION_COOKIE_DOMAIN = smarter_settings.environment_platform_domain
DEBUG = smarter_settings.debug_mode

The docstring on Django’s SECRET_KEY names the whole path a secret takes.

For production deployments the chain of custody should be:

GitHub Secrets -> GitHub Actions -> environment variable -> smarter_settings -> Django SECRET_KEY.

Django still works the way every Django developer expects. from django.conf import settings still answers settings.ALLOWED_HOSTS. It’s just that the value was checked, typed and derived before Django ever saw it. ADR-007 makes it the rule, “All configuration for the platform must be handled through the Settings singleton.”

The model’s configuration

Four lines of SettingsConfigDict set the tone for every field.

model_config = SettingsConfigDict(
    strict=True,
    frozen=True,
    extra="forbid",
    validate_default=True,
)

strict=True refuses to coerce types silently. frozen=True makes the instance immutable. extra="forbid" rejects any field the class doesn’t declare, so a misspelled keyword fails rather than being ignored. And validate_default=True runs the validators on default values too, so a bad default is caught the same way a bad environment variable is.

Types, formats and ranges

Most checks are simply the field’s type, and they cost one line each.

• Secrets are SecretStr. The AWS keys, the Django secret key, the Fernet encryption key, the SMTP password, the GitHub token and the Stripe keys all print as ********** in logs, reprs and tracebacks. Code has to call .get_secret_value() on purpose to read one.
• Emails are EmailStr, and URLs are HttpUrl or AnyUrl. email_admin, branding_support_email and smtp_sender must be real email addresses. marketing_site_url, branding_contact_url and the social links must be real URLs.
• Numbers have bounds. log_level is an integer between 0 and 50, Python’s logging levels. llm_default_max_tokens is at least 1.
• Regular expressions are compiled at startup. sensitive_files_amnesty_patterns arrives as strings and is stored as a List[Pattern], so an invalid pattern fails when the process starts, not in the middle of a file upload.

Some formats need a validator of their own. The SMTP port has to be a number between 1 and 65535.

@before_field_validator("smtp_port")
def validate_smtp_port(cls, v: Optional[int]) -> Optional[int]:
    if v in THE_EMPTY_SET:
        v = settings_defaults.SMTP_PORT
    try:
        retval = int(v)
    except (TypeError, ValueError) as e:
        raise SmarterValueError("Could not convert port number to int.") from e
    if not str(retval).isdigit() or not 1 <= int(retval) <= 65535:
        raise SmarterValueError("Invalid port number")
    return retval

The Fernet key, which encrypts the values of Smarter’s Secrets in the database, has to decode to exactly 32 bytes.

encryption_key = v.get_secret_value()
decoded_key = base64.urlsafe_b64decode(encryption_key)
# Ensure the decoded key is exactly 32 bytes
if len(decoded_key) != 32:
    raise ValueError("Fernet key must be exactly 32 bytes when decoded.")

An API rate limit has to look like a rate, 120/minute, or be one of none, off or disabled, checked against ^\d+/(s|sec|second|m|min|minute|h|hour|d|day)$.

Each of these is a mistake that someone will eventually make in a values file at the end of a long day. In Django settings, each would be a few lines of hand-written parsing in a module that nobody tests. Here, each is a validator next to its field, with a docstring.

Rules between settings

The checks that are hardest to write in Django settings are the ones that involve more than one value, because a settings module has no natural place to put them. A Pydantic validator can read the fields validated before it.

The AWS region must be one of the platform’s allowed regions.

@before_field_validator("aws_region")
def validate_aws_region(cls, v: Optional[str], values: ValidationInfo, **kwargs) -> Optional[str]:
    valid_regions = values.data.get("aws_regions", AWS_REGIONS)
    if v in THE_EMPTY_SET:
        if settings_defaults.AWS_REGION == DEFAULT_MISSING_VALUE:
            return None
        return settings_defaults.AWS_REGION
    if v not in valid_regions:
        raise SmarterValueError(f"aws_region {v} not in aws_regions: {valid_regions}")
    return v

An AWS access key is ignored when an AWS profile is configured, and otherwise must have the shape of a real access key ID.

aws_profile = values.data.get("aws_profile", None)
if aws_profile and len(aws_profile) > 0 and aws_profile != DEFAULT_MISSING_VALUE:
    logger.warning("aws_access_key_id is being ignored. using aws_profile %s.", aws_profile)
    return None

# validate the pattern of the access key id
pattern = r"^AKIA[0-9A-Z]{16}$"
if not re.match(pattern, v.get_secret_value()):
    raise SmarterConfigurationError("aws_access_key_id is not a valid AWS access key ID format.")

Business rules

Some rules aren’t about types at all. They’re about what’s safe to run.

Smarter refuses to start outside local development without a Django secret key. Without one, every pod would generate its own random key, and sessions and password reset links would break between pods and across restarts. A placeholder is worse, because SET-ME-IN-helm/charts/smarter/values.yaml is published in the repository, so anyone could forge signed values with it. is_missing_value() treats any value that starts with SET-ME as missing, and the settings class enforces the rule.

def require_secret_key(self) -> None:
    if self.secret_key_is_set:
        return
    if not self.environment_is_local:
        raise SmarterConfigurationError(
            f"SECRET_KEY is not set. It is required in the {self.environment} environment. "
            "Set SMARTER_SECRET_KEY to a long (50+ characters), random string that is the same for every pod."
        )
    logger.warning("%s SECRET_KEY is not set. Using a randomized value.", logger_prefix)

Locally, a generated key is fine, and only logs a warning. Django’s base.py calls smarter_settings.require_secret_key() right after it reads the key.

Other rules warn rather than fail. ready() reports whether the settings are complete enough for a deployment that real users can reach. A root domain still set to example.com outside local development, or AWS credentials that are neither a profile nor a pair of keys, each print a red banner at startup and a warning in the log. Nothing there is needed to run Smarter on a laptop, so those features degrade gracefully instead of stopping the process.

Derived values for the environment URLs

The most useful thing smarter_settings does is compute values that nobody should have to type. Smarter’s URLs are a good example. An operator sets three environment variables,

and the settings class derives the rest. The platform’s domain depends on the environment.

@cached_property
def environment_platform_domain(self) -> str:
    if self.environment == SmarterEnvironments.PROD:
        return self.root_platform_domain
    if self.environment in SmarterEnvironments.aws_environments:
        return f"{self.environment}.{self.root_platform_domain}"
    if self.environment_is_local:
        return f"localhost:{SMARTER_LOCAL_PORT}"
    # default domain format
    return f"{self.environment}.{self.root_platform_domain}"

The API’s domain follows the same pattern, from api and the platform’s domain. Here’s what those three variables become.

environment_platform_url and environment_api_url go through SmarterValidator.urlify(), which picks http or https by environment and adds the trailing slash, and raises SmarterConfigurationError if it can’t build a valid URL. environment_namespace is the Kubernetes namespace that the environment runs in, and it raises for an environment that has no namespace.

Those derived values then feed others. allowed_hosts is built from the platform and API domains, plus the hostnames without their ports, and validates every entry as a hostname. Django’s ALLOWED_HOSTS is smarter_settings.allowed_hosts. The CSRF and session cookie domains are environment_platform_domain. In the AWS environments, CSRF_TRUSTED_ORIGINS is https:// plus each allowed host, and the default Redis location is a service name inside environment_namespace.

So one variable, ENVIRONMENT=alpha, moves the platform’s URLs, its API host, its allowed hosts, its cookie domains, its CSRF origins and its Kubernetes namespace together, consistently, without anyone writing any of them down. In a Django settings module, each of those would be its own string formatting, and each could disagree with the others.

Immutable, and why that’s good

frozen=True means that once smarter_settings exists, nothing can change it. Assigning to a field raises a validation error. That’s a deliberate constraint, and it pays for itself.

• What was validated is what runs. Every check above happens once, at construction. If code could change a value afterwards, the validation would describe a configuration that no longer exists.
• Derived values stay correct. The URLs, hosts and namespace are cached_property values, computed once and remembered. That’s safe only because their inputs can never change. With mutable settings, a cached URL could silently disagree with the domain it was built from.
• No action at a distance. A setting can’t be changed by a module three imports away, by a test that forgot to restore it, or by a request handler on one thread while another reads it. Configuration is the one piece of state every part of the platform shares, which is exactly why it shouldn’t be writable.
• Tests construct, rather than mutate. A test that needs a different configuration builds its own instance, Settings(aws_regions=["eu-west-1"], aws_region="eu-west-1"), and validates it the same way production does.

A singleton, and why that’s good

smarter_settings is built by one function, cached so it runs once per process.

@lru_cache(maxsize=1)
def get_settings() -> Settings:
    try:
        return Settings()
    except ValidationError as e:
        raise SmarterConfigurationError("Invalid configuration: " + str(e)) from e


smarter_settings = get_settings()

There is one instance per process, and every module imports that one.

• Validation runs once. Reading the environment, applying defaults and running every validator happens a single time, when the module is first imported.
• Every part of the platform agrees. The web server, both Celery workers, the scheduler, management commands and Django’s own settings all read the same object. There’s no second copy that could have been built from a different environment.
• It’s available outside Django. Because the module doesn’t import Django, a Celery worker, a script or a management command’s early setup can import smarter_settings before Django is configured.
• It fails in one place. If the configuration is invalid, the error is raised at import, as a SmarterConfigurationError that names the field and the reason, and the process doesn’t start. A pod that can’t start is visible to Kubernetes at once. A pod that starts with a bad configuration fails later, for a user.

How it works with GitHub Actions, Docker and Helm

Because smarter_settings reads only environment variables, it doesn’t care how they got there. That’s the Twelve-Factor idea of configuration, and it’s why the same class works unchanged with every tool that deploys Smarter.

Locally, with Docker

Each service in docker-compose.yml reads the repository’s .env file, which starts as a copy of the documented .env.example.

env_file:
  - ./.env
environment:
  - PYTHONUNBUFFERED=1
  - SERVICE_TYPE=app

A placeholder left in .env, such as SET-ME-PLEASE, counts as missing, so the default applies instead of the placeholder.

In Kubernetes, with Helm

The Helm chart’s values.yaml has a flat env map,

env:
  AWS_REGION: "ca-central-1"
  DEBUG_MODE: "false"
  ENVIRONMENT: "prod"
  SECRET_KEY: "SET-ME-IN-helm/charts/smarter/values.yaml"
  SMARTER_ANTHROPIC_API_KEY: "SET-ME-IN-helm/charts/smarter/values.yaml"

and every Deployment renders it into its pods’ environment.

env:
  {{- range $key, $value := .Values.env }}
  - name: {{ $key }}
    value: {{ $value | quote }}
  {{- end }}

The placeholders in the published chart are recognized as missing, so a chart installed without a real secret key fails at startup outside local development, rather than running with a key that anyone can read on GitHub.

In CI, with GitHub Actions

Smarter’s deploy workflow keeps its credentials in GitHub’s encrypted secrets, and passes them to a composite action.

aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
openai-api-key: ${{ secrets.OPENAI_API_KEY }}
fernet-encryption-key: ${{ secrets.FERNET_ENCRYPTION_KEY }}

The action writes them into the env map of a values file for that deployment, and runs helm upgrade --install with it. Credentials that already live in the cluster, such as the database’s, are read from Kubernetes Secrets into the job’s environment the same way. From there the path is the one in SECRET_KEY‘s docstring, from GitHub Secrets to GitHub Actions, an environment variable, smarter_settings, Django.

At no point does any of these tools need to know about Pydantic, validators or derived URLs. Docker, Helm and GitHub Actions each do what they’re good at, which is putting strings into a process’s environment. smarter_settings does what they can’t, which is decide whether those strings make sense.

The benefits

Configuration errors fail at startup

An invalid value raises SmarterConfigurationError when the process starts, with the field’s name and the reason. In Kubernetes, that’s a pod that never becomes ready, and a rolling update that stops before it replaces a healthy pod. The alternative, a mistake that surfaces in the first request that needs it, is how a bad SMTP port turns into a week of unsent emails that nobody notices.

One source of truth

Every value has one definition, one default, one validator and one docstring, in one class. A developer who wants to know what LLMCLIENT_CACHE_EXPIRATION does, what type it is and what it defaults to, reads one field. There’s no second copy in a settings module, a Helm template or a script that could drift from it.

Real types everywhere

Code reads smarter_settings.smtp_port as an int, smarter_settings.debug_mode as a bool and smarter_settings.environment_platform_url as a str, and type checkers and editors know it. Nobody writes int(os.environ.get(...)) in the middle of business logic, and nobody discovers at run time that "False" was truthy.

Secrets are protected by default

Every credential is a SecretStr, so it’s masked in logs, reprs, tracebacks and the settings dump. Revealing one takes an explicit call to .get_secret_value(), which is easy to find in a code review. Placeholders that were published in the repository are never accepted as real secrets.

Values are derived, not duplicated

An operator sets the root domain, the subdomain and the environment, and the platform computes its URLs, hosts, cookie domains, CSRF origins and Kubernetes namespace from them. Fewer inputs mean fewer chances to get one wrong, and derived values can’t disagree with each other, because they’re computed by the same code from the same inputs.

Immutability makes it safe to share

Every process, thread and module reads the same configuration, and none of them can change it. Cached derived values stay correct for the life of the process, and a test can’t leak a modified setting into the next test.

It works with any deployment tool

The settings class reads environment variables and nothing else. Docker Compose, Helm, GitHub Actions, a systemd unit or a developer’s shell can all supply them, and none of them needs a special integration. Moving Smarter to a different CI system or cluster changes how the variables are set, not how they’re read.

It documents itself

Every field has a description, a title, examples and a reStructuredText docstring that says its type, its default and what it raises. Sphinx publishes them as the settings reference, so the documentation is the code.

It’s independent of Django

Because the module never imports Django, the same validated configuration is available to Celery workers, management commands and scripts before Django is configured, and could be used by code that isn’t Django at all. Django becomes one consumer of the configuration, rather than its owner.

Takeaways

If you want the same on your own Django project, in rough order of payoff.

• Put a Pydantic Settings class in front of Django’s settings, in a module that doesn’t import Django, and have settings.py read from it.
• Make the instance frozen, strict and extra="forbid", and validate the defaults.
• Store every credential as SecretStr, and treat any placeholder as missing.
• Write a validator for every format a person could get wrong, such as ports, regions, keys, rates, patterns.
• Put rules between settings in validators that read the fields before them.
• Derive URLs, hosts and namespaces from a few inputs, as cached properties, rather than asking operators to type them.
• Build the instance once, with lru_cache, and import that one everywhere.
• Fail at startup, with a message that names the setting.

Closing thoughts

Django’s settings module does one thing well. It hands Django its configuration. Smarter keeps it for exactly that, and puts everything else in front of it, in a class whose only job is to decide whether the environment makes sense. The operator sets a handful of strings in Docker, Helm or GitHub Actions. Pydantic turns them into one typed, validated, immutable object, derives everything that can be derived, and refuses to start if something is wrong. By the time Django reads ALLOWED_HOSTS, every decision about it has already been made, and checked.

• Smarter on GitHub
• smarter.common.conf.settings on GitHub
• Smarter settings reference
• Configuration
• ADR-007: Settings
• Pydantic Settings
• The Twelve-Factor App: Config
• The Smarter blog