Smarter’s @cache_results() Decorator: Persistent Function Caching for Django

Caching a function’s results sounds like a solved problem, and inside one Python process it is, because functools.lru_cache does it in one line. But a Django application doesn’t run in one process. It runs in several web workers and several Celery workers, on several servers, and every one of those processes is thrown away at the next deployment. A result cached in process memory is cached once per process, for the life of that process, and then it’s gone. Each web worker is asynchronous, too. Smarter runs on uvicorn, under ASGI, so one worker serves many requests at once, taking turns on one event loop, and anything that waits on the cache in an async request must not stop the others while it waits. And lru_cache doesn’t work for an async def function at all. It caches the coroutine rather than its result, and a coroutine can be awaited only once, so the second call raises RuntimeError. The results worth caching most, such as database reads that happen on every request and calls to slow, billable third-party APIs, are exactly the ones you want to keep for an hour, shared by every process, across restarts.

This is the problem space that @cache_results() is designed to solve, results that outlive the process, shared by every worker on every server, and by every request, sync or async.

That takes three things that a shared cache doesn’t give you on its own. The first is a cache that outlives the process, which means Redis, through Django’s cache framework, with a timeout and a way to throw away one entry when the data behind it changes. The second is a way to turn a function’s parameters into a key that is unique for every distinct set of arguments, identical for every equivalent one (f(1), f(a=1) and f(1, b=2) when b defaults to 2), and the same in every process, on every server, after every restart. In memory, Python’s own hashing does that. Across processes it can’t, because Python randomizes string hashes per process. The third is support for async code. An async def function’s result has to be cached rather than its coroutine, and an async request has to read and write the cache without holding up the other requests on its event loop.

As far as I know, neither Django nor anyone in the community has solved all three. Django’s cache framework gives you cache.get() and cache.set() and leaves the key to you, so every project writes the same six lines over and over to build a key, get, check, call, set, return. Then it builds the key again somewhere else to invalidate it, hopefully the same way. Django’s cache_page caches whole views, and cached_property caches one attribute of one object for the life of that object. The closest package I know of, Peter Bengtsson’s django-cache-memoize, stores results in Django’s cache and has an .invalidate(), but it builds its key from the str() of the raw arguments, so f(1) and f(a=1) are cached separately, and a dictionary argument’s key depends on the order its keys were inserted. It has no async support either.

Smarter‘s answer is @cache_results(), which began in Smarter and is now a standalone package on PyPI, cache-results, that depends only on Django. Its source is in src/cache_results/decorators.py. It does all three. Results live in Redis beyond the life of any process, each call’s parameters map to a unique, repeatable key, and async functions are cached without blocking the event loop. This post explains how it works, how it’s used, and the technical challenges hiding inside a decorator that looks like it should be twenty lines long.

What it’s for, and what it isn’t

@cache_results() isn’t a general-purpose way to make code faster, the way lru_cache is. It’s for one kind of result, one that is expensive, long-lived and worth sharing, such as a database read that happens on every request, or a slow, billable call to a third-party API, kept for minutes or hours by every process on every server.

That’s because every call pays for the cache, hit or miss. Binding, sorting, JSON and SHA-256 take microseconds, and then comes a round trip over the network to Redis, which takes far longer. lru_cache answers from the process’s own memory, with no network at all. So for something cheap or short-lived, such as a value that takes microseconds to compute or is needed only for the rest of one request, @cache_results() can cost more than the work it saves, and lru_cache, or no cache at all, is the better choice. The module’s docstring says the same.

What it looks like

# pip install cache-results
from cache_results import cache_results

@cache_results(timeout=600)
def expensive_function(x, y):
    # an ORM query, an HTTP request, a slow computation ...
    return x + y

expensive_function(1, 2)             # miss: runs the function, caches 3 for 600 seconds
expensive_function(1, 2)             # hit: returns 3 from Redis
expensive_function(x=1, y=2)         # hit: the same call, written differently
expensive_function.invalidate(1, 2)  # deletes that one entry

It takes three arguments.

The decorated function keeps its name, docstring and signature (it’s wrapped with functools.wraps), and gains one attribute, .invalidate(*args, **kwargs), which takes exactly the arguments you’d pass to the function.

How it works

On every call, the wrapper does four things. It builds a key, looks it up, calls the function on a miss, and stores the result. Each step has a decision in it.

Step 1 builds a key from the call, not from how it was written

These three calls mean the same thing, and should share one cache entry.

def f(a, b=2): ...

f(1)
f(1, b=2)
f(a=1, b=2)

Most memoizers key on the raw args and kwargs, so they give three keys, and f.invalidate(1) clears only one of them. @cache_results() binds the arguments to the function’s signature first, with inspect.signature(func).bind() and apply_defaults(), so every way of writing the call becomes the same dictionary of parameter names and values.

bound = inspect.signature(func).bind(*args, **kwargs)
bound.apply_defaults()
args, kwargs = (), dict(bound.arguments)

Then it makes that dictionary deterministic. Dictionaries stay JSON objects and are serialized with sorted keys, lists stay arrays, and sets are sorted, because their order changes from one process to the next. A class, such as a classmethod’s cls, becomes its module and qualified name. The function’s qualified name and its arguments are serialized to compact JSON, hashed with SHA-256, and the first 32 hex characters are appended to the module and qualified function name.

return f"{func.__module__}.{func.__qualname__}()_" + hashlib.sha256(key_data).hexdigest()[:32]

A real key looks like smarter.apps.account.models.budget.get_resource_lock_message()_ followed by 32 hex characters. The readable prefix means you can find every entry for one function in Redis with a pattern scan. The hash means the key has a fixed length whatever the arguments are, and holds no argument values in plain text.

A function that takes no meaningful arguments, or that is a singleton, can skip all of this with a fixed key, as get_cached_default_account() does in smarter/smarter/apps/account/utils.py.

@cache_results(cache_key="smarter.apps.account.utils.get_cached_default_account")

Step 2 looks it up, and tells a miss from a None

cached_result = lazy_cache.get(computed_cache_key, CACHE_MISS_SENTINEL)
if cached_result is not CACHE_MISS_SENTINEL:
    # cache hit, hooray!

cache.get(key) returns None for a key that isn’t there. If your function legitimately returns None, and lots of lookups do (“no such object”, “not locked”, “no override”), a naive decorator can’t tell “I cached None” from “I never cached anything”, so it calls the function every time. The cache becomes a cost with no benefit, for exactly the answers that are most common.

@cache_results() passes a private sentinel object as the default, so a miss comes back as that object and nothing else can. On the way in, a None result is stored as a sentinel string, and turned back into None on the way out. (More on why it’s a string below.)

Steps 3 and 4 call, store and return

On a miss, the wrapper calls the function, stores the result with the timeout, and returns it. The value is pickled by django-redis, so anything that pickles can be cached, including model instances, lists of them, dictionaries, Pydantic models.

Before it calls the function, the wrapper takes a lock for that one key and looks in the cache again. Threads in one process that miss the same key at the same moment then compute it once, rather than all at once. (Processes on other servers don’t wait for each other.) An async def function gets its own wrapper, which awaits the function and caches its result rather than the coroutine, and reads and writes Redis without blocking the event loop.

invalidate() computes the same key the same way

.invalidate() runs the same key builder on its arguments, so f.invalidate(1) clears the entry made by f(a=1, b=2). It reads the entry once, deletes it if it’s there, and logs which kind of value it removed. It reads once rather than calling has_key() and then get(), because an entry that expires between those two calls would be reported as an invalidated None.

Logging, off unless you ask

Applying one manifest can invalidate a dozen keys, and a busy page can make hundreds of cache hits, so cache logging is off by default. It’s turned on per function with logging_enabled=True, or everywhere with the CACHE_LOGGING waffle switch, without a deployment. Hits log in green, misses in red and invalidations in gray, each with the function, the key and the type of the result, so you can watch a page’s cache behavior in the dev server’s output. The result itself is never logged, and the arguments only with verbose logging.

How it’s used

The uses fall into five patterns.

ORM lookups keyed by a plain id

The most common use is a database read that happens on most requests. In smarter/smarter/apps/llmclient/caching.py for example, each lookup comes in three parts. There’s a private cached function that takes an integer id, a public function that takes the model instance, and a public invalidation function.

@cache_results()
def _get_cached_llmclients_owned_by_user_profile(user_profile_id: int) -> models.QuerySet[LLMClient]:
    user_profile = UserProfile.objects.get(id=user_profile_id)  # type: ignore
    retval = LLMClient.objects.owned_by(user_profile.user)  # type: ignore
    return retval


def get_cached_llmclients_owned_by_user_profile(user_profile: UserProfile) -> models.QuerySet[LLMClient]:
    return _get_cached_llmclients_owned_by_user_profile(user_profile.id)  # type: ignore


def invalidate_cached_llmclients_owned_by_user_profile(user_profile: UserProfile) -> None:
    _get_cached_llmclients_owned_by_user_profile.invalidate(user_profile.id)  # type: ignore

Why split one lookup into a private function and two public ones? Because the private function is the one the decorator sees, and its parameters are its cache key. Giving it a primitive, an integer id, rather than a UserProfile instance, makes everything the decorator does with that argument simple. An integer serializes to itself, so the key is exact, tiny, and the same in every process and every release. A model instance has none of those properties. It’s a large object full of database state, and the key builder refuses it, so the call runs uncached and logs an error. Its str() wouldn’t do as a key either, because it may not be unique and changes when someone edits the model. The id is also all that invalidation needs, so anything that knows it, such as a signal receiver handed the saved row, can clear the entry.

The public functions keep callers from having to think about any of this. Callers pass the object they have, and the public function hands the cache only its id. The MCPClient app’s caching.py has the same three functions for MCPClients.

Hot-path checks whose usual answer is None

Every billable request asks whether a budget has locked the resource. In smarter/smarter/apps/account/models/budget.py, that’s one cached function.

@cache_results(timeout=LOCK_CACHE_TIMEOUT)
def get_resource_lock_message(resource_locator: str) -> Optional[str]:
    """
    The message of a resource's unexpired lock, or None if it is not locked.

    Cached, because it is checked on every request. A lock invalidates it when it is created or removed.
    """

Almost every answer is None. Without the None sentinel, this cache would never hit. And a signal receiver in receivers.py clears the entry the moment a lock appears or disappears, so a 60-second timeout doesn’t mean 60 seconds of spending past the budget.

@receiver(post_save, sender=ResourceLock)
@receiver(post_delete, sender=ResourceLock)
def resource_lock_changed(sender: ResourceLock, instance: ResourceLock, **kwargs):
    get_resource_lock_message.invalidate(instance.resource_locator)

Generic lookups on a base class

MetaDataWithOwnershipModel, the base class of most of Smarter’s resource models, has a get_cached_object() classmethod that every resource inherits. It defines its cached functions inside the method, so that they can use cls.

@cache_results(cls.cache_expiration)
def _get_object_by_pk(pk: int, class_name: str = cls.__name__) -> "MetaDataWithOwnershipModel":

The class_name parameter is never read inside the function. It’s there only to be part of the key, for a reason that’s in the challenges below.

Third-party APIs, with a TTL from the manifest

An ImageSearchPlugin’s manifest says how long its results may be cached (cacheTtl). ImageSearchPlugin.brave_images() in image_search.py defines its cached function inside the method, with that plugin’s TTL, and with every query parameter that’s sent to Brave as its own argument.

@cache_results(timeout=plugin_data.cache_ttl)
def get_brave_images(
    q: str,
    count: int,
    country: Optional[str] = None,
    search_lang: Optional[str] = None,
    safesearch: Optional[str] = None,
    spellcheck: Optional[str] = None,
) -> list[ImageResult]:

It caches the HTTP request, not the plugin’s whole answer. The plugin’s domain filters run afterwards, on every call, against the cached raw result, so a change to a filter applies at once. A failed request raises inside the cached function, and the exception is caught outside it, so a failure is never stored. A cacheTtl of 0 stores nothing, so it needs no special case.

Two TTLs that remember failures for less time than successes

Connecting to a remote MCP server is slow, and when the server is down, retrying on every request is slower. smarter/smarter/apps/mcpclient/caching.py nests two cached functions with different timeouts.

@cache_results(timeout=min(FAILURE_CACHE_TTL, mcpclient.cache_ttl))
def get_cached_connection_result(mcpclient_id: int, fingerprint: str) -> SmarterDictType:
    try:
        return {"catalog": connect(mcpclient).to_dict()}
    except SmarterMCPClientConnectionError as e:
        return {"error": e.message}

@cache_results(timeout=mcpclient.cache_ttl)
def get_cached_catalog_data(mcpclient_id: int, fingerprint: str) -> SmarterDictType:
    result = get_cached_connection_result(mcpclient_id, fingerprint)
    if "error" in result:
        # raise rather than return so that a failure is cached for FAILURE_CACHE_TTL seconds only.
        raise SmarterMCPClientConnectionError(result["error"])
    return result["catalog"]

The inner function caches the outcome, success or error, for at most 60 seconds. The outer one caches only successes, for the MCPClient’s own TTL. The fingerprint argument is a hash of the MCPClient’s connection fields, so editing the server’s URL or headers produces a new key, and the old catalog is simply never asked for again.

The technical challenges

Here’s where a twenty-line decorator stops being twenty lines. Most of the challenges belong to one of the first two problems, outliving the process or mapping parameters to a key.

Outliving the process

A result that’s stored outside the process has to get there through the right cache, survive serialization, and stop being served when the data behind it changes.

The cache has to be the real one

Decorators run when their module is imported. In a Django project, modules are imported while Django itself is still starting, while settings load, while apps are being registered, before apps.ready is true. A caching decorator that grabs django.core.cache at that moment, or a module that imports it at the top and then touches it, is reaching for the cache before Django has finished deciding what the cache is.

What happens then is that Django can fall back to a default, per-process, in-memory cache rather than the Redis cache in CACHES, unexpectedly and silently. Users’ browser session values aren’t stored, and cached results last far less than their timeouts. Nothing raises an exception. A cache that forgets things looks, from the outside, exactly like a cache that’s working and just missing a lot.

So the decorator never touches Django’s cache directly. It goes through lazy_cache, in cache_results/lazy.py, an object with the same API as django.core.cache.cache that does nothing at all until the first get() or set(). Then it does the following.

• checks that Django’s apps are ready, and logs a warning naming the problem if they aren’t,
• imports django.core.cache for the first time,
• writes a test value and reads it back, and logs an error if it doesn’t come back,
• checks that the default cache really is django_redis.cache.RedisCache, and logs a warning if it’s something else.

The decorator adds one more safeguard. A decorated function that’s called before Django’s apps are ready, during startup, simply runs uncached rather than reach for the cache too early.

The rule that keeps this true across a large codebase is short. Outside the tests, only lazy_cache and the reset_cache management command import django.core.cache. Everything else goes through lazy_cache or @cache_results(). The same lazy object also defers importing django-waffle, which caches its own state aggressively and has the same import-order trap.

A cache that outlives the process is only as good as the guarantee that it’s the cache you configured.

A sentinel has to survive the trip through Redis

The miss sentinel can be an object(), because it never leaves the process. It’s only the default for get(). The None sentinel is different. It’s written to Redis, pickled, and unpickled in some other process, and an unpickled object() is a new object, so an identity check against the original would fail. So the None sentinel is a string, and it’s compared by value.

Results have to pickle, and QuerySets pickle by running

Django’s ORM, for all it’s strengths, presents some unique caching challenges. A result that can’t be pickled (an open connection, a generator, a lambda) can’t be cached. QuerySets can, but pickling a QuerySet evaluates it, so what’s cached is the rows as they were at that moment. That’s what you want from a cache, but chaining .filter() onto a cached QuerySet builds a new query and goes back to the database, so you need to ensure that you cache the final query, not the start of one.

Pickle also favors plain data. A dictionary of strings and numbers pickles to about half the size of an object with the same fields, and any release can read it back. A pickled object, a Django model instance and a Pydantic model alike, can only be unpickled while its class still exists at the same import path, so a release that renames or moves the class breaks the entries cached before it. And Smarter’s cache is meant to outlive deployments.

Because Smarter models its structured data with Pydantic, a cached function can store plain JSON (model_dump(mode="json")) and rebuild the model when it reads it, which keeps cache entries small and readable by any release. The dashboard’s token usage chart does this in charges.py, where it caches the JSON of a TokenUsageSeries model, which the API then returns as it is.

@cache_results(timeout=TOKEN_USAGE_CACHE_TIMEOUT)
def token_usage_series(resource_locator: str, periodicity: str) -> dict:
    return build_token_usage_series(resource_locator, periodicity).model_dump(mode="json")

Invalidation has to reach every way the data changes

A timeout bounds how stale a result can get. Invalidation is what makes it correct sooner. Smarter hangs invalidation on model signals (post_save, post_delete) in the app’s receivers.py, as with the budget lock above. But not every change sends a signal. QuerySet.update() and raw SQL don’t, so code that uses them invalidates by hand. A cache entry with no invalidation is a promise that the timeout is short enough.

Mapping parameters to a unique key

A key has to be unique for every distinct call, identical for every equivalent one, and the same everywhere.

Keys have to be the same in every process, forever

An in-process cache can key on anything hashable. A shared cache can’t, because Python’s hash() of a string is randomized per process (PYTHONHASHSEED), so two workers compute different hashes for the same arguments. repr() of most objects includes a memory address. Dictionary order is insertion order, so {"a": 1, "b": 2} and {"b": 2, "a": 1} serialize differently. And a set has no order you can count on at all. That’s why the key goes through sorting, then canonical JSON (sort_keys=True, compact separators), then SHA-256, which is the same on every machine and every Python version. SHA-256 truncated to 128 bits can in theory collide; in practice, it won’t.

Equivalent calls have to produce one key

Without signature binding, the function and its invalidate() can be called in different styles and silently compute different keys. The invalidation “succeeds”, logs that there was nothing to invalidate, and the stale entry lives on until it expires. That’s a bug that no test catches unless it calls the two in different styles on purpose, which is why the binding step exists and why there are tests that do exactly that, in tests/test_decorators.py.

A function’s name isn’t unique

The key starts with func.__module__ and func.__qualname__, the qualified name, so two functions called get in two classes of one module get different keys. But a qualified name says where a function is defined, not who calls it. For a function defined inside a classmethod on a base class, that’s the base class, so _get_object_by_pk is MetaDataWithOwnershipModel.get_cached_object.<locals>._get_object_by_pk whether cls is an LLMClient, a Plugin or a Secret. Without something more in the key, LLMClient 5 and Plugin 5 would share a cache entry, and someone would get the wrong object back. 😱 That’s what the otherwise unused class_name: str = cls.__name__ argument is for. It looks odd, because the function never uses it, and pylint reports it as an unused argument, so a # pylint: disable=W0613 comment above the function says that it’s deliberate.

Closures aren’t part of the key

The same nesting has a second trap. Anything the cached function reads from its enclosing scope, rather than from its parameters, is invisible to the key. In Smarter, the rule is that the decorated function’s parameters must be, one by one, everything that can change its result, meaning every query parameter, filter and flag. That’s why get_brave_images() takes six explicit arguments rather than one search_params dictionary, and why the MCP functions take a fingerprint. The signature is the cache key.

Arguments have to be values

Datetimes, Decimals and UUIDs are written out in a standard form, and so are sets, classes and dataclasses. Any other object that JSON can’t represent is refused. The call runs uncached, and the decorator logs an error that names the function and the argument’s type, so that those calls can be found and fixed. It never falls back to str(). A model instance’s __str__ may not be unique and may change when someone edits the model, and an object without its own __str__ gets a new one, with a memory address in it, in every process. A project can teach the key builder its own types with a KEY_ENCODER in the CACHE_RESULTS setting; Smarter’s accepts Pydantic models. Otherwise, arguments are plain values such as strings, integers, booleans, tuples. That’s also why @cache_results() refuses an ordinary instance method, with a TypeError as soon as its module is imported, because self would be part of the key, and an object has no safe key, so every call would run uncached. The pattern is a module-level function that takes the id, called from the method. A classmethod is fine, since its cls goes into the key as the class’s module and qualified name.

Arguments end up in logs

With verbose cache logging on, every hit and miss logs its arguments. A secret passed as an argument would end up in the logs, and (hashed) in Redis. So secrets are never arguments. The cached function takes a Secret’s id, or nothing, and reads the key inside. For the same reason, a generic HTTP transport whose headers carry credentials is never decorated.

Other challenges

Some things must never be cached

Smarter’s validate_public_url() resolves DNS before every outbound request, to refuse requests to private addresses. Caching it would widen the DNS rebinding window from milliseconds to the cache lifetime. A security check that’s cached is a security check that’s skipped.

A checklist for your own functions

• Cache the expensive call itself (the query, the HTTP request), not a wrapper around a whole workflow.
• Make every input that can change the result an explicit parameter, and make each one a plain value.
• Take an id, not a model instance, and never self.
• Never pass a secret as an argument.
• Raise inside the cached function on failure, and catch outside it, so failures aren’t stored.
• Give failures a shorter TTL than successes with two nested functions, if retrying is expensive.
• Put a version or fingerprint in the arguments when a configuration change should start a fresh cache.
• Wire .invalidate() to the model signals of everything the result depends on, and invalidate by hand after QuerySet.update().
• Add a class name to the arguments of any cached function defined inside a method that subclasses share.
• Don’t cache security checks.

Closing thoughts

None of the individual techniques here is new. Sentinels, canonical JSON, signature binding and signal-based invalidation have all been around for years. What’s unusual is putting all of them in one decorator, so that a contributor writes @cache_results() and gets a result that outlives the process, under a key that means exactly one call, every time. That’s why it’s now its own package, cache-results, on PyPI, so the rest of the Django community can use it too, with pip install cache-results. Its documentation is at docs.cache-results.smarter.sh, and the v0.1 release announcement has the details. ✨

Happy Smartering!

• Smarter on GitHub
• Caching in Smarter
• cache-results on PyPI
• cache-results documentation
• ADR-018: Caching
• Django’s cache framework
• django-cache-memoize
• The Smarter blog