Part 1: Maintaining Long-Term Code Quality With Agentic AI

This is part 1 of Converge or Diverge, a three-part series about AI coding agents in a large codebase: Part 1: Maintaining Long-Term Code Quality With Agentic AI (you are here) · Part 2: The Antipattern That Makes a Codebase Diverge · Part 3: What Claude Code Actually Reads

Updated October 17, 2026: the timeline now runs through v0.33.

(TLDR? Click here to jump directly to the 5 success factors.)

Looking back, I now recognize that the brutal unvarnished truth is that an AI coding agent working in your codebase is guaranteed to take it in one of two directions, and there’s very little middle ground. Every change an agent makes becomes part of what it reads the next time. The effect is compounding, which is the reason I’m compelled to write this article. It learns your conventions by example and copies what it finds, and the code that it writes today is an example that it will copy tomorrow. That feedback loop is the whole story. What matters is what that iterative process does with your mistakes.

In a healthy codebase, the series is telescopic. In a telescoping sum, each new term cancels most of the one before it, and the total collapses to something bounded that depends only on where you started and where you’re going. An agent’s change is checked against tests, framed by types, shaped by an existing pattern and described by documentation, so its errors cancel out before they spread. Each change resolves against what’s already there and leaves nothing loose for the next one to copy. The codebase converges on the quality of its foundation.

Contrastingly, in an unhealthy codebase, the series is geometric, with a ratio greater than one. Each term is the previous one multiplied. The agent copies an inconsistency, then copies its own copy. Nothing catches it, because there’s no test that says what “correct” means and no pattern that says what “normal” looks like. The agent also invents whatever is missing, and its inventions become the examples for its next inventions. At human speed, this kind of decay takes years and you can see it coming. At agent speed it takes weeks, and it compounds. It ends in unbounded chaos: a codebase nobody understands, maintained by an agent that understands it least of all. That’s cataclysmic, it’s apocalyptic, and it will likely end in an involuntary career change on your part.

You don’t choose the ratio when you start the agent. You chose it over the years before that, with every test, type annotation, docstring, linter rule and architecture decision. This is the story about the telescopic outcome that Smarter experienced.

18 days in October …

In early October 2026, Claude Code was given the rest of Smarter’s 2026 feature road map. It finished in 8 days, over about 20 prompts, each of which took between one and four hours to complete. For every feature, Claude designed the change, wrote the code, wrote the unit tests and wrote the Sphinx documentation. Additionally, it wrote the release notes, and it even drafted the blog posts that announced each new minor release. But for the avoidance of any doubt, the author DID write this post, and hopefully you can see the difference ✨.

The road map took six minor releases, v0.15 through v0.20, between September 30 and October 10. Then Claude Code kept going. Over the next seven days, thirteen more minor releases, v0.21 through v0.33, went out: nineteen in eighteen days, plus 45 patch releases along the way. Each one has its own announcement on the Smarter blog, linked from the version numbers below.

The two stretches were different kinds of work. The first eight days were the generative AI road map: new resource kinds, each one a feature a user asks for. The next seven were the work that turns a platform into something other software can build on: a published contract for the API, an SDK generated from it, an extension system, rate limits, a translated console, and the fixes from a real production deploy. That kind of work touches everything, and it usually goes slower. It didn’t. And on the Sunday morning in between, Claude Code spent a few hours cleaning up the loose patterns that the first eight days had multiplied, which is the subject of part 2.

That is an astonishing result, and it would be easy to tell this as a story about code agent LLMs, though that isn’t the case. Claude Code did what it did atop the shoulders of three and a half years of human toil of a most unglamorous variety. That foundation is what made the eighteen days possible, and importantly, that’s the part of this story that you can copy.

The codebase that Claude built

Smarter is not a toy project. It is a large Django platform for building and running LLM applications at scale. It is used as an instructional tool for AI courses at the University of British Columbia, and accordingly it gets put through its paces by students who, by their nature, push the edge cases to breaking point on a routine basis. Its resources (LLMClients, Plugins, Guardrails, Secrets, Custom Domains and so on) are declared in YAML manifests, applied through a CLI or a REST API, and managed in a web console written mostly in React. It runs in Docker locally and on Kubernetes in production.

On October 10, around a fourth of the production code had been written by Claude Code. A week later it’s about a third. That merits a bit of qualifying, because which part it wrote matters a lot more than how much code it wrote.

Who built what?

Broadly speaking, Smarter has two layers, and they were built in very different ways.

The subsystems are the framework that everything else stands on: the manifest and broker machinery, the API’s response contract, multi-tenancy and ownership, authentication, encrypted secrets, settings, middleware, logging, caching and the test base classes. They live in smarter.common, smarter.lib, smarter.apps.account and smarter.apps.secret. They were designed and written by humans, mainly by the author, over a period of years. GitHub Copilot helped with their docstrings and unit tests, but the design, the code and the hard decisions themselves are undeniably the work of real human beings.

On the other hand, the applications are the resource families a Smarter user actually works with: Guardrails, MCPClients, LLMHosts, Proxies, VectorStores, Custom Domains, the new plugin types, the infrastructure service layer, and most of the React web console. And these exist in large part thanks to Claude Code.

git blame makes the split visible:

Two qualifications. First, Claude Code’s share of the subsystems more than doubled in a week. On October 10 its few percent was almost entirely bug fixes and test-driven corrections from the October coverage push, made inside a design it didn’t change. The second week added something new: the agent began extending the subsystems themselves, inside their existing design. The Pydantic models behind the journal’s API contract, the API rate limiter, the database character set conversion and a typing cleanup across 277 files all landed in smarter.lib and smarter.common. Second, the application numbers are a floor rather than a ceiling, because some of Claude Code’s work was committed under the author’s name, and the blame unfairly attributes it to him. 😬

But there’s a third, quieter qualification as well. The new applications weren’t conceived by the agent. In July 2026, the author scaffolded the Guardrail, MCPClient, LLMHost, SkillPlugin, Vectorsearch and Orchestrator apps by hand: the app, its place in the platform, and the outline of what it would become. Claude Code filled in the scaffolds. Humans decided what Smarter would be. The agent decided how each piece would work, within boundaries that humans had drawn. Importantly, this laid out the guardrails for future features that Claude Code was eventually able to shepherd with more complete autonomy.

Now then, at the risk of sounding thoroughly ultracrepidarian, there’s a strong case for delegating design work of this nature to an agent like Claude Code. Agents were trained on exponentially more real-world information and code than what any one human could possibly gain through personal experience. Agents objectively have a clearer understanding of how the rest of the world has already implemented X feature, and why. It therefore behooves you as a project owner to leverage the full benefits of the knowledge and ‘lived experience’ that an agent brings to bear on this aspect of your project.

What “subsystem” means here

It would be easy to read “framework code” as boilerplate. It isn’t. Some of Smarter’s subsystems are extremely non-trivial, and they took years to get right.

The Smarter Application Manifest (SAM) and the broker model. Every Smarter resource is declared in a YAML manifest that deliberately resembles a Kubernetes manifest: apiVersion, kind, metadata, spec, status. Two architecture decision records make that the law. ADR-005 says every resource must have a manifest and that all create, read, update and delete operations go through manifests, never through ad hoc HTTP verbs. ADR-024 says resources are managed through a broker model that follows kubectl as closely as possible: apply, get, describe, delete, deploy, undeploy, logs, example, schema.

Behind those two paragraphs is a lot of machinery. SAMLoader parses and validates manifests, with about 65 commits of history going back to May 2024. Pydantic models give every manifest a typed schema. AbstractBroker is about 2,400 lines that implement the shared behavior of every resource: resolving the account and user, enforcing ownership, journaling the response, mapping errors to HTTP status codes, and refusing to delete a resource that others depend on. It has more than 200 commits in its history, nearly all of them by the author. It has been rewritten, generalized, broken and fixed many times, and every rewrite left behind tests.

When Claude Code built a new resource kind, it didn’t design any of that. It wrote a Pydantic manifest model, a subclass of AbstractBroker, a Django model that inherits the ownership model, and a React list page. The hardest problems had already been solved, once, by a human, in a way that every resource shares.

The journal and the API contract. Every API response in Smarter goes through smarter.lib.journal, which gives success and error responses the same shape and records them. This is why a new resource’s API looks exactly like an old one’s, and why the CLI and the web console can work with resources that didn’t exist when they were written.

Multi-tenancy and ownership. smarter.apps.account holds accounts, user profiles, charges, LLM prices, Stripe billing, and MetaDataWithOwnershipModel, the roughly 1,000-line base model that decides who owns a resource, who can see it, who can change it, and how names are made unique when a resource is cloned. Every resource Claude Code added inherits it. Not one of them had to think about tenancy.

Secrets. smarter.apps.secret stores encrypted credentials that other resources refer to by name. When Claude Code built the Proxy, which forwards requests to LLM provider APIs, the provider API keys became Smarter Secrets with one reference in a manifest. The hard part, keeping keys encrypted, owned and out of developers’ hands, was already done.

Settings, middleware and the rest. smarter.common.conf is a typed settings layer of more than 4,000 lines and more than 230 commits, which validates every environment variable the platform reads. smarter.lib.django holds the middleware (CORS, CSRF, sensitive-file blocking, excessive-404 blocking), the multi-tenant authentication backend, the request mixin that every view uses, and feature flags. smarter.lib.drf holds API token authentication. smarter.lib.unittest holds the test base classes.

Why this split is the point

This is the telescope. Each new application is a small difference term: the part of the feature that is genuinely new. Everything else, tenancy, permissions, the API contract, error handling, secrets, settings, is absorbed by subsystems that already exist and are already tested. The agent writes the difference. The foundation cancels the rest.

Run the same exercise without the subsystems, and every new resource kind invents its own permission checks, its own error format, its own way of storing a key. The agent’s next resource copies the previous one’s inventions, flaws included. That’s the geometric series. The feature list looks the same at the end of the week. The codebase doesn’t.

So the meaningful differentiator is less about “how smart is the agent’s model?” than “what does the agent find when it goes looking?” An LLM working in a codebase of this size can’t hold it in its head, so it only reads the parts that matter for the task at hand, and the quality of what it reads decides the quality of what it writes.

In Smarter’s case, there were four answers to that question, plus a fifth that’s easy to overlook. Each one is a way of keeping the series telescopic.

Success factor #1: unit tests

This was a glacially slow process for much of the life of the project. Smarter’s test base classes evolved artisanally over a number of years. SmarterTestBase, the account fixtures, the manifest broker test harness and the resource view mixins all exist because someone needed them, wrote them, and then wrote them again when the first version turned out to be wrong. From around 2024, GitHub Copilot took over most of the actual test writing duties, eventually achieving a coverage ratio of around 75% which, while minimally acceptable, is hardly sufficient to allow a coding agent to run rough shod through the code base. So, Claude Code took over in early 2026, running sporadically over the course of the year in strategically targeted bug bounty campaigns, ultimately achieving a coverage ratio of around 97% across both Python and React, and growing the test bank fivefold to more than 6,500 tests.

My thoughts have evolved about how much testing is “enough”. Traditionally, people who I respect in the Django community asserted that a coverage ratio in the 80 percentile range was fine given that much of the code is boiler plate. But that was life before agents, which a.) offer the highest potential when operating autonomously, and b.) are dirt cheap compared to human Django programmers. Nowadays, I think more is better. 🤩

Claude Code to a large extent bootstrapped the testing bank that it relies on today. The very unit tests that GitHub CoPilot and Claude Code wrote over the previous three years provide the immediate feedback on which agents most depend in order to work and plan autonomously.

Why do tests matter more to an agent than to you?

Without tests, an agent has to imagine what a correct outcome looks like. It reads the code, forms a theory of what it’s supposed to do, and writes something consistent with that theory. If the theory is wrong, nothing tells it so, and the wrong theory becomes an example for the next change. That’s a geometric term.

Tests make “correct” concrete. A test that fails tells the model exactly what it got wrong, and where, and why. A test that passes tells it that it can stop. The more granular, idempotent and axiomatic your tests are, and the more of your code they cover, the more exactly and the more quickly an LLM understands what “correct” means within the broader context of your project. Tests are the cancellation in the telescope.

but there’s a second benefit that’s easy to miss: tests are executable examples. They’re code samples for the benefit of the agent during its design phases. When Claude Code adds a new resource kind, the test modules for Guardrails and Secrets show it how a resource is created, applied, described and deleted, which fixtures exist, and what has to be cleaned up afterwards. It doesn’t have to guess.

Coverage is also a bug-finding machine

It gets better. The most surprising result of the eight days was how many bugs the coverage campaigns surfaced. It was humbling. Sheesh, it was downright embarrassing! Raising coverage means writing tests for branches nobody has exercised, and branches nobody has exercised are where bugs live. The commit history from those days reads like a bug hunt:

• 23 bugs found while raising coverage of the framework code in smarter.common and smarter.lib to 90%
• 13 account and LLMClient bugs found while raising coverage of the account app to 91%
• 9 API and CLI bugs found while raising coverage of the API app to 93%
• 71 tests that had been marked as expected failures, each one a known bug that nobody had fixed; Claude Code fixed all of them, in batches of 43, 18 and 10
• Smaller batches of five, five and four along the way

That’s roughly 130 bugs bountied in a single week of work! 😱 Some were cosmetic. Several were not. Among the 43 expected-failure bugs, for example:

• An account admin could activate, deactivate or change any account’s API key through a JSON endpoint that didn’t check ownership.
• Any authenticated user could read any account’s LLMClient configuration, because a view loaded the LLMClient by id without checking read permission.
• A middleware meant to block clients that probe for URLs never blocked anyone, because it discarded the 403 response it computed.
• An entire test class never ran, because every test function in it had been dedented out of the class body.

Notice where many of these were: in the human-written subsystems. Years of human work produced a strong foundation, but not a flawless one. The coverage push made the foundation stronger BEFORE the agent built more on top of it, which is exactly the right order. In theory, a human could find these bugs too. But what’s different, and what probably matters most, is that an agent will work through a coverage report line by line, for hours, without getting bored, and every uncovered branch is a question it actually asks: what is this supposed to do, and does it?

Tests touch real billable services and infrastructure

Smarter talks to a lot of LLM providers. One fix for example made the dashboard’s prompt passthrough work across providers, and the tests Claude Code wrote for it send every request template to every active provider, live. Not entirely surprisingly, it found things like one provider’s configured base URL pointing at the wrong API, and another provider’s default model being a faulty copy of some other vendor’s, and that a third returned its model list in a different JSON shape than the code expected. None of that would show up in a mocked test. Smarter’s policy is that these billable test calls are acceptable, and the agent receives instructions accordingly.

Testing from the outside

In the second week, the tests changed point of view. Claude Code wrote a Python SDK and then a Node.js SDK against the platform’s API, as an outside developer would, and the SDK found the bugs that the platform’s own tests had walked past. GET /api/v1/providers/ answered 500. The CLI’s get ignored ?name= for every kind, so the SDK had been filtering by name on the client. Describing a resource that didn’t exist answered 503 instead of 404. And apply returned the broker’s internal state, including a masked API key, the authorization header and the caller’s IP address. All of it is fixed in one commit, with a test that sweeps every resource kind.

The API contract made that permanent. Real responses of every documented operation, errors included, are now validated against the OpenAPI document, and a test walks every route under /api/v1. A new route can’t ship undocumented. A client is a test that reads your API the way a stranger does.

Success factor #2: documentation

Smarter’s Python docstrings, rendered by Sphinx into the technical reference at docs.smarter.sh, were written by GitHub Copilot and curated by the author over a period of about five months in 2025 and 2026, module by module and function by function. Later, the author wrote the free-form narrative pages that give the docs site its overall structure: what each resource is for, how the pieces fit together, how to deploy and operate the platform. Also, there are thirty architecture decision records that explain why the platform is built the way it is — and mind you, agents are attentive to these kinds of details. That is, these are different kinds of documentation, and an agent like Claude Code uses them all.

Docstrings answer local questions. What does this function accept? What does it return? What does it raise? In Smarter’s case they almost always include a code sample, references to where the function is used, and links to related technical documentation. This is tedious work, and it’s the principal reason we’ve leaned on Copilot and other LLMs from the outset. When Claude Code calls a function that it didn’t write, the docstring becomes the contract, because it’s faster and more reliable than reading and understanding the implementation. Plus, it tells the model what the function is supposed to do, which is not always what it does.

Meanwhile, narrative documentation answers the questions that code can’t. Why is a resource declared in a manifest instead of created through a form? Which concerns belong in a broker and which in a model? What does the project value, and what does it refuse to do? This is where qualitative ideas like best practices, strategic vision, values and opinions live, along with the structure and how-to for designing and scaffolding new features. The more complete and descriptive it is, the better it works as a guiding light for an LLM designing a feature that fits your project, rather than a feature that merely works.

The architecture decision records are the strongest form of this. “All CRUD operations must be implemented using YAML manifests” is a sentence that keeps an agent from inventing a second way to do things, and a second way to do things is exactly how a geometric series starts. Think of ADR’s as the individual articles of the “constitution” for your project.

Documentation also becomes part of the output. Every feature in the last eight days shipped with its own documentation page and docstrings, because the existing pages showed what a feature’s documentation looks like. Smarter keeps its docs evergreen: new features are woven into the existing prose rather than appended as “what’s new” sections. That’s a rule Claude Code was taught once and follows every time. An example agent-generated Python docstring, for context:

def cache_results(timeout=smarter_settings.cache_expiration, cache_key: Optional[str] = None, logging_enabled=False):
    """
    A decorator that caches the result of a function based on the arguments passed to it.

    .. important::

        This decorator is intended for expensive and long-lasting, persistable
        caching scenarios such as Django ORM reads, where the results of the
        decorated function should endure application restarts and deployments.
        This caching decorator itself is using Python function caching in order
        to generate cache keys, which is to say that using this decorator could
        be counter-productive for caching scenarios that are better served by
        in-memory caching of short-lived results.

    When the decorated function is called, the decorator first checks if a cached
    result exists for the given arguments. If a cached result is found, it is
    returned immediately. If not, the original function is called, its result
    is cached, and then returned. Smarter's cache infrastructure is based on
    Redis and runs as a remote service that services application restarts,
    deployments, and, it natively services multiple application server instances.

    .. note::

        *One of the challenges with implementing a caching decorator based on Django cache
        regards working around Django's application startup sequence.
        Decorators are imported and applied at module load time,
        which often results in Django's cache
        framework being prematurely imported and initialized while Django itself is still
        running its own application startup process.*

        *This often leads to situations where Django falls back to an
        alternative 'default' memory-based cache backend unexpectedly
        (and silently). When this happens, the fallback cache most likely
        will not persist data as expected, leading to buggy cache misses
        such as users' browser session values not being stored, and cached
        results of this decorator enduring less than specified.*

    **How It works:**

    A cache key is created by building a string of the module name + the function name,
    and then appending a 32-character hash of its serialized positional arguments and sorted keyword pairs.
    This ensures that each unique set of arguments maps to a unique but repeatable cache key.
    Technically speaking, there is a statistical non-zero probability of hash collisions, but,
    the risk of this happening is *EXTREMELY* low.

    :param timeout: The cache timeout in seconds. Defaults to ``smarter_settings.cache_expiration``.
    :type timeout: int
    :param cache_key: An optional pre-computed cache key to use instead of
        generating one from the function arguments. This marginally increases
        performance by skipping the cache key generation step, but should only
        be used if you are certain that the provided cache key is unique and
        correctly represents the function arguments. Defaults to ``None``.
    :type cache_key: Optional[str]
    :param logging_enabled: Whether to enable logging for cache hits and misses. Defaults to ``True``.
    :type logging_enabled: bool
    :return: The decorated function with caching applied.
    :rtype: Callable

    .. note::
        If the function returns ``None``, a sentinel value is cached to distinguish between a cached ``None``
        and a cache miss.

    Usage example::

        @cache_results(timeout=600)
        def expensive_function(x, y, *args, **kwargs):
            # Perform expensive computation ...
            result = "some very expensive computational result"
            return result

        expensive_function.invalidate(1, 2)  # Invalidate cache for specific arguments
    """

There’s a lot to it. Sphinx documentation is literally software in its own right. It compiles. There are syntax and semantics. It is dense and rich in information content. It’s hard to write it well. And upon inspection it’s easy to see how and why agents are able to glean so much insight and direction from meta data of this nature.

Success factor #3: high-quality source code

Beyond the obvious, efficient and bug-free, Smarter has always treated type annotations and strict type checking as a core philosophy; ADR-027 makes it a rule. Pre-commit rules added in early 2024 enforced it strictly, which produced a one-time spike in code quality across the whole repository. Other formatting, style and security linters have run since the project began. Today about two dozen are in use: black, flake8, isort, autoflake, pyupgrade, bandit, codespell and pydocstringformatter for Python; Prettier and ESLint for React; commitlint for commit messages; and a set of checks for credentials, private keys, merge conflicts, YAML, JSON and TOML. An example:

def get_readonly_yaml_file(file_path: str) -> dict[str, Any]:
    """
    Reads a YAML file from the specified path and returns its contents as a Python dictionary.

    :param file_path: The path to the YAML file to be read. This should be a string representing a valid file system path.
    :type file_path: str

    :return: The contents of the YAML file, parsed into a Python dictionary. If the file is empty or contains no valid YAML, ``None`` may be returned.
    :rtype: dict

    .. note::
        This function opens the file in read-only mode with UTF-8 encoding and uses ``yaml.safe_load`` for parsing. Only standard YAML types are supported.

    .. warning::
        If the file does not exist, is not readable, or contains invalid YAML, an exception will be raised. Always validate the file path and contents before use.

    **Example usage:**

    .. code-block:: python

        from smarter.common.utils import get_readonly_yaml_file

        config = get_readonly_yaml_file('/path/to/config.yaml')
        print(config)  # {'key': 'value', ...}

    """
    logger.debug("%s.get_readonly_yaml_file()", logger_prefix)
    with open(file_path, encoding="utf-8") as file:
        return yaml.safe_load(file)

Agents copy what they see

An agent like Claude Code learns your conventions by example. Thus, if half your views check permissions but the other half don’t, it will arbitrarily copy whichever one it happens to read first. If your error handling is inconsistent, its error handling will be too. If, if, if … you get the point. Good code, bad code or otherwise, agents copy what they see. This is the ratio of the series, stated plainly: the agent multiplies whatever it finds.

But equally horrifying, it also invents what’s missing. If a function has no type annotations, the model infers the types from how the function is used, and sometimes it infers wrong. Strong typing eliminates that guesswork. When every function says what it receives and what it returns, the model doesn’t have to reconstruct the data model from scratch before it can change anything, and that has large spillover effects on everything else it does.

Linters do the same job mechanically. A pre-commit hook that rejects a change is a cancellation that doesn’t depend on anyone’s attention.

“Build it like this one”

LLMs perform best when you can tell them, “build it like this one, which already works well.” Smarter’s architecture makes that easy to say. Every resource kind is a Pydantic manifest model plus a broker subclass. Every console list page is a React app with the same toolbar, list view, card view and API contract. Every built-in resource is loaded by a management command that follows the same pattern.

So when the road map called for a new resource kind, the instruction was effectively “look at Guardrail and Secret and do it the same way.” Claude Code read them in full, copied their structure, naming, docstrings, logging and error handling, and followed a checklist of every file a new kind touches. Custom Domains, Budgets, VectorStores, LLMHosts and Proxies were each built that way, in hours rather than weeks.

Consistency is a multiplier, in both directions. Every pattern you’ve applied the same way twice is a pattern an agent can apply a third time without supervision. Every inconsistency you’ve tolerated twice is one it will apply a third time too.

Success factor #4: good prompts, and Skills

The first three factors are properties of the codebase. The fourth is about how you talk to the agent.

Prompts

Each of the roughly 20 prompts behind these releases described one coherent unit of work: a feature, a family of bugs, or a coverage target for one subsystem. Each one said what done looks like. Each one could be verified by the agent itself, by running the tests and the linters, without a human checking intermediate steps. The agent was given broad autonomy on the design, while being hemmed in by explicit guardrails: those of a framework/library (Django, Django REST Framework, React and so on), and Smarter’s own design principles, found in its documentation, tests, code samples and Skills. A prompt that took four hours wasn’t a prompt that needed four hours of supervision.

The output of each prompt was a set of commits whose bodies explain what changed, why, how it was tested, what was left undone and what the user must do before upgrading. Those commit bodies later became the raw material for the release notes and blog posts, which Claude Code also wrote. They also became breadcrumbs for the agent itself on later tasks.

Skills

Agent Skills were added to the repository in September 2026. A skill is a folder of written know-how that the agent loads only when a task needs it: a SKILL.md file with a name and a one-line description, plus optional reference material and helper scripts. On October 10 Smarter had 15 of them, about 2,400 lines in all. A week later it has 24, about 5,800 lines with their reference files and scripts, covering:

• adding a new resource kind, end to end
• building a React list page that Django hosts
• writing Python and React tests, and the traps in each
• running code in the Docker containers
• infrastructure safety
• coding chores like database migrations, management commands, docstrings and Sphinx docs
• linting, commit messages and release announcements
• since October 10: the API contract, translations, the logging style and Python typing

A short CLAUDE.md at the root of the repository holds only the rules that apply to nearly every task, and points to smarter-development, a skill whose table says which narrower skill covers each kind of work. That short file matters more than its size suggests. In part 3 I counted what Claude Code read over these eighteen days: once CLAUDE.md existed, the share of sessions that loaded smarter-development went from about one in seven to about three in four.

Most of these skills weren’t written up front. They accumulated. Each time Claude Code made a mistake that was specific to Smarter, the fix went into a skill so that it wouldn’t happen again. docker cp nests a directory inside the target when the target already exists, and leaves files owned by root: that’s in the Docker skill. SmarterTestBase has no transaction rollback, so a test must delete everything it creates: that’s in the testing skill. Several docker cps in a row can kill the dev server: that’s in there too. In early October the accumulated skills were generalized into a single skills/ folder in the repository, where they’re versioned and reviewed like any other code.

This is the most direct form of cancellation there is. A mistake that’s written down happens once. A mistake that isn’t written down is a term in the series, waiting to be multiplied.

Skills are documentation with a different reader. Your Sphinx docs tell a human how the platform works. Your skills tell an agent how to work on it.

Success factor #5: an environment the agent can run

This one is easy to take for granted. Claude Code can only verify its own work if it can run the code.

Smarter spins up locally with make init and Docker Compose: the Django app, Celery workers, a Celery beat scheduler, MariaDB and Redis. Claude Code syncs its edits into the containers, restarts them, and runs the test suite itself after every change. It doesn’t ask a human to re-run the tests, and it reports failures honestly, including ones that were already there.

During the first eight days, Claude Code also made that environment easier for everyone. A newcomer who ran make init without AWS credentials, SMTP credentials or LLM API keys used to get a broken platform. Claude Code reproduced the problem in an isolated Docker network, with a throwaway database and no credentials, and fixed every step until the whole spin-up exited cleanly. The only required setting now is an encryption key, which make generates. Every missing optional setting produces a banner that says what’s missing and how to set it.

An agent that can run your code converges. An agent that can’t is guessing.

How the five factors fit together

Step back from the list and the five factors sort into three things that the agent needs, all at once. Guidance tells it what good looks like before it starts: the existing code and tests, the docs and decision records, CLAUDE.md and the skills. Feedback tells it whether it worked: the unit tests, the linters and a runtime it can drive by itself. And autonomy lets it decide how to get there, by designing, writing, testing and documenting for hours without anyone checking the intermediate steps.

They aren’t equal partners. Guidance and feedback set the direction, which is the ratio of the series. Autonomy sets the speed, which is the 25-times multiplier. Autonomy without feedback is how you diverge fast. And the loop has two speeds. The inner one, act, verify, fix until green, corrects the change in front of the agent. The outer one, writing the lesson down as a skill, a rule or a test, is what makes the codebase converge across changes, because it changes what the next agent finds when it goes looking. Around all of it is a boundary that humans draw: the subsystems, the scaffolds, the decision records, and every irreversible step.

The inner loop isn’t new. Anthropic describes Claude Code’s own agent loop as gather context, take action, verify work, repeat. What a codebase adds is what each step finds, and the outer arc. Skip the outer arc, and every change passes its own tests while the codebase drifts. Part 2 is an example of exactly that.

What went wrong, and where the human stayed in the loop

It would be disingenuous to present eight days of autonomous work as if everything went famously. In fact, there was a speed bump, to put it mildly. But to spin it into a teachable moment, it’s a real world example of a geometric series. For context, take note that the local containers hold real credentials while running tests. These include, among other things, a kubeconfig for the production Kubernetes cluster and working cloud DNS credentials. Such credentials exist because Smarter performs Terraform-flavored CRUD operations on cloud infrastructure, and all of those operations need to be tested.

Now then, on October 6, a Claude Code release included a seemingly benign 1-line change to a function that verifies DNS records, and which runs on every deployment. Lacking an accurate source of DNS host data, Claude Code drew its own erroneous conclusion, and then — in production — proceeded to propagate incorrect host data onto actual real-world DNS records. One wrong value, copied, then copied again. We were offline for nearly 20 minutes.

In the author’s defense, the fact that those operations were performed on production infrastructure has everything to do with the author’s personal budget constraints, and nothing to do with the veracity of Smarter’s test bank. But I digress. This is a classic example of a seemingly benign coding chore where the agent incorrectly fills in blanks of missing data using its own not-unrealistic suppositions, albeit to catastrophic ends. Alas, the fix was every bit as trivial as the reputational damage was real.

The lasting change was to the skills. There’s now an infrastructure safety skill that says every Kubernetes, DNS, certificate and cloud call must be faked in tests, that commands which write to real infrastructure are handed to the human rather than run, and that the root domain’s A record must never be copied onto a platform host. The error became a test and a written rule. Meanwhile, the lesson learned wasn’t so much that an agent made a mistake. Heck, humans, including yours truly, routinely make mistakes of this magnitude all the time. The lesson was that an agent working at 25 times your pace also makes mistakes at 25 times your pace, so the guard rails have to be in the environment and in the written rules, not in your attention span.

The second week had a speed bump of its own, and this one was mine. In May I had cached Smarter’s reverse() shortcut, which turns a view’s name into a URL, with @cache_results for a year. Its cache key doesn’t change between releases. On October 16, a release moved the LLMTools pages to a new URL, and production’s Redis kept handing out the old one, so the page couldn’t load its lists. A second bug hid the first: the resulting 500 was being returned as a 200, with an error message that pointed everywhere but the cause. Claude Code found both, removed the cache, made the 500 a 500 again, and added a test that reverse() is never cached. The fix was tagged 41 minutes after the release that exposed it.

It’s the DNS lesson again, from the other side. The DNS outage came from a missing source of truth. This one came from a stale one, which is worse, because everything downstream trusts it.

Quantitative results

GitHub’s contributor graph for the repository tells the story in two numbers.

The author (lpm0073) contributed about 750,000 lines (code, documentation and whitespace) over three and a half years, or roughly 4,000 lines a week. About 35% of that is source code, so his long-run pace is about 1,500 lines of source a week. That’s a healthy, sustained pace for one senior developer who is also designing, operating and supporting the platform.

But Claude Code contributed 110,000 lines in 8 days. That’s around 40,000 lines of source in one week, or about 25 times the author’s long-run pace.

Then it did it again. From October 11 to 17, Claude Code added about 102,000 lines, not counting about 108,000 more that were generated (translation catalogs, JSON Schemas and the OpenAPI document). That’s about 25 times the author’s 4,000 lines a week. Counting only source code outside the tests, it’s about 44,000 lines, or about 29 times his 1,500. Across all eighteen days, commits under Claude’s name added about 238,000 lines, and commits under the author’s name about 43,000, most of them documentation.

In fairness, a few caveats belong next to those numbers.

First, lines of code are a poor measure of productivity, and everyone who has written software knows it. The fairer measure is the table at the top of this post: Claude is accredited with nineteen minor releases, a dozen new resource kinds and capabilities, a published API contract and an SDK built from it, an extension system, a console in four languages, roughly 130 bugs fixed in the first week alone, overall coverage ratio raised to 96%, and it wrote really good documentation for all of it as well.

Second, the lines aren’t comparable. the author’s 750,000 include the subsystems: the manifest loader, the broker model, multi-tenancy, the journal, secrets, settings. Those are the lines that took the longest to write and the longest to get right, and they’re the reason Claude Code’s lines could be written so fast.

Third, on October 10 I called this a one-time productivity explosion, not a steady state. The second week argues with that, a little: it ran at the same pace, on different work. So it’s fairer to say that the pace belongs to the conditions, not to the week. It happened because a road map that was well understood met a codebase that was ready for it. Most of the hard thinking about what Smarter should be had already been done, written down and tested. Claude Code turned that understanding into software, very quickly.

A 25-times multiplier is neutral. It makes a telescoping series converge 25 times faster. But it could just as haphazardly make a geometric series diverge 25 times faster. So tread carefully.

If you want the same result on your codebase

None of this requires Smarter’s particular stack. In rough order of payoff:

1. Build your subsystems by hand, and build them well. Tenancy, permissions, your API’s response contract, error handling, secrets, settings. Solve them once, in a way every feature shares, before you let an agent build features on them.
2. Get your test coverage up, and make your tests runnable by the agent. Coverage is where an LLM learns what “correct” means, and raising it is where the agent finds your bugs.
3. Test your API from the outside. Write a client against it, as a stranger would, and validate real responses against a published contract. Smarter’s first SDK found about ten API bugs that the platform’s own tests had walked past.
4. Annotate your types and enforce them in pre-commit. It removes the agent’s single largest source of guesswork.
5. Make your patterns consistent. Pick the best existing implementation of each kind of change and make the others look like it. Then tell the agent which one to copy.
6. Write docstrings for the contract, narrative docs for the intent, and decision records for the law. The agent reads all of them, and it will write them if it sees them.
7. Turn every agent mistake into a written rule. Put it in a skill or in CLAUDE.md, in the repository, under review, so that the mistake happens once.
8. Put the guard rails in the environment. Fake the infrastructure in tests, keep live credentials away from anything that can write to production, and keep humans in charge of the irreversible steps.

If you can’t do most of these yet, do them before you hand an agent the road map, not after. Every one of them lowers the ratio. Skip them, and the agent will raise it for you.

Closing thoughts

Smarter’s feature road map is fairly static. It follows the settled laws of generative AI as the community surfaces, evolves and refines them over time: retrieval augmented generation, function calling, guardrails, the Model Context Protocol, Agent Skills, open-weight models. For now, Smarter has caught up with them, and there’s nothing further in that queue until the generative AI community coalesces around its next protocol, innovation or best practice. So the second week went to the rest of the platform, the parts that let other software build on it, at the same pace and on the same foundation. When the next protocol arrives, the foundation will still be here, and so will the agent. The series has converged so far, and the job now is to keep it that way.

Part 2 is about the antipattern that threatens that, and how to spot it. Part 3 counts what Claude Code actually reads, session by session.

Until then, happy Smartering!

• Smarter on GitHub
• Documentation
• The Smarter blog
• Architecture decision records
• Change log
• Part 2: The Antipattern That Makes a Codebase Diverge
• Part 3: What Claude Code Actually Reads