Milestone 10: Giving the Coding Agent Tools

Milestone 10 adds the first agentic loop to CodeSelf. Up to this point, the project mostly worked in direct-generation mode: render a prompt, generate code, parse it, execute tests, and score the result.

That is a useful baseline, but it does not really support the strategy-development story. If we want to study behaviors like debugging, revising, or using tests, the agent needs a loop where those actions can actually affect the final answer.

Why Agentic Traces Matter

It is easy to overclaim strategy from text alone. A model can say "I will consider edge cases" without actually improving the program. For this project, strategy claims should be grounded in traces:

• Did the agent run public tests?
• Did the first attempt fail?
• Did it revise after seeing feedback?
• Did the revised code pass public tests?
• Did the final submission pass hidden tests?

Milestone 10 adds those trace records.

The Tool Loop

The first agentic loop is deliberately simple:

1. Generate initial code.
2. Run public tests.
3. If public tests fail, revise.
4. Run public tests again.
5. Submit final code.
6. Score final code with hidden tests included.

Hidden tests are never shown during debugging. They are used only in the final submission step. This keeps the loop aligned with the evaluation protocol.

Tools Implemented

I added an AgentToolbox with:

• run_public_tests
• run_custom_tests
• submit_final

Each tool returns a structured result with the tool call, whether it succeeded, an observation string, and payload data such as execution phases and reward breakdowns.

This means traces can be inspected later without rerunning the environment.

A Smoke-Test Repair Helper

Because we still do not have a real model-based repair system installed, I added a tiny deterministic repair helper. It can infer very simple fixes from public assertions, such as:

assert add_one(1) == 2

From that, it can synthesize:

def add_one(x):
    return x + 1

This is not meant to be a serious program-repair algorithm. It is a smoke-test helper that lets the tool loop demonstrate revision behavior before model-based repair is integrated.

Trace Schema

Each trace records:

• task ID
• prompt template
• rendered prompt
• raw completion
• initial code
• final code
• every generation, tool, and revision step
• final execution result
• final reward
• run metadata

The trace viewer can render a JSONL trace as Markdown, which makes debugging much easier.

Strategy Analysis

I added aggregate strategy metrics:

• final pass rate
• mean reward
• mean tool calls
• mean revisions
• revision rate
• public-test pass rate

These metrics are intentionally modest. They do not prove deep emergent strategy, but they do let us compare direct generation against tool-aided self-debugging in a measurable way.

Decisions I Made

I kept the loop dependency-free and model-agnostic. The same trace format can later be used with a real model that proposes revisions, but the current version can run locally with mock or static generators.

I also separated public-test feedback from hidden-test scoring. This is one of the most important design choices in agentic coding evaluation. If hidden tests leak into the loop, the final evaluation becomes meaningless.

What Comes Next

The project now has the infrastructure for agentic traces. The next major options are:

• build PPO comparison scaffolding
• integrate real TRL/model training
• replace the deterministic repair helper with model-generated revisions
• compare direct generation, self-debugging, and Reflexion-style memory

The core lesson from this milestone is that strategy needs instrumentation. If the agent is going to learn debugging behavior, the project needs traces that show what happened, not just final scores.