EntroπaLabs
Language models · Agents

The agent loop, and what it can't do in a browser

A tool-use agent is about forty lines of control flow. Everything difficult about it lives in the tools you are willing to hand it.

26 July 2026 · 8 min read · all writing

A tool-using agent is a much smaller idea than the discourse suggests. Send the conversation and a list of tool schemas to a model; if it asks for a tool, run it and send the result back; repeat until it stops asking. That is the loop, and it fits comfortably on one screen.

The voice agent on this site is a browser port of pyagent, a small terminal coding agent. Porting it was a useful exercise precisely because it forces you to separate the loop — which moves over unchanged — from the tools, which mostly cannot come at all.

The loop

messages.push(user_input)
for step in range(max_steps):
    reply = provider.complete(system, messages, tool_schemas)
    messages.push(assistant(reply.text, reply.tool_calls))
    if not reply.tool_calls:
        return reply.text
    for call in reply.tool_calls:
        result = run_tool(call.name, call.args)
        messages.push(tool_result(call.id, result))

That is the entire agent. Everything else is plumbing. Two details do real work:

The step cap. Without it, a model that misreads a tool result can loop until your budget is gone. Six steps is plenty for a question a person asked out loud.

Errors go back to the model, not to the user. When a tool throws, the exception is caught and returned as the tool result — error: unknown team 'Montrael'. The model reads it, corrects itself, and calls again. Surfacing that exception to the user instead would end the turn on a failure the model could have fixed in one step.

Two wire formats, and only two

pyagent's provider layer exists because tool calling is expressed differently by different APIs. Having implemented both, the split is cleaner than expected: there are two formats in the world.

OpenAI-compatible — which also covers Gemini, Ollama, vLLM, LM Studio and llama.cpp, since they all expose the same endpoint shape — returns a tool_calls array, and the arguments arrive as a JSON string that you have to parse yourself. Results go back as messages with role: "tool".

Anthropic returns tool_use content blocks with the input already a structured object, and results go back as tool_result blocks inside a user message.

OpenAI-compatible Anthropic ───────────────── ───────── tool_calls: [{ content: [{ function: { type: "tool_use", arguments: "{…}" ←JSON input: {…} ←object } }] }] role:"tool" message tool_result block in user turn

Normalise both into one internal shape — {id, name, args} going out, {text, calls} coming back — and the loop and the tools never learn which model is running. That is the whole architectural payoff of the adapter split, and it is worth the fifty lines.

What could not be ported

pyagent's tools are read_file, write_file, edit_file, list_dir and bash. Not one of them can cross into a browser tab. There is no filesystem and no shell — and if there were, handing them to a model on a public web page would be an extraordinary thing to do.

The honest options were to fake the tools, or to find real ones. The tools it got instead query data this site already publishes: today's hit probabilities from Beat the Streak, skater valuations and roster needs from the GM playbook, plus arithmetic and an index of the site itself.

So "who's the best bat today?" spoken aloud produces a genuine streak_board(top=3) call against genuine data, and you watch the trace happen. The loop is demonstrating itself rather than being illustrated.

Voice, with the browser doing the work

pyagent shells out to ffmpeg for recording, Whisper for transcription and piper for speech. The browser has all three built in: the Web Speech API's SpeechRecognition for mic → text and speechSynthesis for the reply. A Web Audio analyser drives the ring around the microphone from the live input level, so you can see it hearing you.

Two caveats. Firefox has no speech recognition at all, so the page says so and typing works everywhere. And recognition is not local — Chrome and Safari stream audio to their vendor's service. Every other part of the page is local; that part is not, and it would be dishonest to imply otherwise.

Whose key, and where it lives

The model call happens in the visitor's browser, with the visitor's own key, held in localStorage. There is no server in this architecture to proxy through, which means the key never reaches this site — and also means it is sitting in browser storage, readable by anything that can run script on the page. The page says so plainly, and offers a forget key button. Use a key you're willing to rotate.

The alternative — my key in a serverless function, so visitors need nothing — would mean paying for strangers' inference and defending an open proxy. For a demonstration of a loop, bring-your-own-key is the honest trade.

With no key at all, the page still runs: the tools execute for real against real data, and only the model is stubbed by matching the question to a tool. The trace you see is genuine, the answer is genuine, and the page labels the model as absent. Same principle as the embedded demo scene in the splat viewer and the synthetic checkpoint in the weights pages — a demo should never be an empty state.

Open the agent