We recently moved most of NextWork’s project generation from Claude to GPT models running on Azure. GPT was faster and cheaper in our testing, and we had substantial Azure credits available to run it at production scale.
This made the decision to switch models fairly easy to justify, but the engineering work required to make the switch was much less straightforward.
At NextWork, generating a project involves several agents rather than one prompt and one response. One agent researches the tools involved, another works out what the learner should build and in what order, others write individual parts of the project, and the finished work goes through validation and repair. Some of those agents use tools while they are reasoning, and their later requests depend on what happened earlier in the same generation process.
Moving from Claude to GPT affected more than generation quality. We had to change:
how our agents used reasoning and tools,
how we preserved their conversation history, and
how requests were retried and scheduled on Azure.
Our GPT agents were running with less reasoning
The first few projects we generated with GPT models completed normally, but the projects were noticeably weaker - they made poor choices about what the learner should build, included less useful research, produced project steps that did not fit together as coherently and didn’t have the NextWork tone of voice.
Our outline agent is responsible for much of that research and planning. On Claude, we gave it a large thinking budget. We configured GPT for high reasoning effort too, but the Azure Chat Completions API we were using did not support function tools and an explicit reasoning setting in the same request.
This meant that the code that converted our agents’ requests into Azure’s Chat Completions format removed the reasoning setting whenever the request also included tools. The generation still completed, but GPT was using Azure’s default reasoning level rather than the high setting we had configured. We were comparing Claude and GPT without actually giving GPT the reasoning effort we intended.
We ended up moving the relevant agents to Azure’s Responses API, which allowed us to use function tools and high reasoning effort in the same request.
GPT handled conversation history differently
Some of our agents gather information from the learner before they start generating the project. They might ask clarifying questions about what the learner wants to build, their preferred tools, budget and experience level. They can also call tools to research information they need before continuing.
After moving to GPT, we found that an agent could start asking questions the learner had already answered earlier in the conversation.
The problem stems from how we stored conversations after a tool call. With Claude, a tool call was usually followed by another message from the model. Our code used that final message to decide that the interaction was complete, but GPT did not always return that final message. It could call a tool and end its turn there.
Even though the tool call had completed successfully, our code saw the missing final message and assumed the interaction had been interrupted, and could remove that part of the conversation when preparing the next request. When GPT received the conversation again, it no longer had some of the information the learner had already provided.
To resolve this, we updated the GPT integration to recognise a tool call as a valid end to the exchange and preserve the full conversation around it. That stopped the agent from losing context and repeating questions it had already asked.
Failed requests needed different retry logic
There are different reasons we might need to try a generation again. If GPT generates a project step that fails one of our checks, we want to give it the feedback and ask it to revise the step.
But Azure can also reject a request before GPT generates anything, such as when we hit a rate limit. We found that these requests could go through the same retry path and receive corrective feedback even though GPT had produced nothing to correct.
That meant the retry could give GPT instructions about a problem that had never occurred instead of simply asking it to try the original generation again. To a learner, it also means a bunch of retried generations without knowing why it was happening.
To solve this, we separated failures based on whether GPT had actually produced a response. If Azure rate-limits, times out or interrupts a request before GPT responds, we resend the original generation request unchanged. If GPT generates a section that fails our checks, we send that section back with the specific feedback it needs to address. Each retry now starts from what actually happened on the previous attempt.
Azure became a bottleneck at higher concurrency
A project uses several agents, and we can have multiple projects generating at the same time. All of those requests share the same Azure capacity.
Azure can reserve capacity based on how long a response could be, not just how long it actually is. Because our project-writing agents can generate long responses, several running at once can hit Azure’s limit even when their final outputs are much shorter. This wasn’t a problem with Claude because its capacity limits were handled differently - several agents could run at once without exhausting the same kind of shared maximum limit.
Limiting each worker separately was not enough because several workers could still exceed the Azure limit together, so we now control how many requests can run across all of the workers sharing the deployment. We also prioritise generations that a learner is actively waiting for over work that can happen in the background.
The next model switch should be easier
Ideally, very little of this work should be visible to a learner. Switching the model behind project generation should not mean having to repeat questions, waiting through unnecessary retries, or receiving a worse project because a reasoning setting changed somewhere underneath the product.
In reality, while our automated checks and manual testing helped, so did the learners who told us when a generated project felt weaker, when an agent repeated a question, or when generation took longer than expected. Thank you to everyone who kept sending us that feedback while we made the switch.
We will keep testing new models as they improve. The work we have done here means the next switch should require fewer changes to the rest of the system, and ideally even less should change for the person building a project on the other side of it.
Happy building,
