Shipping like a startup, regulated like a bank

David Abitbol leads AI operations at Revolut, the European digital bank with 80 million customers in 45 markets. In this 12-minute session from AI Engineer New York, he explains how Revolut gives its engineers effectively unlimited AI coding tokens by running an open-weight model on dedicated GPUs, and what it took to make that fast enough that engineers don’t complain.

The headline: about 60–70% faster delivery for roughly a tenth of the cost of frontier-lab APIs.

What AI did to 2,000 engineers’ lead time

Revolut tracked two numbers across a sample of 2,000 engineers for a year and a half:

• Lead time: from picking up a feature to having it merged and ready to deploy, measured in days.
• Coding time: time spent actually writing the code, measured in hours.

Each step changed the curve differently:

1. Gemini (January 2025). The first time engineers could use AI with private data. Coding time dropped about 20% and lead time about 10%, mostly because people stopped digging through Stack Overflow and documentation.
2. Claude Code (trial from February 2025, at scale from January 2026). The bank was sceptical of the cost, with some engineers spending $1,000 to $10,000 a month for unclear return. So it launched with strict budget limits, and the effect on lead time was small.
3. Self-hosted GLM (from April 2026). Revolut started serving the open-weight GLM model itself, beginning with GLM 5.1, to give engineers unlimited tokens for a fraction of the cost. Coding time improved about 60% and lead time about 70%.

Coding isn’t the bottleneck

The bigger gain was in lead time, not coding time. Writing code isn’t the slowest part of shipping a feature. With unlimited tokens, engineers point the agent at everything around the code too: digging through the inbox, wrangling Jira and Confluence. Nobody saves tokens for the end of the quarter or for only the hardest tasks.

Where Revolut landed:

• Open source is the priority. The economics work and the bill is flat, so there’s no bill shock at the end of the month.
• Frontier models are the exception. Claude and GPT models are still used for the tasks they do best.

What it takes to self-host GLM

• Hardware. Serving 2,000 engineers takes about 56 Nvidia B200 GPUs to host GLM 5.3, which is a large model. Revolut runs them as dedicated endpoints on Together AI.
• Latency target: under 3 seconds to first token. This is empirical: once time to first token (TTFT) goes past 5 seconds, Slack messages start arriving.
• Throughput target: about 200 tokens per second. Going higher doesn’t matter much, because people no longer watch the thinking stream. They run tasks in parallel and check back when they finish. After 5 pm, when load drops, an engineer can get around 800 tokens per second.
• Cost. The blended cost is about 30 cents per million tokens in or out, with cache hits free. The invoice is about a tenth of what Revolut paid Anthropic and OpenAI.

Resist scaling, tune first

Loading weights onto GPUs and starting inference gives terrible performance. Throughput and latency depend heavily on parameters tuned for your workload. Revolut’s own summary: most of the gain came from adjusting hyperparameters, not from adding GPUs.

The chart in the talk tracks demand (requests per second, which drives latency) against TTFT, the number being optimised. Mid-migration, demand was around 3 requests per second and TTFT was over 10 seconds. By August, on the same 56 GPUs, demand was over 7 requests per second and TTFT was around 1.6 seconds.

The parameters that matter

There’s no secret recipe, but these are the settings Revolut tuned:

Production setup

The setup has three layers:

• Text models: a written description in, working code out. GLM is text-only, which is a limitation for some teams. Revolut’s trade-off, which they’d make every time, is to use the smarter model and handle images another way. A local router (OpenCode, for example) sends any image-recognition request to Gemini or a similar model and keeps GLM for the main task.
• Vision models: a mockup in, a working interface out. Where a team genuinely needs vision, a smaller GLM “flash” vision model works well. Revolut’s UI designers use it heavily, and turnaround on UI work has improved a lot.
• Harness: CLI or app. Every endpoint exposes an OpenAI-compatible API, including the Responses API, so it plugs straight into OpenCode, the Codex CLI and the Codex app.

For the CIO: compliance and safety

Using Chinese open-weight models carries a stigma. The answer is in how they’re used: Revolut downloads the weights and runs them on dedicated, US-based infrastructure. No request ever goes to an inference endpoint in China, so data sharing isn’t a concern.

As an EU-regulated bank, Revolut is under one of the strictest compliance regimes. Here’s how it satisfies the regulator:

1. Human in the loop, enforced by a sandbox. Coding tasks aren’t classed as autonomous agents, because a person always approves what the agent does. The sandbox proves it: harnesses like Codex and Claude Code have approval rules, so you set which operations are allowed and which need a human. For example, allow grep but not gradle. That’s what convinces a regulator this isn’t an autonomous agent needing different supervision.
2. Protection from rogue use. The same sandbox limits what a bad actor inside the company could do through the agent.
3. Prompt injection. Less of a threat with today’s smarter models, but still covered by the sandbox.
4. Data residency. Keep the GPUs in a country covered by EU data adequacy arrangements. He named the US, EU, UK and Israel. Inference and GPU providers offer zero data retention by default.
5. Strong OpenTelemetry. If there’s ever an incident, you need logs deep enough to reconstruct what happened. The same data gives you usage analytics.

Measure your own endpoint

The talk’s two targets are easy to check against any OpenAI-compatible endpoint, self-hosted or not. With your endpoint’s base URL, key and model name in environment variables:

export BASE_URL="https://your-endpoint.example.com/v1"
export API_KEY="..."
export MODEL="your-model-name"

curl -sS -N -o /dev/null \
  -w 'first byte: %{time_starttransfer}s  total: %{time_total}s\n' \
  "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MODEL\",\"stream\":true,\"max_tokens\":400,
       \"messages\":[{\"role\":\"user\",\"content\":\"Write a 300-word explanation of TCP slow start.\"}]}"

With "stream": true, the server sends the first chunk as soon as the first token is ready, so time_starttransfer is a close stand-in for time to first token.

How to check it worked: the command prints one line such as first byte: 1.42s total: 3.10s. Revolut’s bar is under 3 seconds for the first number. Run it a few times at your busiest hour and again after hours: the gap between the two shows how much load, rather than the model, is costing you.

How it ties together

• Unlimited tokens changed behaviour. Once engineers stopped rationing, they used agents for the work around the code, and lead time improved more than coding time.
• Open weights made unlimited affordable. A flat bill for dedicated GPUs replaced per-token frontier pricing, at about a tenth of the cost.
• Latency targets set the size. Under 3 seconds to first token and around 200 tokens per second keep 2,000 engineers happy on 56 B200s.
• Tuning came before scaling. Context caps, NVFP4, TP8, cache hits, sticky sessions and PD disaggregation more than doubled capacity on the same hardware.
• Routing covers the model’s gaps. Text and code go to GLM, images to a vision model, and a few tasks to frontier models, all behind one OpenAI-compatible interface.
• The sandbox makes it compliant. Approval rules keep a human in the loop. Data residency, zero retention and OpenTelemetry cover the rest of what a regulator asks.