The Pulse is a series covering events, insights, and trends within Big Tech and startups.Today, we cover:New trend: tech companies moving to open models. Uber, Pinterest, Stripe, Coinbase, Ramp, and AT&T are making large savings on their AI bills by dropping proprietary models and using smart model routing.Automatic software maintenance experiments by Linear and Anthropic. Both startups are experimenting with how far they can push AI agents to automatically fix bugs and remove tech debt. It’
The Pulse is a series covering events, insights, and trends within Big Tech and startups.
Today, we cover:
New trend: tech companies moving to open models. Uber, Pinterest, Stripe, Coinbase, Ramp, and AT&T are making large savings on their AI bills by dropping proprietary models and using smart model routing.
Automatic software maintenance experiments by Linear and Anthropic. Both startups are experimenting with how far they can push AI agents to automatically fix bugs and remove tech debt. It’s working better than anyone might’ve expected in the recent past, but not producing code that can be merged without review.
Frontier AI lab wars: OpenAI pulls models from SpaceX / Cursor. With SpaceX now a frontier model and rival to OpenAI and Anthropic, OpenAI has pulled its GPT models from Cursor. This isn’t an option for Anthropic which is dependent on the SpaceX compute they rent to serve Claude.
HR tech startup’s one-dev-per-project approach. A full-remote HR startup with 70 engineers has a single engineer run each project, and says the approach works well. Will this approach be adopted elsewhere, especially at other full-remote startups?
Industry Pulse. Meta moved over to Slack for better agent interoperability, layoffs at Uber and PagerDuty, Anthropic upsets users by calling a rate limit decrease an “increase”, token usage explodes on OpenRouter, AI drives surging demand for Apple’s Mac Mini & Mac Studio, and more.
1. New trend: moving to open models at tech companies
Update: a week after publishing this article, Ara Krahzian at Ramp has confirmed that AI spend in August, has, indeed, declined at the top 1% of businesses by 10%, based on Ramp data. I’d wager those companies are not spending fewer tokens, but they are optimizing cost, in ways outlined below.
In May, I covered an emerging trend of companies wanting to cut back their AI spending, starting with engineering departments. Different approaches were being tried:
Wow, has it already been five years?! The newsletter hits a big milestone this week, and it wouldn’t have been possible without subscribers. Thanks to everyone who’s read an article or listened to a podcast episode during that time!Before we get into the latest issue, I’d like to point you to two upcoming events in New York City which I’ll be at. It would be great to meet some of you! They’re free to attend:Wednesday, 16 September, 6pm: turbopuffer talks. What happens when you mix a whiteboard,
Wow, has it already been five years?! The newsletter hits a big milestone this week, and it wouldn’t have been possible without subscribers. Thanks to everyone who’s read an article or listened to a podcast episode during that time!
Before we get into the latest issue, I’d like to point you to two upcoming events in New York City which I’ll be at. It would be great to meet some of you! They’re free to attend:
Wednesday, 16 September, 6pm: turbopuffer talks. What happens when you mix a whiteboard, two senior turbopuffer engineers, and myself, with the spotlight on database internals? We’ll draw out turbopuffer’s database architecture and geek out over it, then hang out afterwards. Join us! Sign up here.
Thursday, 17 September, 5:30pm: An evening with The Pragmatic Engineer and WorkOS.A fireside chat between WorkOS founder, Michael Grinich, and me, with him asking the questions. We’ll get into stories about the tech industry – including a few which probably won’t get published! Sign up here.
If you’re at LDX3 New York on 15-16 September (Tue & Wed), I’ll be doing a keynote on Tuesday morning and hanging out around the Antithesis booth during the day. See the full agenda and get tickets.
I checked the calendar and it is indeed half a decade – almost to the day in 2021 – since I published the first-ever issue of The Pragmatic Engineer:
On launch, the paid version of The Pragmatic Engineer cost $100/year, or $10/month (this has since increased to $150/year or $15/month). As a special offer, I’m “resetting” the price of the publication to annual subscribers for $100/year: claim this offer here. The offer ends in a week, on 8 September. Get this offer here.
My personal expectations weren’t high back then; the subscription model for newsletters on platforms like Substack was starting to take off, but the focus was strongly on politics, business, and finance. It wasn’t clear if there was any demand for a publication about software engineering, written by a software engineer.
I soon found out there was demand that surpassed all my expectations – and then some! Fast forward to today; the newsletter has more than 1.1M readers, tens of thousands of paid subscribers, a podcast, and more than 500,000 YouTube subscribers.
But the numbers aren’t the most validating thing for me; that would be the feedback sent in by readers. Via email, in DMs, or in-person at events, it’s great to hear how an article or a podcast we published helped someone try a new approach, or to gain confidence that theirs was the right one, or that an article helped convince a team to change things. It also means a lot to learn that The Pragmatic Engineer helps people feel more confident about keeping up in this fast-changing industry.
Thanks again for your support! It’s the reason why The Pragmatic Engineer is a viable business and a growing publication, and means we can “scale up” our coverage to deliver ever-more detail about how software gets built, today.
Today’s issue covers:
Diving deeper, year after year. The evolution of the Pragmatic Engineer’s coverage over five years, getting in through the “front door” instead of the “back door” for deepdives, launching the podcast, The Pragmatic Summit, and growing our team.
What’s next? What we’re excited about, the second Pragmatic Summit, and how you can expect us to stay focused on how building software is changing, and the ways that successful engineers, teams, and companies adapt.
Cash-prize writing contest: Software is changing faster than ever. Send us your essay about how things are changing for you, for the chance to win cash prizes worth up to $10,000. Read more on how to take part.
1. Diving deeper, year after year
The Pragmatic Engineer was 15 years in the making and not an overnight success; I started to write a blog in 2007 about software development, which was “rebooted” as “The Pragmatic Engineer Blog” in 2015. At the time, it was read by almost nobody!
In 2019, I launched an email digest (“v0” of the newsletter), and a year later, I decided to focus on the newsletter fulltime. This was, after I resigned at Uber, as a manager, following job cuts in 2020. The pandemic hit Uber hard: during those cuts, a quarter of my team was laid off and the rest were disbanded due to the pandemic. I took an employment break, planning to finish ‘The Software Engineer’s Guidebook’ in six months and then start a VC-funded startup.
In the end, finishing the book took another two years, and I didn’t kick off that VC-funded startup I was originally planning to do. Instead, I decided to go all-in on writing a newsletter targeted exclusively at software engineers and engineering leaders.
2021: product-market fit
The Pragmatic Engineer started off with one in-depth article on an interesting topic per week, including these ones:
Growing a junior-heavy team: this was published when it was hard to hire seniors and many teams were junior-heavy
This time, the newsletter took off and crossed 1,000 paid subscribers six weeks after launch. By the end of 2021, it was the #1 paid technology newsletter on Substack (!!), with 2,700 paid subscribers and 30,000 free subscribers. Most surprising was that the growth happened via word-of-mouth, and from me posting about issues on social media, without spending on ads and marketing. Growth has continued since then; for example, as per the Brex Benchmark, The Pragmatic Engineer is the third most expensed newsletter at startups, globally, in 2026, and remains one of the fastest growing ones.
In the second year, I added a Thursday article called ‘The Scoop’ – now ‘The Pulse’ – alongside the research-heavy Tuesday articles. Unlike those articles, The Scoop covered tech news and developments on a weekly basis and wasn’t ‘evergreen’ material. But as I talked with more techies, I got to spot new trends and patterns, often months before mainstream publications covered them.
Recent examples have included us reporting the odd tokenmaxxing trend a month before the Financial Times did so, or covering the extreme work patterns at AI startups months before The Wall Street Journal picked up the story, or The Economist republishing my Trimodal Nature of Software Engineering Compensation diagram four months later across their digital and print editions.
2023: Going direct for engineering deepdives
Until mid-2023, I mostly wrote deepdives on engineering topics without the involvement of companies that weren’t interested in showing me how they did things from the inside.
At the end of 2022, I heard from former colleagues at Uber that the ridesharing giant was moving off its own data centers, and moving onto the cloud, and onboarding to GCP and Oracle. I gathered plenty of details from current and ex-Uber workers, but the infra leadership didn’t engage with me via official channels.
So, I went ahead and published ‘Inside Uber’s move to the Cloud’, which got the majority of details right, except for a few. Inside Uber, the article was criticized for not getting everything correct, but that would’ve involved me talking on the record to Uber’s infra leadership, which they didn’t do!
In 2023, the newsletter had 350,000 readers and was starting to make a name for itself, which began to open some previously closed doors. I increasingly got details from engineers at companies via the “front door” rather than the “back door” route of informal contacts and scraps of information. Uber’s cloud migration was the final deepdive of its type; after that, going through the “front door” became easier, with companies sharing details with me about what they were doing. This change led to more accurate and detailed articles, including:
Since then, deepdives about interesting, cutting-edge tech companies have become a regular part of our publishing schedule. And as a bonus, we’ve figured out a “recipe” for getting access to folks at tech companies who are usually off-limits to the media, which leads to more exclusive content for readers.
Share previously “private” conversations. When doing a deepdive about an interesting company or technology, I usually did a call with engineers. These fascinating, one-hour-long conversations got summarized in a paragraph or two in articles, but I always felt there was more that readers would be interested in.
Meet interesting people. I was spending most of my days behind a keyboard writing deepdives, with the occasional video call when researching engineering teams. I hoped that doing a podcast would just allow me to meet more people!
Simon Willison, one of the most grounded voices in AI engineering, was the first podcast guest, and feedback was warm and positive. I always look forward to talking with guests due to their experience and significant industry contributions; the likes of Grady Booch, Nicole Forsgren, and Mitchell Hashimoto – or for their unusually deep expertise – like context engineering with Dex Horthy, developer productivity with Laura Tacho, and building software without looking at the code with Peter Steinberger. These days, when I go to a conference, more people talk to me about the podcast than the written articles, which is interesting.
Over time, I’ve developed a preference for in-person podcast conversations instead of video calls. When the podcast launched, I had a home studio set up for the remote recording of on-screen meetings:
My podcast studio in Amsterdam. On the left wall: a noise-absorbing panel and a city map
But you might have noticed that these days, podcast episodes are in-person conversations more often than they are video calls. I find that conversation flows better in person, and that the format is more engaging for everyone involved than a conversation on a screen is. It also offers an opportunity to hang out before and after the recording! Of course, the logistics of in-person recording are more complicated due to travel, and when there’s someone I’d really like to get on the show but it’s tricky to arrange, we stick with the remote option.
I’d like to hear your suggestions about future guests for the podcast. Let us know who you’d like to hear from and why. Send suggestions here.
2025: The Pragmatic Engineer Summit
In summer 2025, I attended the LeadDev Conference in London and enjoyed it so much that I asked if we could organize a conference for The Pragmatic Engineer, with deepdives and podcast guests for readers and listeners.
We spent the second half of last year organizing the first-ever Pragmatic Summit, which took place in February this year in San Francisco, with great help from the excellent Statsig team (many of whom now work at OpenAI). With 500 attendees, 15 standout speakers – and with it being the first-ever conference I’d organized – it was a smashing success. A one-minute video recap of the event:
Based on the feedback from attendees, there will be another Pragmatic Summit in San Francisco next year. I’ll share details in the coming weeks; we’re in the middle of putting this event together now!
2026: Growing the team in order to dive deeper
Predictably, the most time-consuming task in The Pragmatic Engineer is working on deepdives. We often spend up to two months on a single deepdive, educating ourselves on the topic being covered, talking with expert engineers, and generally getting deep into a topic in order to deliver a deepdive worth reading.
I say “us” because this year, Jessica Salmon and Ivan Klaric joined the team. Both are software engineers with startup and Big Tech experience, who enjoy spending time getting to understand topics and contributing to longform articles.
2. What’s next?
With a larger team than before, we’ve continued to produce deepdives for readers. A few recent ones:
I’ve found there are no shortcuts for producing an in-depth article on a relevant topic for readers. You simply have to put the time and effort into it. This involves spending a lot of time on thinking and understanding things, going directly to engineers who build what we want to learn about, and then spending even more time on organizing research material into a lengthy article that’s informative and hopefully not dull to read.
Needless to say, we’ve experimented with the analytical powers of AI tools in parts of the research process. The technology is good at gathering publicly-available sources and does a decent job of summarizing them, but that’s been more or less the limits of AI’s usefulness for our purposes to date – except as a spelling and grammar checking tool after a full draft is written by a person.
If anything, AI can easily lead you down the wrong track by theorizing about non-existent connections and confidently espousing theories which some basic critical thinking could easily debunk!
Aside from research and correcting (most) typos, we don’t use AI in this publication. That’s because we are writing for a readership of humans and believe in the value of human voices. What you read, hear, and see in The Pragmatic Engineer comes from me and other engineers, and plenty of thought goes into each sentence.
What comes next?
As a company that doesn’t have any venture funding, we can be ambitious without chasing any “growth goals.” In the near future, our goal is to keep doing what we do, and do it better. This means:
More ambitious deepdives. We’ll keep bringing you deepdives from inside companies and teams building cutting-edge, fascinating software; how do they do it, what is working for them, and why? The more you understand about why engineering approaches work in certain places and situations, the more likely you’ll be able to effectively apply them in your own setting.
In-person events. The Pragmatic Summit returns to San Francisco in February 2027. It’s my long-term goal to have a second summit in Europe, as well. Meanwhile, I’ll keep attending in-person events, like the upcoming shows in New York City with turbopuffer and WorkOS, and let you know in the newsletter when these happen.
Interesting podcast conversations. Recording conversations with software professionals and leaders is something that keeps filling my bucket. Expect more great conversations coming your way.
Keeping up with The Pulse.I spend most of the week talking with engineers on various channels and at companies as part of my efforts to check ‘the pulse’ of the tech business: what’s happening and what’s changing. Writing The Pulse on Thursdays remains a personal highlight of my week.
Building more of our own software stack – on the side. One thing that AI has made easier is context switching between writing and building software. In the past six months, I’ve found myself building more parts of The Pragmatic Engineer backend stack, such as landing pages, through to API endpoints used for group subscriptions, refunds, and more. This year, I’ve probably built more software scratching my own itch at the publication than in previous years combined! Of course, our software stack is not our top focus, but it’s a welcome distraction to work on.
A big theme for the rest of 2026 is how software engineering is changing: tools used for decades like IDEs are falling out of style, and processes long considered as best practices, like code reviews, are becoming optional. Of course, the biggest change is that we’re spending little to no time on typing out the code, which has never been the case since computing existed. Even before computer keyboards, programmers were writing programs on punch cards!
The good news is that we see that fundamentals still matter: many software engineers who were considered standout devs before seem to be even more in demand than ever, while picking up AI engineering appears to be easier than learning a new programming language.
Nonetheless, this change is destabilizing, fast-paced, and no one has figured out the “right” way to build software with AI. We’ll keep reporting on cases of teams and individuals that adapt well, while also paying attention to those things that don’t change, such as how teams, as a “core” unit of a business, seem to be just as important at leading AI labs as they were pre-AI.
3. How software engineering is changing: essay challenge
To finish, we’re delighted to announce an essay challenge on the prescient topic of how software engineering is changing for professionals.
As mentioned, the pace of change in software engineering is only accelerating, with rapid industry-wide adoption of LLMs, AI tooling, and AI infrastructure. At The Pragmatic Engineer, we aim to cover much of what’s going on, and as part of that, we’d love to pull in more perspectives than what our team can cover alone.
Send us an article no more than 10,000 words long on how you see things changing at your startup or tech company. We’ll award $10,000 for the best essay we read, and other leading entries can win smaller prizes. Articles sent to us will be eligible for publication in future editions of the Pragmatic Engineer. See more details here.
So, tell us what’s new, different, better, or worse in your part of the tech industry since AI has been in your workflow.
Submissions close 4 October at midnight (PST). Read all details of the challenge here, and we look forward to reading your article about interesting and consequential changes in your part of the industry.
If you’re thinking of upgrading to the paid version, you can do so for the “launch” price, with a 33% discount on annual plans. This offer ends in a week, on 8 September. Get it here. If you have an L&D budget to expense from, here’s an email template to send to your manager.
Thank you for being a reader of The Pragmatic Engineer; we value your attention and support, and never take it for granted. With that, onwards to the next five, exciting years!
The Pulse is a series covering events, insights, and trends within Big Tech and startups.Today, we cover:Did Meta really decide to reduce team sizes by 60% because of AI? An in-depth report by Reuters details how Meta’s leadership decided to slash team sizes by 60%, hatching plans in January to execute the social media giant’s largest-ever layoffs. But …
Read more
The Pulse is a series covering events, insights, and trends within Big Tech and startups.
Today, we cover:
Did Meta really decide to reduce team sizes by 60% because of AI? An in-depth report by Reuters details how Meta’s leadership decided to slash team sizes by 60%, hatching plans in January to execute the social media giant’s largest-ever layoffs. But …
Stream the latest episodeListen and watch now on YouTube, Apple, and Spotify. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.Brought to You by• Antithesis – turbocharge testing of your systems by running your whole system under aggressive fault injection. There’s good reason teams like Jane Street, Fly.io, and the etcd community rely on Antithesis. Learn more.• Sentry – application monitoring software built by developers, for developers. Sentry’s
Listen and watch now on YouTube, Apple, and Spotify. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.
Brought to You by
• Antithesis– turbocharge testing of your systems by running your whole system under aggressive fault injection. There’s good reason teams like Jane Street, Fly.io, and the etcd community rely on Antithesis. Learn more.
• Sentry– application monitoring software built by developers, for developers. Sentry’s Seer AI agent is one of their new, neat tools, which I’ve used as a way to quickly fix errors on my backend. Check out Sentry.
• turbopuffer– A vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable. I met their team in San Francisco, and am a fan of their “hardcore and whimsical” engineering culture, and how pragmatic their engineering philosophy is. Check them out.
In this episode
There can be few people around who care about software performance more than today’s pod guest, Casey Muratori. He’s a programmer and videogame developer, founder of Molly Rocket, and creator of Handmade Hero – a long-running series about building a game from scratch. He also evangelizes about performance on his Substack, Computer, Enhance.
We got to know each other about three years ago, first via messages, including this one from Casey:
“Why does the industry zeitgeist place so little emphasis on software performance when there seems to be overwhelming evidence that performance is critical to their bottom line?
Like you, I run a Substack for professional programmers, but I focus exclusively on software performance. Although we are quite large by Substack standards, so a certain subset of programmers must believe performance is important, I nonetheless hear lots of dismissive excuses when I post on social media. This happens so frequently, I devoted an entire article to cataloging the extensive pro-performance evidence we already have from the world’s leading software companies: Performance Excuses Debunked.
Strangely, nobody has a rebuttal to why performance is important. When I point people to this, they actually tend to agree. But the prevailing attitude nonetheless stays the same.”
I’m delighted we finally have Casey on the podcast because it’s overdue! In this episode, we discuss why software performance matters, why it’s overlooked, and how developers can get better at writing performant code. We explore why performance should be considered during design, the value of learning to read assembly & understanding how CPUs work, Casey’s critique of ‘clean code’, and why he believes testing shouldn’t drive software design.
We touch on how videogame development has changed, and influential game engines. Casey also tells us why he prefers to write code by hand, not with AI, and more.
Takeaways from the conversation with Casey
1. DirectX might not exist without an “unauthorized” internal Microsoft project. DirectX is a very popular Microsoft library that standardized rendering on top of GPUs, used mostly for games. Casey tells how Chris Hecker built a library for fast on-screen rendering at Microsoft called WinG, which was never authorized; it was a total “Skunk Works” project. DirectX’s roots go back to WinG, which its three founders were testers on.
2. Is performance starting to matter to businesses? Enterprise software buyers care mainly about cost, compliance, and capabilities – but not performance. Even so, there are some products gaining major popularity and market share due to their performance, such as File Pilot (next-gen file explorer) and the Blick video editor. Is the tide turning?
3. Profiler-driven performance optimization is the wrong way to optimize. The standard way of optimizing is to profile the application, tweak hotspots, then check if the stats have improved. But this only finds a local minimum; Casey says every engineer he’s worked with who was a great “optimizer” began by establishing what the hardware could theoretically do, and then did not stop until they’d closed the gap to that performance level.
4. If you care about performance, learn to read assembly (no need to write it). There are about 20-30 instructions you need to learn to be able to read basic assembly. For example, here’s a program that calculates the value of 5 + 3 - 1 (which is 7), then prints it out:
An assembly program calculates 5+3-1 (the first 3 lines after _start), then prints the result to stdout
5. Take a grain of salt with conventional wisdom that premature optimization is the “root of all evil”. Many devs use it as an excuse to delay performance optimization, but Casey says that not optimizing in time could mean that only performance hotspots can be fixed later, and not the architectural issues that create poor performance. Architect your system to be performant, or you’ll have trouble solving problems without a rewrite!
6. Only three things are needed to understand how CPUs work. Casey believes that knowing them means you’ll be able to tell from any CPU announcement roughly how well it performs. Those three pillars of understanding:
How data moves in and out: load/store units and L1–L3 caches
How instructions flow through the pipes: branch prediction, i-cache
Execution unit scheduling: raw throughput per operation type
7. Why are game studios so secretive? Before licensable videogame engines existed, the game engine was a studio’s “core” intellectual property (IP), and every studio built rendering, pathfinding, and other tools from scratch. This is how Blizzard rolled Warcraft 1’s engine into Warcraft 2. Any competitor making a rival game had to start from scratch, which was a reason for game studios to closely guard the secrets of how their own game engines worked.
8. The games industry already had its “AI moment” – and it wasn’t pretty. When game engines became licensable, pretty much any developer could build and publish a game with the likes of Unity and Unreal, on a platform like Steam.
Initially, this change empowered new devs to build interesting games. But soon enough, the market was flooded with tens of thousands of releases per year, which destroyed organic discovery. Without a marketing strategy, the chances of a game gaining traction today are basically zero, says Casey.
9. Old games don’t look dated anymore, and that’s a problem. For decades, graphics were a vital barometer for showing how videogames improved over time; a new release in 1995 was guaranteed to be visually superior to one from 1990. But a new game in 2026 likely doesn’t look much different from one that’s nine years old, and new releases face ongoing competition from older games.
10. Casey’s problem with test-driven development is the “test” bit. Casey believes tests should be a cost/benefit decision, and not put in place by default. For some projects, doing tests upfront – or doing any tests at all, in some cases – is simply a bad choice.
11. One trait of almost every great engineer: refusing to accept programming wisdom untested in the real world. As Casey puts it:
“I find there’s a lot of received programming wisdom that’s just nonsense. Clearly, no one’s ever tested it. In order for something to be received wisdom, you should have to at least demonstrate concrete upsides, but often this cannot be done. I would say focusing on what actually works in practice is a huge plus.”
12. No AI in Casey’s upcoming game. He acknowledges that many developers will disagree, but insists there’s nothing wrong with being outside of mainstream tastes, just like some people chose handmade furniture over the flatpack kind. His reasoning for omitting AI is straightforward:
“I want to program things in a game because I want to program them. If I only wanted output, I’d just get the Unreal Engine.”
The Pragmatic Engineer deepdives relevant for this episode
At a select few tech companies, they write most of their code with their own, custom-built, internal AI coding agents. This is different from most of the industry which uses AI coding agents and harnesses like Codex, Claude Code, Cursor, OpenCode, GitHub Copilot, etc. At Ramp, their own version is called Inspect, while at Block it’s Goose (open source), at Stripe it’s Minions, and River at Shopify.But why not just use what frontier labs and coding harness AI startups already offer; why take the
At a select few tech companies, they write most of their code with their own, custom-built, internal AI coding agents. This is different from most of the industry which uses AI coding agents and harnesses like Codex, Claude Code, Cursor, OpenCode, GitHub Copilot, etc. At Ramp, their own version is called Inspect, while at Block it’s Goose (open source), at Stripe it’s Minions, and River at Shopify.
But why not just use what frontier labs and coding harness AI startups already offer; why take the time and effort?
We reached out to Ramp, a fintech company big on building its internal AI infrastructure, and sat down with the founding team of Inspect and engineering leadership. We talked with CTO Rahul Sengottuvelu, Head of Engineering Hamid Dadkhah, and Zach Bruggeman, principal engineer and founding engineer of Inspect.
Today, we cover:
What is Inspect? Imagine an AI coding agent running on remote sandboxes with access to most internal data sources, and verifying all backend and frontend changes on the remote machine.
Why build your own background coding agent? Engineers and designers at Ramp were dissatisfied with third-party harnesses: they wanted to run more than a few agents in parallel – which local machines don’t support – to have better frontend tooling, and also faced demand for remote development environments.
How Ramp uses Inspect: coding, bugfixing in Slack, debugging, and building internal agents like code review and incident management on top of the Inspect platform
What makes Inspect so popular? The machine in the cloud is a developer machine, plus it has access to numerous internal integrations via API and MCP.
Inside the sandbox. OpenCode, services for development (e.g. Postgres, Redis, RabbitMQ, Temporal), Chromium, and VS Code Server. Plus, we check out smart tricks to make sandboxes spin up in five seconds or less(!!)
Collaboration & feedback. All Inspect sessions are public and open to collaboration, with no opt-outs allowed. More than 150 people at Ramp have contributed to the project.
If you’re like us, you might wonder what the point would be of building your own harness and investing the time and resources in it, given all the choices already out there. This article sets out to answer that question, to understand why other places chose a similar path, and how a non-AI frontier lab can build more efficient tooling than what the frontier AI labs offer. It looks like the “buy, don’t build” tooling convention might not apply to AI tools!
Let’s get into it.
1. What is Inspect?
Inspect is Ramp’s internal background coding agent, shipped and opened internally last November. Engineers at Ramp can use any tool they want, but 75% of merged PRs are now raised by Inspect; a clear indication that many engineers prefer the tool over others:
Inspect’s home page: showing sessions started by the user
Inspect: how the UI looks for engineers inside of Ramp
A couple of things make Inspect different from coding agents like Claude Code and Cursor:
Remote sandboxes: Inspect spins up a sandboxed remote development environment which unlocks unlimited session concurrency, centralized setup configuration, and cross-functional session collaboration.
Internal integrations: Inspect is integrated across the org with the same tools and context that a Ramp engineer has; the only constraint on agents’ ability is model intelligence, not missing tools or access.
Inspect verifies all its changes. As a remote development environment with full tooling access, it can “close the loop” and confirm the changes it makes work:
Frontend work verification: Inspect visually verifies its own work by providing screenshots and live previews to users.
At present, most third-party AI harnesses cannot do these kinds of verifications ‘out of the box’ because they lack internal integrations with things like telemetry and feature flag systems. Also, almost a year ago, Ramp built screenshot verification before it was supported by third-party vendors. Things like this placed Ramp months ahead of nearly all AI coding harnesses, and they could also build a far better feedback loop in their own harness.
Rapid adoption when background agent released
The v1 of Inspect was a Chrome extension for designers to prompt AI to make minor website changes. A few months later, the v2 version with background agents followed.
History of adoption numbers
By January of this year, just two months after the v2 launch, around 60% of PRs at Ramp were authored by Inspect, which increased to 75% by May. At Anthropic, Claude Code won rapid adoption after an internal release, as covered in the deepdive How Claude Code is built.
Then Inspect hit a neat milestone in July, crossing the one million total sessions mark:
Milestone: one million Inspect sessions
2. Why build your own background coding agent?
There are a few reasons why Ramp decided to turn down tried-and-tested products and create their own:
Local machines are limited in how many agents they can run. Ramp found third-party products below expectations; they liked Claude Code on day 1, but were constrained by only being able to run one or two sessions on local machines.
Better frontend tooling. The web engineering team wanted to improve their frontend tooling so designers could make small UI tweaks. There was an opportunity to use AI to automate themselves out of that loop.
Need for remote dev environments. As Ramp scaled, so did the complexity, and with it there was more work at the intersection of systems, like debugging backward compatibility, and broken API contracts. The solution was to create remote dev environments.
Inspect started as a designer’s frontend tool, and a good part of its team were frontend engineers with interests in UX and speedy performance. The v1 was a Chrome extension for visual edits, where a user could highlight an area and tell the AI what minor website changes to make, like copy edits and button placements. The task of building a tool for making UI edits with AI was given to two frontend engineers, Zach Bruggeman and Jason Quense, who aside from their frontend domain knowledge, brought a welcome adversarial perspective, as they were less than fully convinced by AI at that time.
People liked v1 but it wasn’t adopted because engineers already knew how to go to a file and edit a single line of code, so didn’t have a reason to use it, and it also required setting up a local development environment, making it too complicated for non-devs.
For the current iteration of Inspect (released November 2025) the team pivoted. They built Inspect v2 as a remote development environment with a coding agent on top. Setting it up as a remote environment that they could configure centrally removed the need for local setup on each machine. They were also encouraged by seeing that OpenCode, the open-source coding agent which serves as Inspect’s harness, exposed an HTTP API which made it straightforward to set up, and was open-source, good enough, and importantly, offered model agnosticism.
After pivoting, adoption skyrocketed to where it is today:
Daily unique human Inspect users
Adoption numbers today:
75% of all merged PRs come from Inspect sessions
~90% share of PRs merged into the Inspect repo come from an Inspect session
Under 5 seconds to spin up a fully provisioned remote dev environment
5.5 people in the Inspect team: four engineers, a director, and part-time PM
150+ engineers at Ramp who have contributed to the Inspect codebase
3. How Ramp uses Inspect
Having built it, Ramp uses Inspect for a few things:
Coding: obvious use case; engineers prompt Inspect with small and medium-sized coding tasks that can often be one-shot passes. For larger, more complex tasks, devs often use Inspect to kick-start an idea and then take over developing it locally.
Bugfixing in Slack: the @inspect fix this prompt in Slack. Inspect reads all the thread context and raises a pull request (PR) with a fix.
Debugging: Inspect can do things like debug the code (stepping through the code in debugger mode), query the sanitized read-only prod DB replica, find business logic/data mismatches.
Using Inspect to build Inspect: Inspect is used to build itself, and more than 80% of Inspect is written in Inspect sessions.
Platform for agents: Engineers at Ramp have built more than 200 agents running on top of the Inspect platform
Here’s an example of how debugging works. Devs can ask the agent to investigate an issue, and Inspect goes off and pulls data from the correct sources:
Debugging with Inspect: asking the agent about an incorrect allocation. Debugging is done via the web chat interface
The tool goes and makes database or Snowflake queries when helpful:
Making database and Snowflake queries
The debug agent can be long-running while it gathers data from various sources. Finally, it presents its findings:
The debug agent found the root cause: in this case, it was a routing/policy decision, discovered by querying relevant data sources
This debugging example illustrates how much more capable agents can be with the correct access to tools, data, and context.
Some internal agents built on top of Inspect:
ReviewBuddy: Ramp’s own code review system, customizable per team. The difference from third-party AI code review tools is that it’s very aware of Ramp’s context, and the team found it to work better than third-party tools. Built by a single engineer in a week.
Oncall Assistant: connected to all production and observability systems. When the agent detects an incident, it gathers all relevant context and tries to determine the cause. The oncall engineer can choose to join the Inspect session and prompt against this proposed fix.
Testo: a frontend QA tool and browser-based agent that clicks around like a user would, and creates Playwright tests.
Ramp Research: the company’s agentic “data analyst” is connected to all Ramp’s data sources, like Looker, Snowflake and dbt tables. Ping it from Slack about any topic and it gets answers. Before Ramp Research, engineers and data analysts had to know which data tables to query and join. Ramp previously shared more about its Research.
Voice of the Customer: connects to several customer feedback sources like chat, email, App Store reviews, etc. It collects feedback from the last 90 days, and allows prompting against them as a Slack bot
Error automations: automatically create draft pull requests based on alerts from Sentry or Datadog.
Visualized:
Most agentic automations inside Ramp are built on top of Inspect
It’s clever that the Ramp team extended Inspect into a platform, and made it easy to build additional agentic tools, without engineers having to worry about the cloud backend for those tools. Not bad for a tool that started as a simple Chrome extension almost exactly a year ago!
4. Architecture and tech stack
Inspect’s core principle is that agents should have access to the same context and tools as software engineers.Hooking up Inspect to the data sources that engineers would browse with the same tools seems to be a key difference between Inspect and third-party AI harnesses.
The Pulse is a series covering events, insights, and trends within Big Tech and startups.Today, we cover:More on the “great engineering leader career break.” The industry is changing fast, and the VPE and CTO roles also need to adapt. And don’t forget that these are the roles from which you can drive change that reorganizes engineering in ways that work better.We need to talk about migrations with AI. Asana needed to migrate off testing framework Enzyme, but it meant doing a massive rewrite of t
The Pulse is a series covering events, insights, and trends within Big Tech and startups.
Today, we cover:
More on the “great engineering leader career break.” The industry is changing fast, and the VPE and CTO roles also need to adapt. And don’t forget that these are the roles from which you can drive change that reorganizes engineering in ways that work better.
We need to talk about migrations with AI. Asana needed to migrate off testing framework Enzyme, but it meant doing a massive rewrite of test cases. With AI, the project was completed in two weeks: without AI, this work would surely have been kicked down the road. Airbnb and Uber share similar stories, and AI seems like a superb fit for framework migrations.
Are AI startups making the Gartner Magic Quadrant irrelevant? Gartner ranked AWS, Microsoft and IBM above Anthropic, Cursor and OpenAI in their “AI code modernization tools” ranking. This is most likely because the first three pay large sums of money to Gartner, but AI labs and vendors refuse to pay this “Gartner tax.”
Industry Pulse. Another hours-long GitHub outage, GitHub alternatives are here and fighting for market share, Slack launches Slack Code, text generated by Claude to be watermarked, and Uber open sources SubmitQueue.
Before we start: apologies for the numerous typos last week. My editor, Dominic, was on vacation, and numerous typos made it through the spellchecker. A reader asked for cute puppy pictures to accept my apology, and so I updated the post with pictures of our 3-month old puppy.
1. More on the “great engineering leader career break”
Stream the latest episodeListen and watch now on YouTube, Apple, and Spotify. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.Brought to You by• Antithesis – verify your system’s correctness without human review or traditional integration tests – and avoid bugs or outages. Teams like Jane Street, Fly.io, and the etcd community use Antithesis to ship better code, faster. Learn more.• Sentry – application monitoring software built by developers, for
Listen and watch now on YouTube, Apple, and Spotify. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.
Brought to You by
• Antithesis – verify your system’s correctness without human review or traditional integration tests – and avoid bugs or outages. Teams like Jane Street, Fly.io, and the etcd community use Antithesis to ship better code, faster. Learn more.
• Sentry – application monitoring software built by developers, for developers. Sentry’s Seer AI agent is one of their new, neat tools, which I’ve used as a way to quickly fix errors on my backend. Check out Sentry.
Addy Osmani spent more than 14 years at Google, working on Chrome, DevTools, Core Web Vitals, and most recently, AI developer experience.
If you’ve ever opened Chrome DevTools, or optimized a page for Core Web Vitals, you’ve used software built by Addy Osmani. In this episode, I sit down with Addy and we talk about his path from building a web browser aged just 16 to becoming a director at Google. We discuss what he learned from building tools for millions of developers, Google’s engineering culture, and why he continued doing hands-on coding work as a manager. We also get into how he works with AI agents today, the risks of ‘cognitive surrender,’ his approach to ‘loop engineering,’ and why it’s good to develop skills in product management, go-to-market, and other areas.
Takeaways from the conversation with Addy
Here are eleven interesting points from the chat with Addy:
1. Addy built a web browser from scratch, aged just 16. Back then, a pain point was that Addy had to carry floppy disks to his local library to download data. To speed up browsing, he built a browser that opened multiple connections when fetching webpages.
2. Publishing free educational materials helped Addy land a job at Google. A documentary about Google which he watched as a youngster made Addy want to work somewhere like it. Later, Google noticed his work in publishing educational resources about frontend and JavaScript development. The company reached out about a DevRel-and-builder role, and Addy was hired to join the Chrome team.
3. Chrome DevTools was an effort by Google to meet web developers in the browser. Today, DevTools is one of the closest things Google has to an IDE (not counting Antigravity, that is), but the project started as a way to add tools to the browser to help debug web applications. As web engineers started to use more complex frameworks and build chains, DevTools added capabilities like source-map-aware debugging, hiding library code, mobile device emulation, tooling for service workers, and more.
4. Most developers don’t understand memory management. Addy says this is because memory debugging tooling has not advanced in a decade, and remains a hard problem to solve. This is despite making improvements in runtime performance debugging in Chrome DevTools (flame graphs and deep tracing).
5. Becoming accountable on a weekly basis for a top company goal is the biggest difference in a director of engineering at a major tech company. Addy worked his way up from engineer to Director of Engineering at Google, and I asked what the biggest change was when he made it to that level. Being on the hook and reporting regularly on a top company goal was something he found entirely new, Addy said.
6. A big culture shift at Google in the last two years has been VPs and SVPs coding on weekends. Naturally, this is because AI tools make coding much easier. During his last two years at Google, it was common for these folks to talk about their weekend side projects and tools they used to build them.
7. A big risk of AI-assisted development is cognitive surrender. Addy defines cognitive surrender as the erosion of your comprehension of the problems being worked on, and of your own memory of what’s going on. He recommends pushing back against this by understanding every major decision an LLM makes. Unfortunately, his former method of reading the AI’s entire reasoning process is no longer practical given how much output agents can generate, but you’ll still want to understand the most important decisions.
8. Aim for mutual amplification when using AI tools. The aim is to do two things simultaneously:
Help the agent improve throughout the task by having it log its decisions and key learnings
You also improve by reviewing, understanding, and internalizing what the agent does and how you can learn from it
9. Addy believes software engineers will always be important because an AI model cannot be accountable. Accountability for code and software is possible even if the accountable party didn’t write the code, as is the case in projects like Chromium, where designated engineers own parts of the codebase. They’re responsible for approving and rejecting contributions, and for shaping that part of the codebase. Addy reckons that a “what am I accountable for?” mindset will be adopted by many software engineers.
10. Addy is bullish about software engineering’s outlook. Every time the profession has made it easier to create software, we’ve created exponentially more software. Addy predicts the same will happen with AI, and that the total addressable market of people building software will get much bigger.
11. Advice on where to invest efforts as engineers in the coming years. In his words:
“What we are very likely to see happen next with engineering careers (as well as product and other roles) is the unbundling of them, so that an engineer also has product sense, while a product person also has engineering sense, or UX sense.
[You should] think about the non-engineering things if you don’t [usually] have the time to think about product or technical evangelism, or go-to-market approaches, or any other parts of how businesses are successful.
If you can show employers that you are not just a builder, but someone that can help them as roles start to become a little bit fuzzier, then I think that you can be successful in these times. Don’t be just an engineer.”
The Pragmatic Engineer deepdives relevant for this episode
In my ~20 years in this industry, I’ve not seen as many capable engineering leaders opting out or taking prolonged breaks as now, with some high-ranking engineering leaders – CTOs, VPs of Engineering, heads of engineering, etc. – quitting their high-status roles and departing, if not into the sunset, then at least with nothing lined up.To find out what might be behind this spate of sign-outs, I talked with almost 20 engineering leaders currently on a career break – or seriously considering one –
In my ~20 years in this industry, I’ve not seen as many capable engineering leaders opting out or taking prolonged breaks as now, with some high-ranking engineering leaders – CTOs, VPs of Engineering, heads of engineering, etc. – quitting their high-status roles and departing, if not into the sunset, then at least with nothing lined up.
To find out what might be behind this spate of sign-outs, I talked with almost 20 engineering leaders currently on a career break – or seriously considering one – and they let me into their personal reasons for deciding to jam the brakes on their careers. Thanks to everyone who shared their input!
Today, we cover:
Ten of the most common reasons for quitting, sometimes without the next gig lined up:
1. The job got (much) worse
2. The startup is “losing” and becoming worthless
3. Not being AI-native enough for other skills to be relevant
4. Their predecessor saw the “writing on the wall”
5. Long hours – rarely decisive
6. Smaller teams mean less need for leaders
7. Fractional CTO work preferred over fulltime positions
8. AI startups pay ICs more than non-AI startups pay executives
9. Quitting to launch their own business
10. Burnout
“Founder mode” looks here to stay, so how to deal with it? And has it made the CTO and VPE roles become “low ROI”?
‘Work at companies that truly want to drive change’. A personal account from someone who took the VP of Engineering role at Gitpod (later, Ona, now acquired by OpenAI) and enjoyed a rewarding experience. Matt Boyle says he interviewed the employer beforehand on whether their business truly leans into the changes brought by AI.
“Just me?”
I was recently messaged by a head of engineering in San Francisco, who said:
“I’m talking to four startups in San Francisco about the head of engineering roles. Pretty normal.
But one interesting pattern is how founding CTOs/heads of engineering are stepping away to take a full career break. We’re talking about two of these four startups. And these are good startups!
Have you seen this trend? I have a small number of data points here, so you might have a broader view.”
I asked around privately, and it turns out a majority of the CTO-level folks I spoke to are considering the very same thing, or are actually in the process of leaving the office for a long spell away; 6/10 engineering leaders said they’re on the way out.
1. The job got (much) worse
Unrealistic expectations, including about AI, by founders and CEOs are the leading cause of jobs turning bad for CTOs and VPEs right now in 2026:
CTO expected to magically transform the company to be “AI-native”
CTO must make significant engineering cost cuts of up to 20-50%, including morale-sapping job cuts
“Do more with less” equals shipping more with fewer people (e.g., no backfills)
CTO faces pressure on business results as AI coding bills rack up
Founder slop: they want wonky AI prototypes shipped as full-blown products within weeks
Hands-on founders with “AI psychosis” make the job predictably harder, according to one CTO who just signed out of his job:
“Managing ‘AI psychosis’ with founders and executive peers has become very difficult. For example, what do you do when a founder ships a 60,000-line pull request into the product, gleaming with joy at how much more productive they’ve become with AI? They won’t see all the issues with that PR, and how do you bring up that they’ve created a massive amount of tech debt? Especially without looking like a ‘Debbie Downer’.”
Founder slop issues begin when top leadersgetexcited about AI’s capability, then get hands-on and start issuing PRs, and shipping code to production. It can cause issues across the board:
Accountability. Who’s oncall when founder-shipped code breaks? In the “you build it, you own it” culture of startups, it’s confusing when a founder gets hands-on while not owning their work.
Quality out the door: if a founder’s half-baked features are accepted, it sends the wider message that quality does not matter. Some people may adopt this attitude to their own work.
A founder can overrule whatever was previously agreed with the CTO or VPE about what to build next. Vibes the founder has or feels are reason enough.
Another way that leadership roles have diminished is that craft and quality are less important, says a VP of engineering who’s in the process of signing out of their job:
“Shipping software became all about speed. Finding differentiation with your product in the market is brutal, and speed / go-to-market becomes the biggest differentiator. Craft, quality, and care going into the product are taking a backseat.”
Things also go bad when companies don’t ‘get’ AI+engineering, except as a way to cut jobs. CTOs I talked to mentioned the likes of Ramp, Stripe, and Notion as places that understand how to integrate AI into the engineering culture with a growth mindset without forsaking quality. Elsewhere, bad vibes dominate at places where going all-in on AI leads to the cynical conclusion that product management, design, and engineering leadership are irrelevant.
2. The startup is “losing” and becoming worthless
Director+ roles have a few differences from individual-contributor engineering ones:
Larger equity stake in the business. Base salary at these levels is often similar to a staff engineer’s, but usually with more generous equity grants – especially at the VP of Engineering and CTO levels. A good financial outcome depends on the company becoming more valuable, and – in the case of private companies – having a good exit by being acquired or selling shares.
Understanding of the business and competition is a baseline. At Director+ level, a big part of the job is making strategic decisions that grow the business and help the company get ahead. It’s a nice-to-have for an engineer to possess business acumen, but director-and-above folks use it much more than most individual contributors (ICs). Great engineering leaders are good at understanding business performance and outlook.
A company that adopts AI rapidly usually falls into one of three buckets:
“AI-native”, building & selling AI products. The large AI labs and a select few “AI-native” startups are thriving, but many AI startups with VC funding struggle. Engineering leaders know this, and that their equity – usually issued as options – could end up worthless.
Software startups threatened by AI-native businesses. Good businesses in the pre-AI world can be threatened by AI today, like SaaS startups selling seat-based products in areas where agents are taking over the functionality. They have to pivot their businesses or seek an exit. Bending Spoons buying Airtable for less than the company raised is an example of a business threatened by AI and choosing to sell, instead of pivoting the whole business.
Unaffected by AI. Usually stable businesses which do more than software, such as with a real-world side to the operation like manufacturing or distribution.
The majority of software startups fall into one of the first two buckets of being AI-native or under threat. Senior leaders at such companies are in a good position to evaluate whether their company is a “winner” worth staying with.
Leaving due to equity becoming worthless
A CTO who quit their startup told me:
“My company would have needed a massive exit for me to realize any upside. I had an equity grant that was 2% of the common shares. However, this equity was behind an already steep preference stack for investors, post Series A.”
This CTO had a very generous equity grant at 2% of shares, so what made him leave it behind? They laid out how it will be difficult to get any benefit from them because the shares are most likely rendered worthless by rules about the order in which different investors get their share of the pie:
Assume that this company raised a $10M seed round at a $50M valuation, then a $100M Series A at a $500M valuation. So, a total of $110M was raised across two rounds.
Investors typically have a 1x preference. 1x preference would mean that upon any sale, they get the first $110M of the sale.
But in this company, the Series A investors negotiated a 2x preference: so upon a sale, $210M goes to investors first ($10M to the Seed, and $200M to the Series A investors).
The company now needs to sell for at least $210M for common shareholders (like the CTO) to make any money!
If the CTO does not believe a $200M+ exit could happen, then their equity is worthless. A $200M+ exit is typically an acquisition, because a stock market flotation rarely happens at below a $10B+ valuation, these days.
If a VC-funded company does not have the revenue or customers to grow at a fast tick (circa 20-50% per year), then it’s often a struggle to raise the next round of funding, and the business’s actual value usually shrinks to 3-5x of annual revenue. So, if a startup is making $10M per year after raising $110M in funding, and growing 30% year-on-year, then the company is likely worth around $30-50M. Perhaps the right buyer would pay $100M, but if growth slows, the value is likely to drop.
An experienced CTO who takes a step back and assesses things can realize when there’s a high chance of their equity turning into smoke, removing a reason to not sign out of the job. It’s what happened to the CTO above, and when they couldn’t turn the business around, they quit.
Business stops growing
When a VC-funded startup’s business stops growing, the prognosis can be dire in the sense that it’s unlikely to be worth as much as in the previous funding round. This is true even when the startup becomes profitable: this might mean it could theoretically go on forever; but with slow or no growth, it won’t win in another VC funding round.
Here’s a VP of Engineering who saw their startup stop growing, partly due to wrong bets by the CEO:
“My founder/CEO was nontechnical, and was both moving too slow and too fast with AI.
Too slow, as in they did not take the time to understand what our customers wanted. We built a TON of AI stuff, it totally confused them, they churned, growth stalled, word-of-mouth growth was gone. Heck, I don’t think our customers ever wanted or needed anything with AI!
Too fast, as in they deprioritized core systems’ reliability in favor of shipping AI work to prod which did not have any commercial potential. So, our core offering started to have more outages and we lost customers because of this as well.”
I’d add that deprioritizing reliability in favor of building features may be sensible in the early days. The problem seemed to be that this company had not found product-market fit, and the new AI features didn’t resonate with customers. Basically, the CEO lacked customer understanding, business intuition, or both.
So, good on the VPE for getting out when they saw the direction of travel. If the CEO won’t accept input from the VPE – who would’ve at least prioritized reliable operation – then there isn’t much left to stick around for!
3. Not being AI-native enough for other skills to be relevant
The top-paying engineering leadership positions have one thing in common: experience of leading AI-native organisations is expected, and leaders are sought who have turned their current company AI-native, or work at such a place.
It’s new to see people signing out of large companies for feeling like they’re lagging behind in adopting new AI workflows. An ex-engineering director at a large bank told me they quit their job to accelerate their career:
“I was not getting the opportunity to ‘close the loop’ on hypotheses enough. [...] To stay relevant in the industry, I feel like I need to pull out into the “fast lane.”
Like many others, I see the future of software development is with AI. If you don’t get hands-on with your team, working with AI tools day-in, day-out, you’re falling behind.
My plan is to get on the cutting edge of things through a mix of academia and consulting AI companies. I am not saying the plan is perfect, but I need more time to do things differently than I had in my job.”
Consider this: if you stay in your job for two more years, do you expect to find career opportunities at cutting-edge companies in the future? If the answer is “no”, then there’s a risk in just staying put. Joining an uncertain startup or taking a career break to develop AI expertise is also risky, but the outcomes may be more controllable than letting your skillset become outdated, relatively quickly.
But it might actually be necessary to quit in order to get AI experience: you might be able to get this by transferring to an IC role. As Charity Majors, co-founder and CTO of Honeycomb, said in last week’s episode of The Pragmatic Engineer podcast:
“You’ve got to get AI on your resume. You just have to. If you don’t, this is a huge career risk. If you’re working somewhere where you’re not getting these skills, I would do whatever I could to change that [including taking an IC role within the company].”
There are companies where moving from Director+ to individual contributor is possible, even if these companies are the minority. If you happen to work at a place like this: consider if you can and will take advantage of this opportunity.
Most companies say they want to be AI-native, but never do
Claire Vo – founder of ChatPRD and host of ‘How I AI’ podcast, and the former Chief Product & Technology Officer at LaunchDarkly – says most companies will never become “AI native” simply because most VP of Engineering or CTO folks don’t have what it takes to pull off such a transformation. In her words:
“The VPE role used to be primarily about deploying the dark arts to defend engineers from the roadmap, and now everyone thinks that’s BS and leaders are under tremendous pressure to inflect velocity or GTFO (get the f*** out).
Engineers are unhappy (don’t make me tokenmaxx, bro!), product and design sending slop PRs, and everyone good has left for a lab.
Most of these companies’ EPD (Engineering, Product, Design) orgs will never go AI-native, not even close. Most VPEs aren’t good enough at change management to pull it off.”
It looks like there’s a deadlock:
The current engineering org is frustrated by how AI is making engineering culture worse, morale is down, and people are frustrated and confused
To resolve this, drastic changes are needed to how everyone (engineers, product, designers) works
To pull it off, a VP of Engineering or CTO is needed who’s capable of this; someone excellent at change management, who’s ideally done it before.
But most VPEs and CTOs are not experts at large-scale change management, nor have done it before.
According to this, many VPEs and CTOs are doomed to fail at making the change they want, and it’s hard to know if that’s because organizations didn’t support them properly or resisted change.
4. Their predecessor saw the “writing on the wall”
There’s (usually) a honeymoon period in a new job, when we believe in the business we’ve joined and in its direction. But when this phase passes, a fraction or all of the problems described above may emerge, and there’s a decent chance that some of them are why your predecessor signed out:
Has AI helped make the role worse?
Is the equity on course to be worthless?
Is getting AI-native experience actually possible, or is the organization resisting change?
I’ve talked with a CTO who replaced their predecessor and founding CTO. A few years into the job, the predecessor CTO realized their equity in the business was worth almost nothing due to stalled growth, all while they were also being out-competed by AI-native rivals. So, the new CTO also resigned after a short, six-month tenure.
5. Long hours – rarely decisive
Two engineering leaders – a CTO and a VP of Engineering – mentioned “insane working hours” as a factor that contributed to them finally quitting. But there were other things as well:
The business struggling for growth
Their equity grant’s value shrinking to nothing before their eyes
CEO/founder ignoring or overriding efforts to help the business succeed
My sense is that at a thriving business during chaotic times like these, it’s unlikely that long hours alone would spur people to leave, if their contribution to current success counts and is valued. When things are going well, it’s possible to delegate more and take time to recharge batteries. But when things are going badly, it feels like every waking hour needs to be spent on working to turn things around.
6. Smaller teams mean less need for leaders
Several engineering leaders are stepping back into IC roles for more stability because engineering teams are smaller now.
Karthik Hariharan, engineering leader at DoorDash, notes:
“Expectations have been shifting a lot in these roles, and a lot of folks qualified for them have consciously been stepping back into IC roles or joining bigger companies for stability and better compensation.
Engineering teams are also smaller now. A VPE isn’t needed until the team is large enough to require it. A technical founder can run the team for a lot longer these days.”
Some reasons why engineering teams have shrunk:
“Fullstack engineer” is mainstream, and was even before AI. Fullstack engineering was becoming relevant a few years ago in terms of a single engineer working on both the front and backends, instead of having a frontend engineer building the UI, and a backend engineer working on backend services. Fullstack frameworks like Next.js or Ruby on Rails made all this pretty easy before AI. Today with AI coding agents, you can rely on them to write decent code on platforms you’re unfamiliar with. There’s now little to no reason why a project would need multiple devs with different specializations.
It’s normal for one, or a maximum of two fullstack engineers, to be working on any given project at Anthropic as well. Head of Claude Platform, Katelyn Lesse, shared how it works at Anthropic:
“On an individual project, you often cannot have more than two people working on it.
This is because each engineer is already running several agents. And so as an engineer, you’re already fighting against your agents, which are stepping on each other’s toes on implementation. And in this setup, you just cannot have that many humans, who also come with all their agents!”
Frontend-only and native mobile teams are also getting smaller or disappearing. Even at companies where iOS and Android are a big part of the business, more places are building using cross-platform technologies where one engineer can do the work that used to need several. For example, social media app Bluesky had a single engineer build its web, iOS, and Android apps for launch by using React Native and Expo. Bluesky later hired more people to work on the web and apps, but they all work across these three platforms. It’s not the same as hiring separate web engineers, iOS engineers, and Android engineers.
Demand for frontend engineers and native iOS+Android engineers keeps dropping with the trend of smaller engineering teams. Source: The tech jobs market in 2026
Tech companies have been flattening their org structures for three years now. We first covered the trend for fewer middle managers back in 2023, when Meta drastically reduced manager positions. The trend has not stopped, and many – if not most – companies have increased the number of reports each engineering manager has, while reducing the number of layers in their organization.
7. Fractional CTO work preferred over fulltime positions
The Pulse is a series covering events, insights, and trends within Big Tech and startups.Today, we cover:Meta can’t stop the “resignation-wave” it triggered. In what was predictable: Meta’s layoffs and forced reassignments pushed engineers not impacted by either to look for a new job. Meta is now offering large equity retainers to keep these folks, and it doesn’t seem to be working.Grok Bot: the “OpenClaw moment” for managed AI agents? The Cursor team built and released a generic AI harness that
The Pulse is a series covering events, insights, and trends within Big Tech and startups.
Today, we cover:
Meta can’t stop the “resignation-wave” it triggered. In what was predictable: Meta’s layoffs and forced reassignments pushed engineers not impacted by either to look for a new job. Meta is now offering large equity retainers to keep these folks, and it doesn’t seem to be working.
Grok Bot: the “OpenClaw moment” for managed AI agents? The Cursor team built and released a generic AI harness that feels like the “Codex experience, but for knowledge work.” I tried it out, automated a lot of my daily workflows, and am a massive fan. More AI vendors will surely copy this harness.
Apologies for this week’s The Pulse arriving a day later than usual – our family got a puppy this week – who is beyond adorable –, but has kept me up a few nights, indirectly delaying this week’s The Pulse. We’re getting into a rhythm, so things should be back to normal, looking ahead.
Stream the latest episodeListen and watch now on YouTube, Apple, and Spotify. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.Brought to You by• Antithesis – turbocharge testing of your systems by running your whole system under aggressive fault injection. There’s good reason teams like Jane Street, Fly.io, and the etcd community rely on Antithesis. Learn more.• Buildkite – the CI platform trusted by OpenAI, Anthropic, Cursor, Meta, Uber, NVIDIA,
Listen and watch now on YouTube, Apple, and Spotify. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.
Brought to You by
• Antithesis – turbocharge testing of your systems by running your whole system under aggressive fault injection. There’s good reason teams like Jane Street, Fly.io, and the etcd community rely on Antithesis. Learn more.
• Buildkite – the CI platform trusted by OpenAI, Anthropic, Cursor, Meta, Uber, NVIDIA, Airbnb and many more. When CI volume becomes an architecture problem, you deserve better CI. Engineered to reliably manage whatever your coding agents throw at the build queue: today, next year, and beyond. Learn more.
• WorkOS – make your app and agents Enterprise Ready, with SSO, SCIM, RBAC, and more. Get started.
In this episode
In 2025, it was rational to be skeptical about AI, but in 2026 it’s clear that AI is changing all of the industry, and there’s less and less place for skepticism. This take is from one of my favorite voices in software reliability and observability: Charity Majors, CTO and cofounder of Honeycomb, co-author of Observability Engineering. (Note: the second edition of Observability Engineering is out, and it’s pretty much a full rewrite of the book, I recommend grabbing it if you’re building reliable systems)
In this episode, I sat down with Charity to discuss how her thinking on AI has evolved, why she believes it is becoming a foundational part of software engineering, and what that means for how teams build, review, and ship software.
We explore how AI is changing the economics of code generation, why reliability and verification are increasingly the bottlenecks, and why the rise of non-deterministic systems requires more engineering discipline. Charity shares her views on code reviews, observability, DevOps, leadership, and why both AI skeptics and enthusiasts are getting important things right.
Takeaways from the conversation with Charity
Here are 13 parts I found especially interesting, talking with Charity:
1. In March 2025, Charity told the audience at SREcon to try vibe coding, and back then, the response was grumbling. Charity’s point was that people who are skeptical of AI should still learn to use it, because you can complain better if you’ve learned it. At this time, Charity still saw AI having a bigger impact than a new programming language, but was skeptical that it would have a generational impact.
2. Charity’s turning point in seeing AI as a generational change was in November 2025. This was due to Opus 4.5, but Charity argues that the coding harness (Claude Code) made the bigger difference. Because thanks to Claude Code, harnesses went from being more of a shell script to serious infrastructure.
3. The impact of AI on the industry in 2025 was similar to the impact of the cloud in 2010. Looking back, Charity is comfortable saying this: in 2010, it became clear that cloud computing was certainly going mainstream and would change the infra-layer. After 2025, it’s also clear that AI will have a similar impact on the infrastructure of building software.
4. Engineers who were skeptical of AI up to 2025: they had good reason to be so. This was because we’ve seen plenty of technologies and innovations in the past that all promised to transform the software industry, but later fell short. Examples include COBOL (a technology promising that programmers would no longer be needed to create software), neural nets, no-code and low-code tools.
5. The question engineers need to answer: what would it take for you to be fully comfortable shipping code you have not read? Charity believes it is a “when” and not an “if” that professional software engineers will ship code they never looked at – and thus do not understand – to production. Engineering is building the systems that validate this code, and allow shipping with full confidence.
6. AI could have the software industry go through the “pets” to “cattle” change that compute infra went through in the 2010s. Up to now, writing software from scratch was far more expensive than editing existing software. But now, generating hundreds of variants of a function can be done faster than how long it would take you to hand-write it once.
Charity believes that we might be at the beginning of the transition from “pets” to “cattle” that happened at the hardware infrastructure layer. Before the 2010s, configuring and repairing individual servers was commonly done. But with tools like Terraform and Kubernetes, individual servers having issues are no longer fixed up: they are re-created instead. Charity thinks the same might happen with code, sooner rather than later. When there’s an issue with the code, generate new code that solves it, and is verifyably correct.
7. Her contrarian take: code review is overrated, and the least valuable part of what humans add to software engineering. Charity says that humans are good at conversations and deciding what to build, not reading code to check for correctness, syntax and bugs.
8. Charity’s verdict of 20 years of DevOps: it failed. The DevOps feedback was about trying to create a feedback loop that connected people writing the code to the code running in production. She thinks that the “ops people: learn to code!” wave worked, but the “software engineers: understand your code in production” failed, to this day.
9. Non-deterministic systems require more engineering discipline versus before. With code written by AI, we’re reducing the trust in the code (because we no longer wrote it), so we need to increase trust at the other part of the development process. Specifically, at validation: with things like tests, evals, and conformance testing.
10. Charity’s career advice for engineering directors: run towards the waves, and get AI on your resume, immediately. It’s an anxious time to work in tech, thanks to all the change, driven by AI. Charity reminds us that anxiety and excitement are physiologically almost the same, but the difference is agency. When you have no agency, you’re more likely to get anxious, and when you do, you’re more likely to get excited.
So her advice to anxious engineering directors: consider going back to IC work, where you’ll have far more agency. IC work is well-respected, getting back to it has never been easier, but the window to do so is closing. As she put it:
“The next time you’ll have a job interview, you’ll be filtered out if you don’t have AI experience.”
11. On AI fatigue: take back control with small acts! We talked about various types of AI fatigue: reviewing AI slop, getting tired of the AI hype, and getting worn down by “doom trolling” by AI CEOs. Charity finds small acts of taking control back in your work from AI tools help. For example, none of the Honeycomb team uses AI on Wednesdays.
12. Charity would like to see both the “AI-pilled” and the “anti-AI” camps tell the stories better. As she put it:
“There are some really incredible things happening in software right now, for example, with rewrites and with automating away toil. Not a single person that I’ve talked to would give up using AI.
But half of the people are seeing the wins, and they’re not connecting it to the cost, which makes them think that their coworkers are just afraid of getting automated out of existence.
So that’s my beg to everyone who listens to this: tell the whole story! Talk about the costs as well. We’re all in it together.”
13. Charity’s rule on AI writing: do not send any message/email to a human that you yourself have not read in full. She also says that it would take them longer to read whatever you send than it took you to produce it: it’s probably slop!
The Pragmatic Engineer deepdives relevant for this episode
Before we start: I’ll be in New York, on 15 September, presenting the keynote at LDX3 New York, doing a book signing, and hanging out with attendees. The focus of the conference is engineering leadership at a time when things are moving very fast. See the full agenda and get tickets. If you’ll be around – hopefully catch you there!The Pragmatic Engineer is back from our summer break. We resume with a detailed deepdive about the trading industry, and interesting engineering challenges that come w
Before we start: I’ll be in New York, on 15 September, presenting the keynote at LDX3 New York, doing a book signing, and hanging out with attendees. The focus of the conference is engineering leadership at a time when things are moving very fast. See the full agenda and get tickets. If you’ll be around – hopefully catch you there!
The Pragmatic Engineer is back from our summer break. We resume with a detailed deepdive about the trading industry, and interesting engineering challenges that come when working at a company that has no external customers, but where a single, unfortunate enough software bug could wipe out the whole company.
In tech recruitment, proprietary trading companies have a particularly high bar and typically offer compensation on a par with, or even exceeding, Big Tech; right at the top of the market. That’s because for these market makers, success is all about gaining a competitive edge over rivals. Such competitive advantages today includes software that is superior to that at their competitors.
Software engineers tend to know little about trading companies – and this piece aims to change that. Trading companies build bespoke hardware stacks and have larger platform engineering teams than most workplaces. For software engineers, it’s a lucrative niche in terms of compensation, full-stack (hardware to software) work and for engineering challenges, and so we decided to go deeper in this interesting area.
In order to find out more, The Pragmatic Engineer sat down with a leading proprietary trading firm, Optiver. Headquartered in Amsterdam, they also have a large engineering presence in the US and globally. We met engineers and engineering leaders to learn in depth how engineering works in a modern trading business, with contributions from:
Thanks to everyone at Optiver for taking part in this report which lifts the lid on how software engineering is done when even nanoseconds can count. In this article, we look into a software engineering environment that’s distinct from what you expect at most startups and Big Tech. For example:
No external customers. Usually, companies have consumer customers (B2C), business customers (B2B), or both. But not trading houses like Optiver, where their own business is the customer. This is a different reality: there’s no external deadlines and related pressures, but personal motivation to improve is highly valued.
Latency: “enemy number one”. Nearly every major engineering decision at Optiver is made in the interest of minimizing latency – the amount of time between a request and response. This approach is present across the software stack and in kernel-level work. It’s why Optiver manufactures its own hardware.
Today, latency is the floor, and AI models are becoming a differentiator. Gone are the days of having lower latency than the competition allowing for arbitrage opportunities to make risk-free profits. Instead, information models are becoming a differentiator: slow models with a fast trigger sending signals to execute trades, and fast models running at the edge of the network making trade decisions realtime.
Haunted by a bug that nearly killed a business. Among trading houses, there’s a cautionary tale of when a peer company, Knight Capital, nearly went bankrupt after a single bug in a high-frequency trading system triggered a $440M loss.
Different incentives. The business is incentivized to move very fast, but with a high premium on caution in order to avert potential financial disasters on the market. This cautious attitude to risk in concert with chasing speed feels pretty distinct in tech.
I this deepdive, we cover:
Overview of trading & hedge funds. Categories of trading companies, high-frequency trading (HFT), plenty of ML & math, and AI labs poaching HFT talent
Engineering organization. How trading-specific roles work together, platform engineering, the “build and own” culture, and more.
Software tech stack. The three-layer tech stack, languages and tools, CI/CD stack and the data layer.
Hardware engineering, FPGAs and Silicon. Latency progression, custom FPGAs, custom hardware, AMD hardware partnership, and more.
Network & physical infrastructure. Physical infrastructure, dedicated fiber & wavelength leasing, optical cable, radio, data centers & co-locations, and why AI models matter more than ever before.
AI at Optiver. AI tooling stack, future of agentic coding, details about adoption, and how it all looks in practice.
Hiring, career development & culture. Engineering levels at Optiver, goingfrom hiring mostly juniors to hiring experienced engineers today, competition during hiring, and the onboarding feedback loop.
We’re delighted to publish this report, including details never shared before. Let’s dive in!
1. Overview of trading & hedge funds
Here’s a summary of the world of ‘prop shops’; another name for firms like Optiver that invest their own funds in trading financial assets. Below are some useful mental models for understanding the sector.
Sell side: firms sell services or products such as advice, underwriting, research, execution, etc. These are usually investment banks and broker-dealers.
Optiver is on the “buy side”, as a prop shop.
Sources of capital
Trading categories based on capital source
Based on whose money is being traded, there are three main capital sources:
Investment banks serve corporate and institutional clients by raising capital, advising on deals, and executing trades on their behalf. Examples: Goldman Sachs, JPMorgan, Morgan Stanley.
Hedge funds raise money from external investors and trade it on their behalf, charging management & performance fees. Examples: Citadel, Millennium, Two Sigma, Bridgewater.
Proprietary trading firms trade only their own capital, with no clients or external funding. Examples: Optiver, Jane Street, Jump Trading, DRW, Hudson River Trading.
Trading eras
Optiver’s CTO US Alex Itkin pictures the evolution of trading as having unfolded across four eras to date:
Pre-electronic (pre-1990s). Trading was done face-to-face on noisy trading floors and by phone. Prices were shared on reels of ticker tape and printed in newspapers. Investors contacted brokers to place orders.
First wave of electronification (early/mid 1990s). Financial markets moved onto computer screens but orders were still entered manually.
Automated trading (late 1990s to ~2015). Computers did the same as human traders, but faster and at scale. This was the “mechanical” automation era of building automated workflows without data-driven decision-making.
Quantitative trading (~2015 to present). Data-driven decision-making with machine learning models and inference compute, with human decision-making in some key areas.
Each era “weeded” the market. Some companies excelled at automated trading but never made the leap to quantitative trading. According to Itkin, competition has got tougher over time, while the number of serious players has decreased. Today, there are only a handful of really big firms, and one reason for this is cost: investment in research clusters – which serious prop shops all do – requires hundreds of millions of dollars.
Optiver at a glance
Optiver turned 40 years old in March 2026, launching in 1986 at the European Options Exchange. Today, the company has:
~2,200 employees
~950 engineers and ~1,000 traders and researchers
11offices: Amsterdam (HQ), Chicago (US HQ), Austin, New York (2025), London, Sydney, Shanghai, Hong Kong, Singapore, Taipei, and Mumbai.
10M+ trades executed per day, across 100 exchanges
€4.5B ($5.1B) in trading income, and €1.7B ($1.95B) profit, as per 2025 financial results
Optiver is a mix of:
Market maker: providing liquidity on exchanges by quoting ‘buy’ and ‘sell’ prices of financial products and earning the spread between the two.
High-frequency trader: executing automated trading strategies at very low latency
High-Frequency Trading (HFT)
High-frequency trading involves placing high volumes of orders at lightning speed in an effort to take advantage of extremely rapid market movements. In this domain, speed is the biggest advantage, and achieving it obviously involves high-performance computing. The basic trading loop is run millions of times a day. It’s made up of three steps:
Watch the market for new information like price changes
Decide what the information means and the right trade to make
Send a trade to the exchange before competitors do
In trading, timing is everything, and for some types of trade even nanoseconds count. Optiver’s fastest trading system operates in the realm of sub-nanosecond, where measurement noise becomes a challenge in itself. Software, hardware, and physics are all involved, along with microwave and shortwave links between data centers, and custom-manufactured chips.
We go deep into this in the “Hardware Engineering” section below.
However, in this niche, even ultra-low latency is no longer a competitive moat in itself. As competitors have squeezed performance out of their systems, focus has shifted towards fine-tuning of trading strategies. Today, Optiver invests substantially more in building better models than it does in lowering latencies. More on this in the “Network and physical infrastructure” section below.
HFT evolves faster than other industries. Profitable strategies don’t last long, opportunities are fleeting, and innovation is a constant. In this environment, a tool like AI is relatively straightforward to implement because trading houses like Optiver are well used to change in their daily business environment. More on this topic in the ‘Optiver & AI’ section.
Plenty of ML & math
There’s a big role for machine learning (ML) and mathematics in quantitative trading. A good chunk of Optiver’s business is the buying and selling of options, and the pricing of these rests on mathematical theorems like the Black-Scholes model. Traders, quants, and even software engineers building option-pricing strategies must understand the math of this problem space.
Over time, machine learning is becoming more important than math models, but it’s worth keeping in mind that trading is not purely an ML pursuit.
AI infra providers are heavily involved. NVIDIA, Groq, and Cerebras are actively courting trading firms, due to how much money they spend on GPUs. For example, see Hudson River Trading discussing Blackwell deployments at NVIDIA’s GTC conference, or Jump Trading being among the first to deploy next-gen Vera Rubin systems. HFT companies have very clear monetization paths for GPUs and spend large sums on hardware, hence why NVIDIA and other suppliers are keen to partner with them.
AI labs poach trading talent
One new trend is AI labs like Anthropic and OpenAI recruiting from prop shops, defying the assumption that AI labs mostly recruit from Big Tech. There are a few reasons why AI labs seek out talent from the trading world:
Infra expertise. Prop shops like Optiver have spent decades operating their own data centers and deploying on-prem hardware at co-location facilities.
Custom, high-performance hardware. Prop shops also often build their own hardware and their kernel stacks achieve very low latencies. That’s a talent AI labs seek!
Skillsets. The highest-paying destinations for CS majors out of standout colleges are often prop shops, paying top-of-market compensation for standout talent. Outside of select colleges prop shops recruit from, however, there tends to be little awareness about these companies for new grads, or across the industry.
2. Engineering organization
Two eras of Optiver tech
Optiver’s history can be seen as two distinct ages:
Regional systems (“unblock yourself”: 1986-2020): internal systems and platforms were built to serve local needs, such as building support for a market. Systems built exclusively for the US, Europe, or Asia were common.
Global platforms (“build for the whole company”: 2020-present): Optiver recently started to build new systems to work globally across their platform. This global focus is also why the company is investing a lot more in its platform engineering arm. A globalization push started around 2023, and its momentum has been growing.
The benefit of the old “unblock yourself” approach of local teams building whatever they needed, was that it enabled them to move fast and not get held up by dependencies. But this became problematic because of fragmentation and duplication, and the downsides became more visible over time:
Fragmentation: different teams use different technologies, frameworks, and infrastructure
Duplication: teams in different parts of the business independently build the same or very similar services
The career trajectory of Pat Cooney, Optiver’s head of platform engineering, mirrors the shift to a global platform: he was the CTO of Optiver in Europe in the mid-2010s when the business was split by region, and was appointed head of platform engineering in 2025 when that approach was replaced.
Optiver’s approach to continuous integration (CI) has also evolved. Previously, the company had several regional CI services, but from 2025, it started to rebuild its CI system with two new goals:
Build for scale: create a CI system built to scale across regions and stand the test of time
Use from any region: standardize deployment pipelines, so that code built in one location can run anywhere without friction
How roles work together
At Optiver, there are three main areas for tech roles:
Engineering: build and own the full trading-platform stack
Research: quantitative scientists who build models and predictive signals to create and improve trading algorithms. Typically, their background is in math, physics, economics, and statistics
Trading: quantitative traders who watch live markets, adjust trading system parameters in response to conditions, and build tools to automate decisions
In reality, the boundaries between these areas are porous. Yes, people do the job they were hired for, but it’s common to also see researchers roll up their sleeves and take part in implementing a trading strategy, or software engineers conducting research.
At Optiver, folks aren’t tied to one task
Cross-functional collaboration between roles is very common. For example, when developing market signals and associated trading strategies, it’s normal for engineers, researchers, and traders to collaborate on most, if not all, projects.
End-to-end ownership, plus autonomy, is a given. Engineers have autonomy in how they get things done, and they own and solve problems from the ideas stage through to implementation. There is a limited amount of guidance for trading, and it’s down to engineers to find the right solution.
In many ways, this approach to software engineering is pretty similar to startups’: software engineers get limited guidance and lots of autonomy. In order to succeed at tech startups, engineers typically need to understand the business, as well as being excellent at building production-ready software. It’s the same at Optiver, where understanding the business means understanding markets.
Platform engineering
Before Optiver’s globalized platform efforts started seriously in ~2023, regions duplicated effort:
Multiple implementations of identical core logic
Each region had its own systems, frameworks, and infrastructure
Local teams built whatever they needed in an “unblock yourself” culture
But that’s all changed. An obvious sign of global platform efforts is the appointment of Optiver’s first global CTO, Lance Braunstein, who joined with a mandate to scale the platform.
Roughly 30-40% of Optiver’s 950 engineers work on the platform. In contrast, a more typical ratio at other large tech companies is for 15-20% of engineers to be dedicated to platform work.
Prior to the global platform, there was a lot more tolerance of development experience friction; new engineers could spend weeks checking out the codebase and getting their build system to work. This mindset has changed, with the platform team stressing user empathy and reducing friction on engineers’ journeys, like by setting up build pipelines for their software.
Now, the platform is beginning to reimagine itself as built for AI. As agents proliferate at Optiver, users are both humans and automated systems. The goal of this shift is to empower people to decompose work into workstreams and orchestrate agents. Two projects were launched earlier this year by the platform team for agentic work:
AI gateway: gives Optiver engineers access to models
MCP hosting platform: makes it easy for engineers to access internal systems and tools via agents
How trading teams are organized
Trading teams at Optiver have three roles:
Traders decide strategy and make risk decisions
Researchers and quantitative analysts (“quants”) build hypotheses, pricing models, and run evaluations
Engineers build production systems
In practice, these roles overlap. This was true before the AI era, but it seems to be accelerating with AI adoption. Most traders and quants have STEM backgrounds without recent production coding experience. AI enables quantitatively-minded people to automate workflows with agents and to implement strategies.
Trading teams are organized by asset class and strategy. For example (asset classes in italic):
A large team is focused on a broad area like options
A team focused on cash markets and building strategies for exchange-traded funds (ETF) and stocks.
A team focused on machine learning (ML) and trading in the cash market.
Within larger teams, there are horizontal and vertical sub-teams. Horizontal teams take on challenges that impact any trading desk; for example, pricing is a horizontal team as the underlying mechanism is the same whether a soybean or an index fund being priced.
Vertical teams are similar to “tiger teams”, accelerators, and program teams at other companies. They focus on short-term goals attached to a few different desks in a location like the US, Amsterdam, Mumbai or Sydney.
Each team has a trading or research lead and a tech lead, who identify work for the team to do. The overall direction is set by a partnership structure, similar to an investment bank, but partners are not necessarily in charge of teams. At Optiver, partners are collections of senior people responsible for overall strategy.
Regardless of asset class or vertical, every trading team builds a version of a trading loop with four components.
Retrieval of market-related information
Collecting signals to work out which trades to execute
Execution of strategies (sending orders to market)
Intervention via a feedback loop, enabling a trader to monitor the system.
“Build and own” culture
Optiver runs on an ownership culture, with the principle that the best engineers take work personally and care deeply about Optiver’s systems, decisions, and outcomes. Leaders want engineers to treat their projects as if they were CEOs of a company, and be responsible for design, build, rollout, shipping, or support. There is no notion of throwing work over the wall to a QA team.
Optiver’s ownership model:
Traders and engineers define problems together. Engineers design, build, test, deploy, and monitor a solution. There are hundreds of production changes daily
Design reviews for architectural decision-making. When an engineer has a project that entails architectural change to the stack, the engineer is responsible for bringing multiple options with the pros and cons to the team for consultation. The goal is to share information and knowledge, and to make decisions
Optiver pushes new hires and interns to develop ownership. From day one, engineers have something they own and are assigned a real project with mentoring support. Production code changes are an expectation for new hires. Within a year, a new hire becomes the experienced person in their domain, ramping up the next engineer. This is explicitly emphasized in Optiver’s onboarding materials:
Ownership is also baked into the interview process, with explicit questions about problem-solving, talking through trade-offs, and implementation.
Case study: the Options Org
Optiver started life with options trading. The word ‘Optiver’ is actually a Dutch portmanteau of “options” and “trader”, so it’s unsurprising that the options team is among the most developed parts of the operation, with engineers split across multiple locations. The organization is composed of both vertical and horizontal teams.
One of the technical systems for which the Options organization is responsible is the retreat system. When Optiver trades an option, that trade itself changes the price of the next quote on offer. The retreat system has to reprice the entire option surface (i.e., all options related to the one just traded). This is called a ‘retreat’.
In the case of S&P options, the option surface can consist of thousands of options that have to be updated. Ten years ago, the retreat process took seconds; now, through optimizations at every level of the stack, it’s down to nanoseconds.
How the ‘retreat system’ works, at a high-level
Retreat speed matters because everything changes as soon as a trade occurs: the original quote is stale and a trader needs to remove the bid from the exchange before anyone can exploit it. Faster firms can take advantage of others’ stale prices, leading to an adversarial market dynamic.
Horizontal vs vertical team structures
Vertical teams work on specific tactical problems related to local trading desks with a focus on immediate impact. But they are not short-term or temporary teams, even if they work on short-term problems. They’re empowered to solve the most important current problems, end-to-end. On the other hand, horizontal teams serve most desks, and have longer time horizons because they work on cross-cutting problems like pricing, market connectivity, or auto-trading.
3. Software tech stack
Basic trading loop & three-layer tech stack
Most trading software applications or services (aka “apps”) at Optiver can be simplified to the basic trading loop. The exchange where the trading takes place is part of the outside world from which signals are extracted:
The three layers of trading: signals, strategy and execution
Signals
This is the information-gathering phase where services collect market data such as prices and order book information, and also run various data calculations, such as pricing algorithms and machine learning pipelines. These signals are made available to strategy applications/services which decide how to trade.
Strategy
A single trading strategy typically focuses on a particular class of assets and trades, and many different strategies run concurrently. The strategy sets what and how to trade, but doesn’t execute the trade; that’s the next step.
All strategies are enveloped by a risk management system that can block trades and stop individual strategies. To be effective, it has a broader view of the combined risk level of multiple strategies.
Risk mechanisms can include human oversight, with traders tweaking strategy parameters, and also automated monitoring that checks if apps are outputting orders within expected parameters, regardless of what the algorithm wants. The latter approach is essential in low latency strategies where faster-than-human reaction speeds are needed.
Execution
The execution step involves executing trades on exchanges. There’s a hard ‘separation of concerns’ principle where execution steps are only permitted to execute the trade. No additional logic is meant to run there.
Ultra low-latency loop
In some market-making use cases where nanosecond-level latencies matter, much of this process may run within a single chip (FPGA or ASIC) where the strategy part can be memoized with precomputed responses for all expected input patterns. This is then burned into the hardware to minimize latency from when market information arrives until a trading order is issued.
The tech stack’s three layers
All apps implementing the trading loop sit on top of a multi-layered internal platform:
Basic infrastructure layer: the stuff you’d see at most tech companies (CI/CD pipelines, k8s, Kafka, Postgres, etc), but they’re also customizing their stack. They run their own data centers, have custom hardware, custom Linux kernels, customized CI tooling, and databases.
Domain-specific infrastructure contains core trading-specific services such as trading data dictionaries, metadata on securities, and the trade booking system.
The three layers of Optiver’s tech stack. The ‘basic infra platform’ is similar to infrastructure at most other tech companies
Historically, most of this infrastructure was duplicated at each local office level when teams prioritized moving fast and independently over avoiding duplication. A centralized platform team has started consolidating these efforts in recent years.
Roughly 30-40% of the engineering headcount is allocated to the Platform team. This level of investment in the platform is beyond what you’d typically see in a tech company. That’s likely to remain the case for a while longer as they focus on improving the development experience, consolidating duplicated functionality, and catering to the specifics of their tech stack.
Languages and tools
At a glance:
Language choices at Optiver are fairly standard for a financial institution: C++ for low latency applications, and Python for modeling, prototyping and internal tooling work.
However, looking closely at Optiver’s contributions to the Python ecosystem reveals that this language is not just a prototyping tool:
optiver-asyncpg: Optiver’s fork of a performance-focused async Python lib for Postgres
vulcan-py: Optiver’s own dependency manager for Python allows more granular control over indirect dependencies
opti-napalm: Optiver’s fork of a library for automating and simulating various network equipment
Optiver’s internal tooling also has strict performance requirements because traders use internal dashboards and tools to make time-sensitive trading decisions. Avoiding hand-offs between traders and engineers for reimplementation in C++ saves time, and empowers non-engineers to solve their problems directly, in line with the “unblock yourself” ethos.
Rust is starting to play a significant role in research tooling and service orchestration, likely driven by the performance requirements. It’s interesting to see Rust used in areas such as Python, as opposed to it replacing C++, which would be obvious given its focus on performance. It’s likely due to Optiver’s decades’ worth of investment in the low-latency C++ ecosystem, its deep integration with existing internal hardware, and being able to directly control things like memory allocation with C++.
Other languages used in some niche use cases include:
C# for building data-intensive trader-facing GUIs,
Much of the software that Optiver builds interacts with custom hardware, custom Linux kernels, and requires predictable compute performance for predictable results in performance tests. These are all constraints that the CI/CD stack has to operate within.
Optiver’s CI/CD runs on bare metal machines, with custom hardware installed, the right OS tweaks, and a well-understood performance profile. Interestingly, this means Optiver needs to plan capacity in advance for its CI/CD clusters in the same way as it plans capacity for production systems. This is tricky since AI-coding tools started boosting the number of builds an average engineer does in a day.
They chose GitHub Actions as their CI Platform for the seamless development experience with GitHub. Unfortunately, Actions doesn’t provide overall, system-level metrics like queue times and utilizations, which are critical information for planning CI cluster capacity. Therefore, they had to build a bespoke observability layer over GitHub Actions pipelines with GitHub webhooks.
Data
When it comes to databases and storage systems in general, Optiver is a big user of Kafka, Postgres, and Databricks (the company built its entire data platform around this).
A few interesting details show the role of Postgres:
They contributed a new timestamp type to Postgres, allowing timestamps to be expressed with nanosecond precision. Few Postgres applications care about nanosecond-level precision, and this wasn’t available “out of the box”.
They built their own internal version of the NOTIFY - LISTEN mechanism called ‘PG Feed,’ based on Postgres’ write-ahead log. This is used for distributing high-fanout, latency-sensitive messages to clients like pricing and configuration data, whereas using something like Kafka may involve additional disk reads and writes, which imply unwanted latency.
Optiver generally picks industry-standard tooling, but heavily tweaks it to fit their specific performance needs. Not many tech companies of this size tweak Postgres or GitHub Actions, let alone Linux kernels!
Stream the latest episodeListen and watch now on YouTube, Apple and Spotify. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.Brought to You by• Antithesis — Turbocharge testing of your systems by running your whole system under aggressive fault injection. There’s good reason teams like Jane Street, Fly.io, and the etcd community rely on Antithesis. Learn more.• WorkOS – make your app and agents Enterprise Ready, with SSO, SCIM, RBAC, and more. Get
Listen and watch now on YouTube, Apple and Spotify. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.
Brought to You by
• Antithesis — Turbocharge testing of your systems by running your whole system under aggressive fault injection. There’s good reason teams like Jane Street, Fly.io, and the etcd community rely on Antithesis. Learn more.
• WorkOS – make your app and agents Enterprise Ready, with SSO, SCIM, RBAC, and more. Get started.
• turbopuffer– A vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable. I met their team in San Francisco, and am a fan of their “hardcore and whimsical” engineering culture, and how pragmatic their engineering philosophy is. Check them out.
In this episode
There’s a popular theory that AI will finally make formal verification mainstream because mathematical proof of correctness will be needed when machines write most or all of the code. But will this happen? Today, I’m talking with one of the best people to tackle the prediction. Hillel Wayne is a formal methods consultant, educator, and author (his most recent book being Logic for Programmers), who’s deeply interested in software history.
In this episode of The Pragmatic Engineer podcast, I sit down with Hillel to compare software engineering with traditional engineering, discuss where formal methods fit into modern software development, and we explore why they are essential for some of the world’s most complex systems. We cover the formal specification language, TLA+, walk through several formal verification tools, examine why distributed systems are so difficult to reason about, and look into whether AI will make formal methods accessible to more engineering teams.
Takeaways from the conversation with Hillel
1. Are we “real” engineers? After thorough research, Hillel has an answer. For The Crossover Project, Hillel interviewed ~20 people in different fields of traditional engineering and software engineering, and found plenty of similarities and differences. He concluded that the rigor needed in software engineering means we earn the right to the title of “engineer.”
2. Version control is unique to software engineering. Other fields of engineering have change management, but “traditional” engineers wish the concept of version control in software engineering existed in their fields because it’s far more sophisticated.
3. TLA+ is a formal specification language created by Leslie Lamport for designing and verifying systems. Lamport is a mathematician and creator of LaTeX, who wanted to create a language for modeling complex systems. The language represents the state machine of the system and every possible state it can transition to. From the initial state, the system enumerates to get to every reachable state and checks whether properties defined upfront apply to those states. In this episode, Hillel walks us through a demo with TLA+.
4. Amazon used TLA+ to find a bug almost impossible to locate without formal methods. In the paper How AWS uses formal methods, the AWS team shared that they’d found a complicated bug for which the shortest error trace to exhibit was 35 steps (!!). The bug passed unnoticed through extensive design review, code reviews, and testing. AWS concluded they wouldn’t have uncovered it if they’d stuck to conventional testing approaches.
5. Lack of practice makes most engineers bad at dealing with concurrency problems and race conditions. When a system has a race condition due to your code, you usually don’t find out until a few months later – if ever! In contrast, a system modeled in TLA+ can tell you about race conditions as soon as the tool is run, making it a fast feedback loop.
6. Why not use formal verification for everything, then? It’s because specs in the real world are a nightmare to write. Even a simple problem like “find the file in a directory that has the most lines” gets complicated when modeled with formal methods. We would have to answer questions like: ‘do we look at ASCII or UTF-8 new line characters, what about unreadable files, and Symlinks?’ Without formal methods, we can write a simple verification that is right in 99%+ of cases. Formal methods require a lot of extra effort for the less than 1% of exotic use cases!
7. Hillel recommends most engineers adopt property-based testing, and stop there. Property-based tests mean defining properties which the test then throws thousands of inputs at, in order to stress test a system. Hillel is convinced that formal methods are a niche tool for most engineers, whereas property-based testing is the most practical approach for building robust software with this lightweight formal method.
8. AI won’t make formal verification mainstream, but will increase its use. As Hillel says, “AI bringing formal verification up from maybe 0.1% to 0.3% across the industry would still be huge!” He also finds that people who succeed at using AI to generate formal specs are often formal verification experts.
9. Hillel worries about the time-of-check vs time-of-use bug. It makes Hillel want to pull his hair out when he sees a time gap between the time of checking something (e.g., whether a bank account contains sufficient funds for withdrawals), and the action itself (e.g., withdrawing money). This category of bug is hard to defend against and can cause annoying issues in real-world systems.
10. Hillel worries less about job losses from AI and more about software becoming an “ordinary” job. Revisiting his 2025 predictions of the impact of AI on the tech industry, one of Hillel’s concerns is that software engineering in the future will be lower-paid and lower-prestige than today. At present, the range of software careers available is pretty magical, especially compared to “traditional” engineering roles. But will this last?
11. One of Hillel’s coolest projects: verifying train transponders. Beyond databases and distributed systems, he has also formally verified device firmware. One cool project was working on the electric beacons between rail tracks that pass traffic information to the control system. He found a really odd bug in one transponder system, and fixing it made the real-world system more reliable and safe.
12. One thing that software engineering could take from “traditional” engineering: books on “the fundamentals” which every professional in the field should know. One of Hillel’s favorite books is The First Snap-Fit Handbook, a nearly 500-page tome on those little clips that hold battery covers in place. He observes that while most industries have copious documentation for the most mundane topics, within software engineering there’s not even a book on how to version an API! We could learn from other fields about the value of documenting our own craft.
13: The “materials” in software engineering are freakishly consistent. All other engineering professions have to worry about the consistency of their materials; for example, electrical engineers work with resistors that offer resistance within 20% of 100 ohms across a thousand units, and only when operated within a given temperature range. In contrast, a program runs identically on any given computer in software engineering. Hillel argues that the variability we deal with in software, like versions, APIs, bugs with integrations, etc, are largely battles of our own making.
The Pragmatic Engineer deepdives relevant for this episode
Much-improved AI tooling is changing how we build software, and I want to take a peek into how the future of software engineering may unfold under its influence. What better place for that than with tech’s most “AI-pilled” teams: the AI labs themselves.So, I visited the two leading AI labs to see how teams and engineers do things day to day. In this article and in an upcoming follow-up, I’ll share what I learned about how AI is reshaping software engineering principles many of us are accustomed
Much-improved AI tooling is changing how we build software, and I want to take a peek into how the future of software engineering may unfold under its influence. What better place for that than with tech’s most “AI-pilled” teams: the AI labs themselves.
So, I visited the two leading AI labs to see how teams and engineers do things day to day. In this article and in an upcoming follow-up, I’ll share what I learned about how AI is reshaping software engineering principles many of us are accustomed to – and what’s stayed mostly the same despite the AI wave.
In a later article, we’ll compare findings from Anthropic and OpenAI to see what their ways of working might mean for the overall direction of software engineering.
Inside Anthropic’s HQ (left). AI development milestones framed on the wall (right)
Thanks to Anthropic for showing me inside their lab in San Francisco. I talked with four people:
Katelyn Lesse, Head of Engineering for Claude Platform, whose organization owns the infrastructure that Claude runs on
Jarred Sumner, creator of Bun, now at Anthropic on Bun and Claude Code
Thariq Shihipar, who works across Claude Code engineering and education
David Hershey, at Anthropic’s Applied AI organization in a role resembling a sales engineer, working with customers like Cursor, Cognition, and Perplexity
Thanks to them, I got a sense of where things are headed at the leading AI lab – and possibly for the wider industry.
Before we continue, The Pragmatic Engineer will be on summer break for the next week and a half. This means no Thursday article this week, and no articles next week. I appreciate your understanding and support!
Back to today’s deepdive, we cover:
Complex & long: Claude Managed Agents. One of the most complicated projects took the Claude Platform team six months to ship, and created a new primitive to use at the agent infra level. Infra projects still need re-architecting mid-way through and take time to get right.
Twelve-month project done in 11 days: Bun rewrite to Rust. Migrating a 500K+ line project to another language used to take a small team a year, making it impractical. With Fable and $165K of tokens, it recently took the creator of the project less than two weeks.
Changing engineering practices. Inside the AI lab with more than 3,500 employees, prototyping is more fluid, verification is more time-consuming than implementation, code review and testing are increasingly done by AI.
Team-level changes. Design is more ongoing and less upfront, teams work on more projects, a maximum of two engineers per project, and more.
Still the same: two-pizza teams, planning is important, PRDs are relevant in complex projects, context switching is a challenge, the ratio of time spent on coding vs testing not changing that much.
Changing the “standout” software engineer archetype? Deep understanding, including of a layer below what you work on, is valuable, along with the ability to coordinate work.
Will AI replace software engineering? The more hands-on software engineers get with AI at the lab, the less they fear their jobs are going away.
1. Complex & long: Claude Managed Agents
The Claude Platform team’s most complex project in the past year was building Claude Managed Agents, a pre-built harness for production agents that runs in the cloud on infrastructure managed by Anthropic, or on your team’s own infrastructure, with any sandbox you choose. The project took around six months from idea until launch in April. Katelyn Lesse, head of engineering for Claude Platform, shared the story.
With Katelyn Lesse, at Anthropic
Claude Platform
This team sits between the model/accelerator layer (Claude models operate on GPUs) and the product/application layer (with products like Claude Code and Claude Cowork):
Where Claude Platform sits inside of Anthropic
Katelyn on what the Platform team does:
“We’re on the ‘token hot path.’ The prompt comes in, then we tokenize it. Then, things like safeguards and billing all happen within our layer.”
What the Claude team calls “Platform,” I think more of as “API.” Claude Platform operates the API, and owns responsibilities an API would have. Of course, the team does more than that, and Claude Managed Agents is one case we cover here.
The platform layer is being migrated from Python to Rust. Originally, this layer was written for Python for the “usual” reasons at AI companies: it’s a convenient language and AI researchers use Python already, which enables quick iteration. But Python is single-threaded, and at scale, when the API is under high load, it’s not as performant as Rust.
Harness infrastructure demand
The project came together due to customers wanting their own “harness infrastructure”, says Katelyn:
“We started with a model where you get an API to define an agent, then you get an API to start a session with an agent. The reality of what the world wants and needs right now is people running their own infrastructure. So, we started to build a self-hosted sandbox.
But then, what we started to hear from lots of customers is that they’re trying to hack harnesses together, running their own “harness infrastructure,” and this gave us the idea for Claude Managed Agents.”
The largest part: planning
In this project, Katelyn said the single biggest matter was planning:
“There are products you can jump straight to prototyping, but then there are ones where you need to start by architecting it properly. For example, if we build a TypeScript CLI – which is pretty trivial for what needs to be built – we could go straight to prototyping. But with Claude Managed Agents, we needed to first figure out what we are doing.
Of course, we did some upfront prototyping for Managed Agents: hacking and spiking things. But prototyping itself was more about understanding the requirements.
Our planning process looked more like a typical pre-AI planning process. You know how every team has the project, where everyone comes up with some version of the same idea and people keep floating and circling it around until you finally do it? Managed Agents was this for our team. When we started the project, we had documents dating back up to two years about ideas and suggestions.
Post-planning, when the project officially kicked off, a PRD (product requirements document) was created:
“In the end, it was the Product Manager and the Tech Lead on our API Agents team who decided to pull the trigger and kick off this project. We’d get in a room, go through it, and get aligned. But it wasn’t just us: we’d have to align with teams around the business, other cloud providers, and other engineering teams. For example, we have a sandboxing team inside of the Platform org: and so this team was consulted on the design of Managed Agents, given this product would spawn a lot of sandboxes.
Just like before, we had a PRD, it was a Google Doc. We used a Google Doc because we needed to coordinate all interested people. This has not gone away.
Similarly, my product counterpart and I run product reviews.”
Some processes from before AI, like the PRD, are still useful in complex projects today, for getting large groups of people on the same page.
Build for an internal customer first
With planning complete, the team decided to do a “spike” and stress-test the idea and architecture, by building the backend of Claude Code on the web. Remote execution of code with an agent harness was a similarly shaped problem to the one they wanted to solve for customers. The thinking was to start by solving it for the Claude Code team before tackling it in a more generic way for customers.
Internal teams are more fluid than before AI. Katelyn:
“Pre-AI, we might have hit the Claude Code team up with a bunch of big requirements documents, and they would have then hit us back with another set of documents. Now it was much easier: someone on our team built a few components, took it over to the Claude Code team, and they started to hack around it. We could figure out how this component plugs into this part of their product, and the other way around. It was just a faster and easier process, getting this first internal version of the product up and running.
Aligning with other teams on interfaces remains important, and it’s easier.Back in the day, you’d have to come with a fully spec’d interface to use. Now, we could do it a lot more fluidly: we could stand up a stub service that shadowed traffic to start with, and iron out the interfaces with the Claude Code team as we went. They did some hacking on it and gave feedback, we made changes while building out the service under the interface, then went back to make it work.”
They launched a service for Claude Code’s mobile app to spin up a sandbox, boot up Claude Code and run it. The service went to production and the Claude Platform team took the learnings.
Re-architecting midway through
It’s likely a familiar scenario many engineers can relate to, that after planning a project and getting underway, you see that you’re going to need to change the architecture. It happened on this project, too.
The platform team ended up re-architecting Managed Agents based on learnings from the Claude Code “spike.” Re-architecting meant decoupling the “brain” of Claude and its harness from the “hands” (sandboxes & tools that perform actions) and the “session” (the log of events). Each became an interface that made few assumptions about each other.
High-level architecture of Claude Managed Agents after the re-architecture
The team also built an abstraction around vaults and credentials. Credentials can safely be stored inside a vault. All calls using credentials are made via a proxy which has a session token. It is the proxy that fetches the right credentials from the vault: the credentials are never seen by the agent, sandbox, or session. Credentials are only injected at the egress boundary when the service is invoked:
Adding credentials the harness never sees
Internal “dogfooding” helped surface hard problems to solve. A few examples:
Reliability and scalability: these are really hard to do well for agents because if connection to the sandbox is lost, the whole agent dies and you lose state
Credentials and access control: also hard and problematic, especially when first building the service
The project took about six months, by no means a rapid process. Katelyn emphasized that pre-AI, a project like this would have probably been in the realm of two years. Managed Agents is one of the biggest projects the Claude Platform team has built, and more complex than it looks: for example, adding support for running agents on AWS, GCP and Azure.
2. Twelve-month project done in 11 days: Bun rewrite to Rust
As covered before, Jarred Sumner is the creator of Bun, a popular JavaScript runtime with 22 million monthly downloads currently and Claude Code as a dependency.
With Jarred Sumner (left), creator of Bun
Bun is written in Zig, a performant, productive language. However, it’s not memory safe and memory issues kept coming up. Jarred thought that rewriting the project to an also-performant, memory-safe language like Rust could be an option – except that rewrites like this turned out as follies in the past. Jarred (emphasis mine:)
“Historically, rewrites are a terrible idea. Excluding comments, Bun is 535,496 lines of Zig. A rewrite in another language would take a small team of engineers a full year. It would mean freezing bugfixes, security fixes or feature development for that time. The least risky approach to getting something shippable would be a mechanical port from Zig to Rust, with the minimal number of behavioral changes, using the exact same test suite we already use for testing Bun.
Fortunately, Bun’s own test suite is written in TypeScript which means it doesn’t depend on the runtime’s programming language.
A year of zero user-facing impact was not an option we could consider. So, enforcement through code style to fix stability issues was our best bet, and was our plan when we added Rust-inspired smart pointers to Bun’s codebase.
But honestly, I didn’t want to do it. Homegrown smart pointers offer worse ergonomics than Rust, with none of the guarantees.”
But then, Jarred asked if AI could do the heavy lifting and wondered how much the migration could be sped up. In the end, he completed the rewrite from start to merge in 11 days, using 64 parallel agents and $165,000 in tokens at API price. Here’s Jarred on how his AI-heavy rewrite compared:
“By hand, I think this would’ve taken three engineers with full context on the codebase about a year, during which time we wouldn’t be able to improve Node.js compatibility, fix bugs, fix security issues or implement new features. We never would’ve done that. The realistic alternative was to do nothing and keep fixing the bugs at the top of this post forever.”
There was a lot more to the project than typing out the “...make zero mistakes” prompt:
Jarred made a detailed plan and style guide on how to migrate
He set up the project so agents would not use Git worktrees which he found slow, but worked on different files in the same codebase
He created an orchestration system where each AI agent came up with suggestions of what to change, but did not make a change to the file to avoid conflicts; an orchestrator AI agent created the commits
The most time and tokens went on fixing the compile bugs, tests, and verifying that things worked
Bun itself has a very robust test harness: when all tests pass, it’s a high-confidence signal that the rewrite works
Crucially, Jarred is the ultimate domain expert in Bun: he created the project and knows the codebase better than anyone
The rewrite has been shipped to production and powers Claude Code today.
So, what has changed in how teams build software at Anthropic, compared to the pre-AI days? That’s the question of this article, and it seems that many things are different. Let’s go through it:
AI lab-specific practices
Some things as normal as breathing at AI labs like Anthropic stand out as different with an outside perspective:
Everyone runs multiple AI agents all the time. Running 3-10 parallel agents is a given. Folks I talked with had their agents running in the background or cloud.
No token budget, usage not tracked. One major difference between AI labs and everyone else is that there really is no token limit or token leaderboards that promote tokenmaxxing; people already use agents all the time.
Very high autonomy. Work is becoming more structured inside AI labs, but there’s still massive autonomy compared to Big Tech and most startups. When everyone has unlimited tokens, it’s pretty easy to prototype any idea.
Prototyping and “spiking” is far more fluid
It was several times faster to prototype early approaches for Claude Managed Agents. Similarly, “spiking” the Claude Code mobile backend implementation was much faster than pre-AI, Katelyn told me.
Verification takes longer than implementation
Jarred made a point about the split between implementation and validation in his 11-day rewrite to Rust. Roughly, it was:
Implementation of the Rust rewrite took far less time than fixing it up, then validating that it works as expected
The “implementation” part of rewriting the code from Zig to Rust took about 15% of the time, while 85% went on fixing things up: getting it to compile, fixing tests, verifying that it worked.
Most tokens no longer spent on implementation
Thariq:
“We see that few tokens are spent on actual implementation. Most are spent on discovery of unknowns, prototyping, mocking, and then in verification and testing.”
Jarred’s Bun rewrite echoes this: he spent more tokens on fixing up the implementation and verifying that it worked than on the implementation itself!
Code review and more testing by AI
Jarred:
“Critiquing the code and testing it with agents is a new approach we do a lot more of. I think a lot about trust when you merge a lot of code. How do you merge 100+ PRs a day, and make sure the code works? At this pace, you need to trust the code without the ability to read it all yourself. And I think it’s a few things:
Code review: it needs to be really good and automated. I’m clearly tooting our own horn here, but I find Claude’s code review to be really good. Claude’s code review catches bugs that would take me an hour of closely reading the code to figure out. The caveat is that it’s expensive!
Security scanning: for this Rust rewrite we did 11 runs of the Claude Security Scanner.
Fuzz testing: we’ve also been doing different types of fuzzing (fuzz testing), where we had Claude write a fuzzer for things like parser fuzzing.
Running out-of-process testing, where it happens in a different process/session from coding, is one way to build trust in the code. I expect more of this.”
New pattern: fanning out work to AI
Jarred described a new way he works:
“A new approach I’m using is fanning out a lot of the work to many Claudes at the same time. I did this with the Bun rewrite, but I use it for other work. This approach works very well for me, and I feel it’s pretty underused.”
Time-saving automations powered by agents more widespread
Jarred listed several time-saving automations set up by the Bun team to run an active open-source project with a small team, while the team works on Claude Code:
Every time someone files an issue, Claude runs to try and reproduce the issue. If it succeeds, it starts another container, which then tries to fix the issue and submit a PR.
The agent tasked with submitting a PR has to write a test that fails in the system version (the one without the patch) of Bun, and passes in the debug build with the patch, before it is allowed to submit a PR
There are other automations, like if there is no test, the PR is auto-rejected; all linters are run: Claude Code review is run, CodeRabbit’s code review is run, and the agents go back and forth on the GitHub pull request
Auto-merge of pull requests: coming soon?
Pull requests are merged manually when all quality gates pass, but this could become automatic at some point. Once all the above checks pass, all (AI) code review comments are addressed, tests are added to new code, etc. As an interesting aside, a lot of GitHub activity is Claude talking to Claude!
But manual merging may vanish in low-risk cases, at least for the Bun project. Jarred told me:
“Today, a person presses ‘merge’ but within a few months, I expect:
Automated reviewer LGTMs
→ another Claude with a fresh context window judges if it’s simple and low blast-radius
→ if it is: auto-merge!”
Test assumptions with each model generation
Inside Anthropic, the team keeps testing their priors. Thariq gave an interesting example:
“The thing with agents is that you have to revisit any assumptions you have made because it can change with a new model generation. For that reason, we deleted 80% of the Claude Code system prompt recently because the model has gotten smarter.
Using HTML is another assumption we needed to re-examine. HTML is one of those things which Claude is a lot smarter at than many of us expected. I’ve started preferring HTML as an output format over Markdown, and see this being used by others on the Claude Code team.
HTML can convey much richer information compared to markdown, HTML documents are easier to read and share.”
4. Team-level changes
At Anthropic, there are also changes in how engineering teams operate, compared to pre-AI.
The Pulse is a series covering events, insights, and trends within Big Tech and startups. Notice an interesting event or trend? Hit reply and share it with me.Today, we cover:Moving my video podcast off Spotify due to constant reliability issues. Spotify’s podcast platform has become chronically unreliable since the company’s leadership started boasting about high AI adoption. But competitors haven’t had similar issues, and so I have offboarded from Spotify.Will Kimi K3 trigger US push for close
The Pulse is a series covering events, insights, and trends within Big Tech and startups. Notice an interesting event or trend? Hit reply and share it with me.
Today, we cover:
Moving my video podcast off Spotify due to constant reliability issues. Spotify’s podcast platform has become chronically unreliable since the company’s leadership started boasting about high AI adoption. But competitors haven’t had similar issues, and so I have offboarded from Spotify.
Will Kimi K3 trigger US push for closed-source AI models? Moonshot AI’s latest open model, Kimi K3, is on par with Anthropic’s Fable 5. Could it lead to the US government regulating or banning Chinese open models to protect US labs?
AWS laughs off “heart attack” billing error. AWS customers were billed trillions more than they should have been, due to what was likely a conversion error. But instead of sharing an incident report, AWS saw the funny side.
Industry pulse. OpenAI’s unreleased model tried to hack HuggingFace to improve its test scores, X took more than a year to develop its new Android app, Google’s new AI model flops, and more.
1. Moving my video podcast off Spotify due to constant reliability issues
Hi, this is Gergely with the monthly, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. If you’ve been forwarded this email, you can subscribe here.Subscribe nowAfter I recently interviewed Simon Eskildsen, co-founder and CEO of turbopuffer, on the main stage at the AI Engineer’s World Fair, many people at the event told me they found him relatable and inspiring for his choices to stick wit
Hi, this is Gergely with the monthly, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. If you’ve been forwarded this email, you can subscribe here.
After I recently interviewed Simon Eskildsen, co-founder and CEO of turbopuffer, on the main stage at the AI Engineer’s World Fair, many people at the event told me they found him relatable and inspiring for his choices to stick with one company for close to a decade, his belief in the power of “napkin math” to reveal why products run slowly or cost too much, and for his insight about how too much of VC funding is about ego, not business needs.
This article contains the most interesting parts from that conversation; in particular, the concept of “napkin math” – doing quick calculations to get rough answers – as a way to challenge existing systems to improve. The full 55-minute-long video of the discussion at the AI Engineer’s World Fair is available to watch:
Algorithmic programming speedrun. While in high school, Simon competed in the International Olympiad for Informatics (IOI) which pushed him to learn about writing correct programs that are fast and memory-efficient, and more.
Eight years of infra at Shopify. There are many upsides to longer tenure: Simon learned infrastructure concepts, dug deep into databases running across regions, and learned to write software that ages well.
“Napkin math” as a superpower. Simon became obsessed with finding the theoretical limits of compute operations, such as sending over data and reading bytes. This held him in good stead at Shopify, and then at his startup.
Origins of turbopuffer. When ChatGPT took off, context windows were small, and so stuffing them with the right information was key for AI applications. Fast search was needed, but the search solutions were surprisingly expensive. Using “napkin math”, Simon discovered they were far more expensive than necessary.
A new product without VC funding & Cursor as customer no.1. After raising $8M in seed funding, Cursor rolled the dice on the new turbopuffer team after Simon helped with their search & database needs.
Reasons to raise venture capital. Fund R&D, fund growth, stroke founders’ egos, and more.
Disclaimer: turbopuffer is a season sponsor of the podcast, but as with all our deepdives, this article is independent of podcast sponsorships.
1. Algorithmic programming speedrun
A self-taught professional, Simon skipped college to work at Shopify and spent nearly a decade there building a variety of systems. His interest in computers started after initially getting into building websites aged 12. Growing up in Denmark, he dabbled in HTML with tools like Microsoft FrontPage and Adobe’s Dreamweaver.
While still a teenager, Simon “hit a wall” by exhausting the Danish-language part of the internet for learning programming, and got into the World of Warcraft MMO game, which helped him acquire English.
After discovering the International Olympiad for Informatics (IOI), he decided to enter, despite a very competitive, multi-stage selection process open to all Danish high schools. IOI problems are pretty similar to Leetcode problems: algorithmic challenges that value correctness, speed, and memory usage.
Simon cleared the online qualification round and was invited to the Danish Nationals. This was more than a competition: a weekend-long bootcamp to teach participants more advanced programming techniques such as recursion (which Simon already knew), the divide-and-conquer algorithm, and dynamic programming; one of the more tricky concepts to master for algorithmic programming.
I tip my hat to the organizers for creating a challenging bootcamp and offering the opportunity for people like Simon, who wasn’t aware of what “NP complete” meant when he entered. He recalls:
“The routine was that every four hours we’d be introduced to a new “programming concept”, and receive ~2-6 tasks where this, combined with previously introduced concepts, had to be applied. All the solutions had to be submitted to the same site I submitted my qualification solutions to, as it was all part of the final evaluation. The tasks were incredibly challenging, like nothing I had ever tried before.
Sometimes in extreme desperation combined with tiredness from the trip, I’d think about taking the next train home. This feeling would disappear with the utter joy and confidence that arose whenever I would finally solve a task, and creep back once again when I found myself still struggling after an hour on a new problem. But this kept me going. By Saturday afternoon, I had almost managed to get up to speed with the others, and was doing the same tasks as them.”
Ultimately, Simon claimed one of six spots in the Danish national team, making it through to the regional finals.
Aware of how little he knew about programming, Simon doubled down to catch up. He realized that most other participants were better prepared and that he had to do something to survive the next qualifying round. So, he got to work:
Simon’s desk with Donald Knuth’s “The Art of Computer Programming,” and the training week he created
As Simon recalled:
“I armed myself with a borrowed copy of “The Art of Computer Programming”, worked through the exercises, read up on common algorithms on Wikipedia, completed tasks on USACO, and memorized the critical parts of my Vim config for the competition computers. I managed to create quite an intense training weekend for myself.”
The regional finals were even more challenging than the Nationals, and participants struggled to write performant solutions to the problems. Despite putting in the effort, Simon was pretty sure he was out of the competition, and recorded his learnings:
“[From the programming competition] I learned that you must avoid digging holes. Repeatedly, I found myself so fixated on getting a particular idea to work that I’d get absolutely nowhere. Sometimes, you have to bite the bullet, delete your program, find a new sheet of paper, and start from scratch. A good case of this is when you start working around a general solution to solve specific edge-cases. I learned that there is almost always a simple way to solve a problem without explicitly handling edge-cases. If there are two edge-cases, there’s almost certainly two more. The simple solution will handle edge-cases automatically – even those you might not have considered.”
But as it turned out, two months later he was told he had been selected to represent Denmark in the final phase of the competition. Simon continued to push himself out of the comfort zone of web development, using HTML, a bit of PHP, and getting into algorithmic programming.
Another decision that would later bear fruit career-wise was starting a blog while at high school – though he didn’t know it at the time. In 2010, Simon launched his English-language blog but posted only one or two articles per year; mostly short ones describing problems he’d solved:
A five-line script to take screenshots and upload them to Imgur, all from the command line (2010)
Why Simon favors SQL over ORMs: Object-Relational Mappings (ORMs) are frameworks to abstract SQL, these were very popular around that time (2011)
Two posts he published would go on to have an impact on his life. One was about his IOI experience and learnings. The other only came about after he suffered the disaster of fatally dropping his iPhone during his final year in high school.
2. Eight years of infra at Shopify
With his smartphone dead, Simon was forced to switch to an old-school Nokia “dumb” phone. He wrote a short article about the experience titled Why I’m glad my iPhone broke, which went viral on Hacker News, making it to the front page of the site with many comments.
It seems that someone at Shopify, in Canada, read the article and others by him – including the summary of his impressive performance at the International Olympiad for Informatics – as a recruiter from Shopify flew Simon out to interview in Ottawa, Canada, where he was offered a software engineering position at the company.
In his new job on the infrastructure team, Simon made notes about things he didn’t understand and read up on new concepts. As he told me:
“When I started at Shopify, I was insecure about having not studied computer science and my biggest exposure to programming had been the IOI. If nothing else, preparing for the IOI taught me that you can sit down, read a paper, and figure it out if you spend enough time on it. So, I did that repeatedly.
In my first year at Shopify, every time I heard something I didn’t know, I noted it down on a piece of paper. Then, that evening, I would read up about it.
For example, if someone at work mentioned TCP, I assumed that surely they would know exactly what’s in the three-way handshake and how TLS is layered on top. And I also assumed they’d looked at Wireshark and all of that. I don’t think that’s true, but that’s what I thought at the time! So, I dug deep into everything I encountered.”
Working in infrastructure meant solving interesting engineering problems at a time when Shopify was growing 120-140% in load year-on-year; Simon was exposed to problem domains like:
Ruby on Rails and databases: Shopify was already one of the world’s largest Ruby on Rails monolith applications, and working on infrastructure meant being close to the database layer
Sharding and cutting over: as Simon’s manager used to say, “you cannot cache writes”, so Shopify had to move from running on a single group of machines to a shard (partitioned) setup of machines. The team did the cut-over (moving to the shard) just a week before Black Friday, the busiest time of the year.
Multi-data centers: expanding database footprint from one data center (DC) to multiple DCs
Splitting up key services: Shopify had a 128GB machine (massive for the time) running Redis as a key-value store. Nobody dared touch it until one day the service went down. Then, the team split responsibilities up into separate services.
Simon built a framework to simulate networking conditions called toxiproxy. The idea came to him while attempting to write a more thorough, systems-level test to see how Shopify’s application held up during partial outages. He created a matrix of Shopify’s services, and wanted to write a complete test suite for this matrix, to see if the system was resilient enough to handle some parts of the system being down. For example:
“Having tests for the matrix was a must; otherwise we couldn’t guarantee the state of the matrix wouldn’t degrade over time. Since the tools mentioned previously require root access, we investigated proxies to simulate latency and downtime at the TCP level, but didn’t find one that suited our needs. We needed an online API to edit proxies and to support deterministic latencies, which made it suitable for integration testing.”
This was the inspiration that led to toxiproxy. As Simon told me on stage:
“Toxiproxy is a proxy that sits in between the application and the databases. With the proxy in place, you could do things like make an API call to the proxy, instructing it to simulate taking the database down, or making it slow.
Over time, we added a bunch of other ways to inject failures. With this proxy, we did not have to mock low-level drivers, but we could test failure handling really well.”
Toxiproxy was open sourced in 2014, and apparently still runs in Shopify’s CI system 12 years later, to Simon’s knowledge!
The biggest benefit of a long tenure at a company was learning to write software that ages well. Simon told me that this was the lesson that made it so worthwhile, and how often the simple solution someone put together in a week or two outlasted the big, multi-team RFC-driven solutions. That was a lesson he still uses today.
3. “Napkin math” as a superpower
While at Shopify, Simon became interested in figuring out the theoretical limits of certain computer operations. For example, how much bandwidth does DRAM have for memory transfer? How long does a round-trip operation to AWS S3 take, and how much does it cost? What does a gigabyte of memory cost?
He sought the actual numbers, wrote a script to collect them, and created a table with the data:
Simon started to memorize key numbers with flash cards. He wanted to be able to instantly recall all important numbers. As he told me:
“Napkin math was essentially just this table that I maintain on GitHub. There’s probably like 50 of these numbers and then a script that generates them all.
For example, what does a gigabyte of memory cost? $2. What does a gigabyte of S3 cost? Two cents. What does a gigabyte of this cost? 10 cents. What does it cost on spot? What does it cost on a three-year commit? I had a massive table, then I created flashcards for almost every single cell so I know all these numbers.
This was a project I started taking on at Shopify because I would review projects and these numbers would be helpful.”
Napkin math enabled him to challenge design decisions based on benchmarks that had issues. As he told me:
“When reviewing a project, a product team would tell me that they chose Database A over Database B because they benchmarked both, and Database A was better. I hate benchmarks because making design decisions based on benchmarks is not a satisfying answer to me.
For example, you’re saying that with Database A, as per the benchmark, it takes 10 seconds to do this one thing. But it should take 10 milliseconds if you do the napkin math. Say, it’s a search query. Okay, you’re searching for three terms then. Each term has this many documents that match it. That’s this many megabytes. We intersect this many lists. You have DRAM bandwidth on multiple cores, at 100 GB/sec. So if you do the math, it should be 10 milliseconds.
So, now you’re telling me that the benchmark for the same thing takes 10 seconds, which means one of us is wrong! Either there’s a gap in my understanding – which is possible! – or the benchmark measures the wrong thing.
Often, it would be things like the person doing the benchmark and not realizing that the query would be a distributed one, running across a hundred different nodes. And in this case, of course the p99 is going to be very high!”
Simon went even deeper into “napkin math” after leaving Shopify. He investigated whether MySQL’s maximum transactions per second is equivalent to fsyncs per second (the number of file writes per second a system can handle). He discovered that MySQL can handle more writes than the operating system could sync to the file, which was a surprise:
“It takes ~3 seconds to perform 16,000 insertions, or ~5,300 insertions per second. This is 5x more than the 1,000 fsyncs per second our napkin math told us would be the theoretical maximum transactional throughput!
Typically, with napkin math we aim for being within an order of magnitude, which we are. But when I do napkin math, it usually establishes a lower bound for the system, i.e., from first-principles, how fast could this system perform in ideal circumstances?
Rarely is the system 5x faster than the napkin math says. When we identify a significant-looking gap between the real-life performance and the expected performance, I call it the “first-principle gap.” This is where curiosity sets in. It typically means there’s (1) an opportunity to improve the system, or (2) a flaw in our model of the system. In this case, only (2) makes sense, because the system is faster than we predicted.”
Simon started to investigate, and learned that MySQL does grouping of transactions (not doing an fsync for every write), and it also does smart merging of multiple fsync operations that would be processed in parallel, effectively doing a “group commit” to further improve performance. This is a learning he’d later use: grouping writes together on top of S3, to reduce latency and improve cost.
Personally, I find it amazing how much you can learn with some measurements and by asking questions about how a system achieves results that go against what the “napkin math” suggested should be its limit!
4. Origins of turbopuffer
“Napkin math” also played a part in the creation of turbopuffer. Simon told me that three things had to combine for the project to come to life.
1. Search was a difficult project at Shopify. Building search was one of Simon’s final projects at the company, and he did not have a good time with it. He used a popular database vendor but struggled to have it perform napkin math. The query plans were not exposed easily by the database, so he could not figure out what was missing and why it wasn’t performing the napkin math without reading massive amounts of source code. Also, the search infra was difficult to operate.
2. Napkin math became a surprisingly efficient reasoning tool. As a toolkit, napkin math gave Simon a way to reason about what could be achievable with a machine, if utilized perfectly.
3. His friends’ startups needed fast search badly because of AI. In early 2023, Simon helped a few friends with infra at their startups, many of whom had the same problem: LLM context windows were very small (4-8 KB), and they needed to fill them with the right parts of documents, but this demanded very fast search. One startup budgeted that their new search vendor would cost $30K/month, while the existing infra bill was only $5K/month! For a bootstrapped Canadian company, the cost was too high, so they didn’t ship the AI feature.
Simon could not stop thinking about why search was so expensive, and why it didn’t line up with what the “napkin math” predicted the cost should be. Eventually, he laid out a basic architecture to make it fast and cheap:
Store the data to search in AWS S3
Do some clustering and organize the files
Get latency down. S3 is cheap and reliable, but has high latency
Driven by curiosity about how that could work, he set out to build it. As he told me:
“I just became fully obsessed that summer [of 2023] with building it. The first version was the simplest possible thing. I’m a very pragmatic person, so I didn’t get buried in detail. I barely read the literature on log-structured merge (LSM).
The simplest way to do this is to run a clustering algorithm on the vectors. You get the clusters, and then put the clusters in files. The files are called ‘cluster_01’, ‘cluster_02’, ‘cluster_03.’ Then you have a file called centroids of the clusters. Then, you search by downloading centroids, looking into them, then downloading the closest clusters.
There were a few optimizations around merging clusters that were adjacent in files, just to control costs and boost performance. But this was the core of it.”
What about performance? Simon:
“For the first version, I didn’t even implement a dedicated caching layer. I just put the reverse proxy (NGINX) in front of S3, and that was it! The performance improvement came from caching all of the S3 objects. It was the simplest possible layer at the NGINX level.”
5. A new product without VC funding & Cursor as customer no.1
I first heard of turbopuffer last year, when I interviewed Cursor cofounder Sualeh Asif about how they built the AI coding harness. From our previously published Cursor deepdive:
“The ongoing need to re-shard as usage grew became error-prone and frustrating. Sualeh told me the biggest lesson the team learned was to avoid sharding where possible in future. So, they looked for a vector database that could support multi-tenancy without requiring manual sharding.
Turbopuffer was a startup that promised this, so Cursor tried it and migrated a good part of their vector search use cases over.”
So, how did a billion-dollar startup come to bet on a tiny, unproven infra product with zero customers? There were a few factors:
First, Cursor was not yet a billion-dollar company in the fall of 2023, but a relatively fresh startup. The company raised $8M in seed funding in the same month. In 2024, the company was valued at $400M, and then its valuation surged to $29.3B. It was sold to SpaceX for $60B this year.
Secondly, it involved a tweet Simon posted after having had enough of building all summer, and wanting to test the waters. As he told me:
“I was so sick of working on [turbopuffer]. I’d been working on this all summer and didn’t know if anyone cared. I only wanted to work on it if anyone cared. So, I decided to put it on Twitter.
At that point, I had a single TMUX instance running on an 8-core node somewhere in GCP. I was thinking: “if someone goes to prod, I’ll set it up properly on multiple nodes.
But first, let’s see if anyone cares.
Anyone who’s worked in the internals of databases would’ve had too much pride to ship anything like that; I was just releasing it like a SaaS project. Why can’t you work on a database like it’s SaaS?”
Simon had the confidence to launch his product because he knew it was rock-solid and scalable:
“I knew turbopuffer was reliable. It upheld its invariants. For example, you shut down all the VMs and no data is lost. All the writes are committed directly to blob storage.”
Cursor reached out. The company was an eight-person team at the time, fresh from seed funding and growing fast, but with a search problem that they were looking to partner with a startup on. Simon recounts:
“Knowing Cursor’s founders now, I’m sure they must have sat at the dinner table one day and were like: ‘the unit economics of what we have right now, where all the vectors are in DRAM, are not working.’
They probably were asking why someone had not built a solution where you can put all the vectors from the codebase into S3 (to make it cheap), and move the part of the codebase used in memory (to make it fast). Then, everything sits in blob stores and you just hot load it all into cache. When you open the codebase, after a few seconds it’s all in RAM and the queries are as fast as everything else.
Aman (one of the cofounders) was already talking about using S3 as a key-value cache, which at the time, barely anyone was thinking of for unit economics.”
Simon knew nothing about B2B sales at that point, and was not trying to “sell” anything to Cursor. He exchanged a few emails with the Cursor team about their use case of searching many small local codebases. He wanted to help Cursor with the unit economics, while also proving that turbopuffer works. So, he hopped on a flight from Canada to San Francisco, arrived at Cursor’s office and got down to debugging their database provider:
“When I showed up at Cursor’s office, they were having some Postgres problems. I asked, ‘do you guys have pganalyze?’ And they didn’t, so I was like, ‘Okay, let’s get that going. Let’s look at it. “
And the problem was the same thing as it always is with Postgres: autovacuum hadn’t run enough. And so they had all of these going to heap, when they should be doing index scans, etc. So, we were talking about all of that.
I was just helping them: my “database genes” just kicked in. I think this built enough trust with them to believe that if I know enough to help them with their database, maybe I also know how to build one.”
It was around when turbopuffer’s other cofounder, Justine Li, joined that the work with Cursor kicked off. Simon describes Justine as the best engineer he ever worked with at Shopify, and together, they made more performance optimizations. Cursor migrated their local code-base search over to this brand new product.
Simon promised to reduce Cursor’s bill by 95%, and delivered – thanks to napkin math! Cursor was spending around $80K/month on indexing and search; with turbopuffer in place, this dropped to $4K/month. Simon was confident about making this prediction thanks to napkin math! He did the calculations based on the fundamentals; only counting resource usage that Cursor was using, and not taking a margin for turbopuffer’s operating cost at the time.
6. Reasons to raise venture capital
After securing Cursor as the first customer, Simon was not convinced that turbopuffer should raise VC money. He explained his thinking:
“I understood that if you take venture capital, no matter how many smiles there are in the room, everyone’s expecting to earn a big return on some timeline that makes sense to everyone involved. And ‘everyone involved’ are pension funds in Canada.
But at the time, I did not know if turbopuffer could be a billion dollar company. It felt like a very niche kind of search engine. And that was completely fine with me!
So, I just looked at what we invoiced and I looked at my GCP bill. And I was making this equation that:
Customer Invoices >= GCP bill [the turbopuffer costs]
Justine and I were going to optimize the system until these numbers were roughly equal. And if we could get some other workloads over to turbopuffer, we could then start paying ourselves.
But either way, I didn’t know if I could even go and raise a bunch of money. I didn’t have any relationships. I was an outsider who grew up in Aarhus, Denmark, and moved to Canada.”
A few months later, turbopuffer ended up raising a total of $700K, just to be able to hire two engineers until the end of the year on real salaries. Simon found that most VCs did not take them seriously for wanting to raise too little money, and assumed this signified a lack of ambition! Two years later, with less than $1M in initial funding raised, turbopuffer crossed $100M in annual run rate. So, there evidently was ambition!
Since our conversation, Simon has reflected on the topic of VC capital and built a mental model of six reasons that justify raising external funding:
Reason #1: Research and development. This is what turbopuffer raised for: they needed the money to hire two more engineers until the end of the year, in order to build out more of the platform. Then, they would either get more customers and generate enough revenue to not need funding, or shut down the project if it failed to gain traction.
Reason #2: Growth. When you’ve built something and want to tell the world about it, often, you have to spend money on doing that.
Reason #3: Massaging founders’ egos. Simon:
“This is a very, very popular reason! You see big numbers, you get lots of press. But I think it’s a really dangerous reason to raise money.
I wish this reason was talked about more, because you are diluting all of your employees when you do it. And for some people, it can become a status game. It’s not what we are about: we’re here to build a big business together.”
Reason #4: Employee rewards. A startup is a long journey, and everyone wants to work with the best people. But there are necessarily few of these folks, so you want to reward the standout ones by raising money so that employees can sell their shares. This was why, in December 2025, turbopuffer raised a round where they let employees sell some of their equity: it meant they would not have to wait for a future financial event like an IPO.
Reason #5: Strategic partnerships. It might be the case that raising from a VC whose network of connections is key to your business succeeding, or that taking funding from a strategic investor gives you access to their platform or services.
Reason #6: Mergers & acquisitions (M&A). Purchase another company with the funding in order to expand the business.
Simon encourages founders to be honest about why they raise capital funding – and to watch out if the reason is egocentric!
It’s inspiring to see how far it’s possible to get with “napkin math” and by understanding the bottom layers. It didn’t sit right with Simon to make decisions about vendors based on hastily written benchmarks that might measure the wrong thing. By understanding the constraints of data transfer latency and storage cost, he found a way to estimate the theoretical lower bounds of the system.
This helped lead to more informed design decisions at Shopify, and showed him there was an opportunity to build a faster, cheaper search product than the status quo.
Getting “lucky” in business requires a lot of skill, and in-person greatly helps with first impressions. Turbopuffer’s first customer being Cursor sounds almost too good to be true. But the account from Cursor cofounder Sualeh Asif – and now from Simon – reveals the ingredients in more detail:
Spotting a new business need: AI-native startups like Cursor saw their search bills explode, but needed search functionality to offer usable AI products
Offering a product with a magnitude of lower pricing: what people made suddenly pay attention to turbopuffer was the promise of not “just 20-50%” cost savings, but a seemingly radical, 90%+ reduction in costs. When you’re late to enter a market (like search), you need major differentiation, then deliver on it! This is also what Cursor’s attention.
Building trust before a sale: Simon helped the Cursor team fix their existing database before discussing using his product
In-person impressions: if Simon had not flown to San Francisco to meet the Cursor team in person, would they have taken a bet on turbopuffer?
Launching at the earliest opportunity: none of this would have happened if Simon did not announce the first version of the product when he knew it could work, but was still in a pretty unpolished state!
It’s always helpful to be aware of the dynamic of venture funding – some of which are rarely mentioned. Simon knew investors expect returns and growth on timelines which they set. Raising money is helpful in many situations, but problems arise from egotistical reasoning, and Simon believes too many founders prioritize them – knowingly or not.
The Pulse is a series covering events, insights, and trends within Big Tech and startups. Notice an interesting event or trend? Hit reply and share it with me.Today, we cover:Grok’s CLI uploaded all your local files to the cloud, then got caught. Just as devs were starting to use the Grok CLI – thanks to the capable Grok 4.5 coding model – they discover…
Read more
The Pulse is a series covering events, insights, and trends within Big Tech and startups. Notice an interesting event or trend? Hit reply and share it with me.
Today, we cover:
Grok’s CLI uploaded all your local files to the cloud, then got caught. Just as devs were starting to use the Grok CLI – thanks to the capable Grok 4.5 coding model – they discover…
Stream the latest episodeListen and watch now on YouTube, Spotify, and Apple. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.Brought to You by• Antithesis — with Antithesis, you can use AI agents to work on critical systems without worrying about correctness. Going far beyond code review, you can run your complete system in a hostile environment, analyze its behavior, and reproduce every issue perfectly. Teams like Jane Street, Fly.io, and the et
Listen and watch now on YouTube, Spotify, and Apple. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.
Brought to You by
• Antithesis— with Antithesis, you can use AI agents to work on critical systems without worrying about correctness. Going far beyond code review, you can run your complete system in a hostile environment, analyze its behavior, and reproduce every issue perfectly. Teams like Jane Street, Fly.io, and the etcd community use Antithesis to ship better code, faster. Learn more.
• Buildkite —the CI platform trusted by OpenAI, Anthropic, Cursor, Meta, Uber, Ramp, Nvidia, Airbnb and many more. Buildkite was stress-tested at the largest scale inside companies solving some of the hardest engineering problems. It’s built to reliably manage whatever your coding agents throw at the build queue, today, next year and beyond. Learn more.
• Sentry – application monitoring software built by developers, for developers. Sentry’s Seer AI agent is one of their new, neat tools, as a way to debug errors faster. Same with Sentry’s MCP server. Check out Sentry.
In this episode
Knowing how LLM contexts work and how to work around context limitations – aka “context engineering” – is becoming more important for software engineers working with LLMs. Let’s look into what works and what doesn’t, today.
In this episode of The Pragmatic Engineer podcast, I sit down with the CEO and cofounder of HumanLayer, Dex Horthy, who coined the term “context engineering”. We discuss the ideas behind this context engineering, harness engineering, loop engineering, software factories, why his approach to AI-assisted software development has evolved, and how HumanLayer is helping engineering teams automate more of the software development lifecycle without sacrificing code quality.
Key observations from Dex
Here are 12 useful points Dex made in our conversation:
1. Dex talked with ~100 “real” AI Engineers and wrote the popular ‘12-Factor Agents - Principles for building reliable LLM applications’ based on what he learned. Around August 2024, he started to build AI agents when the common approach was to use frameworks like LangChain and CrewAI, and also talked with around 100 AI engineers doing tangible work such as taking on $100K+ contracts to ship AI solutions within enterprises. They tried those frameworks and discarded them in favor of building their own pipelines. Dex shared the learnings from the conversations in his hit book, 12-Factor Agents. It’s a great read!
2. Lesson learned: Shipping unread code spells disaster within months. Dex experimented with having the model write the code and humans not reviewing anything in July 2025. Four months later, they shut things down and threw the whole system out. Production broke, and no matter how much the team prompted Opus 4.1, the model could not find the root cause.
It took days of wading through spaghetti code to discover the primary key wrongly routed through the complete codebase. Once fixed, it took three weeks to re-onboard to a codebase no human had ever read. Today, Dex thinks this problem wouldn’t even take four months to develop, now that newer models produce code a lot faster than a year ago.
3. Today’s coding models are most likely trained in a way that makes codebases worse over time. Dex believes that the reason we see LLMs “degrade” existing codebases is because they are optimized to do well on SWE-bench-style benchmarks. These benchmarks reward reproducing a known fix in codebases like Django, but cannot measure poor architecture decisions. This is because the cost function of bad architecture and bad program design cannot be evaluated by running a unit test.It’s a tricky problem to solve: Dex’s best guess is an eval of a model building 20 features in a row in a codebase without knowing what’s coming next.
4. Context engineering 101: figure out where the “dumb zone” begins. As a rule of thumb, the less of the context window that is used, the better the outcomes are. This is because the attention mechanism is quadratic: the more that goes into the context window, the more compute is required to process it all. We cover more about self-attention scalability challenges in our ChatGPT deepdive.
For a model with a 1M context window, Dex pushes it to around 300-400K when it feels right. For smaller models, he stops at around 100K. You hit the “dumb zone” when its performance starts to degrade because the context window fills up beyond this heuristic limit, and the model begins doing increasingly stupid things like deleting your .env file, for example.
5. A larger context window does not mean a smarter model. Models’ intelligence is behind the ability to use the tokens in the context window, by deciding which parts of the context are relevant for the next decision. You have to get a feel for how much context usage makes sense, and experiment when you hit the “dumb zone.”
6. Frequent, intentional compaction is a technique Dex uses for more complex projects. He will take a long and noisy context, compress it into a Markdown document, then start a new session fresh, pointing the model to this “compressed context” that is in the Markdown.
A workflow he uses:
One session reads a ton of code (filling up its context window while in the “smart zone”), then emits a research document
The next session takes tickets describing the work to be done and turns it into a design document
The following session takes both documents to create a plan
The human is in the loop where it really matters: in this case, reviewing the design document and architecture because Dex finds models to be weak on this
7. Don’t bother optimizing LLM usage until business is booming, there’s massive scale, or high costs. Dex suggests to always start building software with the smartest available model to solve the problem, since engineering time is almost always the bottleneck. Begin to optimize LLM usage and context usage only when at real scale and costs are high enough. That’s when it can be worth using GPT-OSS-120B (1/1,000th the cost of Opus) for the simpler steps in your process.
8. “You’re completely right!” or “you’re right to push back on that” are phrases that mean it’s time to start a new session. These responses mean the LLM session is trajectory-poisoned, and you’re wasting time and tokens to continue. Models are autoregressive, so if you get into this loop of:
Model makes a mistake
→ user “yells”
→ model keeps making mistakes
→ user “yells”
… the model calculates that the next most probable message is to make another mistake!
9. Only four things matter in the context window:
Size: the bigger it is, the more space you should have before hitting the “dumb zone”
Information quality. Once something is in the context window, every subsequent turn treats it as fact. This is why errors can compound.
Missing information: if there’s information missing that the agent would need, the outcome will be worse, as the agent fills in the gap with guesses.
Trajectory. Models are autoregressive, so they predict the next message in the conversation based on previous ones in a kind of thread of reasoning. “Trajectory poisoning” is when the agent gets into a pattern of doing things you don’t want. In this case, it’s time to start over.
10. Slow loops are Dex’s favorite way to do ‘loop engineering.’ The HumanLayer team started with a nightly automation setup that kicks off an agent to fix one thing in the codebase, and open a pull request. In the mornings, they woke up to a PR waiting to be merged. They tweaked it over time and now have four agents open a total of four PRs by the morning, with the focus on code quality improvements. A person still reads all of them before merging.
11. “Token harder” vs. “token smarter”: Dex is in a group chat named ‘Hyper Engineering’, where members share advice on how to max out their Claude subscriptions. This approach, he calls “token harder”. On the other side is “token smarter”: aiming to get maximum value from AI while keeping control. Smarter is harder to pull off.
12. Three ways to run a “software factory.” Here’s options Dex sees as viable:
“Turn the lights off:” go all-in on agentic coding, do not review the code, and pray that AI doesn’t create too much slop. Dex tried this and failed.
Read and review all AI-generated code. This slows things down to human speed. Dex says that this way, you should expect a 30-50% lift in productivity from AI, compared to pre-AI engineering.
Find leverage, but keep people in the loop. Find out where an hour spent in planning could save four hours’ worth of implementation, in terms of fewer bugs. Invest more time in areas with leverage: design, architecture, and key decisions. Then, let the agent generate code and don’t insist on reviewing all of it. In this way, Dex believes you can move 2-3x faster than when devs wrote all code by hand.
The Pragmatic Engineer deepdives relevant for this episode
“Loop engineering” has become a trending topic in the past month, after some high-profile folks at Anthropic and OpenAI revealed that they have stopped writing prompts, and started designing loops. At Anthropic’s developer conference, Boris Cherny, creator of Claude Code, said (emphasis mine):“I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops.”Soon after, Peter Steinberger, creator of OpenClaw, preached loop design in a p
“Loop engineering” has become a trending topic in the past month, after some high-profile folks at Anthropic and OpenAI revealed that they have stopped writing prompts, and started designing loops. At Anthropic’s developer conference, Boris Cherny, creator of Claude Code, said (emphasis mine):
“I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops.”
Soon after, Peter Steinberger, creator of OpenClaw, preached loop design in a post:
Elsewhere, Addy Osmani, formerly of Google, wrote an article, ‘Loop Engineering’:
“Loop engineering is replacing yourself as the person who prompts the agent. You design the system that does it instead.”
That’s three mentions in quick succession of this new approach, which is a novel one and therefore pretty abstract to me. To find out more, I turned online to some of the folks who read these articles. In replies, you told me what “loop engineering” means to you and gave some examples of loops in your work.
Today, we cover:
Where it began: “Ralph Wiggum” loop. A year ago, software engineer Geoffrey Huntley shared how he builds “loops.” In December, the approach went viral and “Ralph loops” were born.
The /goal command ships in all major harnesses. By May this year, the major AI coding harnesses added support to run a loop from a single prompt, using the /goal command.
Loops which devs use: triggers and cron jobs. I asked devs how they use loops. Most use cases involve responding to events or running scheduled jobs. They are useful, but don’t feel like brand new workflows.
Helpful loops for devs. Open PRs for newly-recorded app issues, have notes ready when an oncall joins an outage Slack channel, “babysitting” and fixing nightly end-to-end tests, and more.
Disappointment and “tokenmaxxing”. Several devs reject looping after trying it. Agents drifting, and the “human in the loop” having better results are some reasons. Also, at companies that pay API prices for tokens, loop engineering gets expensive fast.
Was looping a hack while tooling caught up? Distinguished engineer Max Kanat-Alexander believes the “loop” might have just been a temporary hack while the harnesses added the ability to do the same from a single prompt.
Does “context engineering” matter more for devs? Except for engineers building AI infra, there seems little benefit in going deep into loop engineering. Instead, becoming familiar with AI context windows – also part of building loops – could be more useful.
1. Where it began: “Ralph Wiggum” loop
Exactly a year ago, software engineer Geoffrey Huntley published the article ‘Ralph Wiggum as a software engineer’. The name references the naive son of the local police chief in The Simpsons, who is extremely eager to always be helpful. In engineering, “Ralph” is intended to continuously nudge the agent in the right direction. Geoff described it:
“Ralph is a technique. In its purest form, Ralph is a Bash loop:
while :; do cat PROMPT.md | claude-code ; done
Ralph can replace the majority of outsourcing at most companies for greenfield projects. It has defects, but these are identifiable and resolvable through various styles of prompts.
That’s the beauty of Ralph - the technique is deterministically bad in a nondeterministic world.”
The article expands on the idea of the Ralph loop:
Start the agent with a prompt that captures the task and defines a goal
Create a plan of work, each item with a success criteria
Start the loop:
Take one item per loop
When the agent is done, check if the goal is achieved
If not: start the agent again, with a clear context window
Start loops if needed: spawn subagents as and when necessary
Geoff published the experiments he did with this approach, such as building a new programming language last summer, and said it requires skill:
“Engineers are still needed. There is no way this is possible without senior expertise guiding Ralph. Anyone claiming that engineers are no longer required and a tool can do 100% of the work without an engineer is peddling horses***.”
The “Ralph method” blew up late last year with the arrival of better models which were surprisingly capable of building ambitious projects. Software engineer Matt Pocock created a tutorial, ‘Ship working code while you sleep’ with the Ralph Wiggum technique. He said:
“One of the dreams of coding agents is that you can wake up in the morning to working code, where your coding agent has worked through your backlog. It has spit out a whole bunch of code for you to review, and it works.”
Before the Ralph loop, Matt did this in two steps:
Ask the agent to create a detailed plan for the work, with tasks broken out
Then, in a sequential order, have the agent complete each subtask in a separate run
Pre-Ralph: The agent completes one step at a time in a single context window. Source: Matt Pocock
A problem with this approach is there’s no easy way to add new tasks to the “masterplan”. Software engineers know that most plans do need to be modified, often while the work is ongoing. In contrast, with Matt’s take on the Ralph method, the “masterplan” is continuously updated in a “master PRD”. Here’s the prompt he gives the agent:
Choose the next feature: Find the highest-priority feature to work on and work only on that feature
Have tests pass: check that the tests pass (via pnpm test)
Update the master tracker: update the PRD with the work done
Log work: append your progress to the progress.txt file
Commit: make a git commit of the feature
This style of working is more of a “dynamic Kanban”:
“dynamic Kanban” style of working. Source: Matt Pocock
The “Ralph method” is all about working around context window limitations. Back in mid-2025, the maximum size of a context window was around 200,000 tokens. That’s not enough for more ambitious tasks, so it’s necessary to break up agent runs into smaller ones and run them, one by one. In this context, here’s where the Ralph method works:
Have a goal for a project, and keep running (or re-running) agents until this goal is reached
Persist work done in a “compressed” manner on the filesystem (as logs or an updated plan)
Start agents with fresh context to minimize “context rot”
Allow each agent to add or modify the “masterplan” if needed
2. The /goal command ships in all major harnesses
For a few months, building a Ralph loop meant doing it yourself: setting up the loop, state tracking, deciding how the agent can add tasks and when to stop. But things changed once coding harnesses made it easy to run these loops.
April: Codex ships /Goals
About six months after the “Ralph technique” started gaining wider traction, Codex shipped the “Goals” feature in Codex. From the documentation:
“Goals are persistent objectives in Codex that keep a thread working toward a defined outcome across turns. A Goal gives Codex a completion condition: what should be true, how success should be checked, and what constraints must stay intact.”
Also from the docs (emphasis mine):
“A normal prompt says: do this next thing.
A Goal says: keep working until this outcome is true. In a normal request, Codex works through the immediate instruction, reports a result, and waits. With a Goal, Codex has a durable target attached to the thread. After a turn finishes, it can inspect the current evidence and decide whether the objective is satisfied. If the answer is no, and the Goal remains active and within budget, Codex can continue from the latest state.”
/goal Reduce p95 checkout latency below 120 ms on the checkout benchmark while keeping the correctness suite green
That’s a clear enough “end criteria” to just hand off to the agent, which then breaks up the task, spawns subagents, and runs until complete. So, how did OpenAI build Goals? They used files, logs, running tests, and lifecycle controls:
The architecture of the Goals feature. Source: OpenAI
In this way, “Goals” feels awfully similar to a Ralph loop, except compressed into a single command! It feels like the Codex team took the idea of the Ralph loop, built infrastructure around it, took care of coordinating agents by not having them step on one another, dealt with state, running of tests, starting and stopping agents, and then added functionality such as being able to set a budget.
May: Hermes and Claude Code follow with /goal
Three days later (May 2), Hermes agent, a popular AI agent framework and OpenClaw rival, shipped their implementation of /goal. From the docs:
“[/goal] It’s our take on the Ralph loop, directly inspired by Codex CLI 0.128.0’s /goal by Eric Traut (OpenAI). The core idea — keep a goal alive across turns and don’t stop until it’s achieved — is theirs. The implementation here is independent and adapted to Hermes’ architecture.”
Less than two weeks later – on 12 May – Claude Code also shipped their /goal command. It’s identical in what it does to Codex. From Claude Code:
“The /goal command sets a completion condition and Claude keeps working toward it without you prompting each step. After each turn, a small fast model checks whether the condition holds. If not, Claude starts another turn instead of returning control to you. The goal clears automatically once the condition is met.”
A few months before, in March, Claude Code shipped the concept of scheduling an agent with the /loop command. It is basically what JavaScript’s setTimeout() function would be equivalent to: repeat a task after a given interval until the work is done.
By May, running a Ralph loop had become as simple as giving a single command in one of the major agent harnesses. It’s as if AI labs noticed user demand to do more with agentic loops which were hard to set up, and built ways to make it simpler. For instance, in open source agent harnesses like OpenCode, there are plugins like the /goal plugin. For the minimalist coding agent, Pi, the /goal command can be added as a package.
3. Loops which devs use: triggers and cron jobs
By May, we had access to the /goal primitive. So, what use cases were Boris Cherny and Peter Steinberger referring to in terms of spending most of their time on designing loops, instead of writing prompts? I asked around for examples of “loop engineering” from fellow devs. Based on ~210 replies, mostly from X and LinkedIn, it seems that triggers and cron jobs are two very common use cases for loop engineering:
Triggers / automations: an agent kicks off when an event happens. The event could be an error being logged, a new ticket created, customer support feedback received, etc.
Pre-AI, these events were typically triggered by a webhook, and kicked off things like a Slack bot posting in a channel, triggering a system, or being the starting point of a Zapier or an n8n integration.
Cron jobs: many devs see “loop engineering” as kicking off jobs that involve agents on a cadence. This is fundamentally the same as scheduled cron jobs. From director of engineering, Oded Messer:
“The idea is that strategic workflows that are repeatable and automatable can be done so with an agent. OK. But if my strategic workflow is automatable then it either becomes tactical if the AI is capable enough or it’s just a high level old-school-automation I can set up like a cron or a trigger.
The name suggests simplicity and repetition. Sometimes it feels like AI enthusiasts forgot automation was a thing before LLMs.”
4. Helpful loops for devs
Below are some workflows with AI agents that run on a regular basis as a /loop command, or when triggered:
Development-related work
Open PRs for newly recorded app issues. Software engineer Ivan Pantić:
“App encounters a problem and creates a sentry issue. Then: → Cron tells agent to check sentry and open PRs. →If there’s no active PR, agent tackles the issue and creates one → If PRs aren’t reviewed, agent pings the devs via slack
In this flow, there is only one PR open at a time.”
Fix flakey tests. Paul D’Ambra, software engineer at PostHog:
“/loop pull the next flakey test from the trunk API. run it to check if it flakes locally, if it does open a PR with the fix... which netted me 13 PRs to stabilise some of our tests.”
Triaging issues and outages. Software engineer Ivan Abad:
“A new alert/exception pops up in a channel. → The agent investigates the issue → if it is a code change, implements the change → creates the PR → pings the human for review.
Same applies for customer tickets or incidents. By the time you get paged and connect to the incident the agent already triaged everything and often located the root cause.”
Review design plans. Artem Nikitin, software engineer at Elastic:
“What I’m finding myself doing often recently is to review design/implementation plans in a loop.
Usually, agents only find a few issues during a normal run and then find more on subsequent runs.
So I’m now asking them to run in a loop until they find 0 new major issues.”
Daily/nightly work
Daily product improvements. Jack D, founding engineer at Schematic:
“We have a loop which reads the logs for the last 24 hours, user feedback and makes PR with fixes - we still review the PRs it makes though!”
Nightly end-to-end test run babysitted by an agent. Utku K, engineering manager:
“nightly e2e runs on the frontend app. When a test fails, an agent investigates first to work out whether it’s a real regression or a false negative. If it’s a real bug, the agent attempts a fix, reruns the test, and keeps iterating until it passes or hits a retry cap and escalates... Either way it lands as a PR ready for review by morning.”
More complex development work
Build new telemetry integrations and verify they work. Lawrence Jones, software engineer at Incident.io:
“A lot of the AI runbook that we use to build new telemetry integrations uses loops such as executing the query, verifying that it executed correctly and iterating on the query plan/output format/whatever until it is happy with the outcome.”
Do a long-running migration, mostly autonomously. Startup founder Rafel Mendiola:
“Recently, I converted my startup’s codebase from a regular React app to a React Native app using Expo.
There are two ways I could have done it:
In the traditional software engineering style, I could have created a large epic with 50-100 different tickets. I started building myself the infrastructure for that, it felt like it was way too much work.
What I did instead was create a skill that would let an agent figure out a small to medium-sized piece of work or piece of code to convert, given certain detection mechanisms and guidelines, and then do that conversion and keep track of the migration progress. Then I put that skill on a cron job. This was a lot easier to handle cognitively than managing a large migration plan. It also ran every 30 minutes, not nightly or weekly.”
The Pulse is a series covering events, insights, and trends within Big Tech and startups. Notice an interesting event or trend? Hit reply and share it with me.Today, we cover:Bun’s Rust rewrite with Fable: what can we learn? To a sceptic, spending $165K to migrate Bun from Zig to Rust sounds very expensive. But to a realist, shortening a 1-2 year migration down to 11 days opens amazing new opportunities for devs. However, a thoroughly-tested project is required to pull it off.Anthropic’s Fable,
The Pulse is a series covering events, insights, and trends within Big Tech and startups. Notice an interesting event or trend? Hit reply and share it with me.
Today, we cover:
Bun’s Rust rewrite with Fable: what can we learn? To a sceptic, spending $165K to migrate Bun from Zig to Rust sounds very expensive. But to a realist, shortening a 1-2 year migration down to 11 days opens amazing new opportunities for devs. However, a thoroughly-tested project is required to pull it off.
Anthropic’s Fable, OpenAI’s GPT-5.6 Sol, Cursor’s Grok 4.5, Meta’s Muse. Coding LLM wars heat up: Fable is back, OpenAI releases a comparable GPT-5.6 Sol, Cursor offers cheap & very capable Grok 4.5, and Meta is back with its first truly competitive coding model since Llama 3. But how did Gemini slip out of the top-ranked AI coding models?
North Korean hackers keep trying to infiltrate full-remote companies. The founder of a Canadian digital consultancy caught a North Korean dev red-handed, using an AI filter. These events are now so common that it’s hard to trust remote interviewees are who they claim.
Industry Pulse. Meta’s key logging exposed sensitive data, massive cuts at Xbox, Meta could not buy enough AI capacity from Google, Qualcomm acquires Modular, and memory price hikes hit Apple products.
1. Bun’s Rust rewrite with Fable: what can we learn?
Last week in San Francisco, I met Jarred Sumner, creator of JavaScript runtime, Bun, and was keen to learn more about the rewrite of Bun from Zig to Rust. But at the time, Jarred didn’t want to say too much, as the tool used for the migration, Fable, was out of action due to the US government imposing export controls.
Jarred and I at Anthropic’s HQ, last week
Fortunately, the situation is now resolved and Fable is available globally, and Jarred has published a detailed post about the project. Before we get into the migration, some context:
Stream the latest episodeListen and watch now on YouTube, Spotify, and Apple. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.Brought to you by our presenting partner, Antithesis.Verify your system’s correctness by running your whole system in a hostile simulation and finding bugs. I’ve been using Antithesis myself, and I’m impressed with their innovation in building new kinds of debugging tools. Like this:Bug probability analysis in Antithesis’ f
Listen and watch now on YouTube, Spotify, and Apple. See the episode transcript at the top of this page, and timestamps for the episode at the bottom.
Brought to you by our presenting partner, Antithesis.
Verify your system’s correctness by running your whole system in a hostile simulation and finding bugs. I’ve been using Antithesis myself, and I’m impressed with their innovation in building new kinds of debugging tools. Like this:
Bug probability analysis in Antithesis’ fully deterministic, simulated environment. When probability spikes, it’s a good place on the timeline to “rewind” and check logs to find what triggers bugs.
In this special “ask me anything” episode of Pragmatic Engineer podcast, I am in the hot seat facing questions sent in by subscribers that are read out by guest Volodymyr Giginiak, CTO and cofounder of Wordsmith AI, a legal tech startup (note: I’m an investor).
I tackle your questions on the software industry, AI, hiring, engineering organizations, career growth, the business model of the Pragmatic Engineer, and more. We also discuss where software engineering is headed, and I offer advice on some specific situations. Thanks to everyone who sent questions!
Three stories & observations
Story #1: Without the COVID-19 pandemic, The Pragmatic Engineer might not exist. Prior to the global health crisis in 2020, I had no plans to get serious about writing: I enjoyed blogging on The Pragmatic Engineer blog, but intended to remain an engineering manager or software engineer for the foreseeable future.
But then, COVID-19 happened and Uber made layoffs, which led to a quarter of my team being let go, while the rest of us were disbanded into other teams. It was a tough time, and I decided it was a good moment to exit and finish writing a book I had been working on, ‘The Software Engineer’s Guidebook’. After that project was complete, I planned to try and start a VC-funded startup and build something around platform engineering; possibly a system for tracking RFCs at mid-sized and larger companies.
I gave myself around eight months to finish the book, but when that deadline elapsed, it still wasn’t ready. I did write three other books (‘The Tech Resume Inside-Out’, ‘Building Mobile Apps at Scale’, and ‘Growing as a Mobile Engineer’) and still wasn’t convinced by any startup idea. But I did discover that I like to write!
Story #2: I was on track to publish a damning exposé, until one message from an engineer changed my mind.
During the first year of The Pragmatic Engineer, I wrote about conditions for engineers at the Dutch neobank Bunq, based on accounts from disgruntled employees. I had a final draft ready, which I sent to the company to provide a right of reply ahead of publication. Then I received a message.
It was sent by an engineer originally from the Middle East. They told me they had really wanted to break into the European tech industry, but that no company would sponsor their visa, except Bunq. Yes, the company was a tough place to work at as a dev – and this engineer subsequently left for another opportunity – but they appreciated Bunq for taking a chance on them that enabled them to move to Amsterdam and learn how to build fintech with a small team. This engineer now works at Meta and attributes their success to the break Bunq provided.
Based on that interaction, I opted not to publish the article, and it also led me to adopt a new editorial policy that I have followed since: I write about what works inside companies, instead of focusing on what seems to be broken.
Story #3: Being called a “nobody” by a CEO led to my sole investigative piece, which uncovered some pretty interesting details. After I briefly noted Pollen’s poorly-handled layoffs, CEO Callum-Negus Fancey dismissed my report during a company all-hands, and compared The Pragmatic Engineer unfavorably to the BBC as just some minor publication with an agenda against Pollen (why I would have an anti-Pollen bias isn’t clear to me!) To be honest, I took it personally when I heard a recording of this sent to me by employees there, and so started digging around.
I discovered unpaid salaries, silently cancelled health insurance in the US, and the CTO deliberately triggering a $3.2M double charge to customers and never publishing a postmortem, despite engineers requesting one.
It was certainly something, and I published the findings in the article Inside Pollen’s Collapse: “$200M Raised” but Staff Unpaid - Exclusive. To ensure the CEO saw the report on a platform he deemed worthy, I also contributed to the BBC’s documentary: Crashed: $800M Festival Fail, aired in the UK during prime time. By doing all this, I also learned that investigative journalism is just not for me.
Opinion #1: I believe LeetCode-style interviews will stay because they self-select tolerance of corporate nonsense. The existence of data structures and algorithm (DSA) interviews is a bit of a head scratcher because these skills are rarely used at work. However, a candidate who’s willing to grind for weeks or months to prepare for an interview which bears little resemblance to the job, is likely to be someone who understands that sometimes it’s necessary to do pointless work.
This suggests they’ll probably have a much better time in Big Tech than someone who refuses to engage with meaningless tasks. It’s one reason I’ve observed for companies retaining LeetCode-style interviews. Of course, AI solves the puzzles with ease these days, and I expect larger companies to move back to in-person interviewing – all while keeping DSA interview questions.
Opinion #2: MCP became industry standard partly because Anthropic wasn’t a threat – but it couldn’t pull this off today. When MCP launched in November 2024, Anthropic wasn’t yet considered the leading AI lab. GPT-4o was seen as the top-performing multimodal model, followed by Claude 3.5 Sonnet and Gemini 1.5 Pro. At that point, Claude 3.5 Sonnet was seen as the best coding model, but it wasn’t understood how advantageous being good at coding would be for AI in general.
Therefore, OpenAI, Google, Microsoft, and other players could adopt MCP without fear of lock-in, as it came from a promising, but not a dominant lab. When Google launched its Agent2Agent protocol a few months later, no major lab adopted it due to concerns about Google’s dominant position. Today, Anthropic is the leading frontier lab and I reckon this would discourage adoption if MCP was launched in the present climate, for the same reason as nobody adopted the Agent2Agent protocol.
My answer to subscribers with questions about how to create standards is that I see them as emerging somewhat coincidentally, following a technically strong approach and with the right external conditions in place, which are impossible to entirely predict or control.
Opinion #3: My hot AI take: AI doesn’t make work easier, and be mindful of skill atrophy. If you’re using AI and life seems to be getting a lot easier, it raises the question: are you trying hard enough? Personally, using AI forces me to think just as hard, or even harder than before. I choose to use zero AI in my writing for the Pragmatic Engineer, and Grammarly is turned off as well. This is because I don’t want my writing skill to degrade, and would like to keep improving. On the other hand, with coding, I do use AI and accept my hand-coding ability will unavoidably degrade. My tip is to be mindful of the tradeoffs inherent in AI, and to keep using those skills which you value and want to keep sharp, even when AI tools are available.
The Pragmatic Engineer deepdives relevant for this episode
Connexion
