Article summary
Whether you use or even like AI, the genie is out of the bottle. What used to feel like a more personalized Google search can now index your entire codebase, explain how unfamiliar areas are structured, warn you of system design implications, and build features of surprising complexity. (Assuming, of course, you have the vocabulary to prompt it to do so.) From both a capability and flexibility perspective, it’s astonishing technology. I’ve moved beyond the basic question of whether AI can write code; it most certainly can.
Instead, I find myself asking, “If my experience shows it can do all these things, then why do I still feel so skeptical?”
AI Engineers’ World Fair 2026
I’m now on my fourth professional, agentic project, meaning I am mostly not writing my own code anymore. I describe what I want to the LLM and it makes the changes. Then I review. This approach has certain advantages, particularly if you can delegate and parallelize work effectively.
But the lived experience leaves something to be desired. It’s been a gauntlet of relearning what my job as a developer even is anymore. (Admittedly, the natural pains of structural change contribute here.) I’m often battling mental fatigue from deciphering wordy LLM responses, deciding how best to course-correct, and wondering whether my commitment to open-mindedness is costing me my intelligence in the long run.
I keep delivering results on time, but feel less prepared for common questions like, “How long will this feature take to build?” In this paradigm, I’m not fully sure yet. It’s easy to end up acting more like Claude’s secretary, passing along such questions instead of answering as the expert I know I am.
So, when the opportunity was presented to visit the AI Engineer’s World Fair 2026, I was intrigued. It meant I would travel to San Francisco for four packed days of AI keynotes, workshops, panels, expo booths and more, my curiosity won out. I wanted to know how this premier conference would discuss the most controversial technology of our time. I hoped it would reveal how I could improve my day-to-day work a bit, too.
Arrival
Immediately upon arrival, the attention to presentation polish was obvious. Huge, crystal-clear screens rendered gorgeous high-res 3D models. Soaring music and sound effects made speaker transitions feel impactful. One speaker even noted it was a first for them to walk onto a stage through fog machines. Full-on spectacle. And I’ll admit, it did well to capture a feeling of momentum, inevitability. Everything a tech conference should be.
Unfortunately, the wifi couldn’t keep up. At the first demo I attended, the host instructed us to clone their example repo. The network choked when the audience tried. I couldn’t believe it. How much investment had gone into this, and nobody thought to load test the network access points? It’s not like this was some shack in the middle of nowhere. This was the Moscone center in downtown San Francisco! The hiccup left some presenters floundering, unable to quickly autocomplete through their demos using AI like they had practiced. I wondered how many new developers were destined for similar embarrassment due to lower-level issues their AI failed to foresee.
Although that somewhat dampened the atmosphere of prestige carefully cultivated by the lights and theatrics, I was willing to look past it. Live demos are notoriously difficult, and as attendees switched to hotspots or stopped trying to use the wifi, the presentations began to flow better. I still wanted to glean useful bits of knowledge. So I listened patiently as key buzzwords emerged: loops, evals, inference, embeddings, vector-space, forward-deployment, voice agents. All interesting, but how should I apply them? My goal became to find anything I could bring back with me and try the following Monday.
The Takeaways
Evals
My first takeaway came after listening to several talks on “evals.” The broad definition of “evals” are tests used to assess whether the agent does the right thing, on average. You need a bank of human-verified questions/scenarios and answers/outcomes in a measurable format, e.g pass/fail, agree/disagree. You pose each of these questions to the AI model, and test how closely it matches the human answers. The tough pill to swallow is that instead of aiming for a suite of tests that pass 100% of the time, the goal is to set some high threshold of alignment (say, 80%) and declare the evals as passing if the model’s answers agree with the human-verified answers at least that much.
This approach is intended to design tests for the non-determinism of LLMs. (It also can come across as just a worse test suite, depending on your testing setup and philosophy.) I think that the interesting part of this technique is in how failures are addressed. When a model fails to meet the success threshold, the given advice is to look for gaps in the model’s knowledge. Does it typically fail on certain kinds of questions? If so, how can you supply that knowledge to it?
For example, you might update your internal prompts with new paragraphs outlining things like your term glossary or return policy. You can then expand your answer bank in those areas, and confirm that better knowledge leads to a better model. This testing approach is something I think is worth trying out in order to gain confidence in your agentic systems. I’m certainly going to. My current explanation needs further refinement through experience.
LLM Family Preferences
Another useful piece of information is that emerging research shows that models are biased toward content produced by their own LLM family. What this means is that given a group of options, LLMs will skew toward selecting their own material. It’s not just exact model match either. Codex seems to prefer content created by its earlier models over anything generated by Claude, and vice versa. This is a big consideration if you’re designing an LLM-as-judge system. Any time you have an LLM evaluating output from another LLM subsystem, you need to be aware of this bias, and intelligently select the right model for the job.
Homa
As soon as John Ousterhout took the stage and began speaking, I immediately sat up. Unlike most of the talks I’d been listening to, this was instantly recognizable as *Real Engineering*, the type of presentation helmed in the academic sphere. He spoke about how he designed a new protocol, Homa. And based on that design, why it would likely yield incredible performance in data center networking. I am still digesting this myself, and I’m considering writing a follow-up post on this topic alone. But this could allow for compute gains without requiring additional data centers. See this article for more info: The Homa network protocol.
A Gripe
While some of us wait for what feels like an inevitable bubble burst, others gather in huge swaths in support of AI growth and expansion. Having attended AI Engineers’ World Fair, I’m now more aware of the passion and excitement many people from around the globe are bringing to agentic development. And in some ways, it gives me hope. I wish to see this community continue to flourish.
At the same time, I was disappointed to find no mention of some rather important topics. I didn’t encounter anyone interested in discussing the more controversial elements of data centers, their ecological impact, or any concept of an AI bubble. I’d never expect these things to be front-and-center, but represented nowhere? That’s noteworthy.
I bring them up so that I might err on the side of saying too much. Too many of my peers are being forced to adopt technologies they don’t believe in. Not because the technology isn’t sufficient to achieve their ends, but because there is too much being swept under the rug, so to speak.
Data centers are being built in rushed, slimy ways. Plain and simple. These facilities greatly impact their areas. Accordingly, they need to be approved by the people they affect. And research needs to continue on their ecological impact. I would prefer we pause building them and work with the centers we have. I believe that this limitation would ultimately be better for the AI community. It will incentivize optimizations such as Homa and allow us more time to understand this new technology without disrupting other people’s communities. I can report that for the most part, as long as you’re willing to take the time to read through and think about what the LLM says, while being vigilant about course-correction, they are already exceptionally powerful.
Still Skeptical
And so, my skepticism comes from the gap between what the technology can legitimately do, and the often tone-deaf ways AI leaders talk about and push it. In one of the closing keynotes, a speaker called on us to be ambitious and think bigger. In reference to the classic corporate phrase, “Let’s not boil the ocean,” he excitedly declared, “We can boil the ocean now!” This was met with applause, but I’m worried he means literally.