Why I Gave Up on AI Meeting Notes
Plus Code Agents, AutoResearch, and the Loopy Era of AI
“We are deterministic creatures living in a probabilistic universe.” — Amos Tversky
A mix this month: the industry-level logic of where AI compute demand is actually going and the strongest argument there is no bubble, two Karpathy takes, what working with AI does to the user, and three pieces that ask what all this work is for.
I published an article, As We May Work, that looks at how AI is going to impact knowledge workers. If you’re hearing about or using Claude Code/Cursor and Codex, I think it’s a good framing for how to work with those tools effectively.
I’m publishing a daily thought on X, LinkedIn, and Substack Notes - follow along on whichever platform you’re most active on.
Working With AI
I Was an Enthusiastic Early Adopter of AI Scribes. Here’s Why I Stopped
by Dr. Benn Gooch + Venkat Rao’s comment
One of the most widely adopted early use cases I see for people with AI is note taking apps (AKA AI scribes). This is an account of a physician who quickly adopted them with the very reasonable view of reducing his documentation work and spending more time with his patients.
After about 18 months, he realized that writing the clinical note was important cognitive work, not merely administrative burden. Delegating it silently hollowed out the reflection and orientation components of caring for patients.
Rao’s comment reframes the same observation in systems terms: the risk of AI use is “insufficient orientation.” Reorientation is friction, but it’s important and necessary friction.
I think it’s an interesting use case where the problem with AI is not that it got it wrong, but “the AI gets it right in a way that reduces the overall effectiveness of the practitioner.”
This risks romanticizing friction and I think the optimal amount of AI note taking usage is not zero. But, there is a skill in distinguishing friction doing load-bearing work from friction that’s just overhead.
I am experimenting with this right now and testing out using AI note taking but having a manual step where I summarize the call from my perspective and think about the next steps.
Broadly, I think we need to evolve something like a pattern language for working with AI effectively and note taking is a good first form factor.
Forward Deployed, Episode 4: The Special Forces Model
with Chris Papasadero and Noah Brier
The ambient assumption in a lot of AI discourse is that software (and knowledge work more broadly) is a Six-Sigma-able problem. It’s a manufacturing line that can be de-variance’d with enough process.
Brier argues making software (and, by extension, most high level knowledge work) is closer to making a movie than making a car. If that’s right, the search for “the right agent pipeline” shouldn’t look like Ford’s assembly line (stations, handoffs, QC) — it should look more like Warhol’s studio (assistants producing variations, the artist giving direction, dailies as shared context).
In general, I’ve been thinking a lot about the metaphors we are using for describing and working with AI. Most people are approaching it as a replacement for software and junior level employees. That’s a part of it to be sure, but optimizing a Warhol studio with Ford techniques gets you neither variety nor throughput.
Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI
on No Priors
Karpathy’s first public appearance in a while, and two things from it stuck with me.
First, his use of many agents tiled across multiple monitors, each running ~20 min at high-effort mode, 10 repos checked out, rotating through them delegating macro actions. I am wondering if I am under-utilizing agents through micro-managing when I should be assigning more abstract goals and reviewing their output less often. I think this is for sure true of coding tasks, it is less obvious to me in more open-ended domains. I often find that there is a constant dialogue between me and the agents to help flesh out my intentions which are sometimes hard to state explicitly up front.
Second, his observation that the jaggedness of model progress seems somewhat persistent. A key claim from the frontier labs about progress towards “general” intelligence is that doing specific training in one domain will generalize to other domains. In this view, training the models to be better at coding should also make them better at creative writing because there are some underlying inherent properties those two things (and intelligence more generally) share. I would say that’s a pretty core component of the “AGI is around the corner” thesis.
Karpathy argues it’s not really happening. Agentic coding and research have gotten MUCH better than they were a year ago. But, ChatGPT still tells the same stale “scientists don’t trust atoms” joke it told four years ago. Getting better at coding doesn’t seem to lead to telling better jokes. This suggests there is less general intelligence emerging and that models are getting better along relatively narrow and super-optimized tracks. To the extent this is true, it points towards a longer persistence of the freestyle workflow I talked about in As We May Work where humans are very necessary for ‘stitching together’ powerful but narrow-minded AI models.
Under the Hood of AI
Agents Over Bubbles
by Ben Thompson
An obvious question many people are asking around the large tech company valuation is “are we in a bubble?” Good question!
Thompson argues that the rise of Agentic AI suggests companies really can grow into their current valuations.
At least for the time being, AI labs charge for compute/API-usage: the most you use it, the more you pay.
What agents change is that the unit that actually consumes compute isn’t the chatbot user, it’s the agent. One person can only spend so much time sending messages back and forth to a chatbot. In the agentic paradigm, an agent can keep building and testing its own work in ongoing loops. One person may be managing dozens of agents, all burning through tokens at the same time.
That reframes the math in two important ways. First, agents don’t need broad human adoption to justify hyperscaler capex. If only 2% of people become agentic operators, they can still use a huge amount of compute while producing real economic output. That’s not a bubble, that’s a productivity gain (albeit one concentrated in a small group).
Second, the profits concentrate at the integrated model+harness layer rather than the commoditized model layer. In this paradigm, the harness (Claude Code, Codex) is the moat that lets AI companies maintain pricing power even as open source models keep model costs down. This may or may not play out (I’m dubious about how big of a moat the harness is), but it seems to be Anthropic’s strategy and at least a part of OpenAI’s as well.
The companion piece, Mythos, Muse, and the Opportunity Cost of Compute, adds the frame that there will be two AI businesses: a consumer business (ad-supported, aggregation-theory-native, likely with OpenAI vs. Meta fighting for users) and an enterprise business (not aggregation-theory-driven — Anthropic trying to win on product quality, OpenAI on compute capacity).
Deep Dive into LLMs like ChatGPT
by Andrej Karpathy
The clearest end-to-end explainer of how modern AI systems actually work. This video covers pretraining, tokenization, post-training via RLHF, what sampling is doing under the hood and why models hallucinate.
A lot of these technical points are and will be abstracted away for most people using AI, but a basic understanding of how these models are built has been a helpful intuition for thinking about how to work with them.
The models tend to be “jagged” in the sense that they are very good at some things humans are bad at and very bad at some things humans are good at. One example is that they seem to be worse at basic math (algebra/arithmetic) and better at more complex math. Once you understand how tokens work, you can see why this would be and it gives some intuition about what else the models are likely to be relatively good/bad at.
The section on RLHF is particularly useful as it points out that “helpful” is a learned posture on top of a base model rather than an intrinsic property. From this view, sycophancy, jailbreaks, and safety are all parts of the same phenomenon viewed from different angles.
The Longing Under the Work
The Protestant Ethic and the Spirit of Capitalism
by Max Weber (Peter Baehr & Gordon C. Wells translation)
Weber, a German sociologist, wrote this book in the early 20th century as a sociological answer to the question of why the Anglo-sphere (UK and America particularly) had been so economically successful (especially compared with Germany).
His answer was protestantism. He argued the Calvinist notion of predestination created a psychological problem: the core of predestination is that you can’t earn salvation, it’s already decided. This is a kind of stressful state to be in — are you among the chosen or not?
The cultural answer that emerged was that tireless labor in a calling evolved as a sign of selection into salvation.
Calvinists (and other protestant sects that followed) developed the belief that if you work obsessively, reinvest rather than consume, and systematically order your life then you could demonstrate to yourself that you’re among the chosen.
Over time, the theology dropped away, but the behavioral pattern survives. The entrepreneurial American, exemplified in Ben Franklin, had a worldview based on living out a theology they often no longer believed in.
“Innerworldly Protestant asceticism works with all its force against the uninhibited enjoyment of possessions; it discourages consumption... Conversely, it has the effect of liberating the acquisition of wealth from the inhibitions of traditionalist ethics; it breaks the fetters on the striving for gain by not only legalizing it, but seeing it as directly willed by God.”
This hit close to home for me!
Ep.355: What Myths Can Teach Us About the AI Arms Race
with Joshua Schrei on Hidden Forces
Schrei reads the AI race through mythology, specifically the story of the sorcerer’s apprentice and what he calls the chthonic (underworld-deep) drive in young men to tinker and prove power.
In traditional cultures this drive gets tempered by elders: not yet, you have to learn to hold power first, you have to recognize you’re not special.
He argues our civilization has no such guard rails. We celebrate the apprentice’s “look what I can do” and pour venture capital at it.
The technology is here; the practices, norms, and social structures that would make it safe to wield need to evolve still. Containers get built slower than the things they’re meant to contain. Paired with Weber above, this offers a lens on the spiritual content under the work frenzy, the longing that doesn’t get touched by any amount of winning.
Friendship Is the Whole of the Path
by Daniel Thorson
The counter-weight in this issue. Everything else here is AI, work, etc. Thorson writes about the thing that is not so easy to optimize: friendship.
In many cultural and religious traditions, friendship isn’t a slice of the spiritual path, it’s the whole of it.
“There is one mass of suffering, and there is one mass of friendship. The suffering is not distributed. It is shared. Your loneliness and mine are the same loneliness.”
utter crap and a waste of tokens and time.
Agreed.