< So Why Not Spend $2,000?

As might be obvious from this blog, my AI coding usage is increasing. 90%+ of my real coding output now comes from tokens. It has left me with a question about token spend that I can't answer, one that I suspect has been at the back of my mind for 18 months or more.

If tokens are highly correlated with output, then tokens are near enough to output. Twice the tokens gets you twice the output (more or less). For me, in terms of emitting code, that is holding.

Now put that on a team. Say you have 100 engineers and each of them spends $1,000 a month on tokens, at enterprise sticker price[1]. That's $100,000 a month. If tokens track output, then 50 engineers spending $2,000 a month each is roughly equivalent. It's the same $100,000 of tokens and so the same output, from half the engineers.

So why not spend the $2,000? Of course you would. On the spend itself it hardly needs asking.

The work of 100 engineers is now the work of 50[2]. That's people, livelihoods, and careers[3]. The sum doesn't stop at 50 either. Halve the team again and double the tokens again, and you have 25 engineers at $4,000 a month. Do it once more and it's 12 and a half engineers at $8,000 (I'll round that to 12), and it's still the same $100,000.

Does it hold at 25? At 12? Perhaps less so. If you take it to the reductionist conclusion of 1 engineer, you take a total spend of about $21 million a year down to $1.4 million[4].

There's a counterpoint here too. If the sum holds, why not keep the 100 engineers, double the spend, and double the output?

The answer comes down to coherence. 100 engineers producing twice the output is hard to manage. That's twice the context that needs to be shared, which is what two-pizza teams and Dunbar's number are all about. None of that is new. It's the old knowledge coming back. Those limits were never about how fast anyone could write code, they were about how much a group of people can hold together, and tokens don't change that.

The door swings both ways. One engineer can't hold $100,000 a month of output in their head any more than 100 can share twice the context. So the number is somewhere between 1 and 100. I think the sinking feeling we all have is that we know it's probably not 100.

Also read: Rent Versus Buy, The Mythical Machine-Month, and On choosing optimism?.

Discuss and share via the meta page . Filed under AI, Code, and People.

Footnotes

  1. Enterprise means being on the meter, and I wrote up what that does to the way you work in Rent Versus Buy. ↩

  2. Back in January, in The Mythical Machine-Month, I had five engineers outperforming fifty, and that leaned on the same coordination overhead this one ends on. ↩

  3. Back in February, in On choosing optimism?, I wrote that plenty of engineers are right to worry about their careers and that I share the grief in it. ↩

  4. The assumptions: every engineer is on a $200,000 salary with no on-costs, and the token bill stays at $100,000 a month ($1.2 million a year) however many engineers there are. So 100 engineers is $20 million in salaries plus $1.2 million in tokens, and 1 engineer is $200,000 plus the same $1.2 million. ↩