The End of AI Frontiers

In 2023 after the ChatGPT release, it was clear to me that research needed to be more organized to succeed than was typical within Microsoft Research. The amount of support and focus which other research groups were applying was great enough that even with a clever idea pursued by one person it was likely to arrive late and unconvincing because one person could not move fast enough and the baselines which one person could assemble just were not strong enough to really show that an idea was worth taking flight. Several other folks thought similarly and we organized to create the AI Frontiers sub-organization within MSR.

In AI Frontiers, we actively tried to organize coherent research efforts. This was not precisely “top down” or “bottom up”, but rather mostly somewhere in the messy middle. There were disagreements of course, and resolving them was sometimes quite challenging. Compute availability was also an often-severe bottleneck. But at a high level I was under the general impression that the organization was succeeding in doing pioneering things, particularly given the small size of AIF (~50 people) and relative to what we could do previously. Much of the work of AIF was figuring out how to make agents work, for example the Autogen orchestration framework and the Fara computer use agent. In NYC however, several of us were interested in developing new fundamental capabilities such as the Next Latent objective which enables extrapolation, Dion3 which creates an optimized/improved Muon, or the Full Bandwidth Transformer which efficiently makes models function substantially more powerfully via recursion.

We also actively tried to share with the rest of MSR. For example, AIF decided to develop a monorepo that was shared by default with all of Microsoft Research. In this monorepo for example was the proven-capable edge scale pretraining codebase I talked about in the COLM keynote last month. This might seem like a minor detail, but the amount effort required to create a well-formed model creation pipeline is substantial and I’m aware of at least 4 efforts (the other 3 not in public-within-MSR repos) trying to develop model building pipelines.

In early September we were reorged into Microsoft AI. On the surface this could make sense, because putting two different AI organizations together creates opportunities for better concentration of effort, and there were places where AIF was ahead of MAI and of course vice-versa (MAI is much larger). In practice, there are substantial issues because the philosophy of the ex-Inflection leadership of MAI is incompatible. The core philosophy there is scaling, and aggressively not research. This might shock some folks who naturally think that the way you make better models is via research but there is plenty of evidence that bigger models are better models as far as the quality of the predictions they make and from a business perspective returns to scale are far more predictable than returns to research. In a scaling-only environment of course the primary effort is on getting/making/refining data, which is quite a step away from what we were working on. Aside from the incompatible focus, there was plenty of evidence that they just did not really want us.

It’s a tough situation to find yourself in. I consider the scaling-only focus to be both misdirected and not fun. It is an understandable failure mode—scaling is straightforward and it’s common for people to give up on a tradeoff between multiple competing ideals (like innovation and size) by short-sightedly adopting one in totality and dropping the other. But returns to research are real and significant. For example, the Bytedance experiments suggest a variant of the Full Bandwidth Transformer might be comparable to a model of twice the size, implying significant savings in training and inference for the same capability. Do the math—how much is it worth to halve training and inference cost given that the industry has investments approaching a trillion dollars?

So I situated my folks as best I could and left. I am quite fortunate in that there are many other people in the world who are capable of seeing the value of research and innovation which is my intrinsic center of interest. Every other founding AIF leadership team member seems to have come to a similar conclusion.

This does leave a question: Why not go back to MSR? I chatted with folks a bit but could not get away from two serious obstacles. The first is that I am just not comfortable being at a company where the research I do has no path to real use. I could not get innovation into use (or even seriously considered) when I was in MAI, so the prospects for success from MSR seem quite small. The second is that I’m afraid there’s a breakdown of the basic deal in MSR: do good work and good things happen. As far as I can tell we did good work and were non-consensually reorged away. Unfortunately, I would not be surprised to see that kind of thing continue to happen until MSR leadership develops a will to effectively organize to meet the moment.

So AI Frontiers is over. I’m sorry to say that there will not be internships with the folks there in the coming year—I enjoyed working with interns on many occasions. And my best wishes for the folks I leave behind. It is not hard to imagine a Microsoft of the future which becomes attractive for research again if the people in control can develop a little bit more wisdom.