I think precisely because all AIs are relatively the same, it's more important to venture to the cereal model. AI is NOT airlines, because the nature of the work is so much more open-ended.
Airlines' purposes are to get you from point A to point B, and everything else is just pedantic. If you had a good time on the flight is unimportant if you can save a couple hundred bucks, right?
But Generative AI is so open-ended that differentiation is almost inherent to the model. Sure, models right now are blending together in terms of proficiency. But that's why it's better, business-wise, to be hyper-differentiated.
If you can be a model that is 1% better at investment trading than the other models, but 2% worse at writing screenplays, you will not be the "frontier" model, but you will be deployed in investment firms.
Of course, this assumes AI models can be created that are really good at one thing while sacrificing performance in other areas. I think there's some research showing that improving a generalist is better than improving a specialist in terms of actual model performance. So while going for cereal might be better for business, it could be impossible technically, but nonetheless something to look at.
Yes, 100% agreed here. Right now, pretty much all of the large foundation model companies are pursuing the same target - namely, pretraining a model on as much data as they can find, then doing RLHF to improve chat ability and RL on math/coding data to add in thinking capacity. Since they are all broadly working from the same data and using the same techniques, the models are convering in capabilities and we are ending up in an airline industry scenario.
As you mention, the only way out of this scenario is to specialize. I can see this coming from two ways: curating specialized datasets for a particular topic (e.g. investments as you mention) OR building scaffolding around a generic foundation model for that topic (e.g. building tools which the model can interact with to solve a particular problem). We'll likely see companies extract a huge amount of value by doing some combination of these two approaches in particular verticals.
However, I'm not sure that these companies will be the foundation model companies themselves that are able to extract this value. Right now there is so much organizational inertia and so much CapEx in places like OpenAI and Anthropic going towards the large scale pretraining + RL paradigm that I think there will be significant internal resistance to move towards a more specialized paradigm. We'll likely see startups that build on top of these models (or their open source equivalents, like DeepSeek R1) to produce the specialist models that you're describing.
I think precisely because all AIs are relatively the same, it's more important to venture to the cereal model. AI is NOT airlines, because the nature of the work is so much more open-ended.
Airlines' purposes are to get you from point A to point B, and everything else is just pedantic. If you had a good time on the flight is unimportant if you can save a couple hundred bucks, right?
But Generative AI is so open-ended that differentiation is almost inherent to the model. Sure, models right now are blending together in terms of proficiency. But that's why it's better, business-wise, to be hyper-differentiated.
If you can be a model that is 1% better at investment trading than the other models, but 2% worse at writing screenplays, you will not be the "frontier" model, but you will be deployed in investment firms.
Of course, this assumes AI models can be created that are really good at one thing while sacrificing performance in other areas. I think there's some research showing that improving a generalist is better than improving a specialist in terms of actual model performance. So while going for cereal might be better for business, it could be impossible technically, but nonetheless something to look at.
Yes, 100% agreed here. Right now, pretty much all of the large foundation model companies are pursuing the same target - namely, pretraining a model on as much data as they can find, then doing RLHF to improve chat ability and RL on math/coding data to add in thinking capacity. Since they are all broadly working from the same data and using the same techniques, the models are convering in capabilities and we are ending up in an airline industry scenario.
As you mention, the only way out of this scenario is to specialize. I can see this coming from two ways: curating specialized datasets for a particular topic (e.g. investments as you mention) OR building scaffolding around a generic foundation model for that topic (e.g. building tools which the model can interact with to solve a particular problem). We'll likely see companies extract a huge amount of value by doing some combination of these two approaches in particular verticals.
However, I'm not sure that these companies will be the foundation model companies themselves that are able to extract this value. Right now there is so much organizational inertia and so much CapEx in places like OpenAI and Anthropic going towards the large scale pretraining + RL paradigm that I think there will be significant internal resistance to move towards a more specialized paradigm. We'll likely see startups that build on top of these models (or their open source equivalents, like DeepSeek R1) to produce the specialist models that you're describing.