There’s really three components to that total cost. One is the amount of tokens you’re consuming with a specific activity. So the higher computationally intensive areas like building out software or maybe doing a detailed study require more tokens.
There’s also the cost of the model that you’re producing. Some of the latest and greatest frontier capabilities that are pushed out by the frontier AI firms, those latest and greatest models might be five times more expensive than some of the less capable models that are in their portfolio. So the model that you pick actually has a pretty big outcome on cost.
And then the third bit is it does come down to the lack of predictability. For many times, a specific task might take, let’s say 20,000 tokens, but because of some configurations on data, it might get stuck and retry again and again, and that could start to really increase the meter to several hundred thousand tokens to be able to perform that task.
Costs to AI tokens for companies:
- The intensity of the activity.
- The model used.
- Predictability of the tokens needed.