According to the results of a study of 107 companies, the trend of investing in AI infrastructure is advancing faster than their capacity for economic observation and management. Most organizations are still in the early stages of implementing AI, and only 21 percent have deployed AI at scale, while purchasing plans are mainly focused on specialized infrastructure that almost none currently use.
Despite the widespread use of cloud service providers (Google, Microsoft, AWS, Oracle) and API models (such as Gemini, OpenAI, and Anthropic), companies tend to move towards specialized cloud platforms and new accelerators. Interestingly, purchasing decisions are mostly based on integration capability and total cost of ownership, and initial price plays a minor role. However, many companies still cannot accurately measure the actual costs and returns of their AI infrastructure. Also, most purchased GPUs operate at less than half their capacity, resulting in low efficiency.
While 64 percent of organizations intend to add or change infrastructure providers in the next 12 months, most are moving between well-known providers. On the other hand, 83 percent of organizations report their GPUs operate at less than 50 percent capacity, and less than half are able to accurately track costs. The next challenge is shifting inference bottlenecks from processing to memory (bandwidth), which remains unclear for many and its importance is less considered.
These results indicate that the thirst for purchasing AI infrastructure outpaces the ability to measure and optimize costs, and the existing processing gap cannot be resolved by buying more hardware, but rather requires increased transparency and economic monitoring of infrastructures.