Generative AI Tech Stack: Infrastructure, Models, and Apps
Image credit : Andreessen Horowitz
Overview
Generative AI is a rapidly developing market, with hundreds of startups entering the space and focusing on foundational models, AI-native apps, and infrastructure. Models such as Stable Diffusion and ChatGPT are gaining record user growth, demonstrating the significant potential for transformation in this field.
The key question now is where value will accrue in this rapidly developing market. Recent observations reveal that infrastructure vendors seem to be the current winners, capturing most of the revenue. Application companies are struggling with retention, differentiation, and gross margins, while model providers have yet to achieve significant commercial scale.
The generative AI tech stack can be divided into three layers:
Applications: Generative AI models are integrated into user-facing products, either through proprietary model pipelines or third-party APIs.
Models: These power AI products and are made available as proprietary APIs or open-source checkpoints requiring a hosting solution.
Infrastructure: Vendors, such as cloud platforms and hardware manufacturers, handle training and inference workloads for generative AI models.
Though generative AI applications have experienced staggering growth, sustainable and profitable businesses need more than just growth. High gross margins and retention are crucial for long-term success. However, many generative AI apps face varying gross margins, ranging from 50-60% to as high as 90%. Customer acquisition strategies may not be scalable, and retention is already starting to decline. The lack of differentiation among apps using similar AI models makes it challenging to find network effects or unique data/workflows that competitors cannot easily replicate.
It is not yet clear if selling end-user apps is the best path to building a sustainable generative AI business. Margins are expected to improve as language model competition and efficiency increase, and retention should improve as AI tourists leave the market. Vertically integrated apps may have an advantage in driving differentiation, but there is still much to prove.
In summary, the first wave of generative AI applications is achieving scale but faces challenges in retention and differentiation. Long-term customer value and profitability will depend on defensibility, market structure, and unique value propositions. source