Why does Fireworks achieve 5x performance difference compared to cloud providers running the same open source models on identical NVIDIA hardware?
Answered at 1:19 in Everyone Is Still Undersizing the AI Market | Eric Vishria · Invest Like The Best
Why does Fireworks achieve 5x performance difference compared to cloud providers running the same open source models on identical NVIDIA hardware?
Running large language models efficiently is extremely hard. Fireworks achieves 5x performance speed and multiple times throughput difference compared to cloud providers like AWS, Azure, and GCP running the same open source models on the same NVIDIA hardware because these cloud companies pay cloud provider margins on top while Fireworks has specialized expertise in running these models efficiently. The key insight is that these companies are paying the margins of the cloud providers and running on top, making money—showing this is not a commodity pass-through business but requires deep technical expertise.
Full answer and detailsRead the whole answer, key facts and every other question from this episode
In short
Running large language models efficiently is extremely hard.
Eric Vishria, Venture Partner at Sequoia Capital · Everyone Is Still Undersizing the AI Market | Eric Vishria · Invest Like The Best · answer at 1:19
Key takeaways
- Running large language models efficiently is extremely hard.
Quote to remember
“Every single day that you are hitting your plan, you are destroying equity value. Every day you hit that plan, you're fucking up. You're destroying value.”
— Eric Vishria
The full answer · about 2 min
Running large language models efficiently is extremely hard. Fireworks achieves 5x performance speed and multiple times throughput difference compared to cloud providers like AWS, Azure, and GCP running the same open source models on the same NVIDIA hardware because these cloud companies pay cloud provider margins on top while Fireworks has specialized expertise in running these models efficiently. The key insight is that these companies are paying the margins of the cloud providers and running on top, making money—showing this is not a commodity pass-through business but requires deep technical expertise.
Key facts
- Fireworks achieves 5x performance difference versus cloud providers on same models and hardware
- AWS S3 and EC2 launched in 2006
- Bezos discussed AWS importance in 2007 annual letter; investor reaction was poor
- In 2014, smart investors would have gone 0 for 30 correctly predicting AWS long-term margins
- Cloud computing roughly 40-30-20 split between AWS, Azure/GCP by 2026
People, companies and topics mentioned
More answers from this episode
- 3:37What did AWS investor sentiment reveal about predicting long-term margin sustainability in infrastructure businesses in 2007?
- 4:47How did the market size for cloud infrastructure prove wrong the 2014 narrative that AWS would consume all enterprise software opportunities?
- 7:10What does the comparison between cloud adoption timelines and current AI adoption reveal about enterprise readiness in 2025?
- 9:58How do product managers and engineers need to work differently in the AI era compared to traditional SaaS product development?