Listenly

Why does Fireworks achieve 5x performance difference compared to cloud providers running the same open source models on identical NVIDIA hardware?

Answered at 1:19 in Everyone Is Still Undersizing the AI Market | Eric Vishria · Invest Like The Best

— questions searchable right now

Why does Fireworks achieve 5x performance difference compared to cloud providers running the same open source models on identical NVIDIA hardware?

Running large language models efficiently is extremely hard. Fireworks achieves 5x performance speed and multiple times throughput difference compared to cloud providers like AWS, Azure, and GCP running the same open source models on the same NVIDIA hardware because these cloud companies pay cloud provider margins on top while Fireworks has specialized expertise in running these models efficiently. The key insight is that these companies are paying the margins of the cloud providers and running on top, making money—showing this is not a commodity pass-through business but requires deep technical expertise.

Eric Vishria on Invest Like The BestAll answers from this podcast →
Full answer and detailsRead the whole answer, key facts and every other question from this episode

In short

Running large language models efficiently is extremely hard.

Eric Vishria, Venture Partner at Sequoia Capital · Everyone Is Still Undersizing the AI Market | Eric Vishria · Invest Like The Best · answer at 1:19

Key takeaways

  • Running large language models efficiently is extremely hard.

Quote to remember

“Every single day that you are hitting your plan, you are destroying equity value. Every day you hit that plan, you're fucking up. You're destroying value.”

— Eric Vishria

The full answer · about 2 min

Running large language models efficiently is extremely hard. Fireworks achieves 5x performance speed and multiple times throughput difference compared to cloud providers like AWS, Azure, and GCP running the same open source models on the same NVIDIA hardware because these cloud companies pay cloud provider margins on top while Fireworks has specialized expertise in running these models efficiently. The key insight is that these companies are paying the margins of the cloud providers and running on top, making money—showing this is not a commodity pass-through business but requires deep technical expertise.

Key facts

  • Fireworks achieves 5x performance difference versus cloud providers on same models and hardware
  • AWS S3 and EC2 launched in 2006
  • Bezos discussed AWS importance in 2007 annual letter; investor reaction was poor
  • In 2014, smart investors would have gone 0 for 30 correctly predicting AWS long-term margins
  • Cloud computing roughly 40-30-20 split between AWS, Azure/GCP by 2026

People, companies and topics mentioned

Sequoia CapitalFireworksSierraAWS (Amazon Web Services)AzureGCP (Google Cloud Platform)CursorOpenAI

More answers from this episode

 ·  All answers from Invest Like The Best →
—
—
0:00
0:00