Podcast · Tech & Cybersécurité

AI + a16z

By Andreessen Horowitz, Venture Capital Firm at Andreessen Horowitz

Andreessen Horowitz is a leading venture capital firm that has invested in and advised hundreds of technology companies, with deep expertise in AI infrastructure, software economics, and emerging technology trends.

AI + a16z

⏱ 32 min read · Readable by ChatGPT, Gemini, Claude

▶ Listen to the podcast
What AI + a16z covers

This podcast maps the frontier of artificial intelligence across every major industry—from cybersecurity systems being fundamentally rewritten by AI-driven threat detection, to open-source inference engines powering the next generation of software. A16z partners and portfolio founders dissect where AI infrastructure is headed, how self-accelerating systems are reshaping customer data strategy, and why the economics of software itself are changing. The core insight is that AI is no longer a feature; it's the new operating system for how enterprises think, build, and compete.

Key facts

Learn how AI is reshaping infrastructure economics and explore all episodes of AI + a16z to understand where the industry is moving next.

What this podcast really covers

AI + a16z is not a surface-level survey of AI trends. It dives deep into the technical and business infrastructure that determines winners in the AI era. Episodes examine how vLLM optimizes inference performance for open-source models, why knowledge engines are replacing flat vector databases as the next layer of enterprise AI, and how companies like Exa are building search systems specifically designed for AI agents that operate autonomously on behalf of customers.

The podcast distinguishes between hype and substance. It interrogates why continual learning—the ability of AI systems to improve without full retraining—matters for real-world deployment. It explores how startups like Mirendil are building self-accelerating AI systems where the model's output improves the next iteration of itself. And it addresses the uncomfortable truth: as AI agents gain control over customer data, the fight for that data becomes the new competitive battleground.

Who this podcast is essential for

Enterprise technology leaders and CIOs need this podcast because it clarifies how AI infrastructure decisions made today determine operational capability in 18 months. Discussions on cybersecurity rewritten by AI, knowledge engines replacing legacy databases, and the true cost of AI infrastructure provide the strategic context for vendor selection and build-versus-buy decisions.

Founders and product leaders benefit from hearing how companies are positioning themselves in the AI stack. Whether building search for agents, image generation models, or security tools, the podcast reveals which architectural choices are winning and how to differentiate when open-source is the default baseline.

Investors and corporate strategists use this podcast as a real-time map of where capital and talent are concentrating in AI. The a16z perspective includes nuanced takes on economics, distribution, and which problems remain unsolved—insight that shapes portfolio decisions and acquisition strategy.

What the episodes really reveal

The episode catalog shows consistent patterns: infrastructure mattering more than individual models, open-source accelerating commoditization, and the shift from AI as a feature to AI as the fundamental operating system of the business. Every episode on security—from Truffle Security to Socket—treats cybersecurity as a solved problem waiting for AI to rewrite it. Episodes on search, agents, and customer data show that the real competitive value is no longer in the model itself but in the proprietary data and workflows an AI system can control.

There is also a recurring tension: between centralized, closed systems (proprietary models, closed data) and decentralized, open systems (open-weights models, vector databases, modular inference). The podcast argues this is not about ideology—it is about economics. Open-source is winning in commodity layers because it enables faster iteration; closed systems survive only where data moats or proprietary training matter.

What this changes in practice

For technologists, this podcast reframes how to evaluate AI tools. Instead of asking "what model should we use?", the question becomes "how do we own and operate the data and inference system that our business depends on?" This means investing in knowledge engines, vector databases, and inference optimization rather than betting everything on a single third-party API.

For organizations defending against cyber threats, the implication is clear: if you are still building security on rule-based systems, you are already behind. AI-native security is not a future state; it is the baseline. The episodes with Truffle Security and Socket make this concrete—these tools are already changing how vulnerabilities are discovered and how supply-chain attacks are detected.

For executives planning software strategy, the podcast delivers an economic warning: the laws of software are changing. Margin structures, distribution models, and competitive advantage no longer follow the patterns that worked for previous generations of software. AI infrastructure commoditizes implementation, which means margins compress unless you own either the data, the agent that controls behavior, or the integration with customer workflows.

AI is no longer disrupting individual functions—it is rewriting the entire stack, from how security works to how data is stored to how software gets distributed. The winners will be those who understand that AI is not a vertical but a horizontal layer that changes the economics of every business.

Understand the infrastructure reshaping business and listen to all episodes of AI + a16z to stay ahead of where the technology is moving.

Get the insights from the experts shaping AI infrastructure—listen to AI + a16z now.

The podcast answers these questions

What are AI agents and how are they transforming enterprise systems?

AI agents are autonomous software systems that can perceive their environment, make decisions, and take actions without constant human intervention. They are fundamentally changing how enterprises approach customer data management, security workflows, and operational efficiency by moving from reactive systems to predictive, self-directed automation.

How does AI infrastructure affect software economics and distribution?

AI infrastructure—including vector databases, knowledge engines, and open-source models like vLLM—is restructuring how software is built, deployed, and monetized. The shift toward modular, inference-focused architectures and open-weights models is enabling new distribution patterns and lowering barriers to entry for developers building AI-first applications.

Why is cybersecurity being rewritten by artificial intelligence?

Traditional cybersecurity relies on predefined rules and manual threat detection, which cannot scale against modern attack patterns. AI-powered security tools use continuous learning and behavioral analysis to detect novel threats in real time, fundamentally changing how organizations defend their infrastructure and respond to vulnerabilities.

What role do open-source AI models play in democratizing artificial intelligence?

Open-source models like Ideogram's image generation and vLLM's inference engine reduce dependence on closed platforms and enable developers to build custom AI applications without vendor lock-in. This shift accelerates innovation, increases accessibility, and creates new market opportunities for companies building on top of these foundations.

AI + a16z

Discover AI + a16z

Andreessen Horowitz · AI + a16z

Listen to the podcast

Answers from this podcast

7 questions answered · 1 episode indexed

Inside vLLM: The Engine Powering Open-Source AI

How does the funding model for open-source AI development differ from traditional open-source software sustainability?

Open-source software has historically relied on volunteer labor and in-kind donations, but AI cannot operate on this model because frontier model training demands millions or billions of dollars in computing resources . Model labs must adopt revenue-driven funding mechanisms—much like the pharmaceutical industry—to generate income that funds ongoing research and development.

The Scale Gap: Why Volunteers Cannot Fund AI Training

Traditional open-source projects like Linux or Apache succeeded through donated engineering time and corporate sponsorship of employee contributions. This works when the bottleneck is human expertise and code review. AI model development operates under entirely different economics: training a frontier language model requires access to hundreds of thousands of GPUs running continuously , a cost that cannot be absorbed by volunteer effort or modest corporate sponsorship.

As Simon Mo explains in the episode , model labs are actively exploring commercial licensing strategies and usage-based pricing tiers to generate the capital required to continue developing and refining their models. This represents a structural shift in how open-source AI infrastructure must be sustained.

Revenue as Infrastructure: The Pharma R&D Model

The sustainable funding path mirrors how pharmaceutical companies operate: successful products generate revenue that funds future research . A model lab that releases an open-weight model and offers commercial inference services, enterprise licensing, or usage-based access creates a revenue stream that reinvests in the next generation of models.

Without this cycle, model labs face an impossible choice: either shut down when funding runs out, or remain dependent on venture capital and corporate grants indefinitely. The open-source AI ecosystem now requires economic incentives embedded into the infrastructure itself—not altruism alone.

Simon Mo — Co-founder and CEO of Infraact and lead maintainer of vLLM. Mo's team built vLLM, an open source inference engine that runs on half a million GPUs at any moment, starting in 2022 before ChatGPT when they discovered fundamental unsolved problems in how language models are served at scale.

The deeper question explored in this conversation is whether computing costs will ever drop enough to bring open-source AI back to a pure volunteer model—a thought experiment that reveals how hardware economics shape the entire open-source AI landscape today.

Why are open-source model labs implementing commercial licensing terms and usage restrictions on recently released models?

Model labs are implementing commercial licensing to solve a fundamental economics problem: training, research, and data infrastructure for large language models are prohibitively expensive , and traditional open source donation models cannot sustain operations at that scale. Recent models like Meta's LLAMA, Minimax's M2.7, and Kimi now include usage-based restrictions or derivative work limitations to create sustainable funding mechanisms.

The Cost-Sustainability Paradox

Building and maintaining cutting-edge AI models requires resources that far exceed what community donations or volunteer labor can provide. The shift toward commercial terms reflects a hard reality: infrastructure costs for model training and deployment require revenue to be self-sustaining . As discussed in the episode with Simon Mo , model labs cannot continue operating indefinitely without clear funding paths.

Meta's LLAMA exemplifies this approach: the model's licensing terms include commercial agreement thresholds based on user metrics—specifically, daily active users or annual recurring revenue exceeding defined limits trigger commercial licensing obligations. This hybrid approach allows broad access for research and small-scale use while capturing revenue from commercially significant deployments.

Recent Models and Emerging Restrictions

The trend accelerated with newer releases. Minimax's M2.7 and Kimi now include usage-based licensing terms and derivative work restrictions , marking a shift away from permissive open source frameworks. These aren't arbitrary limitations—they directly address the funding gap between the cost of model development and what ad hoc community support can sustain.

The core insight is straightforward: if the donation model worked at AI scale, labs would not need to implement these restrictions. Simon Mo explains in the podcast that open source software has historically relied on volunteer contributions and community funding, but AI model development operates in a completely different economic regime.

Simon Mo — Co-founder and CEO of Infraact, and lead maintainer of vLLM, an open source inference engine running on half a million GPUs at any moment. Mo and his team have been building infrastructure for large-scale model serving since 2022, when they discovered that existing ML infrastructure was fundamentally inadequate for serving large language models efficiently.

The path forward reflects a pragmatic recognition: commercial terms create a pathway for open source model labs to fund ongoing development rather than face unsustainable funding constraints or close entirely. Without these mechanisms, labs would either depend entirely on venture capital or proprietary models, eliminating true open source alternatives altogether.

There's also a broader ecosystem consideration: licensing restrictions preserve the ability for model creators to fund development while still releasing weights publicly. This tension between openness and sustainability is a recurring theme in the AI + a16z episode , illustrating how the industry is navigating the gap between open source ideology and economic reality.

What economic advantages do open-weight models provide in terms of inference speed and customization compared to proprietary models?

Open-weight models unlock 10+ distinct speed tiers that proprietary solutions simply cannot match , enabling inference throughput of 400–500 tokens per second compared to 2–3x slower proprietary fast modes. This granular control over performance and cost is only possible when you run your own infrastructure instead of relying on API access.

The fundamental difference lies in how each model tier is packaged and delivered. Proprietary models typically offer two options: a standard mode and a fast mode. You pick one, accept its speed and cost trade-off, and move forward. Open-weight models work entirely differently because every provider running the same model weights can tune its own inference stack independently.

As Simon Mo explains in the episode , this flexibility is what transforms economics for enterprises. Instead of choosing between expensive (slow, accurate) or cheap (fast, slightly less accurate), you can provision anywhere along a spectrum of 10 different configurations, each optimized for a specific latency-cost balance.

Where the speed advantage compounds

The 2–3x speed advantage manifests concretely in real-world throughput. A proprietary fast mode might achieve a certain tokens-per-second rate, but an optimized open-weight inference setup can push 400–500 tokens per second on the same hardware. This isn't magic—it's the result of running dedicated inference engines like vLLM that handle the specific computational patterns of language models far more efficiently than general-purpose APIs.

When you control your own infrastructure, you're not paying for OpenAI's or Anthropic's operational overhead, their moderation systems, or their API gateway latency. Every millisecond saved directly translates to cost savings at scale. A 2–3x improvement in throughput means you can serve the same traffic volume on one-third the GPUs, or serve three times more traffic on the same hardware budget.

The customization angle runs deeper than speed alone. Enterprises want to add their own guardrails, extend the model pipeline, and understand exact performance characteristics , as discussed in detail in the podcast. Proprietary APIs give you none of that visibility or control. With open-weight models running on your own infrastructure, you audit every layer, inject your own safety filters, and modify the inference process to match your exact business logic.

"Every provider can offer potentially even 10 different levels of speed going from the slowest mode which can be a lot cheaper to 400 tokens per second almost up to 500 in many cases."

Simon Mo — Co-founder and CEO of Infraact, and lead maintainer of vLLM, the open source inference engine now running on half a million GPUs at any moment. Mo founded vLLM in 2022 when his team attempted to make a slow open source demo faster and uncovered a pile of unsolved problems in LLM serving infrastructure. Infraact, his company, builds the infrastructure layer that makes open-source AI models production-ready at enterprise scale.

The real-world implication is that this level of granular control is only achievable when running your own infrastructure —you cannot replicate it through an API, no matter how flexible the provider claims to be. That constraint alone reshapes the economics and strategic calculus for any organization making serious infrastructure bets.

Why do enterprises choose to deploy open-weight models instead of relying on proprietary APIs, beyond cost considerations?

Enterprises deploy open-weight models primarily for infrastructure control, data governance, and compliance assurance —not cost savings alone. With open-weight models, companies can guarantee response times, control data retention, enforce security policies, fine-tune models to their needs, and maintain SLA compliance in ways proprietary APIs do not permit.

The shift toward open-weight models reflects a fundamental change in how enterprises view AI dependencies. For voice agent companies and customer-facing services, controlling the inference engine itself is non-negotiable: they need guaranteed latency and reliability that a third-party API simply cannot promise.

As explained in this episode of AI + a16z , the performance profiles become transparent and reproducible when enterprises own their model infrastructure. This transparency extends beyond speed: companies gain the ability to understand exactly how many tokens their models consume, optimize cost per inference, and modify guardrails to match their specific compliance and safety requirements.

Control as the Core Enterprise Requirement

Data sovereignty and compliance drive a significant portion of enterprise adoption. Unlike proprietary APIs where data flows to third-party servers, open-weight models run entirely on an organization's own hardware, ensuring data never leaves the premises. This is especially critical in regulated industries: financial services, healthcare, and government agencies cannot afford to outsource their intelligence layer.

Performance predictability matters equally. Voice agent companies need guaranteed sub-second response times for phone calls; proprietary APIs offer no such guarantee because you are sharing infrastructure with thousands of other customers. With open-weight models deployed on dedicated hardware, the SLA becomes enforceable and transparent.

Fine-tuning and model customization represent another layer of control unavailable with proprietary APIs. Enterprises can adapt open-weight models to domain-specific language, industry jargon, or customer behavior patterns—then measure the impact of each change on accuracy and cost. This feedback loop is simply impossible with a black-box API.

"Every provider can offer potentially even 10 different levels of speed going from the slowest mode which can be a lot cheaper to 400 tokens per second almost up to 500 in many cases."

Simon Mo — Co-founder and CEO of Infraact and lead maintainer of vLLM. Mo's team created vLLM, an open source inference engine now running on half a million GPUs at any moment, after identifying unsolved problems in LLM serving infrastructure. His work spans the full spectrum of open-source model deployment challenges, from infrastructure optimization to enterprise adoption patterns.

This flexibility in speed-versus-cost tradeoffs is unique to open-weight deployments. With proprietary models, you typically get one "fast mode" and one "standard mode." Open-weight stacks allow operators to tune inference parameters, batch sizes, quantization levels, and hardware allocation to hit any point on the speed-cost curve. Enterprises can serve high-latency queries cheaply and reserve premium speed for customer-facing interactions—something impossible with fixed API pricing tiers.

For a deeper look at how vLLM powers this infrastructure layer , the episode discusses the full stack of considerations enterprises must weigh when building proprietary AI systems.

At what point did open source AI models become critical infrastructure rather than an experimental curiosity?

Open source AI crossed a critical threshold around 2023, when GitHub Copilot and ChatGPT became tools people could not live without , and open-weight models became foundational to daily productivity. The inflection came when application startups realized they could not build meaningful AI products using only closed-source models, forcing them to turn to open source for mid-training, post-training, and inference deployment.

The 2023 inflection: from experiment to necessity

The transition was sharp and driven by practical constraints. Startups discovered they needed control over model customization and deployment that proprietary alternatives could not provide. Closed-source models offered speed and reliability, but they lacked the flexibility required to build differentiated products at scale.

As discussed in the episode with Simon Mo , this shift forced the industry to build entirely new infrastructure layers to support open-source deployment. Open source went from a hobbyist curiosity to the backbone of production AI systems.

Infrastructure as the real bottleneck

Simply releasing model weights was never enough. Making open-source models viable at scale required solving problems that did not exist in the closed-source world—problems around inference efficiency, cost optimization, and multi-model orchestration across heterogeneous hardware.

This is where tools like vLLM, the inference engine now running on half a million GPUs at any moment , became essential. Without such infrastructure, open-source models would have remained a theoretical option, not a practical necessity.

"Every provider can offer potentially even 10 different levels of speed going from the slowest mode which can be a lot cheaper to 400 tokens per second almost up to 500 in many cases."

Simon Mo — Co-founder and CEO of Infraact and lead maintainer of vLLM. Mo and his team created vLLM in 2022, before ChatGPT even launched, when they set out to optimize a slow open-source demo and uncovered fundamental unsolved problems in LLM serving. Today, vLLM supports more than 1,000 model architectures and serves as critical infrastructure for thousands of enterprises.

This flexibility—the ability to trade speed against cost at granular levels—became a competitive advantage that proprietary platforms could not match. Open source enabled providers to offer speed tiers proprietary models reserved for their fastest mode alone.

The deeper reason open source became critical is that it solved a business problem proprietary models created: vendor lock-in and limited control. Enterprise teams discovered they could run, customize, and scale open-weight models independently , reducing dependency on a single provider and giving them the governance controls they needed for regulated industries.

The 2023 threshold was driven by startups needing customization and control that closed-source models could not provide.

Infrastructure tools like vLLM became as critical as the models themselves—no weights-only release was sufficient.

Open source enabled speed-vs-cost tradeoffs that proprietary platforms could not offer, making it cost-competitive for enterprises.

The shift was fundamental: open source went from hobby project to non-negotiable infrastructure for AI-native startups.

How does vLLM function within the modern AI infrastructure stack for enterprises using open-source models?

vLLM is an inference engine that transforms available GPUs into running endpoints for AI intelligence , functioning as critical infrastructure alongside databases and operating systems in modern AI stacks. It supports more than 1,000 model architectures and is currently running on half a million GPUs globally, serving as a performance benchmark for hardware vendors like NVIDIA, AMD, Google, Amazon, and Intel.

GPU acceleration as the foundation

vLLM's core role is managing the computational intensity of LLM serving, which requires rapid execution on accelerators like GPUs or TPUs — a fundamentally different challenge than traditional machine learning inference. Rather than treating inference as a secondary concern, Simon Mo and his team recognized that serving open-source models demands an entirely new infrastructure layer, one that vendors now rely on to validate their hardware performance.

As the episode explains, major chip manufacturers have adopted vLLM as a benchmark tool: when they release new hardware, they test it against vLLM to ensure their latest chips can deliver the performance enterprises demand.

Day Zero support and vendor partnerships

vLLM implements new model architectures immediately upon their public release in a practice called Day Zero model release support . This means enterprises deploying cutting-edge open-source models like Llama, Mistral, or the latest variants can run them on vLLM without waiting for proprietary optimizations or vendor-specific implementations.

The engine's breadth—supporting over 1,000 model architectures—reflects deep collaboration with the open-source community and hardware partners. This positions vLLM not as a niche tool but as the critical infrastructure layer enabling enterprises to run production workloads on open models without proprietary lock-in.

"Every provider can offer potentially even 10 different levels of speed going from the slowest mode which can be a lot cheaper to 400 tokens per second almost up to 500 in many cases."

Simon Mo — Co-founder and CEO of Infraact, lead maintainer of vLLM. Mo founded the vLLM project in 2022, originally aimed at accelerating a slow open-source demo but which evolved into discovering and solving a pile of unsolved problems in LLM serving infrastructure. His work has made vLLM the foundation for inference optimization across the industry, with the engine now running on half a million GPUs at any moment.

This flexibility—enabling different speed tiers from cost-optimized modes to ultra-fast inference—shows how vLLM adapts to real enterprise needs. Organizations can tune throughput and cost based on their use case, something proprietary stacks rarely offer with such granularity.

To understand the broader context of why open-source models require different serving paradigms than proprietary ones, listen to the full discussion between Simon Mo and the a16z team, where they explore the infrastructure economics driving enterprise adoption of open-weight models.

What makes serving a large language model fundamentally different from traditional machine learning workloads?

Serving a large language model requires running it on accelerators like GPUs or TPUs with computationally intensive processes that demand quick and efficient responses —fundamentally different from traditional machine learning workloads. The challenge lies in handling variable input lengths, non-deterministic output distributions, and the complex batching and scheduling requirements embedded in modern inference engines.

The Infrastructure Gap: GPUs, TPUs, and Real-Time Demands

Traditional machine learning models like BERT and ResNet are designed for standardized inputs and outputs. LLMs operate in an entirely different domain. Each request can vary dramatically in length—from a single sentence to thousands of tokens—and the model's response length is unpredictable. This variability forces inference engines to solve scheduling and batching problems that don't exist in conventional ML serving.

The computational intensity compounds this problem. GPUs and TPUs aren't just nice-to-have optimizations; they're absolutely essential because the models themselves are so large that they cannot run efficiently on standard processors. This is why vLLM was designed from the ground up as an inference engine specifically built for LLM workloads , not as a repurposed framework from traditional ML.

Non-Deterministic Outputs and Dynamic Batching

Traditional ML inference is deterministic: the same input always produces the same output, and batch sizes can be fixed. LLMs are fundamentally different. Each token generation is non-deterministic due to sampling and temperature parameters, meaning identical prompts can yield different results. Additionally, variable request lengths make fixed batching impossible—the inference engine must dynamically adjust how it groups requests to maximize throughput without sacrificing latency.

This dynamic behavior is why companies like NVIDIA, AMD, and Google are building inference optimization into their newest chip architectures , and why vLLM has become critical infrastructure that now runs on half a million GPUs at any moment.

"Every provider can offer potentially even 10 different levels of speed going from the slowest mode which can be a lot cheaper to 400 tokens per second almost up to 500 in many cases."

Simon Mo — Co-founder and CEO of Infraact and lead maintainer of vLLM. Mo's team discovered these serving challenges while trying to optimize an open-source demo in 2022, years before ChatGPT democratized LLMs. The resulting vLLM project has become a foundational layer powering modern AI infrastructure across multiple hardware vendors and cloud providers.

The quote reflects a concrete reality: speed optimization in LLM serving is far more complex than in traditional ML , requiring specialized scheduling algorithms that account for all these moving parts simultaneously. This is why traditional machine learning infrastructure simply cannot be adapted to handle LLMs—the problems are fundamentally different at the architectural level.

Découvrir AI + a16z sur Listenly →