Zain Asgar, an adjunct professor at Stanford and a successful former founder, has successfully raised an $80 million Series A round for his new startup. This company aims to ingeniously resolve the AI inference bottleneck issue. Menlo Ventures spearheaded this investment.
The company, Gimlet Labs, asserts that it has developed the first and sole “multi-silicon inference cloud.” This software enables an AI workload to run concurrently across various hardware types, distributing tasks for an AI application among traditional CPUs, specialized AI GPUs, and high-memory systems.
“We fundamentally operate across all available diverse hardware,” Asgar explained to TechCrunch.
Tim Tully, lead investor from Menlo, highlighted in a blog post about the funding that a single AI agent might involve multiple sequential steps, each requiring specific hardware: “Inference is compute-bound; decode is memory-bound; and tool calls are network-bound.”
While no single chip can handle every task perfectly, and as new hardware emerges and older GPUs are repurposed, Tully contends that “the multi-silicon fleet is prepared — it simply lacks the software layer to make it functional.” He believes this is precisely what Gimlet Labs provides.
If the current trend of deploying more compute resources persists, McKinsey projects data center expenditures to reach nearly $7 trillion by 2030. Asgar points out that existing deployed hardware is only utilized “somewhere between 15 to 30 percent” of the time by applications.
“To put it another way: you are squandering hundreds of billions of dollars by leaving resources idle,” he stated. “Our objective was to determine how to make AI workloads at least 10 times more efficient, starting today.”
Therefore, he and his cofounders, Michelle Nguyen, Omid Azizi, and Natalie Serrino, began developing orchestration software designed to segment agentic workloads, enabling their simultaneous distribution across a wide array of hardware.
Gimlet Labs claims its technology reliably boosts AI inference speeds by 3x to 10x while maintaining the same cost and power consumption. Gimlet further states its ability to even segment the underlying model, allowing different parts to run on the most suitable chip architecture.
The company has already formed partnerships with chip manufacturers such as NVIDIA, AMD, Intel, ARM, Cerebras, and d-Matrix.
Gimlet’s offering, available either as standalone software or via an API to its proprietary Gimlet Cloud, is not aimed at average AI app developers. Instead, it targets the largest AI model laboratories and data centers.
The company made its public debut in October, reporting eight-figure revenues from the outset (at least $10 million). Asgar revealed that their customer base has more than doubled in the last four months, now including a significant model developer and an extremely large cloud computing firm, though he chose not to disclose their names.
The cofounders had previously collaborated at Pixie, a startup that developed an open-source observability tool for Kubernetes. Pixie was acquired by New Relic in 2020, merely two months after its launch with a $9 million Series A round led by Benchmark. (Pixie’s technology is now integrated into the open-source organization that manages Kubernetes.)
Following a chance encounter between Asgar and Tully approximately a year ago, and subsequent angel investments from Stanford professors, venture capitalists began reaching out. After the company’s launch, a term sheet arrived on Asgar’s desk. Asgar noted that once VCs learned he was considering offers, “we experienced a significant surge of funding,” and the round was rapidly oversubscribed.
Including the previous seed funding, the startup has now accumulated a total of $92 million. This includes contributions from numerous angel investors such as Sequoia’s Bill Coughran, Stanford Professor Nick McKeown, former VMware CEO Raghu Raghuram, and Intel CEO Lip-Bu Tan. The company currently employs 30 individuals.
Other investors contributing to the round include Factory, who led the seed, Eclipse Ventures, Prosperity7, and Triatomic.