For the development of future autonomous machines, it’s often necessary for one model to inform another.
Businesses creating self-driving vehicles, robots that interact with their physical surroundings, or autonomous construction machinery accumulate immense quantities of video data—potentially millions of hours—for both assessment and training purposes.
Currently, humans are tasked with organizing and cataloging this extensive video footage, which involves watching it all—a process that is not scalable, even with fast-forwarding. NomadicML, a new company co-founded by CEO Mustafa Bal and CTO Varun Krishnan, aims to address this issue for clients whose fleet data largely remains unutilized in archives.
The difficulty intensifies when seeking out “edge cases” – crucial data representing infrequent events that often confuse nascent physical AI models.
Nomadic is tackling this challenge by offering a platform that transforms raw video footage into organized, searchable datasets using various vision language models. This innovation, in turn, facilitates enhanced fleet oversight and the generation of specialized datasets for reinforcement learning and accelerated development cycles.
On Tuesday, the company disclosed an $8.4 million seed funding round, valuing it at $50 million post-investment. TQ Ventures spearheaded this round, with additional backing from Pear VC and Jeff Dean. This capital infusion will enable Nomadic to expand its client base and further enhance its platform. Furthermore, Nomadic secured first place at Nvidia GTC’s recent pitch competition.
Bal informed TechCrunch that he and Krishnan, who first met as computer science undergraduates at Harvard, repeatedly encountered similar technical obstacles during their tenures at companies such as Lyft and Snowflake.
“We offer users insights derived from their specific footage, regardless of what powers their autonomous vehicles or robots,” he stated. “This precise data is what propels autonomous systems development, not arbitrary information.”
Consider, for instance, the task of refining an autonomous vehicle’s ability to proceed through a red light when a police officer directs it, or meticulously identifying every instance of vehicles passing beneath a particular kind of bridge. Nomadic’s platform facilitates the detection of such events for both regulatory compliance and direct integration into training processes.
Leading companies such as Zoox, Mitsubishi Electric, Natix Network, and Zendar are already leveraging this platform to advance their intelligent machine development. Antonio Puglielli, Zendar’s VP of Engineering, noted that Nomadic’s solution enabled his company to accelerate its efforts significantly more efficiently than outsourcing, highlighting its specialized expertise as a key differentiator.
Such model-driven, automated annotation systems are becoming an essential process in physical AI development. Prominent data labeling companies including Scale, Kognic, and Encord are creating AI-powered solutions for this task, and Nvidia has introduced Alpamayo, a suite of open-source models adaptable to this challenge.
Varun asserts that their company’s tool transcends simple labeling, acting instead as an “agentic reasoning system.” He explains, “you describe what it needs and it figures out how to find it,” leveraging various models to comprehend and contextualize actions. Nomadic’s investors anticipate that the startup’s dedication to this specialized infrastructure will lead to success.
Schuster Tanger, a TQ Ventures partner and the lead investor for the round, explained to TechCrunch, “It’s comparable to why Salesforce doesn’t create its own cloud infrastructure or Netflix doesn’t construct its own content distribution networks. The moment an autonomous vehicle company attempts to replicate Nomadic’s functionality in-house, they divert focus from their core winning objective: the robot itself.”
Tanger lauded Nomadic’s team, pointing out Krishnan’s status as an international chess master, ranked 1,549th globally. Krishnan, in turn, proudly mentioned that every one of the company’s approximately twelve engineers has authored scientific publications.
Currently, the team is diligently developing specialized tools, such as one capable of interpreting the physics of lane changes from camera recordings, and another designed to ascertain more exact positions for a robot’s grippers within video. The subsequent hurdle, as identified by Nomadic and its clientele, involves creating comparable tools for non-visual data, like lidar sensor inputs, or integrating sensor information from diverse modalities.
Bal concluded by emphasizing the immense difficulty of “managing terabytes of video, processing it through hundreds of models with over 100 billion parameters, and subsequently extracting precise insights.”