While artificial intelligence is rapidly changing our world, its impact has mostly been confined to digital spaces until now. A growing number of startups are emerging, aiming to extend AI’s capabilities into physical environments.
Among these innovators is Perceptron, established in November 2024 by two ex-Meta research scientists. This company is focused on creating advanced vision models to enable machines to engage more effectively with physical surroundings.
Just this week, Perceptron unveiled Isaac 0.5, their newest model, which its developers state is engineered to empower machines with perception, reasoning, and action capabilities within industrial environments. The software is particularly adept at assisting vision-guided robots in traversing intricate locations such as warehouses and factory floors, and it also aids businesses in deriving valuable visual insights from video footage captured by these robots.
Furthermore, Isaac 0.5 is being made available as an open-weight model, allowing public access to its parameters and training resources for inspection.
Co-founders Armen Aghajanyan and Akshat Shrivastava, both alumni of Meta’s Fundamental AI Research (FAIR) division, envision their software as the cornerstone of future industrial automation.
According to the company, “Current physical AI solutions present a dilemma: either use broad foundation models requiring numerous dedicated cloud GPUs per instance, or opt for specialized models that manage either perception or control, but not both simultaneously.”
Aghajanyan and Shrivastava assert that their tool stands apart from current models by being general-purpose, not constrained to a single, recurring task. They emphasize that the model is designed for adaptability, adjusting to different environments or scenarios.
During an interview, Shrivastava prompted me to consider the steps involved in a seemingly straightforward physical process such as organizing boxes: “Visualize a robot currently sorting packages. What actions would it need to perform?”
This seemingly simple operation, in fact, encompasses numerous stages. Initially, a robot would need to scan the package label, perform spatial analysis to ascertain box locations, and then determine which box to grasp. If handling multiple boxes, it would also need to strategize the picking sequence.
Perceptron’s software is engineered to guide robots through every stage of such processes. While existing industry software can manage many of these individual tasks, few are built with the flexibility to adapt across various situations.
But what fuels this sophisticated algorithmic capability?
Models such as Isaac 0.5 acquire operational proficiency by processing immense volumes of video training data. Perceptron states its new model was trained on a million hours of general video, enabling its algorithm to recognize specific environments, visual elements, and situations. The company also made extensive use of ego video—footage recorded from a human’s perspective, typically via a GoPro or wearable camera, while performing a physical task—and UMI video, which likewise teaches AI systems movements by documenting repetitive human actions.
Although Perceptron remains tight-lipped about its training data sources, Shrivastava indicated that the company has “developed internal petabyte-scale datasets encompassing various modalities, including images, text, video, and robotic trajectories.”
The potential applications for software that enables robots to function proficiently in warehouse environments are clearly immense, and Perceptron believes it is strategically positioned to spearhead this surge in automation. The startup plans to offer its software to diverse vendors, aiming for its AI capabilities to be adopted across numerous industries.
These sectors encompass manufacturing, logistics and warehousing, security, mobility, and even media and entertainment.
Aghajanyan expressed enthusiasm, stating, “There’s truly nothing comparable currently available. We are incredibly excited about this.”
According to Pitchbook, Perceptron secured $16 million in 2024 from Bessemer Venture Partners, The Explorer Fund, and SmartGateVC. TechCrunch has learned that the startup is currently finalizing another funding round.
It’s noted that an earlier iteration of this article contained inaccuracies regarding recent funding information.