Amidst a surge of innovative AI research labs emerging lately, Flapping Airplanes stands out. Driven by its youthful and inquisitive founders, Flapping Airplanes is dedicated to developing AI training methods that require significantly less data. This approach could fundamentally transform the economic viability and power of AI models, and with $180 million in seed funding, they possess ample resources to pursue their vision.
Recently, I interviewed the lab’s three co-founders — siblings Ben and Asher Spector, along with Aidan Smith — to discuss the opportune timing for launching a novel AI lab and their persistent exploration of concepts derived from the human brain.
To begin, I’m curious: why establish this venture now? Given the extensive investments by labs such as OpenAI and DeepMind in model scaling, competition must appear formidable. What made this a propitious time to launch a foundation model company?
Ben: The scope of potential work remains vast. While the progress achieved in the past five to ten years has been remarkable, providing us with invaluable daily tools, we questioned whether this represents the full extent of necessary innovation. Our careful consideration led us to conclude otherwise. Specifically, we identified data efficiency as a critical area for investigation. Contemporary advanced models are trained on the entirety of human knowledge, yet humans demonstrate the ability to learn effectively with significantly less data. This substantial disparity warrants thorough exploration.
Our endeavor represents a focused commitment to three core premises. Firstly, we believe the data efficiency challenge is paramount—a novel and distinct direction where significant advancements are achievable. Secondly, we are confident that solving this problem will yield substantial commercial value and contribute positively to global progress. Finally, we contend that a creative, and even somewhat unconventional or inexperienced, team is best suited to re-examine these foundational issues from a fresh perspective.
Aidan: Indeed. We don’t perceive ourselves as direct competitors with other labs, as our focus is on an entirely distinct set of challenges. The human mind’s learning processes, for instance, diverge dramatically from those of transformers—not necessarily superior, but inherently different. This leads to distinct trade-offs. While Large Language Models (LLMs) excel at memorization and accessing vast knowledge bases, they struggle to acquire new skills rapidly, demanding immense quantities of data for adaptation. Examining the brain reveals algorithms fundamentally unlike gradient descent and other prevalent AI training methods. Consequently, we are cultivating a new generation of researchers to tackle these issues and foster truly innovative thinking within the AI domain.
Asher: The scientific intrigue of this question is profound: why do the intelligent systems we’ve created differ so markedly from human intelligence? What accounts for this discrepancy? And how can understanding this divergence lead to superior systems? Simultaneously, I believe this pursuit holds significant commercial viability and global benefit. Numerous crucial sectors, such as robotics or scientific discovery, operate under severe data constraints. Even in enterprise settings, a model with exponentially greater data efficiency would likely integrate into the economy far more easily. Therefore, adopting a fresh perspective on these methodologies is incredibly exciting for us, prompting us to consider the immense possibilities if we possessed a truly data-efficient model.
This leads to my subsequent question, which also relates to your name, Flapping Airplanes. In AI, there’s an ongoing philosophical debate regarding whether the goal is to replicate human brain functions or to devise a more abstract form of intelligence that follows an entirely different trajectory. Aidan, given your background at Neuralink, a company centered on the human brain, do you view your approach to AI as leaning towards a neuromorphic paradigm?
Aidan: My perspective on the brain is that it serves as an existence proof. It demonstrates that alternative algorithms exist beyond conventional paradigms. The brain operates under remarkable constraints; its underlying hardware exhibits peculiar characteristics. A single action potential takes a millisecond to fire, a duration during which a modern computer can execute countless operations. Therefore, it’s realistic to assume that approaches superior to the brain, and distinctly different from transformer models, are plausible. While deeply inspired by certain brain functions, we do not feel constrained by its exact mechanisms.
Ben: To elaborate, our name, Flapping Airplanes, perfectly encapsulates our philosophy. Consider current systems as akin to large Boeing 787s. We’re not aiming to engineer birds; that’s an excessive leap. Instead, we aspire to construct a type of flapping aircraft. From a computer systems standpoint, the inherent constraints of biological brains and silicon hardware are sufficiently distinct that we should not anticipate identical resulting systems. Given such disparate substrates and genuinely different trade-offs concerning computational cost, locality, and data transfer, it’s natural for these systems to diverge in appearance. However, this divergence doesn’t preclude us from drawing inspiration from the brain and leveraging its intriguing aspects to enhance our own systems.
It appears that contemporary labs now enjoy greater latitude to prioritize research over mere product development, marking a significant distinction for this wave of AI institutions. Some are exclusively research-oriented, while others are “research-focused for now.” How is this balance discussed within Flapping Airplanes?
Asher: I truly wish I could provide a precise timeline, confidently stating that in three years, we’ll have resolved our research challenges and mapped out our commercialization strategy. However, we don’t possess those answers; our pursuit is truth. Nevertheless, our team brings commercial experience; I’ve dedicated considerable time to developing profitable technologies for various companies, and Ben has fostered multiple commercially-driven startups. We are genuinely enthusiastic about commercialization, believing that making valuable creations accessible to users benefits the world. Thus, we are not averse to it. Our imperative, however, is to commence with fundamental research, as engaging in large enterprise contracts prematurely would divert our focus and impede the valuable research we aim to undertake.
Aidan: Indeed, our aim is to explore profoundly divergent approaches, acknowledging that sometimes radical departures may prove less effective than established paradigms. We are investigating a range of distinct trade-offs, anticipating that our solutions will ultimately offer novel differentiations.
Ben: Businesses thrive when intensely focused on excelling in a specific area. While large corporations can manage numerous initiatives simultaneously, a startup must identify and fully commit to its most valuable undertaking. Currently, we generate the most significant value by dedicating ourselves entirely to solving fundamental problems.
I am genuinely optimistic that in the near future, we will have advanced sufficiently to begin applying our innovations in practical, real-world scenarios. Engaging with the real world provides invaluable feedback; it’s a constant source of learning, a vast repository of truth accessible at will. I believe the recent shifts in the economic and financial landscape for such ventures have primarily enabled companies to sustain focus on their core competencies for extended durations. This sustained focus is what excites me most, as it will empower us to produce truly distinctive work.
To clarify my understanding: the palpable excitement and evident investor opportunity have led to a willingness to commit $180 million in seed funding to an entirely new company, comprising highly intelligent yet notably young individuals who haven’t emerged from established tech successes like PayPal. What was your experience engaging with this fundraising process? Did you anticipate such a strong appetite, or did you realize during the process that your vision could be significantly larger than initially conceived?
Ben: I’d describe it as a combination of both. The market has been robust for several months, so the emergence of substantial funding rounds wasn’t entirely unexpected. However, one can never fully predict how the fundraising landscape will react to one’s specific vision. In this regard, it’s crucial to allow the market to provide feedback on your endeavors. Throughout our fundraising journey, we gained considerable insights, which led to adjustments in our ideas and a refinement of our priorities and commercialization timelines.
We were, I believe, somewhat astonished by the strong resonance of our message. While our vision was clear to us, it’s always uncertain whether others will share your conviction or consider your ideas outlandish. We’ve been exceptionally fortunate to secure a cohort of outstanding investors who genuinely connected with our mission, affirming, “Yes, this is precisely what we’ve been seeking.” This outcome was both surprising and gratifying.
Aidan: Indeed, there’s been a growing appetite for a new era of research for some time. Increasingly, we see ourselves as uniquely positioned to spearhead this research-driven age and explore truly radical concepts.
For companies primarily focused on scaling, the barrier to entry for foundation models is substantial, given the immense computational demands of building models at such a scale. Research, by contrast, occupies a middle ground; while you are still building foundation models, if your approach uses less data and is not as scale-dependent, you might gain some relief. To what extent do you anticipate compute costs constraining your operational runway?
Ben: A key benefit of engaging in profound, fundamental research is that, somewhat counterintuitively, pursuing truly radical ideas is often more economical than conducting incremental work. The reason is that incremental advancements necessitate significant scaling to validate their effectiveness, as many small-scale interventions don’t translate successfully to larger scales. Consequently, such work becomes very costly. In contrast, a groundbreaking new concept for an architecture or optimizer is likely to fail early on, sparing the need for extensive scaling. If it’s broken from the outset, that’s a valuable and efficient discovery.
This does not imply that scale is inconsequential for us. In fact, scaling remains a crucial instrument among the various capabilities at our disposal. The ability to expand our ideas is undoubtedly pertinent to our company. Therefore, I would not characterize us as being entirely opposed to scaling. Rather, a significant advantage of our research approach is the capacity to test numerous concepts at a minimal scale before even considering their application at a larger scale.
Asher: Yes, while access to the entire internet is beneficial, it shouldn’t be a prerequisite. We find it genuinely perplexing that achieving human-level intelligence currently demands training on the entirety of the internet’s data.
Consequently, what new possibilities emerge from training models more efficiently on data? One might assume increased model power and intelligence. Do you have concrete visions for its trajectory? Are we anticipating enhanced out-of-distribution generalization, or models that exhibit greater proficiency in specific tasks with reduced training experience?
Asher: Firstly, as our work is scientific, I don’t possess definitive answers, but I can offer three hypotheses. My initial hypothesis posits a broad continuum between mere statistical pattern recognition and truly profound understanding. Current models, I believe, reside somewhere along this spectrum; they haven’t achieved complete deep understanding, yet they also transcend simple pattern matching. It’s plausible that by training models on less data, we compel them to develop exceptionally deep comprehension of their observed information. This could lead to models becoming more intelligent in fascinating ways, perhaps knowing fewer facts but excelling in reasoning. This is one potential hypothesis.
A second hypothesis aligns with your point: currently, imparting new capabilities to models is extremely costly, both operationally and financially, due to the extensive data required. Our work might significantly enhance post-training efficiency, potentially enabling a model to adapt to a new domain with just a handful of examples.
Furthermore, it’s conceivable that this will unlock entirely new AI verticals. Consider certain areas of robotics, for example, where we currently struggle to achieve the capabilities necessary for commercial viability. My belief is that this is a data scarcity issue, not a hardware limitation; the ability to tele-operate robots effectively demonstrates adequate hardware. Many other domains, such as scientific discovery, face similar challenges.
Ben: To elaborate further on the potential global impact of AI, one perspective views it as a deflationary technology. In this view, AI’s role is to automate numerous jobs, making tasks cheaper and allowing robots to perform work previously done by humans, thereby removing it from the economy. While I acknowledge this outcome is likely, it doesn’t represent the most compelling vision of AI for me. The most exciting prospect is an AI that facilitates the creation of entirely new scientific breakthroughs and technologies—innovations that human intelligence alone is insufficient to conceive, but which other systems can achieve.
Regarding this, I believe the initial distinction Asher discussed—the continuum between genuine generalization and data memorization or interpolation—is crucial for generating the profound insights that will drive new advancements in medicine and science. It’s imperative for models to lean heavily towards the creative end of this spectrum. A significant reason for my enthusiasm for our work is that, beyond mere economic benefits, I am deeply committed to the mission of enabling AI to accomplish tasks fundamentally beyond human capability—a goal far more ambitious than simply automating existing jobs.
Understood. Does this stance align you with a specific position in the AGI (Artificial General Intelligence) discourse, or the conversation surrounding out-of-distribution generalization?
Asher: Honestly, I don’t have a precise definition for AGI. What is evident is the rapid progression of capabilities and the immense economic value being generated. However, I don’t believe we are on the verge of creating an omniscient AI, nor do I foresee a singularity within months or even two years that would render humanity obsolete. Fundamentally, I concur with Ben’s earlier sentiment: the world is vast, much work remains, and incredible advancements are continuously being made, to which we are eager to contribute.
Indeed, the concepts surrounding the brain and neuromorphic aspects appear highly pertinent. Are you suggesting that the most relevant comparison for LLMs is the human brain, rather than earlier models like the Mechanical Turk or deterministic computers?
Aidan: I want to stress that the brain is not the ultimate limit; in many respects, it’s merely a baseline. Frankly, I see no indication that the brain isn’t a comprehensible system governed by physical laws. We know it operates under numerous constraints. Therefore, we anticipate developing capabilities that are far more intriguing, distinct, and potentially superior to the brain’s in the long term. We are eager to contribute to that future, regardless of whether it’s classified as AGI or something else.
Asher: I agree that the brain serves as a crucial point of comparison, primarily because it helps us grasp the immense scope of possibilities. It’s easy to observe our current progress and assume we’ve found all the answers, nearing completion. However, adopting a broader perspective reveals a vast amount that remains unknown.
Ben: Our aim isn’t necessarily to be “better,” but rather to be distinct. This is the crucial point I wish to emphasize. All these systems will inevitably involve various trade-offs; an advantage in one area will likely incur a cost in another. The world is expansive, encompassing countless domains with diverse requirements. Therefore, developing more systems and foundational technologies capable of addressing these varied domains will likely facilitate the more effective and rapid integration of AI across the globe.
A distinctive aspect of your strategy is your hiring approach, which recruits exceptionally young individuals, some still pursuing higher education or even high school. What specific qualities or insights resonate with you during an interview that make you decide, “This person needs to be working with us on these research challenges”?
Aidan: It’s when an individual truly captivates you, presenting a wealth of novel ideas and approaching problems with a fresh perspective often unburdened by the extensive literature that can sometimes constrain established researchers. Above all, we prioritize creativity. Our team comprises exceptionally creative individuals, and I consider myself fortunate daily to engage with them in discussing audacious solutions to major AI challenges and envisioning a profoundly different future.
Ben: Personally, the foremost indicator I seek is whether they impart new knowledge during our interaction. If they can teach me something novel, it’s highly probable they can also bring fresh insights to our research. In the realm of research, creative and novel ideas are paramount.
During my undergraduate and doctoral studies, I co-founded an incubator named Prod, which fostered several successful companies. This experience demonstrated unequivocally that young individuals can compete effectively at the highest echelons of industry. A significant breakthrough, frankly, often lies in the realization: “Yes, I am capable of achieving this.” One can absolutely make top-tier contributions.
Naturally, we acknowledge the importance of experience. Individuals with expertise in large-scale systems are highly valued; we have, in fact, hired such talent and are enthusiastic about collaborating with diverse professionals. Our mission, I believe, has also resonated with experienced individuals. However, our primary criterion remains a desire for individuals unafraid to challenge existing paradigms and capable of envisioning entirely new systemic approaches.
I’ve been contemplating the degree of difference we can expect from future AI systems. It’s simple to envision an evolution like Claude Opus merely performing 20% better or executing 20% more tasks. However, if the outcome is fundamentally novel, it becomes challenging to conceptualize its trajectory or final form.
Asher: If you’ve ever had the opportunity to interact with the foundational GPT-4 model, you’d recall its numerous peculiar emergent capabilities. For instance, you could provide a fragment of your unwritten blog post and ask it to identify the author, and it could do so.
Many capabilities like this exist, demonstrating models’ intelligence in ways we cannot fully grasp. Future models will likely exhibit even more unusual forms of intelligence. We should anticipate a truly peculiar future, with even more unconventional architectures. Our objective is to achieve a thousand-fold improvement in data efficiency, not incremental adjustments. Consequently, we should expect equally profound, unforeseen, and transformative changes in capabilities at the extreme end.
Ben: I largely concur with that assessment. My perspective might be slightly more measured regarding how these advancements will ultimately be perceived by the public, much like how the foundational GPT-4 model was refined by OpenAI. As consumers, you want interfaces that don’t feel like peering into an abyss. That’s crucial. However, I fundamentally agree that our research objectives center on developing capabilities that are distinctively different from current possibilities.
Excellent! Are there avenues for individuals to interact with Flapping Airplanes at this stage? Or is it premature, requiring them to simply await the release of your research and models?
Asher: We offer [email protected] for general inquiries or greetings. Additionally, we have [email protected] for those who wish to challenge our approaches. We’ve actually had fascinating exchanges, with individuals submitting lengthy essays detailing why they believe our objectives are unattainable. We’re always open to engaging in such discussions.
Ben: However, no one has managed to persuade us otherwise thus far. Our conviction remains unshaken.
Asher: Secondly, we are actively seeking exceptional individuals driven to transform both the field and the world. If that describes you, we encourage you to connect with us.
Ben: Furthermore, an unconventional background is perfectly acceptable; a dual Ph.D. is not a prerequisite. Our core focus is on identifying individuals who bring diverse perspectives and innovative thought.