The AI personal assistant Instinct, currently in private testing, is generating considerable buzz. While lauded for its impressive capabilities, it has also sparked discussions regarding potential privacy issues. Users have described the agent, which handles various tasks, as “magic-like” and among the “most exciting launches” since OpenClaw. However, some testers have expressed worries about the AI agent’s security framework and its concerning terms of service.
It is important to note that Instinct is still undergoing private testing, so these concerns have not yet reached the broader public.
Based in San Francisco, Instinct was developed by a small team led by former Sierra research scientist Noah Shinn and is operated by Spear Street Technology, according to its terms and California business filings. The company is currently operating in stealth mode, as reported by PitchBook.
The AI agent functions by integrating with your various applications and devices, including email, messaging services, calendar, and access to your device’s audio, location, screen, and more. Users can interact with the agent via text message or WhatsApp, commanding it to execute tasks such as booking appointments and reservations, arranging airport transportation, managing email inboxes, organizing crucial information, handling shopping, finding affordable flights, and many other functions.
While testers generally describe Instinct as exceeding their expectations, some have also voiced apprehension regarding the company’s stance on user privacy and security. This raises a pertinent question: are the benefits of granting AI this level of access and independence truly justifiable?
For example, several individuals have shared screenshots from the company’s Terms of Service. These terms grant Instinct a broad, “perpetual and irrevocable” license to “access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify” any user materials, including for the purpose of training its AI models. The terms further outline how Instinct can gather information from users’ devices, such as screen captures, cursor movements, and keyboard inputs.
Furthermore, the terms permit Instinct to enter into “agreements, commitments, or transactions” on behalf of users, which would be legally binding.
Peter Yang, an early adopter, reported that Instinct initially refused to delete his Gmail records when requested. He later stated that the team resolved the issue by integrating a tool in its settings for deleting external data.
Another user, Claire Vo, discovered that Instinct continued to summarize her inbox even after revoking its access. When she inquired about this, the bot confirmed that her emails were stored in plain text for future searches.
Many others also raised questions about the security model. One tester expressed concern upon realizing that Instinct was capable of extracting a sign-up code from their email to finalize a task, specifically booking a restaurant table through Resy. Alex Cohen, co-founder of Hello Patient, stated he deleted his account after discovering how susceptible Instinct was to phishing:
Furthermore, Katie Jacobs Stanton, founder of Moxxie Ventures, shared that Instinct eroded her trust by sending an email on her behalf without prior consultation.
“We are exchanging privacy and control for highly personalized AI tools (such as AI notetakers and personalized AI agents), often without fully comprehending the trade-offs,” she commented on X, encapsulating the dilemma presented by personal AI agents. She added, “The more powerful these agents become, the more critical trust becomes. Every successful action builds a little more trust. One unauthorized action can reduce that trust to zero.”
Michael Mignano, founder of Anchor (acquired by Spotify) and now a General Partner at Union Square Ventures, observed that products like Instinct will “redefine modern security norms for consumers,” predicting that “people will increasingly provide passwords to third-party apps, often unaware of how or what information these apps are storing.”
Interest in Instinct and personal AI has surged following the emergence of OpenClaw, a personal AI assistant celebrated for its robust capabilities, which led its founder to join OpenAI to contribute to the development of next-generation personal agents. Another assistant relying on messaging, Poke, was recently acquired by Cognition.
Despite the criticisms, the Instinct team has not yet publicly addressed any concerns or complaints on X, maintaining a low profile. (While the bot itself indicates Luca Borletti, also formerly of Sierra, is involved with the company, this has not been officially confirmed.) TechCrunch has learned from multiple investors that Kleiner Perkins and Conviction have invested in the startup, and these funding rounds are now closed.
Requests for comment sent to both the startup’s primary email address and to Shinn directly have yet to receive a response.