Waymo is reportedly experimenting with incorporating Google’s Gemini AI chatbot into its autonomous vehicles. This move aims to provide riders with an AI assistant capable of addressing their questions, as uncovered by researcher Jane Manchun Wong.
“During my examination of Waymo’s mobile application code, I uncovered the comprehensive system prompt for an unlaunched Gemini integration,” Wong stated in a blog post. “This document, internally named ‘Waymo Ride Assistant Meta-Prompt,’ is a specification exceeding 1,200 lines, precisely outlining the anticipated conduct of the AI assistant within a Waymo vehicle.”
Although the feature is not yet available in public versions, Wong indicates that the system prompt reveals it to be “more than a basic chatbot.” The assistant reportedly possesses capabilities to respond to inquiries, control specific interior functions such as climate, and offer reassurance to passengers when needed.
“Currently, we have no specific information to disclose, but our team consistently explores new features to enhance the Waymo riding experience, making it enjoyable, smooth, and practical,” Julia Ilina, a Waymo spokesperson, informed TechCrunch. “It’s uncertain if all these developments will ultimately be rolled out to our riders.”
This integration marks another instance of Gemini being incorporated into the self-driving technology stack of the Alphabet-owned company. Waymo previously stated its utilization of Gemini’s “world knowledge” to educate its autonomous vehicles on how to manage intricate, uncommon, and critical driving situations.
Wong notes that the assistant is designed with a distinct identity and objective: to be “a friendly and helpful AI companion embedded within a Waymo autonomous vehicle.” Its main purpose is “to enrich the rider’s journey by delivering valuable information and support in a secure, comforting, and discreet fashion.” The bot is advised to employ straightforward language, steer clear of technical terms, and limit its replies to one to three sentences for brevity.
As per the system prompts, upon activation by a rider through the in-car display, Gemini can select from a range of pre-approved greetings, customized with the rider’s first name. The system also has the capability to access relevant rider data, such as their cumulative number of Waymo journeys.
Currently, the prompts permit Gemini to interact with and manage interior vehicle features such as temperature, lighting, and audio. Wong highlighted that functions like volume control, alterations to the route, seat positioning, and window operation are conspicuously missing from the capabilities list. Should a rider inquire about a feature beyond Gemini’s control, the bot is instructed to respond with “aspirational phrases,” such as, “That’s not something I can do yet.”
Intriguingly, the assistant is mandated to uphold a distinct separation between its identity as Gemini, the AI bot, and the autonomous driving system, known as the Waymo Driver. Consequently, if asked, “How do you perceive the road?” Gemini is not to state, “I utilize a combination of sensors,” but rather, “The Waymo Driver employs a combination of sensors…”
The system prompts contain several intriguing directives, including guidelines for how the bot should address inquiries about rival companies such as Tesla or the decommissioned Cruise, as well as specific keywords that will cause it to cease speaking.
Furthermore, the assistant is instructed to refrain from speculating, elucidating, affirming, refuting, or remarking on active driving maneuvers or particular driving incidents. Thus, if a passenger inquires about footage showing a Waymo vehicle involved in a collision, the bot is designed to evade a direct response and redirect the conversation.
“You are not designated to act as a representative for the driving system’s capabilities, and you must not assume a defensive or remorseful demeanor,” the prompt specifies.
The in-vehicle assistant is authorized to respond to general knowledge queries, such as those concerning weather, the Eiffel Tower’s height, Trader Joe’s closing times, and the most recent World Series victor. However, it is prohibited from executing real-world tasks like placing food orders, making bookings, or managing emergency situations.
Waymo is not alone in incorporating AI assistants into its autonomous fleet; Tesla is pursuing a comparable initiative with xAI’s Grok. Nevertheless, these two distinct in-car assistants fulfill differing roles. Gemini seems engineered for a more practical, ride-centric approach, whereas Grok is marketed as a conversational companion capable of extended dialogue and recalling past interactions.