Jack Dorsey’s company, Block, unveiled a novel voice model developed in its research laboratory. This model operates locally and exhibits remarkably human-like vocalizations, including laughter, hesitations, and even tears. Block stated that the model, named Dessa, was exclusively trained using human speech and a single NVIDIA GPU.
Dessa functions as a full-duplex model, enabling it to simultaneously engage in speaking and listening, similar to the GPT-Live model introduced by OpenAI earlier this year. While speech models commonly rely on a large language model for information access, Block indicated that Dessa’s understanding is derived directly from its extensive conversational data.
Block highlighted that existing models often produce an artificial and irritating sound due to their reliance on text-based pipelines for speech synthesis. Many models, such as ElevenLabs v4 or Fish Audio Drama 3 , employ voice tags to inject emotion into generated speech. Dorsey’s firm asserts that emotion is inherently integrated into Dessa, though it remains ambiguous whether specific tags guide this emotional expression.
“Engaging with contemporary voice AI can be quite unsatisfactory. It converts your voice into text, discarding a significant amount of data that text alone cannot convey. Even the most advanced AIs globally are unable to perceive your emotions. We consider this absurd, resulting in a substandard user experience, and potentially hazardous,” the company articulated in a blog post.
Researchers at the company shared numerous instances of interactions with Dessa, showcasing the agent’s varied voices, rhythms, and emotional expressions. For example, during one exchange, the bot conveyed feeling fatigued from work and its voice accordingly reflected weariness.
In the subsequent illustration, the bot discussed how the individual made it feel at ease, exhibiting a voice filled with emotion and nearly faltering.
A further example depicts a Block team member engaging in a game with the AI voice assistant.
Block stated that Dessa is capable of operating locally on commercially available computers, including a MacBook M4 Max or superior. Nevertheless, public access to Dessa is currently unavailable, as Block has issued an invitation for safety researchers to rigorously evaluate the model. This initiative is presumably aimed at identifying scenarios where a voice agent might mislead individuals during interactions.
Block primarily operates in the financial sector, offering its services to small enterprises. Square, a subsidiary, currently employs AI-powered voice technology for order processing. However, by leveraging its proprietary, on-device voice model, Block can significantly enhance both business and consumer interactions while reducing operational expenses.