Tesla Adds Smart Assistant With Software Update, Replacing Voice Commands in China [Video]

Tesla Adds Smart Assistant in China

An early version of Tesla's 2023.44 update in China, has transformed its vehicles' voice command feature into a fully functional smart assistant. This assistant extends beyond vehicle control, offering features like weather updates, stock market insights, language translation, and even the recitation of Tang poetry. This development leads to speculation: is Grōk, xAI’s advanced AI model, already along for a ride or is Tesla leveraging a different service?

Grōk's Potential Role in Tesla's AI Evolution

The integration of Grōk AI into Tesla's fleet came up a few months ago when Elon Musk made a post on X that predicted that Grōk (smaller quantized) would run “native” in Tesla using local compute power. Musk then replied: Provided our vehicle AI computer is able to run the model, Tesla will probably have the most amount of true usable inference compute on Earth.

While it is not confirmed if Grōk is powering the new smart assistant in China, the possibility aligns with Tesla's trajectory towards more sophisticated AI applications in its vehicles. China also received another new feature in the update that is unavailable in other regions, a traffic light countdown.

User Impressions: A Glimpse into AI's Future in Automotive

Early feedback from Tesla users in China suggests that the new smart assistant offers a notably enhanced and intelligent experience, differentiating it from the previous basic voice command system. A Not a Tesla App reader told us that this is an authentic voice assistant. You can check the weather, stock prices, and other information.

That sounds like a young Grōk could be involved, considering its capabilities in handling complex interactions and live content processing.

Provided our vehicle AI computer is able to run the model, Tesla will probably have the most amount of true usable inference compute on Earth.

Even in a robotaxi future, the cars will only be used for ~1/3 of hours/week, leaving 2/3 for distributed inference, like SETI.

— Elon Musk (@elonmusk) November 4, 2023

Introducing this advanced assistant raises questions about Tesla's strategy in AI development. While the specifics of Tesla's partnerships in this arena remain unclear, the company's track record suggests a strong preference for in-house development, or at least leveraging products from Musk’s sister companies. The involvement of Grōk, a product of Musk's xAI, further points to Tesla's independent approach to using its AI capabilities.

The Scope of Grōk in Enhancing Tesla's AI Features

If Grōk is part of Tesla's new smart assistant, vehicle-user interaction could significantly leap. Grōk's advanced processing capabilities could enhance various aspects of the driving experience, from improved voice recognition to more intuitive responses, though its exact role in the current update remains speculative.

Musk's recent ventures into AI with xAI and Grōk have been closely watched, with many anticipating how these technologies would integrate into Tesla's product line. Musk's vision of incorporating AI like Grōk into Tesla's vehicles could greatly improve how we interact with the car. However, this integration's exact nature and timeline are still subjects of speculation.

While it's not confirmed if Grōk is powering this feature, the speculation aligns with Tesla's known ambitions in AI. Tesla's introduction of a new smart assistant in China is the latest advancement as Tesla tries to keep up with other vehicles that have already incorporated their own AI assistants like NIO or have leveraged existing services like Google Assistant in vehicles like Polestar.

Tesla’s holiday update, which includes the smart assistant in China started rolling out today. We’ll have to wait and see how well this feature works as it rolls out to more users. It’ll hopefully become available in additional languages and countries in the coming months.

Follow

Follow

Follow

Tesla is Creating a Voice Assistant to Replace Voice Commands [Example of Voice]

It looks like Tesla may soon be replacing its archaic voice commands with something much more useful — a voice assistant.

Greentheonly, who often takes deep dives into Tesla software by decompiling the firmware and looking for changes, has now found the early integration of a voice assistant.

Voice Assistant Already Available in China

Tesla has already replaced its voice commands in China with a voice assistant that can not only control certain aspects of the vehicle using more natural language but can also answer common knowledge questions such as the current weather, stock quotes and more (video below).

Tesla Creating Own Assistant

According to Green's findings, it looks like Tesla may be rolling its own solution instead of integrating something like Amazon Alexa like Rivian has. Tesla created their own solution in China by using some of their own technology and several existing software solutions. However, some of them are only available in China so they don’t scale to support additional regions.

Grōk

The first thought that comes to mind is Tesla leaning on AI efforts by Grōk and xAI, which could help it support common knowledge questions and search the web for answers. However, that only gets Tesla partly there, as Alexa and Google Assistant already support a large ecosystem, one that may not be available if Tesla creates their own voice assistant. With Tesla’s solution, a driver may be limited to what devices they can control through the voice assistant, at least initially. Either way, just a more natural way to activate certain vehicle solutions will be an improvement over what’s available today. The ability to ask the assistant common questions will be the icing on top.

Wake Word

According to Green, the source code hinted at the voice assistant using a wake word, which would replace the need to press the scroll wheel on the steering wheel. Tesla has begun to use the left scroll wheel button for additional things lately such as wiper controls and a multi-function mode, so this may fit in with their plan.

It’s fun to speculate what the wake word would be or whether Tesla would add the ability to create your own. It could be something as simple as ‘Hey, Tesla’, ‘Grok’, or something more humorous as Tesla has a knack for. Unfortunately, there weren't any hints on what the actual wake word would be, but being able to set a destination, play a specific song or ask the assistant for the weather will be a much-needed improvement.

Supported Models

It's not clear which models Tesla would support a voice assistant, but with the release of software v12 with update 2024.14.3, Tesla is starting to leave more features out of older Intel-based vehicles. While a voice assistant may not necessarily be computationally heavy, it may depend on Tesla's implementation and whether voice commands are transcribed on device or carried over the internet for transcription.

Voice Model

In the software, there are references to Microsoft's text-to-speech technology and specifically to a voice named “Jenny Neutral.”

The “Jenny Neutral” voice seems like an acceptable voice, bringing a natural-sounding voice to Tesla's voice assistant, although it doesn't differ much from what you find on other assistants. You can view an example of Microsoft’s “Jenny Neutral” voice in the video below.

Tesla has been more mindful lately about adding information for unreleased features to its software to help prevent leaks, so by the time something like this is found by Green, it usually doesn't take long before it becomes available, although this could also make for an exciting Christmas present.

Follow

Follow

Follow

Tesla Hints at Support for Amazon Music Streaming in Update 2024.14.3

Thanks to Tesla hacker Greentheonly, it looks like Tesla will soon offer a native experience for Amazon Music in their cars.

Green decompiled Tesla's latest update, version 2024.14.3, and found references to Amazon's music service, Amazo Music.

In this same update, Tesla added Audible playback, the audiobook service Amazon bought back in 2008.

For Tesla to add support for these music services, they need to have access to APIs that are provided by the music provider. Like everything else, there's likely a deal Tesla and Amazon struck, not only for Audible which was released in this update, but also for Amazon's other services. Given that there are already references to Amazon Music in 2024.14.3, it may not be long before we see the service launch in an upcoming update.

Prime Video 

Although Green doesn't mention any references to Amazon Prime Video specifically, it's possible we may see that service added as well. Tesla's Theater "apps," are essentially links to a browser without the user interface, so detecting references for Amazon Prime Video would be more difficult to discover in Tesla's source code. Since Tesla already implemented Audible and there are now hints of Amazon Music, it’s looking like we may see support for Amazon Prime Video as well.

Not the First Time Tesla Has Hinted at Amazon Music

Back in 2020, we saw Tesla add hints of supporting Amazon Music, although the code was scrapped in a later revision. The same source code in 2020 had references to Pandora, Audible and Apple Music. Since then we've seen Tesla add support for two of those services, with Pandora being the only one left out.

Top Music Services in the World

According to Exploding Topics, Amazon Music is the fourth most popular music service in the world as of February 2024 with a 13.3% share of the market. Number one is not surprisingly Spotify, with Apple Music and Tencent (Chinese music service) taking the second and third spots respectively.

Who Will Get Access

Amazon Music will actually be a great addition, especially if you don't pay for a separate music service like Spotify or Apple Music, but have Amazon Prime.

In the U.S., LiveOne, previously known as Slacker is the default music service that comes for free with Tesla — although Premium Connectivity is still required to listen while driving. Tesla simply calls it "Streaming" in the vehicle since this was named before Tesla had support for multiple music services. Tesla currently supports LiveOne, Spotify, Apple Music, TIDAL, TuneIn and SiriusXM on the Model S and Model X.

Although LiveOne is free, it has several limitations on audio quality, music curation and song selection. It also doesn’t allow you to jump to certain parts of a song.

Amazon Music offers a much more robust solution, so it's expected to be a nice upgrade for owners who already have access to the music service.

If you subscribe to Amazon Prime and get the free version of Amazon Music, it's expected that you'll have access to it in your vehicle without any additional charges, just like you do on your phone and other devices.

Amazon also offers a premium plan for $10 per month that includes higher-quality audio and access to a larger music library. However, we don't believe the premium plan will be required to use the service in the vehicle.

Tesla update 2024.14.3 is currently rolling out to customers around the world, although it could be several weeks before most customers have access.

Follow

Follow

Follow