How Big Tech use smart speaker recordings... and what you can do about it
An estimated 30% of US adults now own a smart speaker. If you’re new to the world of smart speakers and digital assistants, it’s good to know what you’ve signed up for.
Amazon Echo, Apple HomePod and Google Nest speakers are well beyond the gimmick stage. They let to do plenty of things more quickly than you might manage with a phone, like playing a favorite Spotify playlist, setting an alarm or a reminder.
However, each request passes through the respective company’s servers, and may be stored there or used to build up an ad profile that determines the adverts you see online, or in apps.
While much of this is no different to what happens when you search Google or look up prices for products on Amazon, smart speaker privacy can feel different.
You let these devices into your home and they are, in at least a very limited sense, listening all the time. Otherwise how would they hear you say an “Alexa” or “OK, Google” wake word?
We’re going to take you through what happens when you interact with Amazon, Google and Apple smart speakers and smart displays.
Alexa: Amazon Echo devices
Amazon is the dominant maker of smart speakers, accounting for around 70% of devices sold to date in the US.
Its line-up is huge and filled with Echo products cheap enough to be bought on a whim, particularly during Amazon’s regular sales.
Amazon created the Alexa Privacy Hub to enlighten users and help alleviate privacy concerns. Here’s a look at what Amazon does when you say one of Alexa’s magic words.
What does the device record and when is it supposed to record?
When you activate Alexa by saying the wake work or pressing the button on your Alexa smart speakers, a recording is sent to Amazon’s cloud, where it is processed with other information to formulate a response.
For example, when you request “Alexa, play top hits on Amazon Music,” Alexa uses the recordings of your request and information from Amazon Music to play top hits.
No audio is stored or sent to the cloud unless the device detects the wake word (or Alexa is activated by pressing a button). You will know when Alexa is recording and sending your request because a blue light indicator appears or an audio tone sounds.
Some of the latest Amazon Echo units, including the Echo Show 10 and Echo 4th Generation also offer the option of on-device audio processing.
This means the data used by Amazon’s servers is a transcription of what you say, not the actual recording. The latest processors are smart enough to do the transcription part themselves.
Where are the recordings stored and how are they secured?
All voice recordings streamed to the cloud are encrypted and securely stored on Amazon’s servers, and are associated with your Amazon account.
This allows you to review your voice recordings, access other Amazon services, and helps Alexa give you a more personalized experience.
If the device has a camera, are any video recordings stored?
Video calling sends video to the cloud, Amazon says, but it is only streamed and never stored.
Can I be identified by my recordings?
Amazon says it associates requests with customers’ accounts to allow users to review their voice recordings and access other Amazon services.
You can ask Alexa to read Kindle books or play audiobooks from Audible, for example. Whether this directly addresses our question is open to interpretation.
Who else is listening?
A small fraction of Alexa interactions are monitored by Amazon workers. Here’s the official line: “humans review an extremely small sample of requests to help Alexa understand the correct interpretation of a request and provide the appropriate response in the future.”
We can’t know exactly how this works without being in Amazon’s office, doing the job ourselves. But the concept is real live humans are used to complement the machine learning process that mass analysis of these voice interactions enables.
It’s all about improving the real-world performance of Alexa, making it smarter. You can choose to opt out. In the Alexa app, tap the More tab and then Settings. You’ll find a Help Improve Alexa toggle in Alexa Privacy > Manage Your Alexa Data. Turn it off and your recordings will not be used.
However, Amazon says, “if you turn this off, voice recognition and new features may not work well for you.”
What happens to recordings made by mistake? Are they still analyzed?
If an Echo device starts recording due to something being mistaken for a wake word, Alexa stops processing the audio and streaming to the cloud. A short portion of the audio is stored in customers’ Voice History accessible via the Alexa app or the web.
Reviewing these recordings plays a key role in improving wake word accuracy, Amazon says. Customers who have opted out of the review process will not have these recordings analyzed.
Customers curious to understand what Alexa heard when they made a request can say, “Alexa, tell me what you heard” and the most recent voice request will be read back.
Can a user access Alexa recordings and can they delete them?
Yes, you can review and delete all Alexa voice recordings associated with your account in the Alexa app or on the web. You can also enable voice activated deletion to remove your last request by saying, “Alexa, delete what I just said.”
Alternatively, all the voice recordings for a day can be deleted by saying, “Alexa, delete everything I said today.” When you delete a voice recording, Amazon also deletes any relevant transcript.
How long does Amazon keep customers’ voice recordings? Do recordings auto-delete at any point?
Amazon keeps recordings until customers choose to delete them as standard. You can choose to have recordings saved for three months, 18 months, or not to save recordings at all.
In the Alexa app, go to Settings > Alexa Privacy > Manage Your Alexa Data > Choose How Long to Save Recordings to access these options.
How does the voice data benefit Amazon and customers?
Alexa is designed to learn. It’s a machine learning, artificial intelligence engine. The more data Amazon can use to train these systems, the better Alexa functions.
Customer speech patterns, accents, dialects and vocabulary deepen Alexa’s knowledge. Your data also has value when it comes to advertising and marketing.
Amazon will serve up personalized ads to you on its various networks based on the data it has about you. You can opt out of these in your Amazon account preferences.
Amazon says the majority of Alexa interactions are not used for advertising. But this is in part because most just aren’t relevant in that sense.
The experience on Alexa is similar to what you’d see on the Amazon website or Amazon app. For example, if you play a song on Alexa, you may see recommendations in the Amazon Music app for similar artists. Order paper towels via Alexa and you may see recommendations for similar products on the Amazon website.
Amazon says it doesn’t use other interactions with Alexa, like asking for a recipe or the weather, for product recommendations. Amazon does not allow advertising on Alexa outside of certain third-party skills, or apps, such as streaming radio skills like Pandora or news skills like CNN.
The more you use Alexa, the more it will adapt to your speech patterns, vocabulary and personal preferences. Alexa may make recommendations to customers based on their previous requests or skill usage.
Does Visual ID keep my face on file?
The latest Echo smart displays, like the Echo Show 15, have a feature called Visual ID. This lets it recognize members of your household to offer personalized content when they step up to the display.
Visual ID is disabled by default at present, and Amazon says all face recognition data remains on the device itself and is not stored on Amazon servers.
Do third party apps/other properties owned by Amazon have access to data from voice recordings?
Amazon says it does not share voice recordings with any third parties. When you use a third-party service through Alexa, it exchanges related information with the third party – an email address to make a restaurant reservation, for example.
They can use certain parts of the transcription of a voice recording, but not the recording itself.
You can control which Alexa Skills have access to data in the app – head to Settings > Alexa Privacy > Manage Skill Permission or via the website.
What data does the Echo device collect, other than voice recordings?
In addition to voice recordings, Amazon says it collects other data necessary to provide and improve the service, such device usage and network diagnostics.
Google Assistant: Google Nest speakers
Google’s privacy controls are simplistic considering how wide-reaching their impacts are. Its core settings affect your Google account across devices and services, from phones and tablets to Nest smart speakers and others with Google Assistant support.
You don’t get fine-grain control here, but there are some important options to know about.
What does the device record and when is it supposed to record?
Google Assistant sits in standby mode until it hears the wake words – “Hey Google” or “Okay Google.”
On Google Nest speakers you’ll see the four colored icons light up to indicate it’s listening, while on Google Smart Displays you’ll see an icon on the screen.
If you’re using a third-party speaker such as a Sonos One, you’ll hear a bleep and see an indicator light flash. It depends on what you’re using, but there should always be some audio or visual cue by default.
Assistant uses machine learning to recognize phrases, most likely without sending any data to Google servers.
Where are the recordings stored and how are they secured? Can I be identified?
By default, Google doesn’t retain audio recordings.
Customers can opt in to store audio data via the Voice & Audio Activity (VAA) settings. All voice recordings are encrypted and stored on Google’s servers. Google says it strips personal identifiers from voice recordings and instead attaches a unique number.
Okay, who else is listening?
As part of the Assistant review process some recordings are shared with “language experts” who are tasked with analyzing voice data to improve the service. Google says that 0.2% of all audio recordings captured are listened to by reviewers.
This part of the Assistant process was suspended in 2019 after a huge leak of customer data. However, it was reinstated in 2020.
If the device has a camera, are any video recordings stored?
Third-party cameras that work with Google Assistant do not store data on Google servers, but for the company’s own Nest devices, video footage is stored in your Google account.
You can access, review, and delete this footage – head here for more information on how.
Google’s Nest Hub Max smart display has a built-in camera, which will sometimes be used for Face Match, a tool for determining who is using the device.
Google says that video will be sent from the device to its own servers during the setup process, but not beyond that.
What happens to recordings made by mistake? Are they still analyzed?
Google has taken steps to avoid a repeat of the 2019 blunder that saw more than 1,000 Assistant voice queries leaked by a contractor. Among them were 153 snippets that were clearly recorded by mistake.
Google says it is focusing on tackling the unintended activations or “false accepts” that Google Assistant was apparently processing and have “a number of protections in place to prevent this from occurring.”
While these recordings aren’t directly linked to people’s identities, unintended voice recordings are more likely to contain sensitive information.
Google says Assistant automatically deletes any recordings from unintended activations to ensure they don’t form part of the “expert review process.”
Can a user access recordings and can they delete them? Do they auto-delete at any point?
You can review voice recordings associated with your account and permanently delete past conversations at any time via ‘My Activity.’
Voice commands such as “Hey Google, delete this week’s activity” will also erase conversations. Auto-delete can be enabled at three or 18 monthly intervals.
Here’s a full breakdown on how to delete your Google Assistant voice data.
How long does Google keep customers’ voice recordings?
Voice recordings are not kept by default, but for those who have opted in, recordings are kept until you choose to delete them.
Is data used to give me ads?
Google is adamant that it does not sell any personal information to third parties nor provide them with voice recordings.
What it can do is use the text of Assistant interactions to “inform your interests for ad personalization on Google services.”
So while the audio content of a conversation plays no part in the ads that come your way, the words that are used in the conversation will influence what adverts you see.
Google ad network Adsense generated $238 billion in 2023, and is the most profitable part of parent company Alphabet.
Anyone turned off by this can switch personalized ads based on Google activity off in the Google Account page.
Do third-party apps/other properties owned by Google have access to data from voice recordings?
If you’re using Google Assistant to interact with a third-party service, some information may be given over. Using the example of Uber, Google says it will send information provided in order “to complete a booking or confirm a ride.” In such cases, you must have given Google permission to share that information with the service.
Google elaborates a little more in its terms of service: “We don’t share information that personally identifies you with our advertising partners, such as your name or email, unless you ask us to share it. For example, if you see an ad for a nearby flower shop and select the “tap to call” button, we’ll connect your call and may share your phone number with the flower shop.”
On the Google Home privacy terms page, the section titled ‘Does the third-party service provider get an audio recording of what I said?’ Google’s response is, “Generally no. Google transcribes what you say and sends the text, but not the audio, to the third-party service provider.”
With third-party devices on your network, such as smart security devices, Google says it will share data with those providers to provide a “helpful experience,” but again you must give permission for it to do so.
In sum, voice recordings are never shared with third parties, transcriptions may be, and some personal details such as phone numbers and email addresses can be, but only if you give permission.
But the fact we had to source this information from three different places goes to show that Google still has some way to go in making this clear.
How does the voice data benefit Google and you?
Like Alexa, the more Google Assistant learns about you, the better it is at understanding speech patterns, personal preferences, accents, dialects and intonations.
This goes for things like music preferences; if Google knows your music habits, asking the Assistant to “play music”will make it better at selecting songs you’re going to like.
Siri: Apple HomePod and iOS devices
Apple is big on privacy, and makes it a selling point of life with an iPhone. It has recently made a push to improve the transparency in how your data is used across apps and services, which is commendable.
However, it has had issues of its own over the years. In 2019 a whistleblower report said Apple contractors had access to accidental Siri recordings, caused by smart speaker or Apple Watch activating without deliberate use of the Hey Siri wake word.
Apple announced an update to Siri’s privacy settings in response, on that that centered on a review process it calls ‘grading.’ Previously, a selection of Siri requests, less than 0.2% Apple claims, were flagged for human review. But users can opt out of this now.
This 2019 statement was the last major announcement on how Siri operates behind the scenes, in terms of privacy. Here’s how the process works.
What does the device record and when is it supposed to record?
Siri is designed to activate and send audio to Apple only after it recognizes the “Hey Siri” command. Audio stays on the device until the “Hey Siri” pattern has been through a two-step verification process.
Only then is anything sent to Apple’s voice recognition server for further processing.
What is the policy with contractors and Siri recordings?
As outlined in Apple’s ‘Improving Siri’s Privacy Protections’ news post, the company paused Siri grading as it “reviewed” its review process.
But the practice has since resumed. Users will be able to opt in or out to having some of their audio reviewed with the goal being to help Apple improve Siri.
Only Apple employees will be allowed to listen to audio samples, not contractors, but whether that alleviates privacy concerns is open to question.
What happens if Siri is triggered by mistake?
Of course Siri might, on occasion, mistakenly believe you used the wake word. Apple has updated its audio review process to limit recordings that human reviewers can access. False triggers identified on device will be deleted and not sent to Apple, the company says.
It also says that when a false activation does make it to the Apple servers it will be deleted, except for a subset that will be reviewed by Apple employees to ensure that false trigger identification is functioning correctly.
Any false triggers sent for review will be limited to six seconds or less and reviewers will not have access to any identifiable user data.
How long does Apple keep customers’ voice recordings? Can users access recordings and delete them?
Siri request history is associated with the random identifier for up to six months. This could include transcripts and audio for anyone who opts in to improve Siri.
After six months the user request history is disassociated from the random identifier and could be retained by Apple to help improve Siri.
Any requests that have been reviewed may be kept beyond the two-year period, without the random identifier, again for Siri improvement purposes.
Apple believe their use of random identifiers makes them unique among current digital assistants. Customers who opt-in to the ‘Improve Siri’ feature will be able to delete any Siri audio recordings or transcriptions that are less than six months old.
If Siri and dictation history is deleted within 24 hours of recording neither the audio nor transcription will be sent for review.
If the device has a camera, are any video recordings stored?
Apple does not record your FaceTime calls. It does keep a record of the person on the other end of the call, and their contact information, for 30 days.
The time of the call is recorded too, but not whether it was answered. This is the kind of information you’d probably want access too, and it is stored locally on the device beyond the 30 days it is kept on servers.
Apple states that any information stored about your use of FaceTime will be done “in a way that doesn’t identity you.”
What data do Apple devices collect, other than voice recordings?
Information such as names of contacts, installed Apps and location may also be used to improve Siri. Apple calls this information Siri Data and it is associated with a random, device generated identifier and never linked to an Apple ID or email address.
How does the voice data benefit Apple and you?
Apple says Siri Data and customer requests are never sold to anyone or used to build marketing profiles.
As per the FAQ on Siri Privacy, voice data measures how well Siri responds to requests and to improve the service.
So… which smart speaker is the most private?
If you have worked your way through this entire article, you’ll have seem some patterns. Some of the most contentious stories around digital assistant use relate to the times our data is seen, or heard, by real people.
It is a difficult but necessary part of the process, to guide the development of each digital assistant so these companies don’t have to rely purely on software or machine learning to improve itself. You can only have so much faith in “the machine”.
Apple, Google and Amazon all let you control how much information is kept on servers, to some extent. And, perhaps surprisingly, Google is the least keen on keeping voice recordings in the cloud. However, this may be because it already has the best grasp on voice transcription.
From a wider perspective, Apple is probably the best of the three assistants in privacy terms. “Siri Data and your requests are not used to build a marketing profile, and are never sold to anyone,” says Apple.
This “marketing profile” is a key element of the Google Assistant and Amazon Alexa business strategy. While this can actually be useful to some in practice, it is not a desirable strategy in privacy terms.