Smart home privacy: What Amazon, Google and Apple do with your data

And what you can do about it

How tech companies use smart speaker data
The Ambient is reader-powered. If you click through using links on the site, we may earn an affiliate commission. Learn more

Nearly 70 million Americans own a smart speaker – about 20% of the population. Millions more use Google Assistant, Siri, Alexa (and yes, Cortana and Bixby) on their smartphones. Artificially intelligent voice assistants are part of everyday life.

So the news that Google, Apple and Amazon had some pretty high-profile privacy breaches this summer with customer voice recordings being listened to by third parties caused quite a stir. The tech behemoths retreated, thought long and hard about what they had done and emerged apologetically with a series of rejigged privacy policies.

Here we break down what Google, Amazon and Apple do with your voice recordings and data.

Alexa: Amazon Echo devices

Amazon's vast suite of smart speakers, smart displays, third-party devices with Alexa Voice Services built in – plus the Alexa smartphone and tablet apps – makes Alexa the most dominant AI out there.

Earlier this year, Amazon created the Alexa Privacy Hub to enlighten users and help alleviate privacy concerns. Here's a look at what Amazon does when you say one of Alexa's magic words.

What does the device record and when is it supposed to record?

When you speak to Alexa, a recording is sent to Amazon’s cloud, where it is processed with other information to formulate a response. For example, when you request “Alexa, play top hits on Amazon Music,” Alexa uses the recordings of your request and information from Amazon Music to play top hits.

By default, Echo devices are designed to detect only your chosen wake word. No audio is stored or sent to the cloud unless the device detects the wake word (or Alexa is activated by pressing a button). You will know when Alexa is recording and sending your request because a blue light indicator appears or an audio tone sounds.

Where are the recordings stored and how are they secured?

All voice recordings streamed to the cloud are encrypted and securely stored on Amazon’s servers. All voice requests are associated with your Amazon account. This allows you to review your voice recordings, access other Amazon services, and helps Alexa give you a more personalized experience.

If the device has a camera, are any video recordings stored?

Video calling sends video to the cloud, Amazon says, but it is only streamed and never stored.

Can I be identified by my recordings?

Amazon says it associates requests with customers’ accounts to allow users to review their voice recordings and access other Amazon services. You can ask Alexa to read Kindle books or play audiobooks from Audible, for example. Whether this directly addresses our question is open to interpretation.

Who else is listening?

Amazon has a voice review and annotation program – comprised of Amazon employees and contractors –that analyzes a random sample of recordings to improve Alexa's intelligence. Amazon says it annotates a fraction of 1% of interactions from a random set of customers.

Since the beginning of 2019, analysts transcribed 0.2% of all requests to Alexa. A typical Alexa recording averages 2 seconds. Annotators are subject to privacy agreements and Amazon says there is no direct access to information that can identify customers. Users can opt out of the review and annotation program via the Amazon website or the Alexa app.

What happens to recordings made by mistake? Are they still analyzed?

If an Echo device starts recording due to something being mistaken for a wake word, Alexa stops processing the audio and streaming to the cloud. A short portion of the audio is stored in customers’ Voice History accessible via the Alexa app or the web.

Reviewing these recordings plays a key role in improving wake word accuracy, Amazon says. Customers who have opted out of the review process will not have these recordings analyzed.

Customers curious to understand what Alexa heard when they made a request can say, “Alexa, tell me what you heard” and the most recent voice request will be read back. Later this year, you'll be able to ask, “Alexa, why did you do that?” to hear a short explanation.

Can I opt-out of my voice recordings being analyzed by humans?

Following recent news reports around the voice review program, Amazon now gives you the option to prevent your recordings being analyzed.

Go to the Alexa app and navigate to Settings > Alexa Privacy > Manage Your Alexa Data and turn off the toggle labeled Help Improve Amazon Services and Develop New Features.

Can a user access recordings and can they delete them? Do recordings auto-delete at any point?

Yes, you can review and delete all voice recordings associated with your account in the Alexa app or on the web. You can also enable voice activated deletion to remove your last request by saying, “Alexa, delete what I just said.”

Alternatively, all the voice recordings for a day can be deleted by saying, “Alexa, delete everything I said today.” When you delete a voice recording, Amazon also deletes any relevant transcript. There is also the option to setup auto-delete at either the 3 or 18-month marks via the Alexa Privacy Settings.

How long does Amazon keep customers’ voice recordings?

Amazon keeps recordings until customers choose to delete them.

How does the voice data benefit Amazon and customers?

As an artificial intelligence engine, Alexa is designed to learn. The more data Amazon can use to train these systems, the better Alexa works. Customer speech patterns, accents, dialects and vocabulary deepen Alexa’s knowledge. Your data also has value when it comes to advertising and marketing. Amazon will serve up personalized ads to you on its various properties based on the data it has about you. You can opt out of these in your Amazon account preferences.

The majority of Alexa interactions are not used for advertising, Amazon says. The experience on Alexa is similar to what you’d see on the Amazon website or Amazon app. For example, if you play a song on Alexa, you may see recommendations in the Amazon Music app for similar artists. Order paper towels via Alexa and you may see recommendations for similar products on the Amazon website.

Amazon says it doesn't use other interactions with Alexa, like asking for a recipe or the weather, for product recommendations. Amazon does not allow advertising on Alexa outside of certain third-party skills, or apps, such as streaming radio skills like Pandora or news skills like CNN.

The more you use Alexa, the more it will adapt to your speech patterns, vocabulary and personal preferences. Alexa may make recommendations to customers based on their previous requests or skill usage.

Do third party apps/other properties owned by Amazon have access to data from voice recordings?

Amazon says it does not share voice recordings with any third parties. When you use a third-party service through Alexa, it exchanges related information with the third party - an email address to make a restaurant reservation, for example - but not actual voice recordings.

You can control which Alexa Skills have access to data in the app – head to Settings > Alexa Privacy > Manage Skill Permission or via the website.

What data does the Echo device collect, other than voice recordings?

In addition to voice recordings, Amazon says it collects other data necessary to provide and improve the service, such device usage and network diagnostics.

Smart home privacy: What Amazon, Google and Apple do with your data

Google Assistant: Google Home speakers

In late September Google announced it had conducted a full review of systems and controls for the Google Assistant. “It's clear that we fell short of our high standards in making it easy for you to understand how your data is used, and we apologize,” Google said.

The company has also taken steps to streamline privacy information available to users, with varying degrees of clarity. Here's what the company that once embraced "Don't be evil" as its motto does with your data.

What does the device record and when is it supposed to record?

Google Assistant sits in standby mode until it hears the wake words - “Hey Google” or “Okay Google.” On Google Home speakers you'll see the four colored icons light up to indicate it's listening, while on Google Smart Displays you'll see an icon on the screen.

If you're using a third-party speaker such as a Sonos One, you'll hear a bleep and see an indicator light flash. It depends on what you're using, but there should always be some audio or visual cue by default.

Where are the recordings stored and how are they secured? Can I be identified?

By default, Google doesn’t retain audio recordings. Customers can opt in to store audio data via the Voice & Audio Activity (VAA) settings. All voice recordings are encrypted and stored on Google’s servers. Google says it strips personal identifiers from voice recordings and instead attaches a unique number..

Okay, who else is listening?

Google paused the process of human transcription of audio recordings following widely reported privacy concerns. As part of the Assistant review process some recordings are shared with “language experts” who are tasked with analyzing voice data to improve the service. Google says that 0.2% of all audio recordings captured are listened to by reviewers.

If the device has a camera, are any video recordings stored?

Not with third-party cameras that work with Google Assistant, but for the company's own Nest devices, video footage is stored in your Google account. You can access, review, and delete this footage – head here for more information on how.

Google's Nest Hub Max smart display has a built-in camera, which will sometimes be used for Face Match, a tool for determining who is using the device. Google says that video will be sent from the device to its own servers during the setup process, but not beyond that.

What happens to recordings made by mistake? Are they still analyzed?

Google has taken steps to avoid a repeat of the blunder that saw more than 1,000 Assistant voice queries leaked by a contractor. Among them were 153 snippets that were clearly recorded by mistake.

Google says itis focusing on tackling the unintended activations or "false accepts" that the Google Assistant was apparently processing and have “a number of protections in place to prevent this from occurring.”

While these recordings aren't directly linked to people's identities, unintended voice recordings are more likely to contain sensitive information. Google says Assistant automatically deletes any recordings from unintended activations to ensure they don’t form part of the “expert review process.”

Can a user access recordings and can they delete them? Do they auto-delete at any point?

You can review voice recordings associated with your account and permanently delete past conversations at any time via ‘My Activity.’ Voice commands such as “Hey Google, delete this week’s activity” will also erase conversations. Auto-delete can be enabled at three or 18 monthly intervals. Here's a full breakdown on how to delete your Google Assistant voice data.

How long does Google keep customers’ voice recordings?

Voice recordings are not kept by default, but for those who have opted in, recordings are kept until you choose to delete them.

Is data used to give me ads?

Google is adamant that it does not sell any personal information to third parties nor provide them with voice recordings. What it “may” do is use the text of Assistant interactions to “inform your interests for ad personalization on Google services.”

So while the audio content of a conversation plays no part in the ads that come your way, the words that are used in the conversation could very well influence what adverts you see.

Anyone turned off by this can turn off personalized ads based on Google activity in the Google Account page.

Do third-party apps/other properties owned by Google have access to data from voice recordings?

If you're using Google Assistant to interact with a third-party service, some information may be given over. Using the example of Uber, Google says it will send information provided in order "to complete a booking or confirm a ride." In such cases, you must have given Google permission to share that information with the service.

Google elaborates a little more in its terms of service: "We don’t share information that personally identifies you with our advertising partners, such as your name or email, unless you ask us to share it. For example, if you see an ad for a nearby flower shop and select the “tap to call” button, we’ll connect your call and may share your phone number with the flower shop."

On the Google Home privacy terms page, the section titled 'Does the third-party service provider get an audio recording of what I said?' Google's response is, "Generally no. Google transcribes what you say and sends the text, but not the audio, to the third-party service provider."

With third-party devices on your network, such as smart security devices, Google says it will share data with those providers to provide a "helpful experience," but again you must give permission for it to do so.

In sum, voice recordings are never shared with third parties, transcriptions may be, and some personal details such as phone numbers and email addresses can be, but only if you give permission. But the fact we had to source this information from three different places goes to show that Google still has some way to go in making this clear.

How does the voice data benefit Google and you?

Like Alexa, the more Google Assistant learns about you, the better it is at understanding speech patterns, personal preferences, accents, dialects and intonations. This goes for things like music preferences; if Google knows your music habits, asking the Assistant to “play music”will make it better at selecting songs you’re going to like.

Smart home privacy: What Amazon, Google and Apple do with your data

Siri: Apple HomePod and iOS devices

At the end of August Apple reacted to news reports of contractors listening to Siri audio recordings. We realize we haven’t been fully living up to our high ideals, and for that we apologize,” it said.

Apple announced an update to Siri's privacy settings that centered on a review process it calls ‘grading.’ Previously, a selection of Siri requests, less than 0.2% Apple claims, were flagged for human review. But users can opt out of this now.

Apple says privacy is a “fundamental human right." Here’s what it does with your voice emissions.

What does the device record and when is it supposed to record?

Siri is designed to activate and send audio to Apple only after it recognizes the “Hey Siri” command. Audio stays on the device until the “Hey Siri” pattern has been through a two-step verification process. Only then is anything sent to Apple’s voice recognition server for further processing.

What is the policy with contractors and Siri recordings? As outlined in Apple’s ‘Improving Siri’s Privacy Protections’ news post, the company paused Siri grading as it, err, reviewed its review process. But the practice has since resumed with one of the latest iOS software updates. Users will be able to opt in or out to having some of their audio reviewed with the goal being to help Apple improve Siri. Only Apple employees will be allowed to listen to audio samples, but whether that alleviates privacy concerns is open to question.

What happens if Siri is triggered by mistake?

Of course Siri might, on occasion, mistakenly believe you used the wake word. Apple has updated its audio review process to limit recordings that human reviewers can access. False triggers identified on device will be deleted and not sent to Apple, the company says. It also says that when a false activation does make it to the Apple servers it will be deleted, except for a subset that will be reviewed by Apple employees to ensure that false trigger identification is functioning correctly.

Any false triggers sent for review will be limited to six seconds or less and reviewers will not have access to any identifiable user data.

How long does Apple keep customers’ voice recordings? Can users access recordings and delete them?

Siri request history is associated with the random identifier for up to six months. This could include transcripts and audio for anyone who opts in to improve Siri. After six months the user request history is disassociated from the random identifier and could be retained by Apple to help improve Siri. Any requests that have been reviewed may be kept beyond the two-year period, without the random identifier, again for Siri improvement purposes.

Apple believe their use of random identifiers makes them unique among current digital assistants. Customers who opt-in to the ‘Improve Siri’ feature will be able to delete any Siri audio recordings or transcriptions that are less than six months old. If Siri and dictation history is deleted within 24 hours of recording neither the audio nor transcription will be sent for review.

If the device has a camera, are any video recordings stored?

The only case this might happen is with FaceTime. Here's Apple's lineon the matter: "Apple may record and store information about FaceTime calls, such as who was invited to a call, and your device’s network configurations, and store this information for up to 30 days. Apple doesn’t log whether your call was answered, and can’t access the content of your calls."

Apple state that any information stored about your use of FaceTime will be done “in a way that doesn’t identity you.”

What data do Apple devices collect, other than voice recordings?

Information such as names of contacts, installed Apps and location may also be used to improve Siri. Apple calls this information Siri Data and it is associated with a random, device generated identifier and never linked to an Apple ID or email address.

How does the voice data benefit Apple and you?

Apple says Siri Data and customer requests are never sold to anyone or used to build marketing profiles. As per the FAQ on Siri Privacy, voice data measures how well Siri responds to requests and to improve the service.

So... which smart speaker is the most private?

A lot has changed in the past few months in regards to each company's privacy practices – and a lot hasn't.

The ability for users to opt in or out of having their recordings reviewed by companies is the biggest of those changes. Further, how closely users read privacy small print will always vary, but steps have been taken to inform customers about policy changes and allow for a little bit more autonomy and choice.

And where it previously stood alone, Apple has now joined Google and Amazon in allowing users to delete recordings.

So, what's a smart home user to do if they're concerned about privacy but want the convenience of voice control? When it comes down to it, if you're going to use a smart speaker it's about which company you're more comfortable with, and that is always going to be a personal preference. Whether you opt in or out of services to improve these platforms will also impact how much of your voice is recorded and potentially monitored.

However there are some actions you can take – they require a bit more effort, but just like using a shredder, they'll be worth it in the long run. Delete your voice recordings every day; don't place smart speakers with cameras in bedrooms or bathrooms; mute the nearest speaker if you are having a sensitive conversation, and always remember


TAGGED    amazon    google    apple    smart home    speakers

Related stories

amazon Brilliant Alexa Easter eggs: Funny things to ask your Amazon Echo
google ​Google Nest Wifi review: It's a Mini speaker and much more
speakers Spotify Free users can now get their streaming fix on Sonos speakers
smart home Philips Hue Smart Plug review: For the HomeKit fans only
amazon How to link your Echo and Fire TV to create an Alexa Home Theatre System
google Google's Nest Hubs can now sense when you're stood nearby
What do you think?
Reply to
Your comment