​How Amazon, Google and Apple use your smart speaker data

In a word, adverts

How tech companies use smart speaker data

When you first get your smart speaker home you probably want to hear how it sounds. See if you can get it to control your lights. See what happens when you ask it something silly.

Once that’s out of the way, it’s worth spending a little time learning exactly what happens with the data these speakers collect. It’s so valuable to companies like Google and Amazon, they have at times sold the Amazon Echo Dot and Google Home Mini at loss-masking prices just to hook more of us in.

Why is knowing every inane thing we ask our speakers worth money? As many of you may have already guessed, it comes down to advertising and marketing. Let’s start with a look at Amazon and the Alexa assistant used in the Echo smart speakers.

Amazon Echo and Alexa

How Amazon, Google and Apple use your smart speaker data

Amazon is one of the most transparently ad-obsessed companies involved in tech. It even sells Fire HD tablets and Kindle readers at a reduced price if you can put up with “special offers”, ads in other words, on their standby displays.

It’s no surprise, then, that the value of your data is predominantly about advertising and marketing.

Any voice requests you make through Alexa are linked to your Amazon account. They are stored on servers long after they are made. We have recordings logged that are almost a year old, and its terms of service does not specify a hard cut-off. There isn’t one, in other words.

Read next: What's stopping Amazon and Google recording us 24/7?

Amazon calls voice requests Alexa Interactions. Even messages you recite to your mum over an Echo Dot count as “interactions”. The audio and its transcribed text version will be held in the cloud and treated the same as a request for a weather report on the Malaga holiday you’re about to have.

The traditional worry is that Amazon is “selling” your data, packaging it up just like it does to the millions of items sold at amazon.com. It’s smarter than that, though.

Your data, for the most part, stays with other Amazon subsidiaries. Amazon runs a platter of ad and marketing companies, called AMS, AMG, A9 and AAP. We don’t need to dig too deep into the role of each, but it’s good to understand how far Amazon’s tendrils reach.

Missing manual: Your comprehensive HomePod guide

Amazon also owns places like IMDb, Twitch and Goodreads, and the organisations that manage their display adverts. Its marketing arms also work with other ad networks that place advertisements across the internet. The keywords your Alexa requests highlight, and the demographics they nudge you into will affect the ads you see everywhere, not just on amazon.com.

Amazon likely has more plans for Alexa advertising, and this is where your voice searches become more valuable than the terms you type into the Amazon search bar. After all, what better cue for ads on Amazon's shopping site is there right now than what you search for?

We might not buy a new TV via an Echo Dot, but we can certainly imagine buying shampoo

Alexa has things called skills. These are predominantly voice-based apps you install on your Echo device to give it more abilities.

A company called VoiceLabs was building a voice ads platform for skills publishers until June 2017, and only stopped because a change in Amazon’s terms and conditions killed its platform. It blocked VoiceLabs’s approach to ads. Why would Amazon do this? The official line is that Amazon wants Alexa to be a "delightful" experience for customers and doesn't want to jeopardise this with intrusive adverts. Given the grip it has on Amazon.com ads, we'd also expect it to exercise the same control on future Alexa ads.

According to CNBC, Amazon is in talks with Procter & Gamble and Clorox about Alexa skills ads. Proctor & Gamble owns many popular home brands including Gillette, Head & Shoulders and Olay. We might not buy a new TV via an Echo Dot, but we can certainly imagine buying shampoo.

An Alexa skill won’t suddenly play the equivalent of the ad spot you'd hear on a podcast, but it could more subtly nudge you towards brands in the guise of a helpful suggestion. People working in the field cite the additional context used in voice requests as particularly valuable.

“We used to say that search was the world’s largest consumer database. Voice takes that to a whole other level. In a text environment you might type ‘weather Cologne’ but in a voice context you might ask ‘is it going to rain on Thursday in Cologne’.

“We now know the day and what they are concerned about and we suddenly have rich contextual intent that we can play with,” said iProspect’s chief strategy officer Shenda Loughnane, as reported by Marketing Week.

Amazon’s official line: there are no plans for Alexa ads. We’re not sure we believe them.

Amazon Alexa users worried about the catalogue of interactions stored in the cloud can prune their voice recording history via the Alexa app or online.

Google Home and Assistant

How Amazon, Google and Apple use your smart speaker data

Google is a master at hiding how it makes its money from us as users. Use Gmail and it appears incredibly clean. Use Google Home and its main concern seems to make the most natural-feeling digital assistant possible.

The shadowy behemoth behind tech that seems to fit Google’s old “don’t be evil” tagline is its ad platform. In 2017 Google made 95.4 billion US dollars from advertising. Our data is the fuel for this platform.

To put this into context, in the last quarter of 2017 Google earned “just” $4.69 billion from all device sales and app revenues. And that includes all those Black Friday and Christmas purchases.

Just like Amazon, Google uses data it knows about us to tailor ads we see. Google is perfectly open about this. It’s written in plain English on Google’s website, if on a page that only a fraction of a per cent of Google Home owners will read.

“On surfaces where we show ads, we use data to show you ads that are relevant and useful, and to keep our services free for everyone. Google Home learns over time to provide better and more personalized suggestions and answers,” it reads.

Google’s services employ a lot of cross-pollination, with things you search on Google informing Google Assistant behaviour. And the same effect the other way around.

Requests you make to the Google Assistant can be used to put you in what Google calls an “affinity audience

Requests you make to the Google Assistant are linked to your Google account. They can be used, along with the searches you make, and websites you visit, to put you in what Google calls an “affinity audience”.

These include groups like “gamers” and “auto enthusiasts”. Advertisers will target these groups, and often dig deeper by linking them to keywords, which could come from things you say to Google Home, or type in a Google search bar.

Right now the way I use smart speakers probably wouldn’t see me recommended much more than albums or gig tickets, as I use them predominantly for music. But the end game of the Google Assistant’s natural speech engine is to get us to talk to these listening systems like a person. Read our article about how The Ambient co-founder Paul Lamkin's two-year-old daughter talks to Alexa to see why this isn’t so far-fetched.

The data we give Google is fed into Google AdSense, the largest ad network in the world. Google’s statement “we do not sell your personal information to anyone” may sound reassuring, but this is because, just like Amazon, Google doesn’t need to.

And while, aside from YouTube, Google doesn’t own ad-packed websites that’ll make you say “oh, I didn’t know Google owned that”, adverts powered by Google end up just about everywhere. Well, apart from on Amazon.

In the future, when we end up walking around city centres with bus stops, tube escalators and video-enhanced shop fronts that react to the profile on your phone, wearable or whatever tech apparatus we’ve taken to wearing, they’ll be powered by something like AdSense. If not an actual AdSense subsidiary. Google has already done this for apps, with a platform called Admob (which it acquired in 2009).

If this freaks you out a bit, you can prune your stored voice recording at Google's My Activity page.

You can opt-out of Voice & Audio Activity monitoring altogether too. However, it will make a Google Home speaker barely worth using. Google says it uses data to improve its services, and that’s true, it’s not just a cover for targeting ads at us.

Apple HomePod and Siri

How Amazon, Google and Apple use your smart speaker data

Apple uses privacy as a key selling point of the HomePod smart speaker. Anything you say to it does not end up in some cloud locker, used to sell you trainers and toilet paper when you open up a website.

The things you ask Siri are anonymised and encrypted. Even Apple can’t associate them with your Apple account. As such, they are inaccessible and useless to the sort of ad networks Amazon and Google run.

The HomePod is the best choice for those concerned about “listening devices” in your home.

However, it has a big effect on how “smart” the HomePod seems. Apple has banked on artificial intelligence rather than a catalogue of user data to make HomePod’s Siri useful. Virtually every write-up of the speaker notes it’s not as useful as Alexa or Google Assistant.

It only works with iPhones, and while its sound quality is reportedly fantastic, HomePod is only designed to stream from Apple Music. You don't get voice controls when playing Spotify tunes, just normal AirPlay functionality.

This is Apple’s way of maximising its commercial impact without relying on lucrative data collection. If you buy a HomePod you’re almost certainly going to pay for a $9.99 a month Apple Music subscription. And as the $349 price is a substantial investment, you’re all the more likely to get another iPhone when your contact ends.

Apple wants its fans firmly stuck in its web. As a mostly-Android user who likes good-sounding speakers, it seems a shame. However, it’s not (just) a tactic of Apple’s strategists.

HomePod operates using, effectively, an encrypted Siri tunnel between your iPhone or iPad and the speaker. Short of making HomePod an entirely standalone unit, there’s no other obvious way for it to operate.

Before we paint Apple as some sort of privacy saint, Apple does use some of your data outside of that collected by Siri. It operates an ad network of a much smaller scale than Amazon’s or Google’s. Apple Search Ads lets developers pay to promote their apps on the App Store and determines the ads you see on Apple News.

Here Apple uses your current location, your search requests and what Apple calls the “segment” you fit into. This is your demographic, like the “affinity audience” groups Google applies.

Apple says these segments have to apply to at least 5000 people. It’s not going to pin down “people called Bert who regularly use the Greggs Rewards app and live in Southall”. But considering there are hundreds of millions of iPhones used across the world, 5000 isn’t such a large grouping.

Apple’s approach also used to be somewhat closer to Google’s. From 2010-2016 Apple ran an ad network called iAd. It put ads into iOS apps, a rival to Google’s AdMob. Apple didn’t retire iAd because it decided ads were evil, but because its iOS-only approach left lots of its ad inventory sitting empty. It failed, in other words.

Let’s hope Apple’s famous “walled garden” approach doesn’t sink HomePod too.

TAGGED   speakers

What do you think?

Reply to
Your comment