The following is a conversation with Gustav Sonastrom.

He's the chief research and development officer at Spotify,

leading their product design, data, technology, and engineering teams.

As I've said before, in my research and in life in general,

I love music, listening to it and creating it.

And using technology, especially personalization through machine learning

to enrich the music discovery and listening experience.

That is what Spotify has been doing for years, continually innovating,

defining how we experience music as a society in a digital age.

That's what Gustav and I talk about among many other topics,

including our shared appreciation of the movie True Romance,

in my view, one of the great movies of all time.

This is the Artificial Intelligence podcast.

If you enjoy it, subscribe on YouTube, give five stars on iTunes,

support on Patreon, or simply connect with me on Twitter at Lex Freedman,

spelled F R I D M A N.

And now here's my conversation with Gustav Sonastrom.

Spotify has over 50 million songs in its catalog.

So let me ask the all important question.

I feel like you're the right person to ask.

What is the definitive greatest song of all time?

It varies for me, personally.

So you can't speak definitively for everyone?

I wouldn't believe very much in machine learning if I did,

right? Because everyone had the same taste.

So for you, what is you have to pick? What is the song?

All right. So it's it's pretty easy for me.

There is this song called You're So Cool,

Hans Zimmer, a soundtrack to True Romance.

It was a movie that made a big impression on me,

and it's kind of been following me through my life.

Actually, had to play at my wedding.

I sat with the organist and helped him play it on an organ,

which was a pretty pretty interesting experience.

That is probably my I would say top three movie of all time.

Yeah, this is an incredible movie.

Yeah. And it came out during my formative years.

And as I've discovered in music,

you shape your music taste during those years.

So it definitely affected me quite a bit.

Did it affect you in any other kind of way?

Well, the movie itself affected me back then.

It was a big part of culture.

I didn't really adopt any characters from the movie,

but it was it was a great story of love.

It's fantastic actors.

And and really, I didn't even know who Hans Zimmer was at the time,

but fantastic music.

And so that song has followed me.

And the movie actually has followed me throughout my life.

That was Quentin Tarantino, actually,

I think directed or produced that.

So it's not Stairway to Heaven or Bohemian Rhapsody.

It's those are those are great.

They're not my personal favorites,

but I've realized that people have different tastes.

And that's that's a big part of what we do.

Well, for me, I don't have to stick with Stairway to Heaven.

So 35,000 years ago, I looked this up on Wikipedia,

flute like instruments started being used in caves

as part of hunting rituals and primitive cultural gatherings,

things like that.

This is the birth of music.

Since then, we had a few folks, Beethoven, Elvis, Beatles,

Justin Bieber, of course, Drake.

So in your view, let's start like high level philosophical.

What is the purpose of music on this planet of ours?

I think music has many different purposes.

I think there's there's certainly a big purpose,

which is the same as much of entertainment,

which is escapism and to be able to live in some sort of

other mental state for a while.

But I also think you have the the opposite of escaping,

which is to help you focus on something you are actually doing.

As I think people use music as a tool to to tune the brain

to the activities that they are actually doing.

And it's kind of like in one sense,

maybe it's the rawest signal.

If you if you think about the brain as neural networks,

it's maybe the most efficient hack we can do

to actually actively tune it into some state that you want to be.

You can do it in other ways.

You can tell stories to put people in a certain mood,

but music is probably very effective

to get to a certain mood very fast.

You know, there's a there's a social component historically

to music where people listen to music together.

I was just thinking about this,

that to me, and you mentioned machine learning,

but to me personally, music is a really private thing.

Like I'm speaking for myself.

I listen to music like almost nobody knows

the kind of things I have in my library,

except people who are really close to me

and they really only know a certain percentage.

There's like some weird stuff

that I'm almost probably embarrassed by, right?

It's called the guilty pleasures, right?

Everyone has the guilty pleasures, yeah.

Hopefully they're not too bad.

For me, it's personal.

Do you think of music as something that's social

or as something that's personal?

Or does it vary?

So I think it's the same answer that you use it for both.

We've thought a lot about this

during these 10 years at Spotify, obviously.

In one sense, as you said, music is incredibly social.

You go to concerts and so forth.

On the other hand, it is your escape

and everyone has these things that are very personal to them.

So what we've found is that when it comes to,

to most people claim that they have a friend or two

that they are heavily inspired by and that they listen to.

So I actually think music is very social,

but in a smaller group setting,

it's an intimate form of, it's an intimate relationship.

It's not something that you necessarily share broadly.

Now at concerts, you can argue you do,

but then you've gathered a lot of people

that you have something in common with.

I think this broadcast sharing of music

is something we tried on social networks and so forth,

but it turns out that people aren't super interested in

what their friends listen to.

They're interested in understanding

if they have something in common perhaps with a friend,

but not just as information.

Right, that's really interesting.

I was just thinking that this morning listening to Spotify,

I really have a pretty intimate relationship with Spotify,

with my playlists.

I've had them for many years now

and they've grown with me together.

There's an intimate relationship you have

with a library of music that you've developed.

And we'll talk about different ways we can play with that.

Can you do the impossible task

and try to give a history of music listening

from your perspective, from before the internet

and after the internet?

And just kind of everything leading up to streaming

with Spotify and so on.

I'll try, it could be a 100 year podcast.

Yeah.

I'll try to do a brief version.

There are some things that I think are very interesting

during the history of music,

which is that before recorded music,

to be able to enjoy music,

you actually had to be where the music was produced

because you couldn't record it and time shift it, right?

Creation and consumption had to happen at the same time,

basically concerts.

And so you either had to get to the nearest village

to listen to music.

And while that was cumbersome

and it severely limited the distribution of music,

it also had some different qualities,

which was that the creator could always interact

with the audience.

It was always live.

And also there was no time cap on the music.

So I think it's not a coincidence

that these early classical works,

they're much longer than the three minutes.

The three minutes came in as a restriction

of the first wax disc that could only contain

a three minute song on one side, right?

So actually the recorded music severely limited the,

or put constraints.

I won't say limit.

I mean, constraints are often good,

but it put very hard constraints on the music format.

So you kind of said, like instead of doing this opus

of like many, you know, tens of minutes or something,

now you get three and a half minutes

because then you're out of wax on this disc.

But in return, you get an amazing distribution.

Your reach will widen, right?

Just on that point real quick,

without the mass scale distribution,

there's a scarcity component where you kind of look forward to it.

We had that, it's like the Netflix versus HBO Game of Thrones.

You like wait for the event because you can't really listen to it.

So you like look forward to it and then it's,

you derive perhaps more pleasure

because it's more rare for you to listen to particular piece.

You think there's value to that scarcity?

Yeah, I think that that is definitely a thing.

And there's always this component of

if you have something in infinite amount,

so will you value it as much?

Probably not.

Humanity is always seeking some, is relative.

So you're always seeking something you didn't have

and when you have it, you don't appreciate it as much.

So I think that's probably true.

But I think that's why concerts exist.

So you can actually have both.

But I think net, if you couldn't listen to music

in your car driving, that'd be worse.

That cost will be bigger than the benefit of the anticipation,

I think, that you would have.

So, yeah, it started with live concerts.

Then it's being able to, you know, the photograph invented, right?

You start to be able to record music.

Exactly.

So then you got this massive distribution

that made it possible to create two things.

I think, first of all, cultural phenomenons.

They probably need distribution.

Distribution to be able to happen.

But it also opened access to, you know, for a new kind of artist.

So you started to have these phenomenons,

like Beatles and Elvis and so forth,

that would really a function of distribution, I think.

Obviously, of talent and innovation,

but there was also taking a component.

And of course, the next big innovation to come along

was radio, broadcast radio.

And I think radio is interesting

because it started not as a music medium.

It started as an information medium for news.

And then radio needed to find something to fill the time with

so that they could honestly play more ads and make more money.

And music was free.

So then you had this massive distribution

where you could program to people.

I think those things, that ecosystem,

is what created the ability for hits.

But it was also a very broadcast medium.

So you would tend to get these massive, massive hits,

but maybe not such a long tail.

In terms of choice of everybody listening to the same stuff.

Yeah. And as you said, I think there are some social benefits to that.

I think, for example, there is a high statistical chance

that if I talk about the latest episode of Game of Thrones,

we have something to talk about just statistically.

In the age of individual choice, maybe some of that goes away.

So I do see the value of shared cultural components.

But I also obviously love personalization.

And so let's catch this up to the internet.

So maybe Napster.

Well, first of all, there's like MP3s.

Exactly. There's tape, CDs.

There was a digitalization of music with a CD, really.

It was physical distribution, but the music became digital.

And so they were files, but basically boxed software,

to use a software analogy.

And then you could start downloading these files.

And I think there are two interesting things that happened

back to music used to be longer

before it was constrained by the distribution medium.

I don't think that was a coincidence.

And then really the only music genre to have developed mostly after

music was a file again on the internet is EDM.

And EDM is often much longer than the traditional music.

I think it's interesting to think about the fact that

music is no longer constrained in minutes per song or something.

It's a legacy of an old distribution technology.

And you see some of this new music that breaks the format,

not so much as I would have expected actually by now,

but it still happens.

So first of all, I don't really know what EDM is.

Electronic dance music, you could say.

Avicii was one of the biggest in this genre.

So the main constraint is of time,

something like three, four, five minutes song.

So you could have songs that were eight minutes,

10 minutes, and so forth.

Because it started as a digital product

that you downloaded, so you didn't have this constraint anymore.

So I think it's something really interesting

that I don't think has fully happened yet.

We're kind of jumping ahead a little bit to where we are,

but I think there's tons of formal innovation in music

that should happen now, that couldn't happen

when you needed to really adhere to the distribution constraints.

If you didn't adhere to that, you would get no distribution.

So Björk, for example, an Icelandic artist,

she made a full iPad app as an album.

That was very expensive, even though the App Store

has great distribution, she gets nowhere near the distribution

versus staying within the three minute format.

So I think now that music is fully digital

inside these streaming services,

there is the opportunity to change the format again

and allow creators to be much more creative

without limiting their distribution ability.

That's interesting that you're right.

You're right, it's surprising that we don't see

that taking advantage more often.

It's almost like the constraints of the distribution

from the 50s and 60s have molded the culture

to where we want the three to five minute song,

that anything else, not just...

So we want the song as consumers and as artists,

like, because I write a lot of music

and I never even thought about writing something

longer than 10 minutes.

It's really interesting that those constraints...

Because all your training data

has been three and a half minutes long, right?

It's right.

Okay, so yeah, digitization of data led to then MP3s.

Yeah, so I think you had this file then

that was distributed physically,

but then you had the components of digital distribution

and then the internet happened and there was this vacuum

where you had a format that could be digitally shipped

but there was no business model.

And then all these pirate networks happened.

Napster and in Sweden, Pirate Bay,

which was one of the biggest.

And I think from a consumer point of view,

which leads up to the inception of Spotify

from a consumer point of view,

consumers for the first time had this access model to music

where they could, without any marginal cost,

they could try different tracks.

You could use music in new ways.

There was no marginal cost.

And that was a fantastic consumer experience.

They have access to all the music ever made.

I think it was fantastic.

But it was also horrible for artists

because there was no business model around it.

So they didn't make any money.

So the user need almost drove the user interface

before there was a business model.

And then there were these download stores

that allowed you to download files.

Which was a solution,

but it didn't solve the access problem.

There was still a marginal cost of 99 cents

to try one more track.

And I think that heavily limits how you listen to music.

The example I always give is in Spotify,

a huge amount of people listen to music while they sleep,

while they go to sleep and while they sleep.

If that costed you 99 cents per three minutes,

you probably wouldn't do that.

And you would be much less adventurous

if there was a real dollar cost to explore music.

So the access model is interesting in that

it changes your music behavior.

You can take much more risk

because there's no marginal cost to it.

Maybe let me linger on piracy for a second.

Because I find, especially coming from Russia,

piracy is something that's very interesting.

To me, not me of course ever,

but I have friends who've partook in piracy

of music, software, TV shows, sporting events.

And usually, to me, what that shows

is not that they can actually pay the money

and they're not trying to save money.

They're choosing the best experience.

So what to me, piracy shows

is a business opportunity in all these domains.

And that's where I think you're right.

Spotify stepped in is basically,

piracy is an experience.

You can explore, find music you like.

And actually, the interface of piracy is horrible.

Because it's, I mean...

Bad metadata, long download times, all kinds of stuff.

And what Spotify does is basically

first, rewards artists,

and second, makes the experience

of exploring music much better.

I mean, the same is true, I think, for movies and so on.

Piracy reveals, in the software space, for example,

I'm a huge user and fan of Adobe products.

And there was much more incentive

to pirate Adobe products

before they went to a monthly subscription plan.

And now, all of the said friends

that used to pirate Adobe products that I know,

now actually pay, gladly, for the monthly subscription.

Yeah, I think you're right.

I think it's a sign of an opportunity

for product development.

And that sometimes there's a product market fit

before there's a business model fit in product development.

I think that's a sign of it.

In Sweden, I think it was a bit of both.

There was a culture where we even had

a political party called the Pirate Party.

And this was during the time when people said that

information should be free.

It was somehow wrong to charge for ones and zeros.

So I think people felt that artists

should probably make some money somehow else

in concerts or something.

So at least in Sweden, it was part really social acceptance

even at the political level.

But that also forced Spotify to compete with free,

which I don't think could have happened

anywhere else in the world.

The music industry needed to be doing bad enough

to take that risk.

And Sweden was like the perfect testing ground.

It had government funded high bandwidth, low latency broadband,

which meant that the product would work.

And it was also, there was no music revenue anyway.

So they were kind of like,

I don't think this is going to work, but why not?

So this product is one that I don't think could have happened

in America, the world's largest music market, for example.

So how do you compete with free?

Because that's an interesting world of the internet

where most people don't like to pay for things.

So Spotify steps in and tries to, yes, compete with free.

How do you do it?

So I think two things.

One is people are starting to pay for things on the internet.

I think one way to think about it was that advertising

was the first business model

because no one would put the credit card on the internet.

Transactional with Amazon was the second.

And maybe subscription is the third.

And if you look offline, subscription is the biggest of those.

So that may still happen.

I think people are starting to pay.

But definitely back then, we needed to compete with free.

And the first thing you need to do is obviously

to lower the price to free.

And then you need to be better somehow.

And the way that Spotify was better was on the user experience,

on the actual performance, the latency of, you know,

even if you had high band with broadband,

it would still take you 30 seconds to a minute

to download one of these tracks.

So the Spotify experience of starting within the

perceptual limit of immediacy, about 250 milliseconds,

meant that the whole trick was,

it felt as if you had downloaded all of Pirate Bay.

It was on your hard drive.

It was that fast, even though it wasn't.

And it was still free.

But somehow you were actually still being a legal citizen.

That was the trick that Spotify managed to pull off.

So I've actually heard you say this or write this.

And I was surprised that I wasn't aware of it,

because I just took it for granted.

You know, whenever an awesome thing comes along,

you just like, oh, of course it has to be this way.

That's exactly right.

That it felt like the entire world's libraries

at my fingertips because of that latency being reduced.

What was the technical challenge in reducing the late?

So there was a group of really, really talented engineers.

One of them called Ludwig Strigius.

He wrote the, actually from Gothenburg,

he wrote the initial, the UTorrent Clients,

which is kind of an interesting backstory to Spotify,

you know, that we have one of the top developers

from BitTorrent Clients as well.

So he wrote UTorrent, the world's smallest BitTorrent Clients.

And then he was acquired very early by Daniel and Martin,

who founded Spotify, and they actually sold the UTorrent Client

to BitTorrent, but kept Ludwig.

So Spotify had a lot of experience within peer to peer networking.

So the original innovation was a distribution innovation,

where Spotify built an end to end media distribution system.

Up until only a few years ago, we actually hosted all the music ourselves.

So we had both the server side and the client,

and that meant that we could do things such as having a peer to peer solution

to use local caching on the client side,

because back then the world was mostly desktop.

But we could also do things like hack the TCP protocols,

things like Nagle's algorithm for kind of exponential back off,

or ramp up and just go full throttle and optimize for latency

at the cost of bandwidth.

And all of this end to end control meant that we could do

an experience that felt like a step change.

These days, we actually are on on GCP.

We don't host our own stuff, and everyone is really fast these days.

So that was the initial competitive advantage,

but then obviously you have to move on over time.

And that was over 10 years ago, right?

That was in 2008, the product was launched in Sweden.

It was in a beta, I think, 2007.

And it was on the desktop, right?

It was desktop only.

There's no phone.

There was no phone.

The iPhone came out in 2008,

but the App Store came out one year later, I think.

So the writing was on the wall, but there was no phone yet.

You've mentioned that people would use Spotify

to discover the songs they like,

and then they would torrent those songs

so they can copy it to their phone.

Just hilarious.

Exactly.

Or not torrent.

Seriously, piracy does seem to be a good guide

for business models.

Video content.

As far as I know, Spotify doesn't have video content.

Well, we do have music videos,

and we do have videos on the service,

but the way we think about ourselves is that

we're an audio service,

and we think that if you look at the amount of time

that people spend on audio,

it's actually very similar to the amount of time

that people spend on video.

So the opportunity should be equally big,

but today it's not at all valued.

Video is valued much higher.

So we think it's basically completely undervalued.

We think of ourselves as an audio service,

but within that audio service, I think video

can make a lot of sense.

I think when you're discovering an artist,

you probably do want to see them

and understand who they are,

to understand their identity.

Do you want to see that video every time?

No.

90% of the time the phone is going to be in your pocket.

For podcasters, you use video.

I think that can make a ton of sense.

So we do have video,

but we're an audio service where,

think of it as we call it internally,

backgroundable video.

Video that is helpful,

but isn't the driver of the narrative.

I think also if you look at YouTube,

the way people,

there's quite a few folks who listen to music on YouTube.

So in some sense,

YouTube is a bit of a competitor to Spotify,

which is very strange to me that people use YouTube

to listen to music.

They play essentially the music videos, right?

But don't watch the videos and put it in their pocket.

Well, I think it's similar to what

strangely maybe it's similar to what we were

for the Piracy Networks,

where YouTube for historical reasons

have a lot of music videos.

So you use,

people use YouTube for a lot of the discovery part

of the process, I think.

But then it's not a really good

sort of quote unquote MP3 player,

because it doesn't even background.

Then you have to keep the app in the foreground.

So the consumption,

it's not a good consumption tool,

but it's a decently good discovery.

I mean, I think YouTube is fantastic products.

And I use it for all kinds of purposes.

That's true.

If I were to admit something,

I do use YouTube a little bit for the discovery,

to assist in the discovery process of songs.

And then if I like it,

I'll add it to Spotify.

But that's okay.

That's okay with us.

Okay.

So sorry, we're jumping around a little bit.

So this kind of incredible,

you look at Napster,

you look at the early days of Spotify.

How do you,

one fascinating point is how do you grow

a user base?

So you're there in Sweden.

You have an idea.

I saw the initial sketches that looked terrible.

How do you grow a user base

from a few folks to millions?

I think there are a bunch of tactical answers.

So first of all,

I think you need a great product.

I don't think you take a bad product

and market it to be successful.

So you need a great product.

Sorry to interrupt,

but it's a totally new way to listen to music.

So it's not just,

did people realize immediately that Spotify is a great product?

I think they did.

So back to the point of piracy,

it was a totally new way to listen to music legally,

but people had been used to the access model in Sweden

and the rest of the world for a long time through piracy.

So one way to think about Spotify,

it was just legal and fast piracy.

And so people have been using it for a long time.

So they weren't alien to it.

They didn't really understand how it could be legal

because it seemed too fast and too good to be true,

which I think is a great product proposition

if you can be too good to be true.

But what I saw again and again

was people showing each other,

clicking the song, showing how fast it started

and say, can you believe this?

So I really think it was about speed.

Then we also had an invite program

that was really meant for scaling

because we hosted our own servers.

We needed to control scaling,

but that built a lot of expectation

and I don't want to say hype

because hype implies that it wasn't true.

Expectations, excitement around the product.

And we replicated that when we launched in the US.

We also built up an invite only program first.

There are lots of tactics,

but I think you need a great product

to solve some problem.

And basically the key innovation,

there was technology, but on a meta level,

the innovation was really the access model

versus the ownership model.

And that was tricky.

A lot of people said that they wanted to own their music.

They would never kind of rent it or borrow it.

But I think the fact that we had a free tier

which meant that you get to keep this music for life as well

helped quite a lot.

So this is an interesting psychological point

that maybe you can speak to.

It was a big shift for me.

It's almost like to go to therapy for this.

I think I would describe my early listening experience

and I think a lot of my friends do,

is basically hoarding music.

As you're slowly one song by one song,

or maybe albums,

gathering a collection of music that you love.

And you own it.

Especially with CDs or tape,

you physically had it.

And what Spotify,

what I had to come to grips with,

and it was kind of liberating actually,

is to throw away all the music.

I've had this therapy session with lots of people.

And I think the mental trick is,

so actually we've seen the user data.

When Spotify started,

a lot of people did the exact same thing.

They started hoarding,

as if the music would disappear.

Almost the equivalent of downloading.

And so we had these playlists that had limits

of a few hundred thousand tracks

and we figured no one would ever.

Well, they do.

It's in hundreds and hundreds of thousands of tracks.

And to this day,

some people want to actually save,

quote unquote,

and play the entire catalog.

But I think that the therapy session

goes something like,

instead of throwing away your music,

if you took your files

and you stored them in a locker at Google,

it'd be a streaming service.

It's just that in that locker,

you have all the world's music now for free.

So instead of giving away your music,

you got all the music.

It's yours.

You could think of it as having a copy

of the world's catalog there forever.

So you actually got more music instead of less.

It's just that you just took that hard disk

and you sent it to someone who stored it for you.

And once you go through that mental journey of,

like, still my files, they're just over there.

And I just have 40 million or 50 million or something now.

Then people are like, okay, that's good.

The problem is, I think,

because you paid us a subscription,

if we hadn't had the free tier,

where you would feel like,

even if I don't want to pay anymore,

I still get to keep them.

You keep your playlists forever.

They don't disappear even though you stop paying.

I think that was really important.

If we would have started as, you know,

you can put in all this time,

but if you stop paying, you lose all your work.

I think that would have been a big challenge

and was the big challenge for a lot of our competitors.

That's another reason why I think the free tier

is really important,

that people need to feel the security

that the work they put in, it will never disappear,

even if they decide not to pay.

I like how you put the work you put in.

I actually stopped even thinking of it that way.

Spotify taught me to just enjoy music

as opposed to what I was doing before,

which is like, in an unhealthy way, hoarding music.

Which I found that because I was doing that,

I was listening to a small selection of songs

way too much to where I was getting sick of them.

Whereas Spotify, the more liberating kind of approach,

I was just enjoying, of course,

I listened to Stairway to Heaven over and over,

but because of the extra variety,

I don't get as sick of them.

There's an interesting statistic I saw,

so Spotify has, maybe you can correct me,

but over 50 million songs, tracks,

and over 3 billion playlists.

So 50 million songs and 3 billion playlists.

60 times more playlists than songs.

What do you make of that?

So the way I think about it is that

from a statistical machine learning point of view,

you have all these,

if you want to think about reinforcement learning,

you have this state space of all the tracks,

and you can take different journeys through this world.

And I think of these as like people

helping themselves and each other,

creating interesting vectors through this space of tracks.

And then it's not so surprising that across

many tens of millions of atomic units,

there will be billions of paths that make sense.

And we're probably pretty quite far away from

having found all of them.

So kind of our job now is,

when Spotify started, it was really a search box

that was for the time pretty powerful,

and then I like to refer to this programming language

called playlisting, where if you,

as you probably were pretty good at music,

you knew your new releases, you knew your back catalog,

you knew you're starting with the heaven,

you could create a soundtrack for yourself

using this playlisting tool that's like metaprogramming language

for music to soundtrack your life.

And people who are good at music,

it's back to how do you scale the product.

For people who are good at music,

that wasn't actually enough.

If you had the catalog and a good search tool,

and you can create your own sessions,

you could create really good a soundtrack for your entire life.

Probably perfectly personalized because you did it yourself.

The problem was, many people aren't that good at music.

They just can't spend the time.

Even if you're very good at music, it's going to be hard to keep up.

So what we did to try to scale this

was to essentially try to build,

you can think of them as agents,

this friend that some people had

that helped them navigate this music catalog.

That's what we're trying to do for you.

But also, there is something like

200 million active users on Spotify.

So from the machine learning perspective,

you have these 200 million people plus.

They're creating,

it's really interesting to think of playlists

as, I don't know if you meant it that way,

but it's almost like a programming language.

It's released a trace of exploration

of those individual agents, the listeners.

And you have all this new tracks coming in.

So it's a fascinating space that is ripe for machine learning.

So how can playlists be used as data

in terms of machine learning

to help Spotify organize the music?

So we found in our data,

not surprising that people who playlisted lots,

they retained much better.

We had a great experience.

And so our first attempt was to playlist for users.

And so we acquired this company called Tunigo of editors

and professional playlisters

and kind of leveraged the maximum of human intelligence

to help build kind of these vectors

through the track space for people.

And that brought in the product.

Then the obvious next,

and we used statistical means

where they could see when they created a playlist,

how did that playlist perform?

They could see skips of the songs,

they could see how the songs perform,

and they manually iterated the playlist

to maximize performance for a large group of people.

But there were never enough editors to playlists for you personally.

So the promise of machine learning

was to go from kind of group personalization,

using editors and tools and statistics

to individualization.

And then what's so interesting

about three billion playlists we have is,

the truth is we lucked out.

This was not a priori strategy,

as is often the case.

It looks really smart in hindsight,

but it was dumb luck.

We looked at these playlists

and we had some people in the company,

a person named Erik Bernhardtson,

who was really good at machine learning

already back then, in like 2007, 2008.

Back then it was mostly collaborative filtering and so forth,

we realized that what this is,

is people are grouping tracks for themselves

that have some semantic meaning to them.

And then they actually label it

with a playlist name as well.

So in a sense, people were grouping tracks

along semantic dimensions and labeling them.

And so could you use that information

to find that latent embedding?

And so we started playing around

with collaborative filtering

and we saw tremendous success with it.

Basically trying to extract some of these dimensions.

And if you think about it, it's not surprising at all.

It would be quite surprising

if playlists were actually random,

if they had no semantic meaning.

For most people, they grouped these tracks for some reason.

So we just happened across this incredible data set

where people had taken these tens of millions of tracks

and grouped them along different semantic vectors.

And the semantics being outside the individual user,

so it's some kind of universal.

There's a universal embedding

that holds across people on this earth.

Yes, I do think that the embeddings you find

are going to be reflective of the people who play listed.

So if you have a lot of indie lovers who play list

your embedding is going to perform better there.

But what we found was that, yes,

there were these latent similarities.

They were very powerful.

And it was interesting because

I think that the people who play listed the most initially

were the so called music aficionados

who were really into music.

And they often had a certain...

Their taste was often geared towards a certain type of music.

And so what surprised us,

if you look at the problem from the outside,

you might expect that the algorithms

would start performing best with mainstreamers first

and now feels like an easier problem to solve mainstream taste

than really particular taste.

It was a complete opposite for us.

The recommendations performed fantastically

for people who saw themselves as having very unique taste.

That's probably because all of them play listed

and they didn't perform so well for mainstreamers.

They actually thought they were a bit too particular

and unorthodox.

So we had a complete opposite of what we expected.

Success within the hardest problem first

and then had to try to scale to more mainstream recommendations.

So you've also acquired

Echo Nest

that analyzes song data.

So

in your view, maybe you can talk about

so what kind of data is there

from a machine learning perspective?

There's a huge amount,

what we're talking about, play listing

and just user data

of what people are listening to, the playlist they're constructing

and so on.

And then there is the actual data within a song

what makes a song,

I don't know, the actual waveforms.

How do you mix the two?

How much values are in each?

To me, it seems like user data

is a romantic notion

that the song itself would contain useful information

but if I were to guess,

user data would be much more powerful.

Playlists would be much more powerful.

Yeah, so we use both.

Our biggest success initially was with

playlist data without understanding

anything about the structure of the song.

But when we acquired the Echo Nest,

they had the inverse problem.

They actually didn't have any play data.

They were a provider of recommendations

but they didn't actually have any play data.

So they looked at the structure of songs

sonically

and they looked at Wikipedia for cultural references

and so forth, right?

And did a lot of NLU and so forth.

They had that skill into the company

and combined our user data

with their content based.

So you can think of it as we were user based

and they were content based in their recommendations.

And we combined those two.

And for some cases where you have a new song

that has no play data,

obviously you have to try to go by

either who the artist is

or the sonic information in the song

or what it's similar to.

And we do a lot in both.

But I would say yes.

The user data captures things that

have to do with culture in the greater society

that you would never see

in the content itself.

But that said, we have seen

we have a research lab in Paris

when we can talk about

more about that on

kind of machine learning on the creator side.

What it can do for creators, not just for the consumers.

But where we looked at

how does the structure of a song actually affect

the listening behavior.

And it turns out that

we can predict things like skips

based on the song itself.

We could say that

maybe you should move that chorus a bit

because your skip is going to go up here.

There is a lot of latent structure in the music

which is not surprising

because it is some sort of mind hack.

So there should be structure.

That's probably what we respond to.

You just blew my mind actually

from the creator perspective.

That probably most creators

are not taken advantage of.

So I have recently

got to interact with a few

folks, YouTubers

who are

obsessed with this idea

of what do I

do to make sure people

keep watching the video.

And then you look at the analytics

of which point do people turn it off

and so on. First of all,

I don't think that's healthy.

It's because you can do it a little too much.

But it is a really

powerful tool for helping the creative process.

You just made me

realize you could do the same thing for

creation of music.

So is that something you've looked

into?

Can you speak

to how much opportunity there is for that kind of thing?

I listened to the podcast

with Zirash and I thought it was

fantastic and reacted to the same thing

where he said he posted something

in the morning, immediately

watched the feedback where the drop off was

and then responded to that in the afternoon.

Which is quite different

from how people make podcasts, for example.

The feedback loop is almost

non existent. So if we

back out at one level, I think

actually both for music

and podcasts, which we also

do at Spotify, I think there's

tremendous opportunity just

for the creation workflow.

I think it's really interesting

speaking to you, because you're

a musician, a developer and a

podcaster, if you think about those

three different roles,

if you make the leap as a musician,

if you think about it

as a software tool chain, really,

your door with the stems,

that's the IDE, right?

That's where you work in source code format

with what you're creating.

Then you sit around and you play with that

and when you're happy you compile that thing into

some sort of, you know, AAC,

or something, you do that because

you get distribution. There are so many run times

for that MP3 across the world in car stairs and stuff.

So you kind of compile this executable and you ship it out

in kind of an old fashioned

box software analogy.

And then you hope for the best,

right? But as

a software developer,

you would never do that. First you go

on GitHub and you collaborate with other creators.

And then, you know,

you think it would be crazy to just ship one version

of your software without doing an A.B. test,

without any feedback loop.

And then, issue tracking.

Exactly. And then you would look at the

feedback loops and try to optimize that thing, right?

So I think if you think

about it as a very specific software tool chain,

it looks

quite arcane.

The tools that a music creator has versus

what a software developer has.

So that's kind of how we think about it.

Why wouldn't

a music creator

have something like GitHub where you could collaborate

much more easily? So we bought

this company called Soundtrap,

which has a kind of

Google Docs for music approach where you can

collaborate with other people on the kind of

source code format with Stems.

And I think introducing things like

AI tools there to help you

as you're creating music,

both in

helping you

put a component to your music,

like drums or something,

help you

master and mix automatically,

help you understand how this track will perform.

Exactly what you would expect as a software

developer. I think it makes a lot of sense.

And I think the same goes for

a podcaster. I think podcasters will expect

to have the same kind of feedback loop that Sirosh

has. Like,

why wouldn't you? Maybe it's not healthy, but

Sorry, I wanted to

criticize the fact because you can overdo it.

Because a lot of the

and we're in a new era

of that.

So you can become

addicted to it and

therefore

what people say you become a slave to the YouTube

algorithm.

It's always a danger

of a new technology as opposed to

say if you're creating a song

becoming too obsessed

about the

intro riff to the song

that keeps people listening versus

actually the entirety of the creation process. It's a balance.

But the fact that

there's zero, I mean you're blowing my mind right now

because you're

completely right that there's no

signal whatsoever. There's no feedback

whatsoever on the creation process

and musical podcasting

almost at all.

And are you saying

that Spotify

is hoping to help create tools

to, not tools, but

No, tools actually.

Actually tools for creators.

Absolutely.

So we have

we've made some acquisitions the last few years around music creation

this company called Soundtrap

which is a digital audio workstation

that is browser based

and their focus was really the Google Docs approach.

We can collaborate with people much more easily

than you could in previous tools.

So we have some of these tools

that we're working with that we want to make accessible

and then we can connect it

with our consumption data.

We can create this feedback loop where

we could help you understand

we could help you create

and help you understand how you will perform.

We also acquired this other company

with a podcasting called Anchor

which is one of the biggest podcasting tools

mobile focused.

So really focused on simple creation

or easy access to creation

but that also gives us this feedback loop

and even before that

we invested in something called

Spotify for Artists

and Spotify for Podcasters

which is an app that you can download

you can verify that you are that creator

and then you get

things that

software developers have had for years.

You can see where

if you look at your podcast for example on Spotify

or a song that you released

you can see how it's performing

which citizen it's performing in

who's listening to it, what's the demographic breakup

so similar in the sense that

you can understand how you're actually doing on the platform.

So we definitely want

to build tools. I think

you also interviewed

the head of research for Adobe

and I think that's

back to Photoshop that you like.

I think that's an interesting analogy as well.

Photoshop I think has been

very innovative in helping

photographers and artists

and I think there should be

the same kind of tools for

music creators where you could get

AI assistants for example as you're creating music

as you can do

with Adobe where you can

I want a sky over here and you can get help creating that sky.

The really fascinating thing is

what Adobe

doesn't have

is a distribution for the content

you create. So you don't have

the data of if I create

if I

you know whatever creation I make

in Photoshop or Premiere

I can't get like immediate feedback

like I can on YouTube for example about

the way people are responding

and if Spotify is creating those tools

that's a really exciting

actually world

but

let's talk a little about podcasts

so I have trouble

talking to one person

so it's a bit terrifying

and kind of hard to fathom

but on average

60 to 100,000

people will listen to this episode

okay so

it's intimidating

so I hosted on Blueberry

I don't know if I'm pronouncing

that correctly actually

it looks like most people listen to it on Apple

Podcast, Cast Box and Pocket

Cast and only about

a thousand

listen on Spotify

just my podcast right

so

where

do you see a time when Spotify

will dominate this so Spotify

is relatively new

in podcasting

so yeah in podcasting

what's the deal with podcasting in Spotify

how serious

is Spotify about podcasting

do you see a time where everybody would listen

to you know probably

a huge amount of people, majority perhaps

listen to music on Spotify

do you see a time

when the same is true for

podcasting

well I certainly hope so that is our mission

our mission as a company is actually to

enable a million creators to live

off of their art and a billion people inspired

by it and what I think is interesting about that mission

is it actually puts the creators

first even though it started as a

consumer focused company and it says

to be able to live off of their art not just

make some money off of their art as well

so it's quite an ambitious

product and

so we think about creators of all kinds

and

we kind of expanded our mission from

being music to being audio

a while back

and that's not

so much because

we think we made that decision

we think that decision

was made for us

we think the world made that decision

whether we like it or not

when you put in your headphones

you're going to make a choice between

music

and a new episode of

your podcast or something else

we're in that world whether we like it or not

that's how radio work

so we decided that

we think it's about audio you can see the

rights of audio books and so forth

we think audio is this great opportunity

so we decided to enter it and obviously

Apple

and Apple podcast is absolutely

dominating in

podcasting and we didn't have

a single podcast only like two years ago

what we did though was

we

we looked at this and said

bring something to this

we want to do this but back to the original

Spotify we had to do something that consumers actually value

to be able to do this and the reason

we've gone from not existing at all

to being the

quite a wide margin the second largest podcast

consumption

still wide gap to iTunes but we're growing

quite fast

I think it's because when we looked at the consumer

problem

people said surprisingly that they wanted their podcasts

and music in the same

in the same application

so what we did was we took a little bit of a different

approach what we said instead of building a separate podcast

app

we thought is there a consumer problem to solve here

because the others are very successful already

and we thought there was in making a more

seamless experience where you can have your podcasts

and your music

in the same application

because we think it's audio to you

and that has been successful and that meant that

we actually had 200 million people

to offer this to instead of starting from zero

so I think we have a good

chance because we're taking a different approach

than the competition and back to the other thing

I mentioned about

creators

because we're looking at the end to end flow

I think there's a tremendous amount of innovation

to do around podcast as a format

when we have creation tools and consumption

I think we could

start improving what podcasting is

I mean podcast is this

this opaque big like one to

your file

that you're streaming

which it really doesn't make that much sense in 2019

that it's not interactive

there's no feedback loops nothing like that

so I think if we're going to win it's going to have to be

because we build a better product

for creators and for consumers

so we'll see but it's certainly our goal

we have a long way to go

well the creators part is really exciting

you got me hooked there

it's the only stats I have

Blueberry just recently added the stats of

it's listen to the end

or not

and that's like a huge improvement

but that's still

nowhere to where you could possibly

go into statistics

just download the Spotify podcasters up and verify

and then you'll know where people dropped out in this episode

oh wow okay

the moment I started talking

I might be depressed by this

but okay so one

one other question

the original

Spotify for music

and I have a question about

podcasting in this line

is the idea of albums

I have

music aficionados

friends who are really

big fans of music

often really enjoy albums

listening to entire albums of

an artist correct me if I'm wrong

but I feel like

Spotify has helped

replace the idea of an album with playlists

so you create your own albums

that's kind of the way

at least I've experienced music

and I've really enjoyed it that way

one of the things that was missing

in podcasting for me

I don't know if it's missing I don't know

it's an open question for me

but the way I listen to podcasts is the way I would listen to albums

so I take

Joe Rogan Experience

and that's an album and I listen

I put that on

and I listen one episode after the next

and there's a sequence and so on

is there room for

doing

what Spotify did for music

but creating

playlists

sort of this kind of playlisting

idea of breaking apart from podcasting

from individual

podcasts and creating kind of

this interplay

or have you thought about that

it's a great question so I think in

music

you're right basically you bought an album

so it was like you bought a small catalog of

10 tracks

again it was actually a lot of consumption

you think it's about what you like

but it's based on the business model

you paid for this 10 track

service and then you listen to that for a while

and then when everything was flat

priced you tended to listen differently

now so I think

the album is still tremendously important that's why we have it

and you can save albums and so forth

and you have a huge amount of people who really listen

according to albums and I like that because

it is a creator format you can tell a longer

story over several tracks

and so some people listen to just one track

some people actually want to hear that whole

story

now in podcast I think

I think it's different

you can argue that podcasts might be more

like shows on Netflix

you have like a full season of Narcos

and you're probably not going to do like one

episode of Narcos and then one of House of Cards

like you know

there's a narrative there

and you love the cast

and you love these characters

so I think people love shows

and I think they will

listen to those shows

I do think you follow a bunch of shows at the same time

so there's certainly an opportunity to bring you the

latest episode of

whatever the 5, 6, 10 things that you're into

but I think

I think people are going to listen

to specific hosts

and love those hosts

for a long time because I think there's something

different with podcasts

where this format

of the

experience of the audience

is actually standing here right between us

whereas if you look at something on TV

the audio actually would come from

you would sit over there and the audio would come to you

from both of us as if you were watching

not as if you were part of the conversation

so my experience of having to listen to podcasts

like yours and Joe Rogan

I feel like I know all of these people

and I have no idea who I am

but I feel like I've listened to so many hours of them

it's very different from me watching

like a TV show or an interview

so I think you kind of

fall in love with people

and experience in a different way

so I think shows and hosts

are going to be very important

I don't think that's going to go away into some sort of thing

where you don't even know who you're listening to

I don't think that's going to happen

what I do think is, I think there's a tremendous

discovery opportunity

in podcasts because

the catalogue is growing quite quickly

and

I think podcasts is only

a few like five, six hundred thousand

shows right now

if you look back to YouTube as another analogy

for creators, no one really knows

if you would lift the lid on

YouTube but it's probably billions

of episodes

and so I think the podcast catalogue would probably grow

tremendously because the creation

tools are getting easier

and you're going to have this

discovery opportunity that I think is really big

so a lot of people tell me that they love their shows

but discovering

podcasts kind of suck

it's really hard to get into new show

they're usually quite long, it's a big time investment

so I think there's plenty of opportunity

in the discovery part

yeah for sure, a hundred percent

and even the dumbest

there's so many low hanging fruit too

for example

just knowing

what episode to listen to first

to try out a podcast

exactly because most podcasts

don't have an order to them

they can be listened to out of order

and

sorry to say

some are better than others

episodes so some episodes of Joe Rogan

are better than others

and it's nice to know

which you should listen to

to try it out and there's as far

as I know almost no information

in terms of

like upvotes

on how good an episode is

so I think part of the problem is

it's kind of like music

there isn't one answer, people use music for different things

and there's actually many different types of music

there's workout music and there's classical piano music

and focus music and

and so forth

I think the same with podcasts, some podcasts are sequential

they're supposed to be listened to

in order

it's actually telling and narrative

podcasts are one topic

kind of like yours but different guests

so you could jump in anywhere

some podcasts actually have completely different topics

and for those podcasts it might be that

we should recommend

one episode because it's about AI

from someone

but then they talk about something that you're not interested in

the rest of the episode

so I think what we're spending a lot of time on now

is just first understanding the domain

and creating kind of the knowledge graph

of

how do these objects relate

and how do people consume and I think we'll find that it's going to be

it's going to be different

I'm excited

Spotify is the first

people I'm aware of that are

trying to do this

for podcasting

podcasting has been like a wild west

until now

we want to be very careful though

because it's been a very good wild west

I think it's this fragile ecosystem

and

we want to make sure that

you don't barge in and say

we're going to internetize this thing

and you have to think about

the creators, you have to understand

how they get distribution today

who listens to how they make money today

try to

make sure that their business model works, that they understand

I think it's back to doing something

improving their products

like feedback loops and distribution

jumping back into terms of this

fascinating world of

recommender system and listening to music

and using machine learning to analyze things

do you think it's better

to

what currently

correct me if I'm wrong but

Spotify lets people pick what they listen to

for the most part

there's a discovery process but you kind of

organize playlists

is it better to let people pick

what they listen to or recommend

what they should listen to

something like stations by Spotify

that I saw that you're playing around with

maybe you can tell me

what's the status of that

this is a Pandora style app

that just kind of as opposed to you select

the music you listen to

it kind of feeds you

the music you listen to

what's the status of stations by Spotify

what's its future

the store is Spotify as we have grown

has been that we made it more accessible

to different audiences

and

stations is another one of those where

the question is

some people want to be very specific

they actually want to start with heaven right now

that needs to be very easy to do

and some people or even the same person

at some point might say

I want to feel upbeat

or I want to feel happy

or I want songs to sing in the car

so they put in the information

at a very different level

and then we need to translate that into

musically

so stations is a test to

create like a consumption input vector

that is much simpler where you can just tune it a little bit

and see if that increases the overall reach

but we're trying to kind of serve

the entire gamut of super advanced

so called music aficionados

all the way to

to people who

they love listening to music but it's not

their number one priority in life

they're not going to sit and follow every new release

from every new artist

to influence music at

at a different level

so we're trying, you can think of it as different

products and I think when

one of the interesting things

to answer your question on

if it's better to lift the user choose

or to play I think the answer is

the challenge when you

when machine learning kind of came along

there was a lot of thinking about

what does product development mean

in a machine learning context

people like Andrew Eng for example

when he went to Baidu

he started doing a lot of practical machine learning

went from academia and he thought a lot about this

and he had this notion that

you know product manager, designer

and they used to work around this wireframe

kind of describe what the product should look like