back to index

Michael Kearns: Algorithmic Fairness, Privacy & Ethics | Lex Fridman Podcast #50


small model | large model

link |
00:00:00.000
The following is a conversation with Michael Kearns.
link |
00:00:03.120
He's a professor at the University of Pennsylvania
link |
00:00:06.240
and a coauthor of the new book, Ethical Algorithm,
link |
00:00:09.520
that is the focus of much of this conversation.
link |
00:00:12.640
It includes algorithmic fairness, bias, privacy,
link |
00:00:16.640
and ethics in general.
link |
00:00:18.080
But that is just one of many fields
link |
00:00:20.000
that Michael is a world class researcher in,
link |
00:00:22.480
some of which we touch on quickly,
link |
00:00:24.880
including learning theory
link |
00:00:26.240
or the theoretical foundation of machine learning,
link |
00:00:29.120
game theory, quantitative finance,
link |
00:00:31.280
computational social science, and much more.
link |
00:00:34.000
But on a personal note,
link |
00:00:35.600
when I was an undergrad, early on,
link |
00:00:38.320
I worked with Michael
link |
00:00:39.520
on an algorithmic trading project
link |
00:00:41.280
and competition that he led.
link |
00:00:43.120
That's when I first fell in love
link |
00:00:44.720
with algorithmic game theory.
link |
00:00:46.800
While most of my research life
link |
00:00:48.400
has been in machine learning
link |
00:00:49.520
and human robot interaction,
link |
00:00:51.440
the systematic way that game theory
link |
00:00:53.600
reveals the beautiful structure
link |
00:00:55.120
in our competitive and cooperating world of humans
link |
00:00:58.480
has been a continued inspiration to me.
link |
00:01:01.120
So for that and other things,
link |
00:01:03.520
I'm deeply thankful to Michael
link |
00:01:05.600
and really enjoyed having this conversation
link |
00:01:07.920
again in person after so many years.
link |
00:01:11.040
This is the Artificial Intelligence Podcast.
link |
00:01:13.760
If you enjoy it, subscribe on YouTube,
link |
00:01:16.080
give it five stars on Apple Podcast,
link |
00:01:18.320
support on Patreon,
link |
00:01:19.760
or simply connect with me on Twitter
link |
00:01:21.680
at Lex Friedman, spelled F R I D M A N.
link |
00:01:25.760
This episode is supported
link |
00:01:27.280
by an amazing podcast called Pessimists Archive.
link |
00:01:31.040
Jason, the host of the show,
link |
00:01:32.720
reached out to me looking to support this podcast,
link |
00:01:35.360
and so I listened to it, to check it out.
link |
00:01:38.160
And by listened, I mean I went through it,
link |
00:01:40.720
Netflix binge style, at least five episodes in a row.
link |
00:01:44.800
It's not one of my favorite podcasts,
link |
00:01:46.480
and I think it should be one of the top podcasts
link |
00:01:48.720
in the world, frankly.
link |
00:01:50.480
It's a history show
link |
00:01:51.760
about why people resist new things.
link |
00:01:54.080
Each episode looks at a moment in history
link |
00:01:56.400
when something new was introduced,
link |
00:01:59.040
something that today we think of as commonplace,
link |
00:02:01.680
like recorded music, umbrellas, bicycles, cars,
link |
00:02:04.560
chess, coffee, the elevator,
link |
00:02:06.960
and the show explores why it freaked everyone out.
link |
00:02:10.160
The latest episode on mirrors and vanity
link |
00:02:12.800
still stays with me as I think about vanity
link |
00:02:15.360
in the modern day of the Twitter world.
link |
00:02:18.640
That's the fascinating thing about the show,
link |
00:02:20.880
is that stuff that happened long ago,
link |
00:02:22.640
especially in terms of our fear of new things,
link |
00:02:25.040
repeats itself in the modern day,
link |
00:02:26.880
and so has many lessons for us to think about
link |
00:02:29.280
in terms of human psychology
link |
00:02:31.040
and the role of technology in our society.
link |
00:02:34.160
Anyway, you should subscribe
link |
00:02:35.600
and listen to Pessimist Archive.
link |
00:02:37.760
I highly recommend it.
link |
00:02:39.840
And now, here's my conversation with Michael Kearns.
link |
00:02:44.880
You mentioned reading Fear and Loathing in Las Vegas
link |
00:02:47.840
in high school, and having a more,
link |
00:02:50.800
or a bit more of a literary mind.
link |
00:02:52.560
So, what books, non technical, non computer science,
link |
00:02:56.880
would you say had the biggest impact on your life,
link |
00:02:59.360
either intellectually or emotionally?
link |
00:03:02.400
You've dug deep into my history, I see.
link |
00:03:04.640
Went deep.
link |
00:03:05.680
Yeah, I think, well, my favorite novel is
link |
00:03:08.240
Infinite Jest by David Foster Wallace,
link |
00:03:10.960
which actually, coincidentally,
link |
00:03:13.360
much of it takes place in the halls of buildings
link |
00:03:15.840
right around us here at MIT.
link |
00:03:18.000
So that certainly had a big influence on me.
link |
00:03:20.000
And as you noticed, like, when I was in high school,
link |
00:03:22.320
I actually even started college as an English major.
link |
00:03:25.440
So, I was very influenced by sort of that genre of journalism
link |
00:03:29.120
at the time, and thought I wanted to be a writer,
link |
00:03:30.880
and then realized that an English major teaches you to read,
link |
00:03:33.840
but it doesn't teach you how to write,
link |
00:03:35.360
and then I became interested in math
link |
00:03:36.960
and computer science instead.
link |
00:03:38.560
Well, in your new book, Ethical Algorithm,
link |
00:03:41.600
you kind of sneak up from an algorithmic perspective
link |
00:03:45.920
on these deep, profound philosophical questions
link |
00:03:48.400
of fairness, of privacy.
link |
00:03:55.120
In thinking about these topics,
link |
00:03:56.400
how often do you return to that literary mind that you had?
link |
00:04:01.440
Yeah, I'd like to claim there was a deeper connection,
link |
00:04:05.200
but, you know, I think both Aaron and I
link |
00:04:07.920
kind of came at these topics first and foremost
link |
00:04:10.320
from a technical angle.
link |
00:04:11.280
I mean, you know, I kind of consider myself primarily
link |
00:04:14.720
and originally a machine learning researcher,
link |
00:04:17.600
and I think as we just watched, like the rest of the society,
link |
00:04:20.400
the field technically advance, and then quickly on the heels
link |
00:04:23.440
of that kind of the buzzkill of all of the antisocial behavior
link |
00:04:27.120
by algorithms, just kind of realized
link |
00:04:29.360
there was an opportunity for us to do something about it
link |
00:04:31.840
from a research perspective.
link |
00:04:34.000
You know, more to the point in your question,
link |
00:04:36.320
I mean, I do have an uncle who is literally a moral philosopher,
link |
00:04:41.600
and so in the early days of my life,
link |
00:04:43.280
he was a philosopher, and so in the early days
link |
00:04:46.000
of our technical work on fairness topics,
link |
00:04:48.800
I would occasionally, you know, run ideas behind him.
link |
00:04:51.280
So, I mean, I remember an early email I sent to him
link |
00:04:53.440
in which I said, like, oh, you know,
link |
00:04:55.040
here's a specific definition of algorithmic fairness
link |
00:04:57.840
that we think is some sort of variant of Rawlsian fairness.
link |
00:05:02.320
What do you think?
link |
00:05:03.920
And I thought I was asking a yes or no question,
link |
00:05:06.880
and I got back your kind of classical philosopher's
link |
00:05:09.120
response saying, well, it depends.
link |
00:05:10.800
Hey, then you might conclude this, and that's when I realized
link |
00:05:14.400
that there was a real kind of rift between the ways
link |
00:05:19.680
philosophers and others had thought about things
link |
00:05:21.840
like fairness, you know, from sort of a humanitarian perspective
link |
00:05:25.680
and the way that you needed to think about it
link |
00:05:27.840
as a computer scientist if you were going to kind of
link |
00:05:30.720
implement actual algorithmic solutions.
link |
00:05:34.080
But I would say the algorithmic solutions take care
link |
00:05:39.200
of some of the low hanging fruit.
link |
00:05:41.360
Sort of the problem is a lot of algorithms,
link |
00:05:44.560
when they don't consider fairness,
link |
00:05:47.280
they are just terribly unfair.
link |
00:05:50.240
And when they don't consider privacy,
link |
00:05:51.680
they're terribly, they violate privacy.
link |
00:05:55.200
Sort of the algorithmic approach fixes big problems.
link |
00:05:59.680
But there's still, when you start pushing into the gray area,
link |
00:06:04.000
that's when you start getting into this philosophy
link |
00:06:06.240
of what it means to be fair, starting from Plato,
link |
00:06:09.680
what is justice kind of questions?
link |
00:06:12.400
Yeah, I think that's right.
link |
00:06:13.280
And I mean, I would even not go as far as you want to say
link |
00:06:16.960
that sort of the algorithmic work in these areas
link |
00:06:19.520
is solving like the biggest problems.
link |
00:06:22.720
And, you know, we discuss in the book,
link |
00:06:24.320
the fact that really we are, there's a sense in which
link |
00:06:27.520
we're kind of looking where the light is in that,
link |
00:06:30.640
you know, for example, if police are racist
link |
00:06:34.880
in who they decide to stop and frisk,
link |
00:06:37.760
and that goes into the data,
link |
00:06:39.120
there's sort of no undoing that downstream
link |
00:06:41.520
by kind of clever algorithmic methods.
link |
00:06:45.760
And I think, especially in fairness,
link |
00:06:47.520
I mean, I think less so in privacy,
link |
00:06:49.840
where we feel like the community kind of really has settled
link |
00:06:52.880
on the right definition, which is differential privacy.
link |
00:06:56.320
If you just look at the algorithmic fairness literature
link |
00:06:58.960
already, you can see it's going to be much more of a mess.
link |
00:07:01.600
And, you know, you've got these theorems saying,
link |
00:07:03.360
here are three entirely reasonable,
link |
00:07:06.640
desirable notions of fairness.
link |
00:07:09.760
And, you know, here's a proof that you cannot simultaneously
link |
00:07:12.720
have all three of them.
link |
00:07:14.560
So I think we know that algorithmic fairness
link |
00:07:17.680
compared to algorithmic privacy
link |
00:07:19.280
is going to be kind of a harder problem.
link |
00:07:21.600
And it will have to revisit, I think,
link |
00:07:23.440
things that have been thought about by,
link |
00:07:26.080
you know, many generations of scholars before us.
link |
00:07:29.520
So it's very early days for fairness, I think.
link |
00:07:32.240
TK So before we get into the details
link |
00:07:34.560
of differential privacy, and on the fairness side,
link |
00:07:37.280
let me linger on the philosophy a bit.
link |
00:07:39.520
Do you think most people are fundamentally good?
link |
00:07:43.600
Or do most of us have both the capacity
link |
00:07:46.640
for good and evil within us?
link |
00:07:48.320
SB I mean, I'm an optimist.
link |
00:07:50.240
I tend to think that most people are good
link |
00:07:52.480
and want to do right.
link |
00:07:55.600
And that deviations from that are, you know,
link |
00:07:58.480
kind of usually due to circumstance,
link |
00:08:00.240
not due to people being bad at heart.
link |
00:08:02.960
TK With people with power,
link |
00:08:05.520
are people at the heads of governments,
link |
00:08:08.160
people at the heads of companies,
link |
00:08:10.320
people at the heads of, maybe, so financial power markets,
link |
00:08:15.040
do you think the distribution there is also,
link |
00:08:19.040
most people are good and have good intent?
link |
00:08:21.040
SB Yeah, I do.
link |
00:08:22.800
I mean, my statement wasn't qualified to people
link |
00:08:26.560
not in positions of power.
link |
00:08:28.560
I mean, I think what happens in a lot of the, you know,
link |
00:08:30.640
the cliche about absolute power corrupts absolutely.
link |
00:08:34.080
I mean, you know, I think even short of that,
link |
00:08:37.680
you know, having spent a lot of time on Wall Street,
link |
00:08:40.720
and also in arenas very, very different from Wall Street,
link |
00:08:44.160
like academia, you know, one of the things
link |
00:08:47.920
I think I've benefited from by moving between
link |
00:08:50.960
two very different worlds is you become aware
link |
00:08:53.760
that, you know, these worlds kind of develop
link |
00:08:57.120
their own social norms, and they develop
link |
00:08:59.040
their own rationales for, you know,
link |
00:09:02.080
behavior, for instance, that might look
link |
00:09:03.920
unusual to outsiders.
link |
00:09:05.280
But when you're in that world,
link |
00:09:07.120
it doesn't feel unusual at all.
link |
00:09:09.600
And I think this is true of a lot of,
link |
00:09:11.680
you know, professional cultures, for instance.
link |
00:09:15.440
And, you know, so then your maybe slippery slope
link |
00:09:18.480
is too strong of a word.
link |
00:09:19.600
But, you know, you're in some world
link |
00:09:21.040
where you're mainly around other people
link |
00:09:23.200
with the same kind of viewpoints and training
link |
00:09:25.520
and worldview as you.
link |
00:09:27.360
And I think that's more of a source of,
link |
00:09:30.640
of, you know, kind of abuses of power
link |
00:09:34.560
than sort of, you know, there being good people
link |
00:09:36.720
and evil people, and that somehow the evil people
link |
00:09:40.640
are the ones that somehow rise to power.
link |
00:09:43.040
Oh, that's really interesting.
link |
00:09:44.160
So it's the, within the social norms
link |
00:09:46.880
constructed by that particular group of people,
link |
00:09:50.400
you're all trying to do good.
link |
00:09:52.720
But because as a group, you might be,
link |
00:09:54.640
you might drift into something
link |
00:09:56.080
that for the broader population,
link |
00:09:58.160
it does not align with the values of society.
link |
00:10:00.480
That kind of, that's the word.
link |
00:10:01.680
Yeah, I mean, or not that you drift,
link |
00:10:03.520
but even the things that don't make sense
link |
00:10:07.440
to the outside world don't seem unusual to you.
link |
00:10:11.280
So it's not sort of like a good or a bad thing,
link |
00:10:13.360
but, you know, like, so for instance,
link |
00:10:14.800
you know, on, in the world of finance, right?
link |
00:10:18.160
There's a lot of complicated types of activity
link |
00:10:21.280
that if you are not immersed in that world,
link |
00:10:22.960
you cannot see why the purpose of that,
link |
00:10:25.760
you know, that activity exists at all.
link |
00:10:27.440
It just seems like, you know, completely useless
link |
00:10:30.640
and people just like, you know, pushing money around.
link |
00:10:33.440
And when you're in that world, right,
link |
00:10:34.800
you're, and you learn more,
link |
00:10:36.640
your view does become more nuanced, right?
link |
00:10:39.600
You realize, okay, there is actually a function
link |
00:10:41.680
to this activity.
link |
00:10:43.840
And in some cases, you would conclude that actually,
link |
00:10:46.640
if magically we could eradicate this activity tomorrow,
link |
00:10:50.240
it would come back because it actually is like
link |
00:10:52.720
serving some useful purpose.
link |
00:10:54.400
It's just a useful purpose that's very difficult
link |
00:10:56.880
for outsiders to see.
link |
00:10:59.200
And so I think, you know, lots of professional work
link |
00:11:02.720
environments or cultures, as I might put it,
link |
00:11:06.320
kind of have these social norms that, you know,
link |
00:11:08.800
don't make sense to the outside world.
link |
00:11:10.160
Academia is the same, right?
link |
00:11:11.280
I mean, lots of people look at academia and say,
link |
00:11:13.600
you know, what the hell are all of you people doing?
link |
00:11:16.480
Why are you paid so much in some cases
link |
00:11:18.960
at taxpayer expenses to do, you know,
link |
00:11:21.840
to publish papers that nobody reads?
link |
00:11:24.400
You know, but when you're in that world,
link |
00:11:25.920
you come to see the value for it.
link |
00:11:27.680
And, but even though you might not be able to explain it
link |
00:11:30.240
to, you know, the person in the street.
link |
00:11:33.040
Right.
link |
00:11:33.360
And in the case of the financial sector,
link |
00:11:36.000
tools like credit might not make sense to people.
link |
00:11:39.200
Like, it's a good example of something that does seem
link |
00:11:41.600
to pop up and be useful or just the power of markets
link |
00:11:45.120
and just in general capitalism.
link |
00:11:47.120
Yeah.
link |
00:11:47.360
In finance, I think the primary example
link |
00:11:49.360
I would give is leverage, right?
link |
00:11:51.040
So being allowed to borrow, to sort of use ten times
link |
00:11:56.320
as much money as you've actually borrowed, right?
link |
00:11:58.480
So that's an example of something that before I had
link |
00:12:00.720
any experience in financial markets,
link |
00:12:02.320
I might have looked at and said,
link |
00:12:03.440
well, what is the purpose of that?
link |
00:12:05.280
That just seems very dangerous and it is dangerous
link |
00:12:08.400
and it has proven dangerous.
link |
00:12:10.480
But, you know, if the fact of the matter is that,
link |
00:12:13.280
you know, sort of on some particular time scale,
link |
00:12:16.560
you are holding positions that are,
link |
00:12:19.680
you know, very unlikely to, you know,
link |
00:12:23.040
lose, you know, your value at risk or variance
link |
00:12:26.800
is like one or five percent, then it kind of makes sense
link |
00:12:30.320
that you would be allowed to use a little bit more
link |
00:12:32.160
than you have because you have, you know,
link |
00:12:35.120
some confidence that you're not going to lose
link |
00:12:37.760
it all in a single day.
link |
00:12:39.840
Now, of course, when that happens,
link |
00:12:42.960
we've seen what happens, you know, not too long ago.
link |
00:12:45.920
But, you know, but the idea that it serves
link |
00:12:48.800
no useful economic purpose under any circumstances
link |
00:12:52.800
is definitely not true.
link |
00:12:54.800
We'll return to the other side of the coast,
link |
00:12:57.680
Silicon Valley, and the problems there as we talk about privacy,
link |
00:13:02.560
as we talk about fairness.
link |
00:13:05.360
At the high level, and I'll ask some sort of basic questions
link |
00:13:09.360
with the hope to get at the fundamental nature of reality.
link |
00:13:12.560
But from a very high level, what is an ethical algorithm?
link |
00:13:18.160
So I can say that an algorithm has a running time
link |
00:13:20.960
of using big O notation n log n.
link |
00:13:24.400
I can say that a machine learning algorithm
link |
00:13:27.760
classified cat versus dog with 97 percent accuracy.
link |
00:13:31.440
Do you think there will one day be a way to measure
link |
00:13:36.320
sort of in the same compelling way as the big O notation
link |
00:13:39.920
of this algorithm is 97 percent ethical?
link |
00:13:44.000
First of all, let me riff for a second on your specific n log n example.
link |
00:13:48.800
So because early in the book when we're just kind of trying to describe
link |
00:13:51.920
algorithms period, we say like, okay, you know,
link |
00:13:54.640
what's an example of an algorithm or an algorithmic problem?
link |
00:13:58.560
First of all, like it's sorting, right?
link |
00:14:00.160
You have a bunch of index cards with numbers on them
link |
00:14:02.240
and you want to sort them.
link |
00:14:03.760
And we describe, you know, an algorithm that sweeps all the way through,
link |
00:14:07.360
finds the smallest number, puts it at the front,
link |
00:14:09.680
then sweeps through again, finds the second smallest number.
link |
00:14:12.640
So we make the point that this is an algorithm
link |
00:14:14.800
and it's also a bad algorithm in the sense that, you know,
link |
00:14:17.760
it's quadratic rather than n log n,
link |
00:14:20.640
which we know is kind of optimal for sorting.
link |
00:14:23.920
And we make the point that sort of like, you know,
link |
00:14:26.080
so even within the confines of a very precisely specified problem,
link |
00:14:31.680
there, you know, there might be many, many different algorithms
link |
00:14:35.200
for the same problem with different properties.
link |
00:14:37.520
Like some might be faster in terms of running time,
link |
00:14:40.400
some might use less memory, some might have, you know,
link |
00:14:43.520
better distributed implementations.
link |
00:14:46.240
And so the point is that already we're used to, you know,
link |
00:14:50.560
in computer science thinking about trade offs
link |
00:14:53.520
between different types of quantities and resources
link |
00:14:56.800
and there being, you know, better and worse algorithms.
link |
00:15:00.960
And our book is about that part of algorithmic ethics
link |
00:15:08.480
that we know how to kind of put on that same kind of quantitative footing right now.
link |
00:15:13.520
So, you know, just to say something that our book is not about,
link |
00:15:17.440
our book is not about kind of broad, fuzzy notions of fairness.
link |
00:15:22.400
It's about very specific notions of fairness.
link |
00:15:25.840
There's more than one of them.
link |
00:15:28.240
There are tensions between them, right?
link |
00:15:30.880
But if you pick one of them, you can do something akin to saying
link |
00:15:35.680
that this algorithm is 97% ethical.
link |
00:15:39.200
You can say, for instance, the, you know, for this lending model,
link |
00:15:44.080
the false rejection rate on black people and white people is within 3%, right?
link |
00:15:51.040
So we might call that a 97% ethical algorithm and a 100% ethical algorithm
link |
00:15:57.040
would mean that that difference is 0%.
link |
00:15:59.920
In that case, fairness is specified when two groups, however,
link |
00:16:04.640
they're defined are given to you.
link |
00:16:06.720
That's right.
link |
00:16:07.280
So the, and then you can sort of mathematically start describing the algorithm.
link |
00:16:11.760
But nevertheless, the part where the two groups are given to you,
link |
00:16:20.080
I mean, unlike running time, you know, we don't in computer science
link |
00:16:24.480
talk about how fast an algorithm feels like when it runs.
link |
00:16:29.200
True.
link |
00:16:29.760
We measure it and ethical starts getting into feelings.
link |
00:16:33.040
So, for example, an algorithm runs, you know, if it runs in the background,
link |
00:16:38.160
it doesn't disturb the performance of my system.
link |
00:16:40.480
It'll feel nice.
link |
00:16:41.600
I'll be okay with it.
link |
00:16:42.560
But if it overloads the system, it'll feel unpleasant.
link |
00:16:45.280
So in that same way, ethics, there's a feeling of how socially acceptable it is.
link |
00:16:50.320
How does it represent the moral standards of our society today?
link |
00:16:55.200
So in that sense, and sorry to linger on that first of high,
link |
00:16:59.040
low philosophical questions.
link |
00:17:00.640
Do you have a sense we'll be able to measure how ethical an algorithm is?
link |
00:17:05.920
First of all, I didn't, certainly didn't mean to give the impression that you can kind of
link |
00:17:09.680
measure, you know, memory speed trade offs, you know, and that there's a complete mapping from
link |
00:17:16.320
that onto kind of fairness, for instance, or ethics and accuracy, for example.
link |
00:17:22.880
In the type of fairness definitions that are largely the objects of study today and starting
link |
00:17:28.960
to be deployed, you as the user of the definitions, you need to make some hard decisions before you
link |
00:17:35.360
even get to the point of designing fair algorithms.
link |
00:17:40.240
One of them, for instance, is deciding who it is that you're worried about protecting,
link |
00:17:45.840
who you're worried about being harmed by, for instance, some notion of discrimination or
link |
00:17:50.560
unfairness.
link |
00:17:52.160
And then you need to also decide what constitutes harm.
link |
00:17:55.520
So, for instance, in a lending application, maybe you decide that, you know, falsely rejecting
link |
00:18:02.320
a creditworthy individual, you know, sort of a false negative, is the real harm and that false
link |
00:18:08.960
positives, i.e. people that are not creditworthy or are not gonna repay your loan, that get a loan,
link |
00:18:14.560
you might think of them as lucky.
link |
00:18:17.120
And so that's not a harm, although it's not clear that if you don't have the means to repay a loan,
link |
00:18:22.720
that being given a loan is not also a harm.
link |
00:18:26.880
So, you know, the literature is sort of so far quite limited in that you sort of need to say,
link |
00:18:33.600
who do you want to protect and what would constitute harm to that group?
link |
00:18:37.920
And when you ask questions like, will algorithms feel ethical?
link |
00:18:42.080
One way in which they won't, under the definitions that I'm describing, is if, you know, if you are
link |
00:18:47.440
an individual who is falsely denied a loan, incorrectly denied a loan, all of these definitions
link |
00:18:54.320
basically say like, well, you know, your compensation is the knowledge that we are also
link |
00:19:00.240
falsely denying loans to other people, you know, in other groups at the same rate that we're doing
link |
00:19:05.120
it to you.
link |
00:19:05.680
And, you know, and so there is actually this interesting even technical tension in the field
link |
00:19:12.800
right now between these sort of group notions of fairness and notions of fairness that might
link |
00:19:18.400
actually feel like real fairness to individuals, right?
link |
00:19:22.160
They might really feel like their particular interests are being protected or thought about
link |
00:19:27.360
by the algorithm rather than just, you know, the groups that they happen to be members of.
link |
00:19:33.360
Is there parallels to the big O notation of worst case analysis?
link |
00:19:37.920
So, is it important to looking at the worst violation of fairness for an individual?
link |
00:19:45.760
Is it important to minimize that one individual?
link |
00:19:48.080
So like worst case analysis, is that something you think about or?
link |
00:19:52.320
I mean, I think we're not even at the point where we can sensibly think about that.
link |
00:19:56.960
So first of all, you know, we're talking here both about fairness applied at the group level,
link |
00:20:03.280
which is a relatively weak thing, but it's better than nothing.
link |
00:20:08.000
And also the more ambitious thing of trying to give some individual promises, but even
link |
00:20:14.960
that doesn't incorporate, I think something that you're hinting at here is what I might
link |
00:20:18.640
call subjective fairness, right?
link |
00:20:20.720
So a lot of the definitions, I mean, all of the definitions in the algorithmic fairness
link |
00:20:25.200
literature are what I would kind of call received wisdom definitions.
link |
00:20:28.400
It's sort of, you know, somebody like me sits around and things like, okay, you know, I
link |
00:20:33.440
think here's a technical definition of fairness that I think people should want or that they
link |
00:20:37.840
should, you know, think of as some notion of fairness, maybe not the only one, maybe
link |
00:20:41.840
not the best one, maybe not the last one.
link |
00:20:44.320
But we really actually don't know from a subjective standpoint, like what people really
link |
00:20:52.480
think is fair.
link |
00:20:53.360
You know, we just started doing a little bit of work in our group at actually doing kind
link |
00:21:01.120
of human subject experiments in which we, you know, ask people about, you know, we ask
link |
00:21:09.120
them questions about fairness, we survey them, we, you know, we show them pairs of individuals
link |
00:21:15.120
in, let's say, a criminal recidivism prediction setting, and we ask them, do you think these
link |
00:21:20.320
two individuals should be treated the same as a matter of fairness?
link |
00:21:24.320
And to my knowledge, there's not a large literature in which ordinary people are asked
link |
00:21:31.760
about, you know, they have sort of notions of their subjective fairness elicited from
link |
00:21:37.040
them.
link |
00:21:38.160
It's mainly, you know, kind of scholars who think about fairness kind of making up their
link |
00:21:43.840
own definitions.
link |
00:21:44.400
And I think this needs to change actually for many social norms, not just for fairness,
link |
00:21:50.320
right?
link |
00:21:50.560
So there's a lot of discussion these days in the AI community about interpretable AI
link |
00:21:56.560
or understandable AI.
link |
00:21:58.560
And as far as I can tell, everybody agrees that deep learning or at least the outputs
link |
00:22:04.880
of deep learning are not very understandable, and people might agree that sparse linear
link |
00:22:11.840
models with integer coefficients are more understandable.
link |
00:22:15.520
But nobody's really asked people.
link |
00:22:17.440
You know, there's very little literature on, you know, sort of showing people models
link |
00:22:21.280
and asking them, do they understand what the model is doing?
link |
00:22:25.280
And I think that in all these topics, as these fields mature, we need to start doing more
link |
00:22:32.560
behavioral work.
link |
00:22:34.400
Yeah, which is one of my deep passions is psychology.
link |
00:22:38.160
And I always thought computer scientists will be the best future psychologists in a sense
link |
00:22:44.480
that data is, especially in this modern world, the data is a really powerful way to understand
link |
00:22:51.680
and study human behavior.
link |
00:22:53.360
And you've explored that with your game theory side of work as well.
link |
00:22:56.720
Yeah, I'd like to think that what you say is true about computer scientists and psychology
link |
00:23:02.240
from my own limited wandering into human subject experiments.
link |
00:23:07.520
We have a great deal to learn, not just computer science, but AI and machine learning more
link |
00:23:11.600
specifically, I kind of think of as imperialist research communities in that, you know, kind
link |
00:23:17.040
of like physicists in an earlier generation, computer scientists kind of don't think of
link |
00:23:22.800
any scientific topic that's off limits to them.
link |
00:23:25.440
They will like freely wander into areas that others have been thinking about for decades
link |
00:23:30.880
or longer.
link |
00:23:31.440
And, you know, we usually tend to embarrass ourselves in those efforts for some amount
link |
00:23:37.840
of time.
link |
00:23:38.320
Like, you know, I think reinforcement learning is a good example, right?
link |
00:23:41.840
So a lot of the early work in reinforcement learning, I have complete sympathy for the
link |
00:23:48.160
control theorists that looked at this and said like, okay, you are reinventing stuff
link |
00:23:53.120
that we've known since like the forties, right?
link |
00:23:55.600
But, you know, in my view, eventually this sort of, you know, computer scientists have
link |
00:24:01.120
made significant contributions to that field, even though we kind of embarrassed ourselves
link |
00:24:06.320
for the first decade.
link |
00:24:07.520
So I think if computer scientists are gonna start engaging in kind of psychology, human
link |
00:24:12.080
subjects type of research, we should expect to be embarrassing ourselves for a good 10
link |
00:24:18.080
years or so, and then hope that it turns out as well as, you know, some other areas that
link |
00:24:23.600
we've waded into.
link |
00:24:25.600
So you kind of mentioned this, just to linger on the idea of an ethical algorithm, of idea
link |
00:24:30.400
of groups, sort of group thinking and individual thinking.
link |
00:24:33.760
And we're struggling that.
link |
00:24:35.040
One of the amazing things about algorithms and your book and just this field of study
link |
00:24:39.280
is it gets us to ask, like forcing machines, converting these ideas into algorithms is
link |
00:24:46.640
forcing us to ask questions of ourselves as a human civilization.
link |
00:24:50.160
So there's a lot of people now in public discourse doing sort of group thinking, thinking like
link |
00:24:58.320
there's particular sets of groups that we don't wanna discriminate against and so on.
link |
00:25:02.000
And then there is individuals, sort of in the individual life stories, the struggles
link |
00:25:08.560
they went through and so on.
link |
00:25:10.000
Now, like in philosophy, it's easier to do group thinking because you don't, it's very
link |
00:25:16.480
hard to think about individuals.
link |
00:25:17.920
There's so much variability, but with data, you can start to actually say, you know what
link |
00:25:23.840
group thinking is too crude.
link |
00:25:26.400
You're actually doing more discrimination by thinking in terms of groups and individuals.
link |
00:25:30.880
Can you linger on that kind of idea of group versus individual and ethics?
link |
00:25:36.720
And is it good to continue thinking in terms of groups in algorithms?
link |
00:25:41.680
So let me start by answering a very good high level question with a slightly narrow technical
link |
00:25:49.360
response, which is these group definitions of fairness, like here's a few groups, like
link |
00:25:54.480
different racial groups, maybe gender groups, maybe age, what have you.
link |
00:25:59.440
And let's make sure that for none of these groups, do we have a false negative rate,
link |
00:26:06.480
which is much higher than any other one of these groups.
link |
00:26:09.200
Okay, so these are kind of classic group aggregate notions of fairness.
link |
00:26:13.760
And you know, but at the end of the day, an individual you can think of as a combination
link |
00:26:18.000
of all of their attributes, right?
link |
00:26:19.360
They're a member of a racial group, they have a gender, they have an age, and many other
link |
00:26:26.800
demographic properties that are not biological, but that are still very strong determinants
link |
00:26:33.840
of outcome and personality and the like.
link |
00:26:36.720
So one, I think, useful spectrum is to sort of think about that array between the group
link |
00:26:43.920
and the specific individual, and to realize that in some ways, asking for fairness at
link |
00:26:49.600
the individual level is to sort of ask for group fairness simultaneously for all possible
link |
00:26:56.800
combinations of groups.
link |
00:26:57.840
So in particular, you know, if I build a predictive model that meets some definition of fairness,
link |
00:27:06.480
definition of fairness by race, by gender, by age, by what have you, marginally, to get
link |
00:27:14.160
it slightly technical, sort of independently, I shouldn't expect that model to not discriminate
link |
00:27:20.960
against disabled Hispanic women over age 55, making less than $50,000 a year annually,
link |
00:27:27.440
even though I might have protected each one of those attributes marginally.
link |
00:27:32.480
So the optimization, actually, that's a fascinating way to put it.
link |
00:27:35.680
So you're just optimizing, the one way to achieve the optimizing fairness for individuals
link |
00:27:42.160
is just to add more and more definitions of groups that each individual belongs to.
link |
00:27:46.080
That's right.
link |
00:27:47.080
So, you know, at the end of the day, we could think of all of ourselves as groups of size
link |
00:27:50.320
one because eventually there's some attribute that separates you from me and everybody else
link |
00:27:55.400
in the world, okay?
link |
00:27:57.020
And so it is possible to put, you know, these incredibly coarse ways of thinking about fairness
link |
00:28:03.560
and these very, very individualistic specific ways on a common scale.
link |
00:28:09.960
And you know, one of the things we've worked on from a research perspective is, you know,
link |
00:28:14.160
so we sort of know how to, you know, in relative terms, we know how to provide fairness guarantees
link |
00:28:20.520
at the core system of the scale.
link |
00:28:22.760
We don't know how to provide kind of sensible, tractable, realistic fairness guarantees at
link |
00:28:28.240
the individual level, but maybe we could start creeping towards that by dealing with more
link |
00:28:33.120
refined subgroups.
link |
00:28:35.040
I mean, we gave a name to this phenomenon where, you know, you protect, you enforce
link |
00:28:41.000
some definition of fairness for a bunch of marginal attributes or features, but then
link |
00:28:46.580
you find yourself discriminating against a combination of them.
link |
00:28:49.980
We call that fairness gerrymandering because like political gerrymandering, you know, you're
link |
00:28:55.480
giving some guarantee at the aggregate level, but when you kind of look in a more granular
link |
00:29:01.400
way at what's going on, you realize that you're achieving that aggregate guarantee by sort
link |
00:29:06.440
of favoring some groups and discriminating against other ones.
link |
00:29:10.880
And so there are, you know, it's early days, but there are algorithmic approaches that
link |
00:29:15.940
let you start creeping towards that, you know, individual end of the spectrum.
link |
00:29:22.440
Does there need to be human input in the form of weighing the value of the importance of
link |
00:29:30.740
each kind of group?
link |
00:29:33.000
So for example, is it like, so gender, say crudely speaking, male and female, and then
link |
00:29:42.400
different races, are we as humans supposed to put value on saying gender is 0.6 and race
link |
00:29:51.980
is 0.4 in terms of in the big optimization of achieving fairness?
link |
00:29:59.200
Is that kind of what humans are supposed to do here?
link |
00:30:01.720
I mean, of course, you know, I don't need to tell you that, of course, technically one
link |
00:30:05.320
could incorporate such weights if you wanted to into a definition of fairness.
link |
00:30:10.720
You know, fairness is an interesting topic in that having worked in the book being about
link |
00:30:19.680
both fairness, privacy, and many other social norms, fairness, of course, is a much, much
link |
00:30:24.820
more loaded topic.
link |
00:30:27.160
So privacy, I mean, people want privacy, people don't like violations of privacy, violations
link |
00:30:32.180
of privacy cause damage, angst, and bad publicity for the companies that are victims of them.
link |
00:30:40.680
But sort of everybody agrees more data privacy would be better than less data privacy.
link |
00:30:48.020
And you don't have these, somehow the discussions of fairness don't become politicized along
link |
00:30:53.780
other dimensions like race and about gender and, you know, whether we, and, you know,
link |
00:31:01.900
you quickly find yourselves kind of revisiting topics that have been kind of unresolved forever,
link |
00:31:10.760
like affirmative action, right?
link |
00:31:12.560
Sort of, you know, like, why are you protecting, and some people will say, why are you protecting
link |
00:31:16.400
this particular racial group?
link |
00:31:20.320
And others will say, well, we need to do that as a matter of retribution.
link |
00:31:26.240
Other people will say, it's a matter of economic opportunity.
link |
00:31:30.040
And I don't know which of, you know, whether any of these are the right answers, but you
link |
00:31:34.920
sort of, fairness is sort of special in that as soon as you start talking about it, you
link |
00:31:39.840
inevitably have to participate in debates about fair to whom, at what expense to who
link |
00:31:46.360
else.
link |
00:31:47.360
I mean, even in criminal justice, right, you know, where people talk about fairness in
link |
00:31:56.180
criminal sentencing or, you know, predicting failures to appear or making parole decisions
link |
00:32:02.840
or the like, they will, you know, they'll point out that, well, these definitions of
link |
00:32:08.340
fairness are all about fairness for the criminals.
link |
00:32:13.640
And what about fairness for the victims, right?
link |
00:32:16.120
So when I basically say something like, well, the false incarceration rate for black people
link |
00:32:22.840
and white people needs to be roughly the same, you know, there's no mention of potential
link |
00:32:28.300
victims of criminals in such a fairness definition.
link |
00:32:33.180
And that's the realm of public discourse.
link |
00:32:34.960
I should actually recommend, I just listened to people listening, Intelligence Squares
link |
00:32:41.200
debates, US edition just had a debate.
link |
00:32:45.080
They have this structure where you have old Oxford style or whatever they're called, debates,
link |
00:32:50.080
you know, it's two versus two and they talked about affirmative action and it was incredibly
link |
00:32:55.680
interesting that there's really good points on every side of this issue, which is fascinating
link |
00:33:03.000
to listen to.
link |
00:33:04.000
Yeah, yeah, I agree.
link |
00:33:05.680
And so it's interesting to be a researcher trying to do, for the most part, technical
link |
00:33:12.400
algorithmic work, but Aaron and I both quickly learned you cannot do that and then go out
link |
00:33:17.980
and talk about it and expect people to take it seriously if you're unwilling to engage
link |
00:33:22.640
in these broader debates that are entirely extra algorithmic, right?
link |
00:33:28.160
They're not about, you know, algorithms and making algorithms better.
link |
00:33:31.200
They're sort of, you know, as you said, sort of like, what should society be protecting
link |
00:33:35.160
in the first place?
link |
00:33:36.160
When you discuss the fairness, an algorithm that achieves fairness, whether in the constraints
link |
00:33:42.320
and the objective function, there's an immediate kind of analysis you can perform, which is
link |
00:33:48.520
saying, if you care about fairness in gender, this is the amount that you have to pay for
link |
00:33:56.520
it in terms of the performance of the system.
link |
00:33:59.280
Like do you, is there a role for statements like that in a table, in a paper, or do you
link |
00:34:03.960
want to really not touch that?
link |
00:34:06.680
No, no, we want to touch that and we do touch it.
link |
00:34:09.800
So I mean, just again, to make sure I'm not promising your viewers more than we know how
link |
00:34:16.680
to provide, but if you pick a definition of fairness, like I'm worried about gender discrimination
link |
00:34:21.760
and you pick a notion of harm, like false rejection for a loan, for example, and you
link |
00:34:27.100
give me a model, I can definitely, first of all, go audit that model.
link |
00:34:30.960
It's easy for me to go, you know, from data to kind of say like, okay, your false rejection
link |
00:34:36.640
rate on women is this much higher than it is on men, okay?
link |
00:34:41.880
But once you also put the fairness into your objective function, I mean, I think the table
link |
00:34:47.240
that you're talking about is what we would call the Pareto curve, right?
link |
00:34:51.640
You can literally trace out, and we give examples of such plots on real data sets in the book,
link |
00:34:58.740
you have two axes.
link |
00:34:59.760
On the X axis is your error, on the Y axis is unfairness by whatever, you know, if it's
link |
00:35:06.360
like the disparity between false rejection rates between two groups.
link |
00:35:12.240
And you know, your algorithm now has a knob that basically says, how strongly do I want
link |
00:35:17.080
to enforce fairness?
link |
00:35:19.400
And the less unfair, you know, if the two axes are error and unfairness, we'd like to
link |
00:35:24.680
be at zero, zero.
link |
00:35:26.260
We'd like zero error and zero unfairness simultaneously.
link |
00:35:31.280
Anybody who works in machine learning knows that you're generally not going to get to
link |
00:35:34.840
zero error period without any fairness constraint whatsoever.
link |
00:35:38.840
So that's not going to happen.
link |
00:35:41.060
But in general, you know, you'll get this, you'll get some kind of convex curve that
link |
00:35:46.480
specifies the numerical trade off you face.
link |
00:35:49.960
You know, if I want to go from 17% error down to 16% error, what will be the increase in
link |
00:35:57.920
unfairness that I experienced as a result of that?
link |
00:36:02.960
And so this curve kind of specifies the, you know, kind of undominated models.
link |
00:36:09.520
Models that are off that curve are, you know, can be strictly improved in one or both dimensions.
link |
00:36:14.480
You can, you know, either make the error better or the unfairness better or both.
link |
00:36:18.840
And I think our view is that not only are these objects, these Pareto curves, you know,
link |
00:36:26.000
with efficient frontiers as you might call them, not only are they valuable scientific
link |
00:36:34.360
objects, I actually think that they in the near term might need to be the interface between
link |
00:36:41.320
researchers working in the field and stakeholders in given problems.
link |
00:36:46.180
So you know, you could really imagine telling a criminal jurisdiction, look, if you're concerned
link |
00:36:55.320
about racial fairness, but you're also concerned about accuracy.
link |
00:36:58.820
You want to, you know, you want to release on parole people that are not going to recommit
link |
00:37:05.200
a violent crime and you don't want to release the ones who are.
link |
00:37:08.600
So you know, that's accuracy.
link |
00:37:10.600
But if you also care about those, you know, the mistakes you make not being disproportionately
link |
00:37:15.120
on one racial group or another, you can show this curve.
link |
00:37:19.160
I'm hoping that in the near future, it'll be possible to explain these curves to non
link |
00:37:23.980
technical people that are the ones that have to make the decision, where do we want to
link |
00:37:29.520
be on this curve?
link |
00:37:30.520
Like, what are the relative merits or value of having lower error versus lower unfairness?
link |
00:37:38.440
You know, that's not something computer scientists should be deciding for society, right?
link |
00:37:43.560
That, you know, the people in the field, so to speak, the policymakers, the regulators,
link |
00:37:49.400
that's who should be making these decisions.
link |
00:37:51.680
But I think and hope that they can be made to understand that these trade offs generally
link |
00:37:56.600
exist and that you need to pick a point and like, and ignoring the trade off, you know,
link |
00:38:03.280
you're implicitly picking a point anyway, right?
link |
00:38:06.760
You just don't know it and you're not admitting it.
link |
00:38:09.400
Just to linger on the point of trade offs, I think that's a really important thing to
link |
00:38:12.740
sort of think about.
link |
00:38:15.400
So you think when we start to optimize for fairness, there's almost always in most system
link |
00:38:22.360
going to be trade offs.
link |
00:38:25.080
Can you like, what's the trade off between just to clarify, there have been some sort
link |
00:38:30.200
of technical terms thrown around, but sort of a perfectly fair world.
link |
00:38:39.240
Why is that?
link |
00:38:40.760
Why will somebody be upset about that?
link |
00:38:43.760
The specific trade off I talked about just in order to make things very concrete was
link |
00:38:47.400
between numerical error and some numerical measure of unfairness.
link |
00:38:53.360
What is numerical error in the case of...
link |
00:38:56.400
Just like say predictive error, like, you know, the probability or frequency with which
link |
00:39:01.000
you release somebody on parole who then goes on to recommit a violent crime or keep incarcerated
link |
00:39:08.480
somebody who would not have recommitted a violent crime.
link |
00:39:10.920
So in the case of awarding somebody parole or giving somebody parole or letting them
link |
00:39:17.480
out on parole, you don't want them to recommit a crime.
link |
00:39:21.480
So it's your system failed in prediction if they happen to do a crime.
link |
00:39:26.600
Okay, so that's one axis.
link |
00:39:30.280
And what's the fairness axis?
link |
00:39:31.800
So then the fairness axis might be the difference between racial groups in the kind of false
link |
00:39:39.640
positive predictions, namely people that I kept incarcerated predicting that they would
link |
00:39:47.840
recommit a violent crime when in fact they wouldn't have.
link |
00:39:51.200
Right.
link |
00:39:52.200
And the unfairness of that, just to linger it and allow me to in eloquently to try to
link |
00:40:00.840
sort of describe why that's unfair, why unfairness is there.
link |
00:40:06.360
The unfairness you want to get rid of is that in the judge's mind, the bias of having being
link |
00:40:13.280
brought up to society, the slight racial bias, the racism that exists in the society, you
link |
00:40:18.480
want to remove that from the system.
link |
00:40:21.760
Another way that's been debated is sort of equality of opportunity versus equality of
link |
00:40:28.720
outcome.
link |
00:40:30.440
And there's a weird dance there that's really difficult to get right.
link |
00:40:35.120
And we don't, affirmative action is exploring that space.
link |
00:40:40.200
Right.
link |
00:40:41.200
And then this also quickly bleeds into questions like, well, maybe if one group really does
link |
00:40:48.840
recommit crimes at a higher rate, the reason for that is that at some earlier point in
link |
00:40:55.240
the pipeline or earlier in their lives, they didn't receive the same resources that the
link |
00:41:00.200
other group did.
link |
00:41:02.560
And so there's always in kind of fairness discussions, the possibility that the real
link |
00:41:08.480
injustice came earlier, right?
link |
00:41:11.040
Earlier in this individual's life, earlier in this group's history, et cetera, et cetera.
link |
00:41:16.360
And so a lot of the fairness discussion is almost, the goal is for it to be a corrective
link |
00:41:20.840
mechanism to account for the injustice earlier in life.
link |
00:41:25.440
By some definitions of fairness or some theories of fairness, yeah.
link |
00:41:29.640
Others would say like, look, it's not to correct that injustice, it's just to kind of level
link |
00:41:35.120
the playing field right now and not falsely incarcerate more people of one group than
link |
00:41:40.720
another group.
link |
00:41:41.720
But I mean, I think just it might be helpful just to demystify a little bit about the many
link |
00:41:46.960
ways in which bias or unfairness can come into algorithms, especially in the machine
link |
00:41:54.940
learning era, right?
link |
00:41:55.940
I think many of your viewers have probably heard these examples before, but let's say
link |
00:42:00.680
I'm building a face recognition system, right?
link |
00:42:04.160
And so I'm kind of gathering lots of images of faces and trying to train the system to
link |
00:42:12.000
recognize new faces of those individuals from training on a training set of those faces
link |
00:42:17.340
of individuals.
link |
00:42:19.080
And it shouldn't surprise anybody or certainly not anybody in the field of machine learning
link |
00:42:24.860
if my training data set was primarily white males and I'm training the model to maximize
link |
00:42:34.960
the overall accuracy on my training data set, that the model can reduce its error most by
link |
00:42:44.060
getting things right on the white males that constitute the majority of the data set, even
link |
00:42:48.800
if that means that on other groups, they will be less accurate, okay?
link |
00:42:53.640
Now, there's a bunch of ways you could think about addressing this.
link |
00:42:57.720
One is to deliberately put into the objective of the algorithm not to optimize the error
link |
00:43:05.760
at the expense of this discrimination, and then you're kind of back in the land of these
link |
00:43:09.060
kind of two dimensional numerical trade offs.
link |
00:43:13.140
A valid counter argument is to say like, well, no, you don't have to, there's no, you know,
link |
00:43:18.660
the notion of the tension between error and accuracy here is a false one.
link |
00:43:22.840
You could instead just go out and get much more data on these other groups that are in
link |
00:43:27.760
the minority and, you know, equalize your data set, or you could train a separate model
link |
00:43:34.580
on those subgroups and, you know, have multiple models.
link |
00:43:38.800
The point I think we would, you know, we tried to make in the book is that those things have
link |
00:43:43.120
cost too, right?
link |
00:43:45.160
Going out and gathering more data on groups that are relatively rare compared to your
link |
00:43:51.200
plurality or more majority group that, you know, it may not cost you in the accuracy
link |
00:43:55.520
of the model, but it's going to cost, you know, it's going to cost the company developing
link |
00:43:59.460
this model more money to develop that, and it also costs more money to build separate
link |
00:44:04.460
predictive models and to implement and deploy them.
link |
00:44:07.500
So even if you can find a way to avoid the tension between error and accuracy in training
link |
00:44:14.100
a model, you might push the cost somewhere else, like money, like development time, research
link |
00:44:20.720
time and the like.
link |
00:44:22.920
There are fundamentally difficult philosophical questions, in fairness, and we live in a very
link |
00:44:30.200
divisive political climate, outraged culture.
link |
00:44:34.160
There is alt right folks on 4chan, trolls.
link |
00:44:38.560
There is social justice warriors on Twitter.
link |
00:44:43.320
There's very divisive, outraged folks on all sides of every kind of system.
link |
00:44:49.920
How do you, how do we as engineers build ethical algorithms in such divisive culture?
link |
00:44:57.280
Do you think they could be disjoint?
link |
00:44:59.540
The human has to inject your values, and then you can optimize over those values.
link |
00:45:04.700
But in our times, when you start actually applying these systems, things get a little
link |
00:45:09.560
bit challenging for the public discourse.
link |
00:45:13.100
How do you think we can proceed?
link |
00:45:14.920
Yeah, I mean, for the most part in the book, a point that we try to take some pains to
link |
00:45:21.000
make is that we don't view ourselves or people like us as being in the position of deciding
link |
00:45:29.560
for society what the right social norms are, what the right definitions of fairness are.
link |
00:45:34.960
Our main point is to just show that if society or the relevant stakeholders in a particular
link |
00:45:41.660
domain can come to agreement on those sorts of things, there's a way of encoding that
link |
00:45:47.160
into algorithms in many cases, not in all cases.
link |
00:45:50.720
One other misconception that hopefully we definitely dispel is sometimes people read
link |
00:45:55.640
the title of the book and I think not unnaturally fear that what we're suggesting is that the
link |
00:46:00.880
algorithms themselves should decide what those social norms are and develop their own notions
link |
00:46:05.760
of fairness and privacy or ethics, and we're definitely not suggesting that.
link |
00:46:10.160
The title of the book is Ethical Algorithm, by the way, and I didn't think of that interpretation
link |
00:46:13.920
of the title.
link |
00:46:14.920
That's interesting.
link |
00:46:15.920
Yeah, yeah.
link |
00:46:16.920
I mean, especially these days where people are concerned about the robots becoming our
link |
00:46:21.080
overlords, the idea that the robots would also sort of develop their own social norms
link |
00:46:25.980
is just one step away from that.
link |
00:46:29.360
But I do think, obviously, despite disclaimer that people like us shouldn't be making those
link |
00:46:35.240
decisions for society, we are kind of living in a world where in many ways computer scientists
link |
00:46:40.880
have made some decisions that have fundamentally changed the nature of our society and democracy
link |
00:46:46.820
and sort of civil discourse and deliberation in ways that I think most people generally
link |
00:46:53.240
feel are bad these days, right?
link |
00:46:55.720
But they had to make, so if we look at people at the heads of companies and so on, they
link |
00:47:01.120
had to make those decisions, right?
link |
00:47:02.800
There has to be decisions, so there's two options, either you kind of put your head
link |
00:47:08.440
in the sand and don't think about these things and just let the algorithm do what it does,
link |
00:47:14.000
or you make decisions about what you value, you know, of injecting moral values into the
link |
00:47:19.320
algorithm.
link |
00:47:20.320
Look, I never mean to be an apologist for the tech industry, but I think it's a little
link |
00:47:26.760
bit too far to sort of say that explicit decisions were made about these things.
link |
00:47:31.120
So let's, for instance, take social media platforms, right?
link |
00:47:34.920
So like many inventions in technology and computer science, a lot of these platforms
link |
00:47:40.160
that we now use regularly kind of started as curiosities, right?
link |
00:47:45.120
I remember when things like Facebook came out and its predecessors like Friendster,
link |
00:47:49.240
which nobody even remembers now, people really wonder, like, why would anybody want to spend
link |
00:47:55.620
time doing that?
link |
00:47:56.620
I mean, even the web when it first came out, when it wasn't populated with much content
link |
00:48:01.480
and it was largely kind of hobbyists building their own kind of ramshackle websites, a lot
link |
00:48:07.100
of people looked at this and said, well, what is the purpose of this thing?
link |
00:48:09.960
Why is this interesting?
link |
00:48:11.000
Who would want to do this?
link |
00:48:12.880
And so even things like Facebook and Twitter, yes, technical decisions were made by engineers,
link |
00:48:18.120
by scientists, by executives in the design of those platforms, but, you know, I don't
link |
00:48:23.520
think 10 years ago anyone anticipated that those platforms, for instance, might kind
link |
00:48:32.240
of acquire undue, you know, influence on political discourse or on the outcomes of elections.
link |
00:48:42.200
And I think the scrutiny that these companies are getting now is entirely appropriate, but
link |
00:48:47.600
I think it's a little too harsh to kind of look at history and sort of say like, oh,
link |
00:48:53.080
you should have been able to anticipate that this would happen with your platform.
link |
00:48:56.320
And in this sort of gaming chapter of the book, one of the points we're making is that,
link |
00:49:00.600
you know, these platforms, right, they don't operate in isolation.
link |
00:49:05.200
So unlike the other topics we're discussing, like fairness and privacy, like those are
link |
00:49:09.360
really cases where algorithms can operate on your data and make decisions about you
link |
00:49:13.600
and you're not even aware of it, okay?
link |
00:49:16.300
Things like Facebook and Twitter, these are, you know, these are systems, right?
link |
00:49:20.280
These are social systems and their evolution, even their technical evolution because machine
link |
00:49:25.960
learning is involved, is driven in no small part by the behavior of the users themselves
link |
00:49:31.680
and how the users decide to adopt them and how to use them.
link |
00:49:35.680
And so, you know, I'm kind of like who really knew that, you know, until we saw it happen,
link |
00:49:44.600
who knew that these things might be able to influence the outcome of elections?
link |
00:49:48.340
Who knew that, you know, they might polarize political discourse because of the ability
link |
00:49:55.120
to, you know, decide who you interact with on the platform and also with the platform
link |
00:50:00.840
naturally using machine learning to optimize for your own interest that they would further
link |
00:50:05.080
isolate us from each other and, you know, like feed us all basically just the stuff
link |
00:50:10.080
that we already agreed with.
link |
00:50:12.080
So I think, you know, we've come to that outcome, I think, largely, but I think it's
link |
00:50:18.120
something that we all learned together, including the companies as these things happen.
link |
00:50:24.240
You asked like, well, are there algorithmic remedies to these kinds of things?
link |
00:50:29.940
And again, these are big problems that are not going to be solved with, you know, somebody
link |
00:50:35.360
going in and changing a few lines of code somewhere in a social media platform.
link |
00:50:40.040
But I do think in many ways, there are definitely ways of making things better.
link |
00:50:44.960
I mean, like an obvious recommendation that we make at some point in the book is like,
link |
00:50:49.360
look, you know, to the extent that we think that machine learning applied for personalization
link |
00:50:55.280
purposes in things like newsfeed, you know, or other platforms has led to polarization
link |
00:51:03.480
and intolerance of opposing viewpoints.
link |
00:51:07.940
As you know, right, these algorithms have models, right, and they kind of place people
link |
00:51:11.880
in some kind of metric space, and they place content in that space, and they sort of know
link |
00:51:17.700
the extent to which I have an affinity for a particular type of content.
link |
00:51:22.180
And by the same token, they also probably have that same model probably gives you a
link |
00:51:26.400
good idea of the stuff I'm likely to violently disagree with or be offended by, okay?
link |
00:51:32.760
So you know, in this case, there really is some knob you could tune that says like, instead
link |
00:51:37.440
of showing people only what they like and what they want, let's show them some stuff
link |
00:51:43.040
that we think that they don't like, or that's a little bit further away.
link |
00:51:46.160
And you could even imagine users being able to control this, you know, just like everybody
link |
00:51:51.680
gets a slider, and that slider says like, you know, how much stuff do you want to see
link |
00:51:58.240
that's kind of, you know, you might disagree with, or is at least further from your interest.
link |
00:52:02.960
It's almost like an exploration button.
link |
00:52:05.720
So just get your intuition.
link |
00:52:08.360
Do you think engagement, so like you staying on the platform, you're staying engaged.
link |
00:52:15.160
Do you think fairness, ideas of fairness won't emerge?
link |
00:52:19.920
Like how bad is it to just optimize for engagement?
link |
00:52:23.740
Do you think we'll run into big trouble if we're just optimizing for how much you love
link |
00:52:28.440
the platform?
link |
00:52:29.440
Well, I mean, optimizing for engagement kind of got us where we are.
link |
00:52:34.800
So do you, one, have faith that it's possible to do better?
link |
00:52:39.960
And two, if it is, how do we do better?
link |
00:52:44.240
I mean, it's definitely possible to do different, right?
link |
00:52:47.060
And again, you know, it's not as if I think that doing something different than optimizing
link |
00:52:51.700
for engagement won't cost these companies in real ways, including revenue and profitability
link |
00:52:57.880
potentially.
link |
00:52:58.880
In the short term at least.
link |
00:53:00.600
Yeah.
link |
00:53:01.600
In the short term.
link |
00:53:02.600
Right.
link |
00:53:03.600
And again, you know, if I worked at these companies, I'm sure that it would have seemed
link |
00:53:08.920
like the most natural thing in the world also to want to optimize engagement, right?
link |
00:53:12.640
And that's good for users in some sense.
link |
00:53:14.600
You want them to be, you know, vested in the platform and enjoying it and finding it useful,
link |
00:53:19.600
interesting, and or productive.
link |
00:53:21.660
But you know, my point is, is that the idea that there is, that it's sort of out of their
link |
00:53:27.080
hands as you said, or that there's nothing to do about it, never say never, but that
link |
00:53:31.560
strikes me as implausible as a machine learning person, right?
link |
00:53:34.560
I mean, these companies are driven by machine learning and this optimization of engagement
link |
00:53:39.600
is essentially driven by machine learning, right?
link |
00:53:42.040
It's driven by not just machine learning, but you know, very, very large scale A, B
link |
00:53:47.120
experimentation where you kind of tweak some element of the user interface or tweak some
link |
00:53:53.080
component of an algorithm or tweak some component or feature of your click through prediction
link |
00:53:59.520
model.
link |
00:54:01.200
And my point is, is that anytime you know how to optimize for something, you, you know,
link |
00:54:06.360
by def, almost by definition, that solution tells you how not to optimize for it or to
link |
00:54:10.600
do something different.
link |
00:54:13.240
Engagement can be measured.
link |
00:54:16.200
So sort of optimizing for sort of minimizing divisiveness or maximizing intellectual growth
link |
00:54:25.320
over the lifetime of a human being are very difficult to measure.
link |
00:54:30.160
That's right.
link |
00:54:31.160
And I'm not claiming that doing something different will immediately make it apparent
link |
00:54:38.240
that this is a good thing for society and in particular, I mean, I think one way of
link |
00:54:42.320
thinking about where we are on some of these social media platforms is that, you know,
link |
00:54:47.400
it kind of feels a bit like we're in a bad equilibrium, right?
link |
00:54:50.880
That these systems are helping us all kind of optimize something myopically and selfishly
link |
00:54:55.920
for ourselves and of course, from an individual standpoint at any given moment, like why would
link |
00:55:02.280
I want to see things in my newsfeed that I found irrelevant, offensive or, you know,
link |
00:55:07.280
or the like, okay?
link |
00:55:09.240
But you know, maybe by all of us, you know, having these platforms myopically optimized
link |
00:55:15.320
in our interests, we have reached a collective outcome as a society that we're unhappy with
link |
00:55:20.920
in different ways.
link |
00:55:21.920
Let's say with respect to things like, you know, political discourse and tolerance of
link |
00:55:26.360
opposing viewpoints.
link |
00:55:28.160
And if Mark Zuckerberg gave you a call and said, I'm thinking of taking a sabbatical,
link |
00:55:34.840
could you run Facebook for me for six months?
link |
00:55:37.440
What would you, how?
link |
00:55:39.240
I think no thanks would be my first response, but there are many aspects of being the head
link |
00:55:45.720
of the entire company that are kind of entirely exogenous to many of the things that we're
link |
00:55:51.200
discussing here.
link |
00:55:52.200
Yes.
link |
00:55:53.200
And so I don't really think I would need to be CEO of Facebook to kind of implement the,
link |
00:55:58.680
you know, more limited set of solutions that I might imagine.
link |
00:56:02.720
But I think one concrete thing they could do is they could experiment with letting people
link |
00:56:08.940
who chose to, to see more stuff in their newsfeed that is not entirely kind of chosen to optimize
link |
00:56:17.020
for their particular interests, beliefs, et cetera.
link |
00:56:22.500
So the, the kind of thing, so I could speak to YouTube, but I think Facebook probably
link |
00:56:27.240
does something similar is they're quite effective at automatically finding what sorts of groups
link |
00:56:34.880
you belong to, not based on race or gender or so on, but based on the kind of stuff you
link |
00:56:40.260
enjoy watching in the case of YouTube.
link |
00:56:43.120
Sort of, it's a, it's a difficult thing for Facebook or YouTube to then say, well, you
link |
00:56:50.160
know what?
link |
00:56:51.160
We're going to show you something from a very different cluster.
link |
00:56:54.700
Even though we believe algorithmically, you're unlikely to enjoy that thing sort of that's
link |
00:57:00.020
a weird jump to make.
link |
00:57:02.340
There has to be a human, like at the very top of that system that says, well, that will
link |
00:57:07.020
be longterm healthy for you.
link |
00:57:09.840
That's more than an algorithmic decision.
link |
00:57:11.880
Or that same person could say that'll be longterm healthy for the platform or for the platform's
link |
00:57:18.460
influence on society outside of the platform, right?
link |
00:57:22.560
And it, you know, it's easy for me to sit here and say these things, but conceptually
link |
00:57:27.020
I do not think that these are kind of totally or should, they shouldn't be kind of completely
link |
00:57:32.920
alien ideas, right?
link |
00:57:34.800
That, you know, you could try things like this and it wouldn't be, you know, we wouldn't
link |
00:57:40.880
have to invent entirely new science to do it because if we're all already embedded in
link |
00:57:45.820
some metric space and there's a notion of distance between you and me and every other,
link |
00:57:50.520
every piece of content, then, you know, we know exactly, you know, the same model that
link |
00:57:56.060
tells, you know, dictates how to make me really happy also tells how to make me as unhappy
link |
00:58:03.340
as possible as well.
link |
00:58:04.960
Right.
link |
00:58:05.960
The focus in your book and algorithmic fairness research today in general is on machine learning,
link |
00:58:11.000
like we said, is data, but, and just even the entire AI field right now is captivated
link |
00:58:16.800
with machine learning, with deep learning.
link |
00:58:19.720
Do you think ideas in symbolic AI or totally other kinds of approaches are interesting,
link |
00:58:25.480
useful in the space, have some promising ideas in terms of fairness?
link |
00:58:31.400
I haven't thought about that question specifically in the context of fairness.
link |
00:58:35.240
I definitely would agree with that statement in the large, right?
link |
00:58:39.040
I mean, I am, you know, one of many machine learning researchers who do believe that the
link |
00:58:46.840
great successes that have been shown in machine learning recently are great successes, but
link |
00:58:51.400
they're on a pretty narrow set of tasks.
link |
00:58:53.280
I mean, I don't, I don't think we're kind of notably closer to general artificial intelligence
link |
00:59:00.480
now than we were when I started my career.
link |
00:59:03.360
I mean, there's been progress and I do think that we are kind of as a community, maybe
link |
00:59:08.640
looking a bit where the light is, but the light is shining pretty bright there right
link |
00:59:12.000
now and we're finding a lot of stuff.
link |
00:59:13.760
So I don't want to like argue with the progress that's been made in areas like deep learning,
link |
00:59:18.520
for example.
link |
00:59:19.880
This touches another sort of related thing that you mentioned and that people might misinterpret
link |
00:59:25.080
from the title of your book, ethical algorithm.
link |
00:59:27.420
Is it possible for the algorithm to automate some of those decisions?
link |
00:59:31.800
Sort of a higher level decisions of what kind of, like what, what should be fair, what should
link |
00:59:37.400
be fair.
link |
00:59:38.720
The more you know about a field, the more aware you are of its limitations.
link |
00:59:43.400
And so I'm a, I'm pretty leery of sort of trying, you know, there's, there's so much
link |
00:59:47.840
we don't all, we already don't know in fairness, even when we're the ones picking the fairness
link |
00:59:53.760
definitions and, you know, comparing alternatives and thinking about the tensions between different
link |
00:59:58.960
definitions that the idea of kind of letting the algorithm start exploring as well.
link |
01:00:05.160
I definitely think, you know, this is a much narrower statement.
link |
01:00:08.560
I definitely think that kind of algorithmic auditing for different types of unfairness,
link |
01:00:12.440
right?
link |
01:00:13.440
So like in this gerrymandering example where I might want to prevent not just discrimination
link |
01:00:18.680
against very broad categories, but against combinations of broad categories.
link |
01:00:23.960
You know, you quickly get to a point where there's a lot of, a lot of categories.
link |
01:00:27.700
There's a lot of combinations of end features and, you know, you can use algorithmic techniques
link |
01:00:33.520
to sort of try to find the subgroups on which you're discriminating the most and try to
link |
01:00:38.000
fix that.
link |
01:00:39.000
That's actually kind of the form of one of the algorithms we developed for this fairness
link |
01:00:42.460
gerrymandering problem.
link |
01:00:44.240
But I'm, I'm, you know, partly because of our technological, you know, our sort of our
link |
01:00:49.440
scientific ignorance on these topics right now.
link |
01:00:53.400
And also partly just because these topics are so loaded emotionally for people that
link |
01:00:58.360
I just don't see the value.
link |
01:01:00.440
I mean, again, never say never, but I just don't think we're at a moment where it's
link |
01:01:03.920
a great time for computer scientists to be rolling out the idea like, hey, you know,
link |
01:01:08.600
you know, not only have we kind of figured fairness out, but, you know, we think the
link |
01:01:12.520
algorithm should start deciding what's fair or giving input on that decision.
link |
01:01:16.880
I just don't, it's like the cost benefit analysis to the field of kind of going there
link |
01:01:22.080
right now just doesn't seem worth it to me.
link |
01:01:24.520
That said, I should say that I think computer scientists should be more philosophically,
link |
01:01:29.200
like should enrich their thinking about these kinds of things.
link |
01:01:32.280
I think it's been too often used as an excuse for roboticists working on autonomous vehicles,
link |
01:01:38.020
for example, to not think about the human factor or psychology or safety in the same
link |
01:01:43.720
way like computer science design algorithms that have been sort of using it as an excuse.
link |
01:01:47.440
And I think it's time for basically everybody to become a computer scientist.
link |
01:01:51.640
I was about to agree with everything you said except that last point.
link |
01:01:54.440
I think that the other way of looking at it is that I think computer scientists, you know,
link |
01:01:59.760
and many of us are, but we need to weigh it out into the world more, right?
link |
01:02:06.120
I mean, just the influence that computer science and therefore computer scientists have had
link |
01:02:12.520
on society at large just like has exponentially magnified in the last 10 or 20 years or so.
link |
01:02:21.520
And you know, before when we were just tinkering around amongst ourselves and it didn't matter
link |
01:02:26.560
that much, there was no need for sort of computer scientists to be citizens of the world more
link |
01:02:32.360
broadly.
link |
01:02:33.440
And I think those days need to be over very, very fast.
link |
01:02:36.760
And I'm not saying everybody needs to do it, but to me, like the right way of doing it
link |
01:02:40.720
is to not to sort of think that everybody else is going to become a computer scientist.
link |
01:02:44.120
But you know, I think people are becoming more sophisticated about computer science,
link |
01:02:49.200
even lay people.
link |
01:02:50.200
You know, I think one of the reasons we decided to write this book is we thought 10 years
link |
01:02:55.520
ago I wouldn't have tried this just because I just didn't think that sort of people's
link |
01:03:00.400
awareness of algorithms and machine learning, you know, the general population would have
link |
01:03:06.240
been high.
link |
01:03:07.240
I mean, you would have had to first, you know, write one of the many books kind of just explicating
link |
01:03:12.060
that topic to a lay audience first.
link |
01:03:14.720
Now I think we're at the point where like lots of people without any technical training
link |
01:03:18.900
at all know enough about algorithms and machine learning that you can start getting to these
link |
01:03:22.800
nuances of things like ethical algorithms.
link |
01:03:26.000
I think we agree that there needs to be much more mixing, but I think a lot of the onus
link |
01:03:31.780
of that mixing needs to be on the computer science community.
link |
01:03:35.360
Yeah.
link |
01:03:36.360
So just to linger on the disagreement, because I do disagree with you on the point that I
link |
01:03:41.920
think if you're a biologist, if you're a chemist, if you're an MBA business person, all of those
link |
01:03:50.780
things you can, like if you learned a program, and not only program, if you learned to do
link |
01:03:57.160
machine learning, if you learned to do data science, you immediately become much more
link |
01:04:02.160
powerful in the kinds of things you can do.
link |
01:04:04.200
And therefore literature, like library sciences, like, so you were speaking, I think, I think
link |
01:04:11.600
it holds true what you're saying for the next few years.
link |
01:04:14.760
But long term, if you're interested to me, if you're interested in philosophy, you should
link |
01:04:21.520
learn a program, because then you can scrape data and study what people are thinking about
link |
01:04:27.700
on Twitter, and then start making philosophical conclusions about the meaning of life.
link |
01:04:33.760
I just feel like the access to data, the digitization of whatever problem you're trying to solve,
link |
01:04:41.440
will fundamentally change what it means to be a computer scientist.
link |
01:04:44.200
I mean, a computer scientist in 20, 30 years will go back to being Donald Knuth style theoretical
link |
01:04:51.200
computer science, and everybody would be doing basically, exploring the kinds of ideas that
link |
01:04:56.560
you explore in your book.
link |
01:04:57.560
It won't be a computer science major.
link |
01:04:58.880
Yeah, I mean, I don't think I disagree enough, but I think that that trend of more and more
link |
01:05:05.000
people in more and more disciplines adopting ideas from computer science, learning how
link |
01:05:11.600
to code, I think that that trend seems firmly underway.
link |
01:05:14.560
I mean, you know, like an interesting digressive question along these lines is maybe in 50
link |
01:05:21.000
years, there won't be computer science departments anymore, because the field will just sort
link |
01:05:27.080
of be ambient in all of the different disciplines.
link |
01:05:30.840
And people will look back and having a computer science department will look like having an
link |
01:05:35.720
electricity department or something that's like, you know, everybody uses this, it's
link |
01:05:39.480
just out there.
link |
01:05:40.480
I mean, I do think there will always be that kind of Knuth style core to it, but it's not
link |
01:05:45.180
an implausible path that we kind of get to the point where the academic discipline of
link |
01:05:50.180
computer science becomes somewhat marginalized because of its very success in kind of infiltrating
link |
01:05:56.160
all of science and society and the humanities, etcetera.
link |
01:06:00.720
What is differential privacy, or more broadly, algorithmic privacy?
link |
01:06:07.720
Algorithmic privacy more broadly is just the study or the notion of privacy definitions
link |
01:06:15.040
or norms being encoded inside of algorithms.
link |
01:06:19.580
And so, you know, I think we count among this body of work just, you know, the literature
link |
01:06:27.520
and practice of things like data anonymization, which we kind of at the beginning of our discussion
link |
01:06:33.980
of privacy say like, okay, this is sort of a notion of algorithmic privacy.
link |
01:06:38.600
It kind of tells you, you know, something to go do with data, but, you know, our view
link |
01:06:44.840
is that it's, and I think this is now, you know, quite widespread, that it's, you know,
link |
01:06:50.120
despite the fact that those notions of anonymization kind of redacting and coarsening are the most
link |
01:06:57.320
widely adopted technical solutions for data privacy, they are like deeply fundamentally
link |
01:07:03.700
flawed.
link |
01:07:05.120
And so, you know, to your first question, what is differential privacy?
link |
01:07:11.240
Differential privacy seems to be a much, much better notion of privacy that kind of avoids
link |
01:07:16.680
a lot of the weaknesses of anonymization notions while still letting us do useful stuff with
link |
01:07:24.520
data.
link |
01:07:25.520
What is anonymization of data?
link |
01:07:27.480
So by anonymization, I'm, you know, kind of referring to techniques like I have a database.
link |
01:07:34.000
The rows of that database are, let's say, individual people's medical records, okay?
link |
01:07:40.240
And I want to let people use that data.
link |
01:07:43.840
Maybe I want to let researchers access that data to build predictive models for some disease,
link |
01:07:49.480
but I'm worried that that will leak, you know, sensitive information about specific people's
link |
01:07:56.200
medical records.
link |
01:07:57.640
So anonymization broadly refers to the set of techniques where I say like, okay, I'm
link |
01:08:01.680
first going to like, I'm going to delete the column with people's names.
link |
01:08:06.160
I'm going to not put, you know, so that would be like a redaction, right?
link |
01:08:09.760
I'm just redacting that information.
link |
01:08:12.040
I am going to take ages and I'm not going to like say your exact age.
link |
01:08:17.040
I'm going to say whether you're, you know, zero to 10, 10 to 20, 20 to 30, I might put
link |
01:08:23.120
the first three digits of your zip code, but not the last two, et cetera, et cetera.
link |
01:08:27.520
And so the idea is that through some series of operations like this on the data, I anonymize
link |
01:08:31.800
it.
link |
01:08:32.800
You know, another term of art that's used is removing personally identifiable information.
link |
01:08:38.880
And you know, this is basically the most common way of providing data privacy, but that it's
link |
01:08:45.600
in a way that still lets people access the, some variant form of the data.
link |
01:08:50.240
So at a slightly broader picture, as you talk about what does anonymization mean when you
link |
01:08:56.080
have multiple database, like with a Netflix prize, when you can start combining stuff
link |
01:09:01.440
together.
link |
01:09:02.440
So this is exactly the problem with these notions, right?
link |
01:09:05.400
Is that notions of a anonymization, removing personally identifiable information, the kind
link |
01:09:10.900
of fundamental conceptual flaw is that, you know, these definitions kind of pretend as
link |
01:09:16.000
if the data set in question is the only data set that exists in the world or that ever
link |
01:09:21.240
will exist in the future.
link |
01:09:23.640
And of course, things like the Netflix prize and many, many other examples since the Netflix
link |
01:09:28.080
prize, I think that was one of the earliest ones though, you know, you can reidentify
link |
01:09:33.320
people that were, you know, that were anonymized in the data set by taking that anonymized
link |
01:09:38.540
data set and combining it with other allegedly anonymized data sets and maybe publicly available
link |
01:09:43.240
information about you.
link |
01:09:44.480
You know,
link |
01:09:45.480
for people who don't know the Netflix prize was, was being publicly released this data.
link |
01:09:50.880
So the names from those rows were removed, but what was released is the preference or
link |
01:09:55.640
the ratings of what movies you like and you don't like.
link |
01:09:58.720
And from that combined with other things, I think forum posts and so on, you can start
link |
01:10:03.360
to figure out
link |
01:10:04.360
I guess it was specifically the internet movie database where, where lots of Netflix users
link |
01:10:10.400
publicly rate their movie, you know, their movie preferences.
link |
01:10:15.280
And so the anonymized data and Netflix, when it's just this phenomenon, I think that we've
link |
01:10:21.840
all come to realize in the last decade or so is that just knowing a few apparently irrelevant
link |
01:10:29.920
innocuous things about you can often act as a fingerprint.
link |
01:10:33.100
Like if I know, you know, what, what rating you gave to these 10 movies and the date on
link |
01:10:39.000
which you entered these movies, this is almost like a fingerprint for you in the sea of all
link |
01:10:43.480
Netflix users.
link |
01:10:44.480
There were just another paper on this in science or nature of about a month ago that, you know,
link |
01:10:49.760
kind of 18 attributes.
link |
01:10:51.240
I mean, my favorite example of this is, was actually a paper from several years ago now
link |
01:10:57.120
where it was shown that just from your likes on Facebook, just from the time, you know,
link |
01:11:03.400
the things on which you clicked on the thumbs up button on the platform, not using any information,
link |
01:11:09.520
demographic information, nothing about who your friends are, just knowing the content
link |
01:11:14.720
that you had liked was enough to, you know, in the aggregate accurately predict things
link |
01:11:20.680
like sexual orientation, drug and alcohol use, whether you were the child of divorced parents.
link |
01:11:27.280
So we live in this era where, you know, even the apparently irrelevant data that we offer
link |
01:11:32.080
about ourselves on public platforms and forums often unbeknownst to us, more or less acts
link |
01:11:38.760
as signature or, you know, fingerprint.
link |
01:11:42.480
And that if you can kind of, you know, do a join between that kind of data and allegedly
link |
01:11:46.980
anonymized data, you have real trouble.
link |
01:11:50.720
So is there hope for any kind of privacy in a world where a few likes can identify you?
link |
01:11:58.380
So there is differential privacy, right?
link |
01:12:00.380
What is differential privacy?
link |
01:12:01.380
Yeah, so differential privacy basically is a kind of alternate, much stronger notion
link |
01:12:06.100
of privacy than these anonymization ideas.
link |
01:12:10.280
And, you know, it's a technical definition, but like the spirit of it is we compare two
link |
01:12:18.760
alternate worlds, okay?
link |
01:12:20.320
So let's suppose I'm a researcher and I want to do, you know, there's a database of medical
link |
01:12:26.120
records and one of them is yours, and I want to use that database of medical records to
link |
01:12:31.600
build a predictive model for some disease.
link |
01:12:33.800
So based on people's symptoms and test results and the like, I want to, you know, build a
link |
01:12:39.440
probably model predicting the probability that people have disease.
link |
01:12:42.180
So, you know, this is the type of scientific research that we would like to be allowed
link |
01:12:46.400
to continue.
link |
01:12:48.060
And in differential privacy, you ask a very particular counterfactual question.
link |
01:12:53.400
We basically compare two alternatives.
link |
01:12:57.480
One is when I do this, I build this model on the database of medical records, including
link |
01:13:04.760
your medical record.
link |
01:13:07.200
And the other one is where I do the same exercise with the same database with just your medical
link |
01:13:15.320
record removed.
link |
01:13:16.320
So basically, you know, it's two databases, one with N records in it and one with N minus
link |
01:13:22.280
one records in it.
link |
01:13:23.840
The N minus one records are the same, and the only one that's missing in the second
link |
01:13:27.960
case is your medical record.
link |
01:13:30.420
So differential privacy basically says that any harms that might come to you from the
link |
01:13:40.580
analysis in which your data was included are essentially nearly identical to the harms
link |
01:13:47.640
that would have come to you if the same analysis had been done without your medical record
link |
01:13:52.720
included.
link |
01:13:53.720
So in other words, this doesn't say that bad things cannot happen to you as a result of
link |
01:13:58.280
data analysis.
link |
01:13:59.760
It just says that these bad things were going to happen to you already, even if your data
link |
01:14:05.080
wasn't included.
link |
01:14:06.080
And to give a very concrete example, right, you know, like we discussed at some length,
link |
01:14:12.360
the study that, you know, in the 50s that was done that established the link between
link |
01:14:17.800
smoking and lung cancer.
link |
01:14:19.960
And we make the point that, like, well, if your data was used in that analysis and, you
link |
01:14:25.200
know, the world kind of knew that you were a smoker because, you know, there was no stigma
link |
01:14:28.980
associated with smoking before those findings, real harm might have come to you as a result
link |
01:14:35.160
of that study that your data was included in.
link |
01:14:37.760
In particular, your insurer now might have a higher posterior belief that you might have
link |
01:14:42.440
lung cancer and raise your premium.
link |
01:14:44.360
So you've suffered economic damage.
link |
01:14:47.820
But the point is, is that if the same analysis has been done with all the other N minus one
link |
01:14:54.960
medical records and just yours missing, the outcome would have been the same.
link |
01:14:58.800
Or your data wasn't idiosyncratically crucial to establishing the link between smoking and
link |
01:15:05.560
lung cancer because the link between smoking and lung cancer is like a fact about the world
link |
01:15:10.440
that can be discovered with any sufficiently large database of medical records.
link |
01:15:14.820
But that's a very low value of harm.
link |
01:15:17.320
Yeah.
link |
01:15:18.320
So that's showing that very little harm is done.
link |
01:15:20.560
Great.
link |
01:15:21.560
But how what is the mechanism of differential privacy?
link |
01:15:24.760
So that's the kind of beautiful statement of it.
link |
01:15:27.600
It's the mechanism by which privacy is preserved.
link |
01:15:30.440
Yeah.
link |
01:15:31.440
So it's basically by adding noise to computations, right?
link |
01:15:34.600
So the basic idea is that every differentially private algorithm, first of all, or every
link |
01:15:40.400
good differentially private algorithm, every useful one, is a probabilistic algorithm.
link |
01:15:45.380
So it doesn't, on a given input, if you gave the algorithm the same input multiple times,
link |
01:15:51.000
it would give different outputs each time from some distribution.
link |
01:15:55.760
And the way you achieve differential privacy algorithmically is by kind of carefully and
link |
01:15:59.820
tastefully adding noise to a computation in the right places.
link |
01:16:05.400
And to give a very concrete example, if I wanna compute the average of a set of numbers,
link |
01:16:11.600
the non private way of doing that is to take those numbers and average them and release
link |
01:16:17.220
like a numerically precise value for the average.
link |
01:16:21.880
In differential privacy, you wouldn't do that.
link |
01:16:24.200
You would first compute that average to numerical precisions, and then you'd add some noise
link |
01:16:29.520
to it, right?
link |
01:16:30.520
You'd add some kind of zero mean, Gaussian or exponential noise to it so that the actual
link |
01:16:37.520
value you output is not the exact mean, but it'll be close to the mean, but it'll be close...
link |
01:16:44.120
The noise that you add will sort of prove that nobody can kind of reverse engineer any
link |
01:16:50.560
particular value that went into the average.
link |
01:16:53.440
So noise is a savior.
link |
01:16:56.200
How many algorithms can be aided by adding noise?
link |
01:17:01.640
Yeah, so I'm a relatively recent member of the differential privacy community.
link |
01:17:07.040
My co author, Aaron Roth is really one of the founders of the field and has done a great
link |
01:17:12.440
deal of work and I've learned a tremendous amount working with him on it.
link |
01:17:15.520
It's a pretty grown up field already.
link |
01:17:17.240
Yeah, but now it's pretty mature.
link |
01:17:18.480
But I must admit, the first time I saw the definition of differential privacy, my reaction
link |
01:17:22.080
was like, wow, that is a clever definition and it's really making very strong promises.
link |
01:17:28.360
And I first saw the definition in much earlier days and my first reaction was like, well,
link |
01:17:34.920
my worry about this definition would be that it's a great definition of privacy, but that
link |
01:17:38.960
it'll be so restrictive that we won't really be able to use it.
link |
01:17:43.180
We won't be able to compute many things in a differentially private way.
link |
01:17:47.200
So that's one of the great successes of the field, I think, is in showing that the opposite
link |
01:17:51.920
is true and that most things that we know how to compute, absent any privacy considerations,
link |
01:18:00.980
can be computed in a differentially private way.
link |
01:18:02.920
So for example, pretty much all of statistics and machine learning can be done differentially
link |
01:18:08.240
privately.
link |
01:18:09.320
So pick your favorite machine learning algorithm, back propagation and neural networks, cart
link |
01:18:15.120
for decision trees, support vector machines, boosting, you name it, as well as classic
link |
01:18:21.060
hypothesis testing and the like in statistics.
link |
01:18:24.920
None of those algorithms are differentially private in their original form.
link |
01:18:29.720
All of them have modifications that add noise to the computation in different places in
link |
01:18:35.700
different ways that achieve differential privacy.
link |
01:18:39.120
So this really means that to the extent that we've become a scientific community very dependent
link |
01:18:47.460
on the use of machine learning and statistical modeling and data analysis, we really do have
link |
01:18:53.400
a path to provide privacy guarantees to those methods and so we can still enjoy the benefits
link |
01:19:02.760
of the data science era while providing rather robust privacy guarantees to individuals.
link |
01:19:10.760
So perhaps a slightly crazy question, but if we take the ideas of differential privacy
link |
01:19:16.160
and take it to the nature of truth that's being explored currently.
link |
01:19:20.680
So what's your most favorite and least favorite food?
link |
01:19:24.880
Hmm.
link |
01:19:25.880
I'm not a real foodie, so I'm a big fan of spaghetti.
link |
01:19:29.880
Spaghetti?
link |
01:19:30.880
Yeah.
link |
01:19:31.880
What do you really don't like?
link |
01:19:35.840
I really don't like cauliflower.
link |
01:19:37.280
Wow, I love cauliflower.
link |
01:19:39.280
Okay.
link |
01:19:40.280
Is there one way to protect your preference for spaghetti by having an information campaign
link |
01:19:46.400
bloggers and so on of bots saying that you like cauliflower?
link |
01:19:51.280
So like this kind of the same kind of noise ideas, I mean if you think of in our politics
link |
01:19:56.640
today there's this idea of Russia hacking our elections.
link |
01:20:01.920
What's meant there I believe is bots spreading different kinds of information.
link |
01:20:07.200
Is that a kind of privacy or is that too much of a stretch?
link |
01:20:10.480
No it's not a stretch.
link |
01:20:12.160
I've not seen those ideas, you know, that is not a technique that to my knowledge will
link |
01:20:19.320
provide differential privacy, but to give an example like one very specific example
link |
01:20:24.400
about what you're discussing is there was a very interesting project at NYU I think
link |
01:20:30.240
led by Helen Nissenbaum there in which they basically built a browser plugin that tried
link |
01:20:38.720
to essentially obfuscate your Google searches.
link |
01:20:41.640
So to the extent that you're worried that Google is using your searches to build, you
link |
01:20:46.440
know, predictive models about you to decide what ads to show you which they might very
link |
01:20:51.480
reasonably want to do, but if you object to that they built this widget you could plug
link |
01:20:56.040
in and basically whenever you put in a query into Google it would send that query to Google,
link |
01:21:01.280
but in the background all of the time from your browser it would just be sending this
link |
01:21:06.200
torrent of irrelevant queries to the search engine.
link |
01:21:11.800
So you know it's like a weed and chaff thing so you know out of every thousand queries
link |
01:21:16.840
let's say that Google was receiving from your browser one of them was one that you put in
link |
01:21:21.560
but the other 999 were not okay so it's the same kind of idea kind of you know privacy
link |
01:21:27.300
by obfuscation.
link |
01:21:29.680
So I think that's an interesting idea, doesn't give you differential privacy.
link |
01:21:34.920
It's also I was actually talking to somebody at one of the large tech companies recently
link |
01:21:39.260
about the fact that you know just this kind of thing that there are some times when the
link |
01:21:45.560
response to my data needs to be very specific to my data right like I type mountain biking
link |
01:21:53.120
into Google, I want results on mountain biking and I really want Google to know that I typed
link |
01:21:58.420
in mountain biking, I don't want noise added to that.
link |
01:22:01.880
And so I think there's sort of maybe even interesting technical questions around notions
link |
01:22:06.180
of privacy that are appropriate where you know it's not that my data is part of some
link |
01:22:10.800
aggregate like medical records and that we're trying to discover important correlations
link |
01:22:15.800
and facts about the world at large but rather you know there's a service that I really want
link |
01:22:20.960
to you know pay attention to my specific data yet I still want some kind of privacy guarantee
link |
01:22:26.120
and I think these kind of obfuscation ideas are sort of one way of getting at that but
link |
01:22:30.200
maybe there are others as well.
link |
01:22:32.160
So where do you think we'll land in this algorithm driven society in terms of privacy?
link |
01:22:36.520
So sort of China like Kai Fuli describes you know it's collecting a lot of data on its
link |
01:22:44.960
citizens but in the best form it's actually able to provide a lot of sort of protect human
link |
01:22:52.360
rights and provide a lot of amazing services and it's worst forms that can violate those
link |
01:22:57.320
human rights and limit services.
link |
01:23:01.080
So where do you think we'll land because algorithms are powerful when they use data.
link |
01:23:08.400
So as a society do you think we'll give over more data?
link |
01:23:12.900
Is it possible to protect the privacy of that data?
link |
01:23:16.400
So I'm optimistic about the possibility of you know balancing the desire for individual
link |
01:23:24.400
privacy and individual control of privacy with kind of societally and commercially beneficial
link |
01:23:32.360
uses of data not unrelated to differential privacy or suggestions that say like well
link |
01:23:37.840
individuals should have control of their data.
link |
01:23:40.560
They should be able to limit the uses of that data.
link |
01:23:43.600
They should even you know there's you know fledgling discussions going on in research
link |
01:23:48.200
circles about allowing people selective use of their data and being compensated for it.
link |
01:23:54.680
And then you get to sort of very interesting economic questions like pricing right.
link |
01:23:59.480
And one interesting idea is that maybe differential privacy would also you know be a conceptual
link |
01:24:05.360
framework in which you could talk about the relative value of different people's data
link |
01:24:09.120
like you know to demystify this a little bit.
link |
01:24:12.080
If I'm trying to build a predictive model for some rare disease and I'm trying to use
link |
01:24:17.320
machine learning to do it, it's easy to get negative examples because the disease is rare
link |
01:24:22.480
right.
link |
01:24:23.740
But I really want to have lots of people with the disease in my data set okay.
link |
01:24:30.880
And so somehow those people's data with respect to this application is much more valuable
link |
01:24:35.380
to me than just like the background population.
link |
01:24:37.840
And so maybe they should be compensated more for it.
link |
01:24:43.160
And so you know I think these are kind of very, very fledgling conceptual questions
link |
01:24:48.800
that maybe we'll have kind of technical thought on them sometime in the coming years.
link |
01:24:54.000
But I do think we'll you know to kind of get more directly answer your question.
link |
01:24:56.760
I think I'm optimistic at this point from what I've seen that we will land at some you
link |
01:25:02.760
know better compromise than we're at right now where again you know privacy guarantees
link |
01:25:08.640
are few far between and weak and users have very, very little control.
link |
01:25:15.400
And I'm optimistic that we'll land in something that you know provides better privacy overall
link |
01:25:20.320
and more individual control of data and privacy.
link |
01:25:22.820
But you know I think to get there it's again just like fairness it's not going to be enough
link |
01:25:27.740
to propose algorithmic solutions.
link |
01:25:29.560
There's going to have to be a whole kind of regulatory legal process that prods companies
link |
01:25:34.880
and other parties to kind of adopt solutions.
link |
01:25:38.880
And I think you've mentioned the word control a lot and I think giving people control that's
link |
01:25:43.040
something that people don't quite have in a lot of these algorithms and that's a really
link |
01:25:48.200
interesting idea of giving them control.
link |
01:25:50.540
Some of that is actually literally an interface design question sort of just enabling because
link |
01:25:57.920
I think it's good for everybody to give users control.
link |
01:26:00.440
It's almost not a trade off except that you have to hire people that are good at interface
link |
01:26:06.160
design.
link |
01:26:07.160
Yeah.
link |
01:26:08.160
I mean the other thing that has to be said right is that you know it's a cliche but you
link |
01:26:13.080
know we as the users of many systems platforms and apps you know we are the product.
link |
01:26:21.720
We are not the customer.
link |
01:26:23.120
The customer are advertisers and our data is the product.
link |
01:26:26.760
Okay.
link |
01:26:27.760
So it's one thing to kind of suggest more individual control of data and privacy and
link |
01:26:32.640
uses but this you know if this happens in sufficient degree it will upend the entire
link |
01:26:40.480
economic model that has supported the internet to date.
link |
01:26:44.520
And so some other economic model will have to be you know we'll have to replace it.
link |
01:26:50.040
So the idea of markets you mentioned by exposing the economic model to the people they will
link |
01:26:56.480
then become a market.
link |
01:26:57.920
They could be participants in it.
link |
01:27:00.280
And you know this isn't you know this is not a weird idea right because there are markets
link |
01:27:04.680
for data already.
link |
01:27:05.720
It's just that consumers are not participants and there's like you know there's sort of
link |
01:27:10.080
you know publishers and content providers on one side that have inventory and then their
link |
01:27:14.780
advertisers on the others and you know you know Google and Facebook are running you know
link |
01:27:19.680
they're pretty much their entire revenue stream is by running two sided markets between those
link |
01:27:25.540
parties right.
link |
01:27:27.380
And so it's not a crazy idea that there would be like a three sided market or that you know
link |
01:27:32.800
that on one side of the market or the other we would have proxies representing our interest.
link |
01:27:37.080
It's not you know it's not a crazy idea but it would it's not a crazy technical idea but
link |
01:27:43.080
it would have pretty extreme economic consequences.
link |
01:27:49.920
Speaking of markets a lot of fascinating aspects of this world arise not from individual human
link |
01:27:55.520
beings but from the interaction of human beings.
link |
01:27:59.880
You've done a lot of work in game theory.
link |
01:28:02.080
First can you say what is game theory and how does it help us model and study?
link |
01:28:07.360
Yeah game theory of course let us give credit where it's due.
link |
01:28:11.080
You know it comes from the economist first and foremost but as I've mentioned before
link |
01:28:16.300
like you know computer scientists never hesitate to wander into other people's turf and so
link |
01:28:22.000
there is now this 20 year old field called algorithmic game theory.
link |
01:28:26.520
But you know game theory first and foremost is a mathematical framework for reasoning
link |
01:28:33.240
about collective outcomes in systems of interacting individuals.
link |
01:28:40.240
You know so you need at least two people to get started in game theory and many people
link |
01:28:46.040
are probably familiar with Prisoner's Dilemma as kind of a classic example of game theory
link |
01:28:50.560
and a classic example where everybody looking out for their own individual interests leads
link |
01:28:57.000
to a collective outcome that's kind of worse for everybody than what might be possible
link |
01:29:02.560
if they cooperated for example.
link |
01:29:05.200
But cooperation is not an equilibrium in Prisoner's Dilemma.
link |
01:29:09.780
And so my work in the field of algorithmic game theory more generally in these areas
link |
01:29:16.120
kind of looks at settings in which the number of actors is potentially extraordinarily large
link |
01:29:24.720
and their incentives might be quite complicated and kind of hard to model directly but you
link |
01:29:31.160
still want kind of algorithmic ways of kind of predicting what will happen or influencing
link |
01:29:36.120
what will happen in the design of platforms.
link |
01:29:39.880
So what to you is the most beautiful idea that you've encountered in game theory?
link |
01:29:47.160
There's a lot of them.
link |
01:29:48.160
I'm a big fan of the field.
link |
01:29:50.760
I mean you know I mean technical answers to that of course would include Nash's work just
link |
01:29:56.400
establishing that you know there is a competitive equilibrium under very very general circumstances
link |
01:30:02.640
which in many ways kind of put the field on a firm conceptual footing because if you don't
link |
01:30:09.840
have equilibrium it's kind of hard to ever reason about what might happen since you know
link |
01:30:14.280
there's just no stability.
link |
01:30:16.200
So just the idea that stability can emerge when there's multiple.
link |
01:30:20.680
Not that it will necessarily emerge just that it's possible right.
link |
01:30:23.840
Like the existence of equilibrium doesn't mean that sort of natural iterative behavior
link |
01:30:28.580
will necessarily lead to it.
link |
01:30:30.640
In the real world.
link |
01:30:31.640
Yeah.
link |
01:30:32.640
Maybe answering a slightly less personally than you asked the question I think within
link |
01:30:35.960
the field of algorithmic game theory perhaps the single most important kind of technical
link |
01:30:43.760
contribution that's been made is the realization between close connections between machine
link |
01:30:49.640
learning and game theory and in particular between game theory and the branch of machine
link |
01:30:53.840
learning that's known as no regret learning and this sort of provides a very general framework
link |
01:31:00.600
in which a bunch of players interacting in a game or a system each one kind of doing
link |
01:31:07.460
something that's in their self interest will actually kind of reach an equilibrium and
link |
01:31:12.440
actually reach an equilibrium in a you know a pretty you know a rather you know short
link |
01:31:18.960
amount of steps.
link |
01:31:21.400
So you kind of mentioned acting greedily can somehow end up pretty good for everybody.
link |
01:31:30.120
Or pretty bad.
link |
01:31:31.320
Or pretty bad.
link |
01:31:32.320
Yeah.
link |
01:31:33.320
It will end up stable.
link |
01:31:34.320
Yeah.
link |
01:31:35.320
Right.
link |
01:31:36.320
And and you know stability or equilibrium by itself is neither is not necessarily either
link |
01:31:41.500
a good thing or a bad thing.
link |
01:31:43.220
So what's the connection between machine learning and the ideas.
link |
01:31:45.840
Well I think we kind of talked about these ideas already in kind of a non technical way
link |
01:31:50.960
which is maybe the more interesting way of understanding them first which is you know
link |
01:31:57.200
we have many systems platforms and apps these days that work really hard to use our data
link |
01:32:04.840
and the data of everybody else on the platform to selfishly optimize on behalf of each user.
link |
01:32:12.120
OK.
link |
01:32:13.120
So you know let me let me give I think the cleanest example which is just driving apps
link |
01:32:17.960
navigation apps like you know Google Maps and Waze where you know miraculously compared
link |
01:32:24.040
to when I was growing up at least you know the objective would be the same when you wanted
link |
01:32:28.680
to drive from point A to point B spend the least time driving not necessarily minimize
link |
01:32:33.440
the distance but minimize the time.
link |
01:32:35.840
Right.
link |
01:32:36.840
And when I was growing up like the only resources you had to do that were like maps in the car
link |
01:32:41.080
which literally just told you what roads were available and then you might have like half
link |
01:32:46.540
hourly traffic reports just about the major freeways but not about side roads.
link |
01:32:51.680
So you were pretty much on your own.
link |
01:32:54.040
And now we've got these apps you pull it out and you say I want to go from point A to point
link |
01:32:57.920
B and in response kind of to what everybody else is doing if you like what all the other
link |
01:33:03.360
players in this game are doing right now here's the you know the route that minimizes your
link |
01:33:09.280
driving time.
link |
01:33:10.280
So it is really kind of computing a selfish best response for each of us in response to
link |
01:33:16.560
what all of the rest of us are doing at any given moment.
link |
01:33:20.240
And so you know I think it's quite fair to think of these apps as driving or nudging
link |
01:33:26.280
us all towards the competitive or Nash equilibrium of that game.
link |
01:33:32.560
Now you might ask like well that sounds great why is that a bad thing.
link |
01:33:36.320
Well you know it's known both in theory and with some limited studies from actual like
link |
01:33:45.400
traffic data that all of us being in this competitive equilibrium might cause our collective
link |
01:33:52.660
driving time to be higher maybe significantly higher than it would be under other solutions.
link |
01:33:59.760
And then you have to talk about what those other solutions might be and what the algorithms
link |
01:34:04.320
to implement them are which we do discuss in the kind of game theory chapter of the
link |
01:34:07.760
book.
link |
01:34:09.880
But similarly you know on social media platforms or on Amazon you know all these algorithms
link |
01:34:17.040
that are essentially trying to optimize our behalf they're driving us in a colloquial
link |
01:34:22.000
sense towards some kind of competitive equilibrium and you know one of the most important lessons
link |
01:34:26.920
of game theory is that just because we're at equilibrium doesn't mean that there's not
link |
01:34:30.360
a solution in which some or maybe even all of us might be better off.
link |
01:34:35.720
And then the connection to machine learning of course is that in all these platforms I've
link |
01:34:39.080
mentioned the optimization that they're doing on our behalf is driven by machine learning
link |
01:34:44.320
you know like predicting where the traffic will be predicting what products I'm going
link |
01:34:48.040
to like predicting what would make me happy in my newsfeed.
link |
01:34:52.220
Now in terms of the stability and the promise of that I have to ask just out of curiosity
link |
01:34:56.720
how stable are these mechanisms that you game theory is just the economist came up with
link |
01:35:02.600
and we all know that economists don't live in the real world just kidding sort of what's
link |
01:35:08.040
do you think when we look at the fact that we haven't blown ourselves up from the from
link |
01:35:15.720
a game theoretic concept of mutually shared destruction what are the odds that we destroy
link |
01:35:21.000
ourselves with nuclear weapons as one example of a stable game theoretic system?
link |
01:35:28.400
Just to prime your viewers a little bit I mean I think you're referring to the fact
link |
01:35:32.080
that game theory was taken quite seriously back in the 60s as a tool for reasoning about
link |
01:35:38.400
kind of Soviet US nuclear armament disarmament detente things like that.
link |
01:35:45.160
I'll be honest as huge of a fan as I am of game theory and its kind of rich history it
link |
01:35:52.320
still surprises me that you know you had people at the RAND Corporation back in those days
link |
01:35:57.700
kind of drawing up you know two by two tables and one the row player is you know the US
link |
01:36:02.800
and the column player is Russia and that they were taking seriously you know I'm sure if
link |
01:36:08.240
I was there maybe it wouldn't have seemed as naive as it does at the time you know.
link |
01:36:12.840
Seems to have worked which is why it seems naive.
link |
01:36:15.440
Well we're still here.
link |
01:36:16.440
We're still here in that sense.
link |
01:36:17.960
Yeah even though I kind of laugh at those efforts they were more sensible then than
link |
01:36:22.600
they would be now right because there were sort of only two nuclear powers at the time
link |
01:36:26.540
and you didn't have to worry about deterring new entrants and who was developing the capacity
link |
01:36:32.480
and so we have many you know it's definitely a game with more players now and more potential
link |
01:36:39.120
entrants.
link |
01:36:40.120
I'm not in general somebody who advocates using kind of simple mathematical models when
link |
01:36:46.200
the stakes are as high as things like that and the complexities are very political and
link |
01:36:51.840
social but we are still here.
link |
01:36:55.760
So you've worn many hats one of which the one that first caused me to become a big fan
link |
01:37:00.600
of your work many years ago is algorithmic trading.
link |
01:37:04.460
So I have to just ask a question about this because you have so much fascinating work
link |
01:37:08.520
there in the 21st century what role do you think algorithms have in space of trading
link |
01:37:15.820
investment in the financial sector?
link |
01:37:19.080
Yeah it's a good question I mean in the time I've spent on Wall Street and in finance you
link |
01:37:27.160
know I've seen a clear progression and I think it's a progression that kind of models the
link |
01:37:31.320
use of algorithms and automation more generally in society which is you know the things that
link |
01:37:38.640
kind of get taken over by the algos first are sort of the things that computers are
link |
01:37:44.240
obviously better at than people right so you know so first of all there needed to be this
link |
01:37:50.320
era of automation right where just you know financial exchanges became largely electronic
link |
01:37:56.200
which then enabled the possibility of you know trading becoming more algorithmic because
link |
01:38:01.720
once you know that exchanges are electronic an algorithm can submit an order through an
link |
01:38:06.720
API just as well as a human can do at a monitor quickly can read all the data so yeah and
link |
01:38:11.800
so you know I think the places where algorithmic trading have had the greatest inroads and
link |
01:38:18.800
had the first inroads were in kind of execution problems kind of optimized execution problems
link |
01:38:24.560
so what I mean by that is at a large brokerage firm for example one of the lines of business
link |
01:38:30.440
might be on behalf of large institutional clients taking you know what we might consider
link |
01:38:36.320
difficult trade so it's not like a mom and pop investor saying I want to buy a hundred
link |
01:38:40.280
shares of Microsoft it's a large hedge fund saying you know I want to buy a very very
link |
01:38:45.940
large stake in Apple and I want to do it over the span of a day and it's such a large volume
link |
01:38:52.760
that if you're not clever about how you break that trade up not just over time but over
link |
01:38:57.260
perhaps multiple different electronic exchanges that all let you trade Apple on their platform
link |
01:39:02.560
you know you will you will move you'll push prices around in a way that hurts your your
link |
01:39:07.760
execution so you know this is the kind of you know this is an optimization problem this
link |
01:39:11.600
is a control problem right and so machines are better we we know how to design algorithms
link |
01:39:19.800
you know that are better at that kind of thing than a person is going to be able to do because
link |
01:39:23.600
we can take volumes of historical and real time data to kind of optimize the schedule
link |
01:39:29.320
with which we trade and you know similarly high frequency trading you know which is closely
link |
01:39:35.080
related but not the same as optimized execution where you're just trying to spot very very
link |
01:39:41.480
temporary you know mispricings between exchanges or within an asset itself or just predict
link |
01:39:48.520
directional movement of a stock because of the kind of very very low level granular buying
link |
01:39:54.800
and selling data in the in the exchange machines are good at this kind of stuff it's kind of
link |
01:40:00.440
like the mechanics of trading what about the can machines do long terms of prediction yeah
link |
01:40:08.080
so I think we are in an era where you know clearly there have been some very successful
link |
01:40:13.280
you know quant hedge funds that are you know in what we would traditionally call you know
link |
01:40:19.760
still in this the stat arb regime like so you know what's that stat arb referring to
link |
01:40:24.520
statistical arbitrage but but for the purposes of this conversation what it really means
link |
01:40:28.920
is making directional predictions in asset price movement or returns your prediction
link |
01:40:35.840
about that directional movement is good for you know you you have a view that it's valid
link |
01:40:42.100
for some period of time between a few seconds and a few days and that's the amount of time
link |
01:40:48.440
that you're going to kind of get into the position hold it and then hopefully be right
link |
01:40:51.920
about the directional movement and you know buy low and sell high as the cliche goes.
link |
01:40:57.360
So that is a you know kind of a sweet spot I think for quant trading and investing right
link |
01:41:04.300
now and has been for some time when you really get to kind of more Warren Buffett style timescales
link |
01:41:11.920
right like you know my cartoon of Warren Buffett is that you know Warren Buffett sits and thinks
link |
01:41:16.800
what the long term value of Apple really should be and he doesn't even look at what Apple
link |
01:41:22.360
is doing today he just decides you know you know I think that this is what its long term
link |
01:41:27.400
value is and it's far from that right now and so I'm going to buy some Apple or you
link |
01:41:31.960
know short some Apple and I'm going to I'm going to sit on that for 10 or 20 years okay.
link |
01:41:37.880
So when you're at that kind of timescale or even more than just a few days all kinds of
link |
01:41:45.600
other sources of risk and information you know so now you're talking about holding things
link |
01:41:51.600
through recessions and economic cycles, wars can break out.
link |
01:41:56.080
So there you have to understand human nature at a level that.
link |
01:41:59.080
Yeah and you need to just be able to ingest many many more sources of data that are on
link |
01:42:03.820
wildly different timescales right.
link |
01:42:06.380
So if I'm an HFT I'm a high frequency trader like I don't I don't I really my main source
link |
01:42:13.120
of data is just the data from the exchanges themselves about the activity in the exchanges
link |
01:42:18.000
right and maybe I need to pay you know I need to keep an eye on the news right because you
link |
01:42:22.880
know that can cause sudden you know the CEO gets caught in a scandal or you know gets
link |
01:42:29.040
run over by a bus or something that can cause very sudden changes but you know I don't need
link |
01:42:33.720
to understand economic cycles I don't need to understand recessions I don't need to worry
link |
01:42:38.480
about the political situation or war breaking out in this part of the world because you
link |
01:42:43.600
know all I need to know is as long as that's not going to happen in the next 500 milliseconds
link |
01:42:49.720
then you know my model is good.
link |
01:42:52.440
When you get to these longer timescales you really have to worry about that kind of stuff
link |
01:42:55.760
and people in the machine learning community are starting to think about this.
link |
01:42:59.280
We held a we jointly sponsored a workshop at Penn with the Federal Reserve Bank of Philadelphia
link |
01:43:06.760
a little more than a year ago on you know I think the title is something like machine
link |
01:43:10.960
learning for macroeconomic prediction.
link |
01:43:14.120
You know macroeconomic referring specifically to these longer timescales and you know it
link |
01:43:19.440
was an interesting conference but it you know my it left me with greater confidence that
link |
01:43:26.800
we have a long way to go to you know and so I think that people that you know in the grand
link |
01:43:32.440
scheme of things you know if somebody asked me like well whose job on Wall Street is safe
link |
01:43:37.480
from the bots I think people that are at that longer you know timescale and have that appetite
link |
01:43:42.840
for all the risks involved in long term investing and that really need kind of not just algorithms
link |
01:43:49.320
that can optimize from data but they need views on stuff they need views on the political
link |
01:43:54.480
landscape economic cycles and the like and I think you know they're they're they're pretty
link |
01:44:01.000
safe for a while as far as I can tell.
link |
01:44:02.680
So Warren Buffett's job is not seeing you know a robo Warren Buffett anytime soon.
link |
01:44:08.320
Give him comfort.
link |
01:44:10.080
Last question.
link |
01:44:11.160
If you could go back to if there's a day in your life you could relive because it made
link |
01:44:18.320
you truly happy.
link |
01:44:21.280
Maybe you outside family what otherwise you know what what what day would it be.
link |
01:44:29.100
But can you look back you remember just being profoundly transformed in some way or blissful.
link |
01:44:40.720
I'll answer a slightly different question which is like what's a day in my my life or
link |
01:44:44.840
my career that was kind of a watershed moment.
link |
01:44:49.040
I went straight from undergrad to doctoral studies and you know that's not at all atypical
link |
01:44:55.760
and I'm also from an academic family like my my dad was a professor my uncle on his
link |
01:45:00.440
side as a professor both my grandfathers were professors.
link |
01:45:03.440
All kinds of majors to philosophy.
link |
01:45:05.640
Yeah they're kind of all over the map yeah and I was a grad student here just up the
link |
01:45:10.820
river at Harvard and came to study with Les Valiant which was a wonderful experience.
link |
01:45:15.840
But you know I remember my first year of graduate school I was generally pretty unhappy and
link |
01:45:21.600
I was unhappy because you know at Berkeley as an undergraduate you know yeah I studied
link |
01:45:25.720
a lot of math and computer science but it was a huge school first of all and I took
link |
01:45:29.960
a lot of other courses as we've discussed I started as an English major and took history
link |
01:45:34.020
courses and art history classes and had friends you know that did all kinds of different things.
link |
01:45:40.200
And you know Harvard's a much smaller institution than Berkeley and its computer science department
link |
01:45:44.840
especially at that time was was a much smaller place than it is now.
link |
01:45:48.720
And I suddenly just felt very you know like I'd gone from this very big world to this
link |
01:45:54.000
highly specialized world and now all of the classes I was taking were computer science
link |
01:45:59.400
classes and I was only in classes with math and computer science people.
link |
01:46:04.600
And so I was you know I thought often in that first year of grad school about whether I
link |
01:46:09.960
really wanted to stick with it or not and you know I thought like oh I could you know
link |
01:46:14.760
stop with a master's I could go back to the Bay Area and to California and you know this
link |
01:46:19.860
was in one of the early periods where there was you know like you could definitely get
link |
01:46:23.660
a relatively good job paying job at one of the one of the tech companies back you know
link |
01:46:28.680
that were the big tech companies back then.
link |
01:46:31.440
And so I distinctly remember like kind of a late spring day when I was kind of you know
link |
01:46:36.880
sitting in Boston Common and kind of really just kind of chewing over what I wanted to
link |
01:46:40.440
do with my life and I realized like okay and I think this is where my academic background
link |
01:46:45.220
helped me a great deal.
link |
01:46:46.220
I sort of realized you know yeah you're not having a great time right now this feels really
link |
01:46:50.420
narrowing but you know that you're here for research eventually and to do something original
link |
01:46:56.320
and to try to you know carve out a career where you kind of you know choose what you
link |
01:47:02.320
want to think about you know and have a great deal of independence.
link |
01:47:06.260
And so you know at that point I really didn't have any real research experience yet I mean
link |
01:47:10.920
it was trying to think about some problems with very little success but I knew that like
link |
01:47:15.840
I hadn't really tried to do the thing that I knew I'd come to do and so I thought you
link |
01:47:23.320
know I'm going to stick through it for the summer and you know and that was very formative
link |
01:47:30.080
because I went from kind of contemplating quitting to you know a year later it being
link |
01:47:37.160
very clear to me I was going to finish because I still had a ways to go but I kind of started
link |
01:47:42.360
doing research it was going well it was really interesting and it was sort of a complete
link |
01:47:46.520
transformation you know it's just that transition that I think every doctoral student makes
link |
01:47:52.400
at some point which is to sort of go from being like a student of what's been done before
link |
01:48:00.040
to doing you know your own thing and figure out what makes you interested in what your
link |
01:48:04.120
strengths and weaknesses are as a researcher and once you know I kind of made that decision
link |
01:48:09.280
on that particular day at that particular moment in Boston Common you know I'm glad
link |
01:48:15.120
I made that decision.
link |
01:48:16.240
And also just accepting the painful nature of that journey.
link |
01:48:19.400
Yeah exactly exactly.
link |
01:48:21.400
In that moment said I'm gonna I'm gonna stick it out yeah I'm gonna stick around for a while.
link |
01:48:26.880
Well Michael I've looked off do you work for a long time it's really nice to talk to you
link |
01:48:30.880
thank you so much.
link |
01:48:31.880
It's great to get back in touch with you too and see how great you're doing as well.
link |
01:48:34.360
Thanks a lot.
link |
01:48:35.360
Thank you.