back to indexMichael Kearns: Algorithmic Fairness, Privacy & Ethics | Lex Fridman Podcast #50
link |
The following is a conversation with Michael Kearns.
link |
He's a professor at the University of Pennsylvania
link |
and a coauthor of the new book, Ethical Algorithm,
link |
that is the focus of much of this conversation.
link |
It includes algorithmic fairness, bias, privacy,
link |
and ethics in general.
link |
But that is just one of many fields
link |
that Michael is a world class researcher in,
link |
some of which we touch on quickly,
link |
including learning theory
link |
or the theoretical foundation of machine learning,
link |
game theory, quantitative finance,
link |
computational social science, and much more.
link |
But on a personal note,
link |
when I was an undergrad, early on,
link |
I worked with Michael
link |
on an algorithmic trading project
link |
and competition that he led.
link |
That's when I first fell in love
link |
with algorithmic game theory.
link |
While most of my research life
link |
has been in machine learning
link |
and human robot interaction,
link |
the systematic way that game theory
link |
reveals the beautiful structure
link |
in our competitive and cooperating world of humans
link |
has been a continued inspiration to me.
link |
So for that and other things,
link |
I'm deeply thankful to Michael
link |
and really enjoyed having this conversation
link |
again in person after so many years.
link |
This is the Artificial Intelligence Podcast.
link |
If you enjoy it, subscribe on YouTube,
link |
give it five stars on Apple Podcast,
link |
support on Patreon,
link |
or simply connect with me on Twitter
link |
at Lex Friedman, spelled F R I D M A N.
link |
This episode is supported
link |
by an amazing podcast called Pessimists Archive.
link |
Jason, the host of the show,
link |
reached out to me looking to support this podcast,
link |
and so I listened to it, to check it out.
link |
And by listened, I mean I went through it,
link |
Netflix binge style, at least five episodes in a row.
link |
It's not one of my favorite podcasts,
link |
and I think it should be one of the top podcasts
link |
in the world, frankly.
link |
It's a history show
link |
about why people resist new things.
link |
Each episode looks at a moment in history
link |
when something new was introduced,
link |
something that today we think of as commonplace,
link |
like recorded music, umbrellas, bicycles, cars,
link |
chess, coffee, the elevator,
link |
and the show explores why it freaked everyone out.
link |
The latest episode on mirrors and vanity
link |
still stays with me as I think about vanity
link |
in the modern day of the Twitter world.
link |
That's the fascinating thing about the show,
link |
is that stuff that happened long ago,
link |
especially in terms of our fear of new things,
link |
repeats itself in the modern day,
link |
and so has many lessons for us to think about
link |
in terms of human psychology
link |
and the role of technology in our society.
link |
Anyway, you should subscribe
link |
and listen to Pessimist Archive.
link |
I highly recommend it.
link |
And now, here's my conversation with Michael Kearns.
link |
You mentioned reading Fear and Loathing in Las Vegas
link |
in high school, and having a more,
link |
or a bit more of a literary mind.
link |
So, what books, non technical, non computer science,
link |
would you say had the biggest impact on your life,
link |
either intellectually or emotionally?
link |
You've dug deep into my history, I see.
link |
Yeah, I think, well, my favorite novel is
link |
Infinite Jest by David Foster Wallace,
link |
which actually, coincidentally,
link |
much of it takes place in the halls of buildings
link |
right around us here at MIT.
link |
So that certainly had a big influence on me.
link |
And as you noticed, like, when I was in high school,
link |
I actually even started college as an English major.
link |
So, I was very influenced by sort of that genre of journalism
link |
at the time, and thought I wanted to be a writer,
link |
and then realized that an English major teaches you to read,
link |
but it doesn't teach you how to write,
link |
and then I became interested in math
link |
and computer science instead.
link |
Well, in your new book, Ethical Algorithm,
link |
you kind of sneak up from an algorithmic perspective
link |
on these deep, profound philosophical questions
link |
of fairness, of privacy.
link |
In thinking about these topics,
link |
how often do you return to that literary mind that you had?
link |
Yeah, I'd like to claim there was a deeper connection,
link |
but, you know, I think both Aaron and I
link |
kind of came at these topics first and foremost
link |
from a technical angle.
link |
I mean, you know, I kind of consider myself primarily
link |
and originally a machine learning researcher,
link |
and I think as we just watched, like the rest of the society,
link |
the field technically advance, and then quickly on the heels
link |
of that kind of the buzzkill of all of the antisocial behavior
link |
by algorithms, just kind of realized
link |
there was an opportunity for us to do something about it
link |
from a research perspective.
link |
You know, more to the point in your question,
link |
I mean, I do have an uncle who is literally a moral philosopher,
link |
and so in the early days of my life,
link |
he was a philosopher, and so in the early days
link |
of our technical work on fairness topics,
link |
I would occasionally, you know, run ideas behind him.
link |
So, I mean, I remember an early email I sent to him
link |
in which I said, like, oh, you know,
link |
here's a specific definition of algorithmic fairness
link |
that we think is some sort of variant of Rawlsian fairness.
link |
What do you think?
link |
And I thought I was asking a yes or no question,
link |
and I got back your kind of classical philosopher's
link |
response saying, well, it depends.
link |
Hey, then you might conclude this, and that's when I realized
link |
that there was a real kind of rift between the ways
link |
philosophers and others had thought about things
link |
like fairness, you know, from sort of a humanitarian perspective
link |
and the way that you needed to think about it
link |
as a computer scientist if you were going to kind of
link |
implement actual algorithmic solutions.
link |
But I would say the algorithmic solutions take care
link |
of some of the low hanging fruit.
link |
Sort of the problem is a lot of algorithms,
link |
when they don't consider fairness,
link |
they are just terribly unfair.
link |
And when they don't consider privacy,
link |
they're terribly, they violate privacy.
link |
Sort of the algorithmic approach fixes big problems.
link |
But there's still, when you start pushing into the gray area,
link |
that's when you start getting into this philosophy
link |
of what it means to be fair, starting from Plato,
link |
what is justice kind of questions?
link |
Yeah, I think that's right.
link |
And I mean, I would even not go as far as you want to say
link |
that sort of the algorithmic work in these areas
link |
is solving like the biggest problems.
link |
And, you know, we discuss in the book,
link |
the fact that really we are, there's a sense in which
link |
we're kind of looking where the light is in that,
link |
you know, for example, if police are racist
link |
in who they decide to stop and frisk,
link |
and that goes into the data,
link |
there's sort of no undoing that downstream
link |
by kind of clever algorithmic methods.
link |
And I think, especially in fairness,
link |
I mean, I think less so in privacy,
link |
where we feel like the community kind of really has settled
link |
on the right definition, which is differential privacy.
link |
If you just look at the algorithmic fairness literature
link |
already, you can see it's going to be much more of a mess.
link |
And, you know, you've got these theorems saying,
link |
here are three entirely reasonable,
link |
desirable notions of fairness.
link |
And, you know, here's a proof that you cannot simultaneously
link |
have all three of them.
link |
So I think we know that algorithmic fairness
link |
compared to algorithmic privacy
link |
is going to be kind of a harder problem.
link |
And it will have to revisit, I think,
link |
things that have been thought about by,
link |
you know, many generations of scholars before us.
link |
So it's very early days for fairness, I think.
link |
TK So before we get into the details
link |
of differential privacy, and on the fairness side,
link |
let me linger on the philosophy a bit.
link |
Do you think most people are fundamentally good?
link |
Or do most of us have both the capacity
link |
for good and evil within us?
link |
SB I mean, I'm an optimist.
link |
I tend to think that most people are good
link |
and want to do right.
link |
And that deviations from that are, you know,
link |
kind of usually due to circumstance,
link |
not due to people being bad at heart.
link |
TK With people with power,
link |
are people at the heads of governments,
link |
people at the heads of companies,
link |
people at the heads of, maybe, so financial power markets,
link |
do you think the distribution there is also,
link |
most people are good and have good intent?
link |
I mean, my statement wasn't qualified to people
link |
not in positions of power.
link |
I mean, I think what happens in a lot of the, you know,
link |
the cliche about absolute power corrupts absolutely.
link |
I mean, you know, I think even short of that,
link |
you know, having spent a lot of time on Wall Street,
link |
and also in arenas very, very different from Wall Street,
link |
like academia, you know, one of the things
link |
I think I've benefited from by moving between
link |
two very different worlds is you become aware
link |
that, you know, these worlds kind of develop
link |
their own social norms, and they develop
link |
their own rationales for, you know,
link |
behavior, for instance, that might look
link |
unusual to outsiders.
link |
But when you're in that world,
link |
it doesn't feel unusual at all.
link |
And I think this is true of a lot of,
link |
you know, professional cultures, for instance.
link |
And, you know, so then your maybe slippery slope
link |
is too strong of a word.
link |
But, you know, you're in some world
link |
where you're mainly around other people
link |
with the same kind of viewpoints and training
link |
and worldview as you.
link |
And I think that's more of a source of,
link |
of, you know, kind of abuses of power
link |
than sort of, you know, there being good people
link |
and evil people, and that somehow the evil people
link |
are the ones that somehow rise to power.
link |
Oh, that's really interesting.
link |
So it's the, within the social norms
link |
constructed by that particular group of people,
link |
you're all trying to do good.
link |
But because as a group, you might be,
link |
you might drift into something
link |
that for the broader population,
link |
it does not align with the values of society.
link |
That kind of, that's the word.
link |
Yeah, I mean, or not that you drift,
link |
but even the things that don't make sense
link |
to the outside world don't seem unusual to you.
link |
So it's not sort of like a good or a bad thing,
link |
but, you know, like, so for instance,
link |
you know, on, in the world of finance, right?
link |
There's a lot of complicated types of activity
link |
that if you are not immersed in that world,
link |
you cannot see why the purpose of that,
link |
you know, that activity exists at all.
link |
It just seems like, you know, completely useless
link |
and people just like, you know, pushing money around.
link |
And when you're in that world, right,
link |
you're, and you learn more,
link |
your view does become more nuanced, right?
link |
You realize, okay, there is actually a function
link |
And in some cases, you would conclude that actually,
link |
if magically we could eradicate this activity tomorrow,
link |
it would come back because it actually is like
link |
serving some useful purpose.
link |
It's just a useful purpose that's very difficult
link |
for outsiders to see.
link |
And so I think, you know, lots of professional work
link |
environments or cultures, as I might put it,
link |
kind of have these social norms that, you know,
link |
don't make sense to the outside world.
link |
Academia is the same, right?
link |
I mean, lots of people look at academia and say,
link |
you know, what the hell are all of you people doing?
link |
Why are you paid so much in some cases
link |
at taxpayer expenses to do, you know,
link |
to publish papers that nobody reads?
link |
You know, but when you're in that world,
link |
you come to see the value for it.
link |
And, but even though you might not be able to explain it
link |
to, you know, the person in the street.
link |
And in the case of the financial sector,
link |
tools like credit might not make sense to people.
link |
Like, it's a good example of something that does seem
link |
to pop up and be useful or just the power of markets
link |
and just in general capitalism.
link |
In finance, I think the primary example
link |
I would give is leverage, right?
link |
So being allowed to borrow, to sort of use ten times
link |
as much money as you've actually borrowed, right?
link |
So that's an example of something that before I had
link |
any experience in financial markets,
link |
I might have looked at and said,
link |
well, what is the purpose of that?
link |
That just seems very dangerous and it is dangerous
link |
and it has proven dangerous.
link |
But, you know, if the fact of the matter is that,
link |
you know, sort of on some particular time scale,
link |
you are holding positions that are,
link |
you know, very unlikely to, you know,
link |
lose, you know, your value at risk or variance
link |
is like one or five percent, then it kind of makes sense
link |
that you would be allowed to use a little bit more
link |
than you have because you have, you know,
link |
some confidence that you're not going to lose
link |
it all in a single day.
link |
Now, of course, when that happens,
link |
we've seen what happens, you know, not too long ago.
link |
But, you know, but the idea that it serves
link |
no useful economic purpose under any circumstances
link |
is definitely not true.
link |
We'll return to the other side of the coast,
link |
Silicon Valley, and the problems there as we talk about privacy,
link |
as we talk about fairness.
link |
At the high level, and I'll ask some sort of basic questions
link |
with the hope to get at the fundamental nature of reality.
link |
But from a very high level, what is an ethical algorithm?
link |
So I can say that an algorithm has a running time
link |
of using big O notation n log n.
link |
I can say that a machine learning algorithm
link |
classified cat versus dog with 97 percent accuracy.
link |
Do you think there will one day be a way to measure
link |
sort of in the same compelling way as the big O notation
link |
of this algorithm is 97 percent ethical?
link |
First of all, let me riff for a second on your specific n log n example.
link |
So because early in the book when we're just kind of trying to describe
link |
algorithms period, we say like, okay, you know,
link |
what's an example of an algorithm or an algorithmic problem?
link |
First of all, like it's sorting, right?
link |
You have a bunch of index cards with numbers on them
link |
and you want to sort them.
link |
And we describe, you know, an algorithm that sweeps all the way through,
link |
finds the smallest number, puts it at the front,
link |
then sweeps through again, finds the second smallest number.
link |
So we make the point that this is an algorithm
link |
and it's also a bad algorithm in the sense that, you know,
link |
it's quadratic rather than n log n,
link |
which we know is kind of optimal for sorting.
link |
And we make the point that sort of like, you know,
link |
so even within the confines of a very precisely specified problem,
link |
there, you know, there might be many, many different algorithms
link |
for the same problem with different properties.
link |
Like some might be faster in terms of running time,
link |
some might use less memory, some might have, you know,
link |
better distributed implementations.
link |
And so the point is that already we're used to, you know,
link |
in computer science thinking about trade offs
link |
between different types of quantities and resources
link |
and there being, you know, better and worse algorithms.
link |
And our book is about that part of algorithmic ethics
link |
that we know how to kind of put on that same kind of quantitative footing right now.
link |
So, you know, just to say something that our book is not about,
link |
our book is not about kind of broad, fuzzy notions of fairness.
link |
It's about very specific notions of fairness.
link |
There's more than one of them.
link |
There are tensions between them, right?
link |
But if you pick one of them, you can do something akin to saying
link |
that this algorithm is 97% ethical.
link |
You can say, for instance, the, you know, for this lending model,
link |
the false rejection rate on black people and white people is within 3%, right?
link |
So we might call that a 97% ethical algorithm and a 100% ethical algorithm
link |
would mean that that difference is 0%.
link |
In that case, fairness is specified when two groups, however,
link |
they're defined are given to you.
link |
So the, and then you can sort of mathematically start describing the algorithm.
link |
But nevertheless, the part where the two groups are given to you,
link |
I mean, unlike running time, you know, we don't in computer science
link |
talk about how fast an algorithm feels like when it runs.
link |
We measure it and ethical starts getting into feelings.
link |
So, for example, an algorithm runs, you know, if it runs in the background,
link |
it doesn't disturb the performance of my system.
link |
I'll be okay with it.
link |
But if it overloads the system, it'll feel unpleasant.
link |
So in that same way, ethics, there's a feeling of how socially acceptable it is.
link |
How does it represent the moral standards of our society today?
link |
So in that sense, and sorry to linger on that first of high,
link |
low philosophical questions.
link |
Do you have a sense we'll be able to measure how ethical an algorithm is?
link |
First of all, I didn't, certainly didn't mean to give the impression that you can kind of
link |
measure, you know, memory speed trade offs, you know, and that there's a complete mapping from
link |
that onto kind of fairness, for instance, or ethics and accuracy, for example.
link |
In the type of fairness definitions that are largely the objects of study today and starting
link |
to be deployed, you as the user of the definitions, you need to make some hard decisions before you
link |
even get to the point of designing fair algorithms.
link |
One of them, for instance, is deciding who it is that you're worried about protecting,
link |
who you're worried about being harmed by, for instance, some notion of discrimination or
link |
And then you need to also decide what constitutes harm.
link |
So, for instance, in a lending application, maybe you decide that, you know, falsely rejecting
link |
a creditworthy individual, you know, sort of a false negative, is the real harm and that false
link |
positives, i.e. people that are not creditworthy or are not gonna repay your loan, that get a loan,
link |
you might think of them as lucky.
link |
And so that's not a harm, although it's not clear that if you don't have the means to repay a loan,
link |
that being given a loan is not also a harm.
link |
So, you know, the literature is sort of so far quite limited in that you sort of need to say,
link |
who do you want to protect and what would constitute harm to that group?
link |
And when you ask questions like, will algorithms feel ethical?
link |
One way in which they won't, under the definitions that I'm describing, is if, you know, if you are
link |
an individual who is falsely denied a loan, incorrectly denied a loan, all of these definitions
link |
basically say like, well, you know, your compensation is the knowledge that we are also
link |
falsely denying loans to other people, you know, in other groups at the same rate that we're doing
link |
And, you know, and so there is actually this interesting even technical tension in the field
link |
right now between these sort of group notions of fairness and notions of fairness that might
link |
actually feel like real fairness to individuals, right?
link |
They might really feel like their particular interests are being protected or thought about
link |
by the algorithm rather than just, you know, the groups that they happen to be members of.
link |
Is there parallels to the big O notation of worst case analysis?
link |
So, is it important to looking at the worst violation of fairness for an individual?
link |
Is it important to minimize that one individual?
link |
So like worst case analysis, is that something you think about or?
link |
I mean, I think we're not even at the point where we can sensibly think about that.
link |
So first of all, you know, we're talking here both about fairness applied at the group level,
link |
which is a relatively weak thing, but it's better than nothing.
link |
And also the more ambitious thing of trying to give some individual promises, but even
link |
that doesn't incorporate, I think something that you're hinting at here is what I might
link |
call subjective fairness, right?
link |
So a lot of the definitions, I mean, all of the definitions in the algorithmic fairness
link |
literature are what I would kind of call received wisdom definitions.
link |
It's sort of, you know, somebody like me sits around and things like, okay, you know, I
link |
think here's a technical definition of fairness that I think people should want or that they
link |
should, you know, think of as some notion of fairness, maybe not the only one, maybe
link |
not the best one, maybe not the last one.
link |
But we really actually don't know from a subjective standpoint, like what people really
link |
You know, we just started doing a little bit of work in our group at actually doing kind
link |
of human subject experiments in which we, you know, ask people about, you know, we ask
link |
them questions about fairness, we survey them, we, you know, we show them pairs of individuals
link |
in, let's say, a criminal recidivism prediction setting, and we ask them, do you think these
link |
two individuals should be treated the same as a matter of fairness?
link |
And to my knowledge, there's not a large literature in which ordinary people are asked
link |
about, you know, they have sort of notions of their subjective fairness elicited from
link |
It's mainly, you know, kind of scholars who think about fairness kind of making up their
link |
And I think this needs to change actually for many social norms, not just for fairness,
link |
So there's a lot of discussion these days in the AI community about interpretable AI
link |
or understandable AI.
link |
And as far as I can tell, everybody agrees that deep learning or at least the outputs
link |
of deep learning are not very understandable, and people might agree that sparse linear
link |
models with integer coefficients are more understandable.
link |
But nobody's really asked people.
link |
You know, there's very little literature on, you know, sort of showing people models
link |
and asking them, do they understand what the model is doing?
link |
And I think that in all these topics, as these fields mature, we need to start doing more
link |
Yeah, which is one of my deep passions is psychology.
link |
And I always thought computer scientists will be the best future psychologists in a sense
link |
that data is, especially in this modern world, the data is a really powerful way to understand
link |
and study human behavior.
link |
And you've explored that with your game theory side of work as well.
link |
Yeah, I'd like to think that what you say is true about computer scientists and psychology
link |
from my own limited wandering into human subject experiments.
link |
We have a great deal to learn, not just computer science, but AI and machine learning more
link |
specifically, I kind of think of as imperialist research communities in that, you know, kind
link |
of like physicists in an earlier generation, computer scientists kind of don't think of
link |
any scientific topic that's off limits to them.
link |
They will like freely wander into areas that others have been thinking about for decades
link |
And, you know, we usually tend to embarrass ourselves in those efforts for some amount
link |
Like, you know, I think reinforcement learning is a good example, right?
link |
So a lot of the early work in reinforcement learning, I have complete sympathy for the
link |
control theorists that looked at this and said like, okay, you are reinventing stuff
link |
that we've known since like the forties, right?
link |
But, you know, in my view, eventually this sort of, you know, computer scientists have
link |
made significant contributions to that field, even though we kind of embarrassed ourselves
link |
for the first decade.
link |
So I think if computer scientists are gonna start engaging in kind of psychology, human
link |
subjects type of research, we should expect to be embarrassing ourselves for a good 10
link |
years or so, and then hope that it turns out as well as, you know, some other areas that
link |
So you kind of mentioned this, just to linger on the idea of an ethical algorithm, of idea
link |
of groups, sort of group thinking and individual thinking.
link |
And we're struggling that.
link |
One of the amazing things about algorithms and your book and just this field of study
link |
is it gets us to ask, like forcing machines, converting these ideas into algorithms is
link |
forcing us to ask questions of ourselves as a human civilization.
link |
So there's a lot of people now in public discourse doing sort of group thinking, thinking like
link |
there's particular sets of groups that we don't wanna discriminate against and so on.
link |
And then there is individuals, sort of in the individual life stories, the struggles
link |
they went through and so on.
link |
Now, like in philosophy, it's easier to do group thinking because you don't, it's very
link |
hard to think about individuals.
link |
There's so much variability, but with data, you can start to actually say, you know what
link |
group thinking is too crude.
link |
You're actually doing more discrimination by thinking in terms of groups and individuals.
link |
Can you linger on that kind of idea of group versus individual and ethics?
link |
And is it good to continue thinking in terms of groups in algorithms?
link |
So let me start by answering a very good high level question with a slightly narrow technical
link |
response, which is these group definitions of fairness, like here's a few groups, like
link |
different racial groups, maybe gender groups, maybe age, what have you.
link |
And let's make sure that for none of these groups, do we have a false negative rate,
link |
which is much higher than any other one of these groups.
link |
Okay, so these are kind of classic group aggregate notions of fairness.
link |
And you know, but at the end of the day, an individual you can think of as a combination
link |
of all of their attributes, right?
link |
They're a member of a racial group, they have a gender, they have an age, and many other
link |
demographic properties that are not biological, but that are still very strong determinants
link |
of outcome and personality and the like.
link |
So one, I think, useful spectrum is to sort of think about that array between the group
link |
and the specific individual, and to realize that in some ways, asking for fairness at
link |
the individual level is to sort of ask for group fairness simultaneously for all possible
link |
combinations of groups.
link |
So in particular, you know, if I build a predictive model that meets some definition of fairness,
link |
definition of fairness by race, by gender, by age, by what have you, marginally, to get
link |
it slightly technical, sort of independently, I shouldn't expect that model to not discriminate
link |
against disabled Hispanic women over age 55, making less than $50,000 a year annually,
link |
even though I might have protected each one of those attributes marginally.
link |
So the optimization, actually, that's a fascinating way to put it.
link |
So you're just optimizing, the one way to achieve the optimizing fairness for individuals
link |
is just to add more and more definitions of groups that each individual belongs to.
link |
So, you know, at the end of the day, we could think of all of ourselves as groups of size
link |
one because eventually there's some attribute that separates you from me and everybody else
link |
in the world, okay?
link |
And so it is possible to put, you know, these incredibly coarse ways of thinking about fairness
link |
and these very, very individualistic specific ways on a common scale.
link |
And you know, one of the things we've worked on from a research perspective is, you know,
link |
so we sort of know how to, you know, in relative terms, we know how to provide fairness guarantees
link |
at the core system of the scale.
link |
We don't know how to provide kind of sensible, tractable, realistic fairness guarantees at
link |
the individual level, but maybe we could start creeping towards that by dealing with more
link |
refined subgroups.
link |
I mean, we gave a name to this phenomenon where, you know, you protect, you enforce
link |
some definition of fairness for a bunch of marginal attributes or features, but then
link |
you find yourself discriminating against a combination of them.
link |
We call that fairness gerrymandering because like political gerrymandering, you know, you're
link |
giving some guarantee at the aggregate level, but when you kind of look in a more granular
link |
way at what's going on, you realize that you're achieving that aggregate guarantee by sort
link |
of favoring some groups and discriminating against other ones.
link |
And so there are, you know, it's early days, but there are algorithmic approaches that
link |
let you start creeping towards that, you know, individual end of the spectrum.
link |
Does there need to be human input in the form of weighing the value of the importance of
link |
each kind of group?
link |
So for example, is it like, so gender, say crudely speaking, male and female, and then
link |
different races, are we as humans supposed to put value on saying gender is 0.6 and race
link |
is 0.4 in terms of in the big optimization of achieving fairness?
link |
Is that kind of what humans are supposed to do here?
link |
I mean, of course, you know, I don't need to tell you that, of course, technically one
link |
could incorporate such weights if you wanted to into a definition of fairness.
link |
You know, fairness is an interesting topic in that having worked in the book being about
link |
both fairness, privacy, and many other social norms, fairness, of course, is a much, much
link |
more loaded topic.
link |
So privacy, I mean, people want privacy, people don't like violations of privacy, violations
link |
of privacy cause damage, angst, and bad publicity for the companies that are victims of them.
link |
But sort of everybody agrees more data privacy would be better than less data privacy.
link |
And you don't have these, somehow the discussions of fairness don't become politicized along
link |
other dimensions like race and about gender and, you know, whether we, and, you know,
link |
you quickly find yourselves kind of revisiting topics that have been kind of unresolved forever,
link |
like affirmative action, right?
link |
Sort of, you know, like, why are you protecting, and some people will say, why are you protecting
link |
this particular racial group?
link |
And others will say, well, we need to do that as a matter of retribution.
link |
Other people will say, it's a matter of economic opportunity.
link |
And I don't know which of, you know, whether any of these are the right answers, but you
link |
sort of, fairness is sort of special in that as soon as you start talking about it, you
link |
inevitably have to participate in debates about fair to whom, at what expense to who
link |
I mean, even in criminal justice, right, you know, where people talk about fairness in
link |
criminal sentencing or, you know, predicting failures to appear or making parole decisions
link |
or the like, they will, you know, they'll point out that, well, these definitions of
link |
fairness are all about fairness for the criminals.
link |
And what about fairness for the victims, right?
link |
So when I basically say something like, well, the false incarceration rate for black people
link |
and white people needs to be roughly the same, you know, there's no mention of potential
link |
victims of criminals in such a fairness definition.
link |
And that's the realm of public discourse.
link |
I should actually recommend, I just listened to people listening, Intelligence Squares
link |
debates, US edition just had a debate.
link |
They have this structure where you have old Oxford style or whatever they're called, debates,
link |
you know, it's two versus two and they talked about affirmative action and it was incredibly
link |
interesting that there's really good points on every side of this issue, which is fascinating
link |
Yeah, yeah, I agree.
link |
And so it's interesting to be a researcher trying to do, for the most part, technical
link |
algorithmic work, but Aaron and I both quickly learned you cannot do that and then go out
link |
and talk about it and expect people to take it seriously if you're unwilling to engage
link |
in these broader debates that are entirely extra algorithmic, right?
link |
They're not about, you know, algorithms and making algorithms better.
link |
They're sort of, you know, as you said, sort of like, what should society be protecting
link |
in the first place?
link |
When you discuss the fairness, an algorithm that achieves fairness, whether in the constraints
link |
and the objective function, there's an immediate kind of analysis you can perform, which is
link |
saying, if you care about fairness in gender, this is the amount that you have to pay for
link |
it in terms of the performance of the system.
link |
Like do you, is there a role for statements like that in a table, in a paper, or do you
link |
want to really not touch that?
link |
No, no, we want to touch that and we do touch it.
link |
So I mean, just again, to make sure I'm not promising your viewers more than we know how
link |
to provide, but if you pick a definition of fairness, like I'm worried about gender discrimination
link |
and you pick a notion of harm, like false rejection for a loan, for example, and you
link |
give me a model, I can definitely, first of all, go audit that model.
link |
It's easy for me to go, you know, from data to kind of say like, okay, your false rejection
link |
rate on women is this much higher than it is on men, okay?
link |
But once you also put the fairness into your objective function, I mean, I think the table
link |
that you're talking about is what we would call the Pareto curve, right?
link |
You can literally trace out, and we give examples of such plots on real data sets in the book,
link |
you have two axes.
link |
On the X axis is your error, on the Y axis is unfairness by whatever, you know, if it's
link |
like the disparity between false rejection rates between two groups.
link |
And you know, your algorithm now has a knob that basically says, how strongly do I want
link |
to enforce fairness?
link |
And the less unfair, you know, if the two axes are error and unfairness, we'd like to
link |
We'd like zero error and zero unfairness simultaneously.
link |
Anybody who works in machine learning knows that you're generally not going to get to
link |
zero error period without any fairness constraint whatsoever.
link |
So that's not going to happen.
link |
But in general, you know, you'll get this, you'll get some kind of convex curve that
link |
specifies the numerical trade off you face.
link |
You know, if I want to go from 17% error down to 16% error, what will be the increase in
link |
unfairness that I experienced as a result of that?
link |
And so this curve kind of specifies the, you know, kind of undominated models.
link |
Models that are off that curve are, you know, can be strictly improved in one or both dimensions.
link |
You can, you know, either make the error better or the unfairness better or both.
link |
And I think our view is that not only are these objects, these Pareto curves, you know,
link |
with efficient frontiers as you might call them, not only are they valuable scientific
link |
objects, I actually think that they in the near term might need to be the interface between
link |
researchers working in the field and stakeholders in given problems.
link |
So you know, you could really imagine telling a criminal jurisdiction, look, if you're concerned
link |
about racial fairness, but you're also concerned about accuracy.
link |
You want to, you know, you want to release on parole people that are not going to recommit
link |
a violent crime and you don't want to release the ones who are.
link |
So you know, that's accuracy.
link |
But if you also care about those, you know, the mistakes you make not being disproportionately
link |
on one racial group or another, you can show this curve.
link |
I'm hoping that in the near future, it'll be possible to explain these curves to non
link |
technical people that are the ones that have to make the decision, where do we want to
link |
Like, what are the relative merits or value of having lower error versus lower unfairness?
link |
You know, that's not something computer scientists should be deciding for society, right?
link |
That, you know, the people in the field, so to speak, the policymakers, the regulators,
link |
that's who should be making these decisions.
link |
But I think and hope that they can be made to understand that these trade offs generally
link |
exist and that you need to pick a point and like, and ignoring the trade off, you know,
link |
you're implicitly picking a point anyway, right?
link |
You just don't know it and you're not admitting it.
link |
Just to linger on the point of trade offs, I think that's a really important thing to
link |
sort of think about.
link |
So you think when we start to optimize for fairness, there's almost always in most system
link |
going to be trade offs.
link |
Can you like, what's the trade off between just to clarify, there have been some sort
link |
of technical terms thrown around, but sort of a perfectly fair world.
link |
Why will somebody be upset about that?
link |
The specific trade off I talked about just in order to make things very concrete was
link |
between numerical error and some numerical measure of unfairness.
link |
What is numerical error in the case of...
link |
Just like say predictive error, like, you know, the probability or frequency with which
link |
you release somebody on parole who then goes on to recommit a violent crime or keep incarcerated
link |
somebody who would not have recommitted a violent crime.
link |
So in the case of awarding somebody parole or giving somebody parole or letting them
link |
out on parole, you don't want them to recommit a crime.
link |
So it's your system failed in prediction if they happen to do a crime.
link |
Okay, so that's one axis.
link |
And what's the fairness axis?
link |
So then the fairness axis might be the difference between racial groups in the kind of false
link |
positive predictions, namely people that I kept incarcerated predicting that they would
link |
recommit a violent crime when in fact they wouldn't have.
link |
And the unfairness of that, just to linger it and allow me to in eloquently to try to
link |
sort of describe why that's unfair, why unfairness is there.
link |
The unfairness you want to get rid of is that in the judge's mind, the bias of having being
link |
brought up to society, the slight racial bias, the racism that exists in the society, you
link |
want to remove that from the system.
link |
Another way that's been debated is sort of equality of opportunity versus equality of
link |
And there's a weird dance there that's really difficult to get right.
link |
And we don't, affirmative action is exploring that space.
link |
And then this also quickly bleeds into questions like, well, maybe if one group really does
link |
recommit crimes at a higher rate, the reason for that is that at some earlier point in
link |
the pipeline or earlier in their lives, they didn't receive the same resources that the
link |
And so there's always in kind of fairness discussions, the possibility that the real
link |
injustice came earlier, right?
link |
Earlier in this individual's life, earlier in this group's history, et cetera, et cetera.
link |
And so a lot of the fairness discussion is almost, the goal is for it to be a corrective
link |
mechanism to account for the injustice earlier in life.
link |
By some definitions of fairness or some theories of fairness, yeah.
link |
Others would say like, look, it's not to correct that injustice, it's just to kind of level
link |
the playing field right now and not falsely incarcerate more people of one group than
link |
But I mean, I think just it might be helpful just to demystify a little bit about the many
link |
ways in which bias or unfairness can come into algorithms, especially in the machine
link |
learning era, right?
link |
I think many of your viewers have probably heard these examples before, but let's say
link |
I'm building a face recognition system, right?
link |
And so I'm kind of gathering lots of images of faces and trying to train the system to
link |
recognize new faces of those individuals from training on a training set of those faces
link |
And it shouldn't surprise anybody or certainly not anybody in the field of machine learning
link |
if my training data set was primarily white males and I'm training the model to maximize
link |
the overall accuracy on my training data set, that the model can reduce its error most by
link |
getting things right on the white males that constitute the majority of the data set, even
link |
if that means that on other groups, they will be less accurate, okay?
link |
Now, there's a bunch of ways you could think about addressing this.
link |
One is to deliberately put into the objective of the algorithm not to optimize the error
link |
at the expense of this discrimination, and then you're kind of back in the land of these
link |
kind of two dimensional numerical trade offs.
link |
A valid counter argument is to say like, well, no, you don't have to, there's no, you know,
link |
the notion of the tension between error and accuracy here is a false one.
link |
You could instead just go out and get much more data on these other groups that are in
link |
the minority and, you know, equalize your data set, or you could train a separate model
link |
on those subgroups and, you know, have multiple models.
link |
The point I think we would, you know, we tried to make in the book is that those things have
link |
Going out and gathering more data on groups that are relatively rare compared to your
link |
plurality or more majority group that, you know, it may not cost you in the accuracy
link |
of the model, but it's going to cost, you know, it's going to cost the company developing
link |
this model more money to develop that, and it also costs more money to build separate
link |
predictive models and to implement and deploy them.
link |
So even if you can find a way to avoid the tension between error and accuracy in training
link |
a model, you might push the cost somewhere else, like money, like development time, research
link |
time and the like.
link |
There are fundamentally difficult philosophical questions, in fairness, and we live in a very
link |
divisive political climate, outraged culture.
link |
There is alt right folks on 4chan, trolls.
link |
There is social justice warriors on Twitter.
link |
There's very divisive, outraged folks on all sides of every kind of system.
link |
How do you, how do we as engineers build ethical algorithms in such divisive culture?
link |
Do you think they could be disjoint?
link |
The human has to inject your values, and then you can optimize over those values.
link |
But in our times, when you start actually applying these systems, things get a little
link |
bit challenging for the public discourse.
link |
How do you think we can proceed?
link |
Yeah, I mean, for the most part in the book, a point that we try to take some pains to
link |
make is that we don't view ourselves or people like us as being in the position of deciding
link |
for society what the right social norms are, what the right definitions of fairness are.
link |
Our main point is to just show that if society or the relevant stakeholders in a particular
link |
domain can come to agreement on those sorts of things, there's a way of encoding that
link |
into algorithms in many cases, not in all cases.
link |
One other misconception that hopefully we definitely dispel is sometimes people read
link |
the title of the book and I think not unnaturally fear that what we're suggesting is that the
link |
algorithms themselves should decide what those social norms are and develop their own notions
link |
of fairness and privacy or ethics, and we're definitely not suggesting that.
link |
The title of the book is Ethical Algorithm, by the way, and I didn't think of that interpretation
link |
That's interesting.
link |
I mean, especially these days where people are concerned about the robots becoming our
link |
overlords, the idea that the robots would also sort of develop their own social norms
link |
is just one step away from that.
link |
But I do think, obviously, despite disclaimer that people like us shouldn't be making those
link |
decisions for society, we are kind of living in a world where in many ways computer scientists
link |
have made some decisions that have fundamentally changed the nature of our society and democracy
link |
and sort of civil discourse and deliberation in ways that I think most people generally
link |
feel are bad these days, right?
link |
But they had to make, so if we look at people at the heads of companies and so on, they
link |
had to make those decisions, right?
link |
There has to be decisions, so there's two options, either you kind of put your head
link |
in the sand and don't think about these things and just let the algorithm do what it does,
link |
or you make decisions about what you value, you know, of injecting moral values into the
link |
Look, I never mean to be an apologist for the tech industry, but I think it's a little
link |
bit too far to sort of say that explicit decisions were made about these things.
link |
So let's, for instance, take social media platforms, right?
link |
So like many inventions in technology and computer science, a lot of these platforms
link |
that we now use regularly kind of started as curiosities, right?
link |
I remember when things like Facebook came out and its predecessors like Friendster,
link |
which nobody even remembers now, people really wonder, like, why would anybody want to spend
link |
I mean, even the web when it first came out, when it wasn't populated with much content
link |
and it was largely kind of hobbyists building their own kind of ramshackle websites, a lot
link |
of people looked at this and said, well, what is the purpose of this thing?
link |
Why is this interesting?
link |
Who would want to do this?
link |
And so even things like Facebook and Twitter, yes, technical decisions were made by engineers,
link |
by scientists, by executives in the design of those platforms, but, you know, I don't
link |
think 10 years ago anyone anticipated that those platforms, for instance, might kind
link |
of acquire undue, you know, influence on political discourse or on the outcomes of elections.
link |
And I think the scrutiny that these companies are getting now is entirely appropriate, but
link |
I think it's a little too harsh to kind of look at history and sort of say like, oh,
link |
you should have been able to anticipate that this would happen with your platform.
link |
And in this sort of gaming chapter of the book, one of the points we're making is that,
link |
you know, these platforms, right, they don't operate in isolation.
link |
So unlike the other topics we're discussing, like fairness and privacy, like those are
link |
really cases where algorithms can operate on your data and make decisions about you
link |
and you're not even aware of it, okay?
link |
Things like Facebook and Twitter, these are, you know, these are systems, right?
link |
These are social systems and their evolution, even their technical evolution because machine
link |
learning is involved, is driven in no small part by the behavior of the users themselves
link |
and how the users decide to adopt them and how to use them.
link |
And so, you know, I'm kind of like who really knew that, you know, until we saw it happen,
link |
who knew that these things might be able to influence the outcome of elections?
link |
Who knew that, you know, they might polarize political discourse because of the ability
link |
to, you know, decide who you interact with on the platform and also with the platform
link |
naturally using machine learning to optimize for your own interest that they would further
link |
isolate us from each other and, you know, like feed us all basically just the stuff
link |
that we already agreed with.
link |
So I think, you know, we've come to that outcome, I think, largely, but I think it's
link |
something that we all learned together, including the companies as these things happen.
link |
You asked like, well, are there algorithmic remedies to these kinds of things?
link |
And again, these are big problems that are not going to be solved with, you know, somebody
link |
going in and changing a few lines of code somewhere in a social media platform.
link |
But I do think in many ways, there are definitely ways of making things better.
link |
I mean, like an obvious recommendation that we make at some point in the book is like,
link |
look, you know, to the extent that we think that machine learning applied for personalization
link |
purposes in things like newsfeed, you know, or other platforms has led to polarization
link |
and intolerance of opposing viewpoints.
link |
As you know, right, these algorithms have models, right, and they kind of place people
link |
in some kind of metric space, and they place content in that space, and they sort of know
link |
the extent to which I have an affinity for a particular type of content.
link |
And by the same token, they also probably have that same model probably gives you a
link |
good idea of the stuff I'm likely to violently disagree with or be offended by, okay?
link |
So you know, in this case, there really is some knob you could tune that says like, instead
link |
of showing people only what they like and what they want, let's show them some stuff
link |
that we think that they don't like, or that's a little bit further away.
link |
And you could even imagine users being able to control this, you know, just like everybody
link |
gets a slider, and that slider says like, you know, how much stuff do you want to see
link |
that's kind of, you know, you might disagree with, or is at least further from your interest.
link |
It's almost like an exploration button.
link |
So just get your intuition.
link |
Do you think engagement, so like you staying on the platform, you're staying engaged.
link |
Do you think fairness, ideas of fairness won't emerge?
link |
Like how bad is it to just optimize for engagement?
link |
Do you think we'll run into big trouble if we're just optimizing for how much you love
link |
Well, I mean, optimizing for engagement kind of got us where we are.
link |
So do you, one, have faith that it's possible to do better?
link |
And two, if it is, how do we do better?
link |
I mean, it's definitely possible to do different, right?
link |
And again, you know, it's not as if I think that doing something different than optimizing
link |
for engagement won't cost these companies in real ways, including revenue and profitability
link |
In the short term at least.
link |
In the short term.
link |
And again, you know, if I worked at these companies, I'm sure that it would have seemed
link |
like the most natural thing in the world also to want to optimize engagement, right?
link |
And that's good for users in some sense.
link |
You want them to be, you know, vested in the platform and enjoying it and finding it useful,
link |
interesting, and or productive.
link |
But you know, my point is, is that the idea that there is, that it's sort of out of their
link |
hands as you said, or that there's nothing to do about it, never say never, but that
link |
strikes me as implausible as a machine learning person, right?
link |
I mean, these companies are driven by machine learning and this optimization of engagement
link |
is essentially driven by machine learning, right?
link |
It's driven by not just machine learning, but you know, very, very large scale A, B
link |
experimentation where you kind of tweak some element of the user interface or tweak some
link |
component of an algorithm or tweak some component or feature of your click through prediction
link |
And my point is, is that anytime you know how to optimize for something, you, you know,
link |
by def, almost by definition, that solution tells you how not to optimize for it or to
link |
do something different.
link |
Engagement can be measured.
link |
So sort of optimizing for sort of minimizing divisiveness or maximizing intellectual growth
link |
over the lifetime of a human being are very difficult to measure.
link |
And I'm not claiming that doing something different will immediately make it apparent
link |
that this is a good thing for society and in particular, I mean, I think one way of
link |
thinking about where we are on some of these social media platforms is that, you know,
link |
it kind of feels a bit like we're in a bad equilibrium, right?
link |
That these systems are helping us all kind of optimize something myopically and selfishly
link |
for ourselves and of course, from an individual standpoint at any given moment, like why would
link |
I want to see things in my newsfeed that I found irrelevant, offensive or, you know,
link |
or the like, okay?
link |
But you know, maybe by all of us, you know, having these platforms myopically optimized
link |
in our interests, we have reached a collective outcome as a society that we're unhappy with
link |
in different ways.
link |
Let's say with respect to things like, you know, political discourse and tolerance of
link |
opposing viewpoints.
link |
And if Mark Zuckerberg gave you a call and said, I'm thinking of taking a sabbatical,
link |
could you run Facebook for me for six months?
link |
What would you, how?
link |
I think no thanks would be my first response, but there are many aspects of being the head
link |
of the entire company that are kind of entirely exogenous to many of the things that we're
link |
And so I don't really think I would need to be CEO of Facebook to kind of implement the,
link |
you know, more limited set of solutions that I might imagine.
link |
But I think one concrete thing they could do is they could experiment with letting people
link |
who chose to, to see more stuff in their newsfeed that is not entirely kind of chosen to optimize
link |
for their particular interests, beliefs, et cetera.
link |
So the, the kind of thing, so I could speak to YouTube, but I think Facebook probably
link |
does something similar is they're quite effective at automatically finding what sorts of groups
link |
you belong to, not based on race or gender or so on, but based on the kind of stuff you
link |
enjoy watching in the case of YouTube.
link |
Sort of, it's a, it's a difficult thing for Facebook or YouTube to then say, well, you
link |
We're going to show you something from a very different cluster.
link |
Even though we believe algorithmically, you're unlikely to enjoy that thing sort of that's
link |
a weird jump to make.
link |
There has to be a human, like at the very top of that system that says, well, that will
link |
be longterm healthy for you.
link |
That's more than an algorithmic decision.
link |
Or that same person could say that'll be longterm healthy for the platform or for the platform's
link |
influence on society outside of the platform, right?
link |
And it, you know, it's easy for me to sit here and say these things, but conceptually
link |
I do not think that these are kind of totally or should, they shouldn't be kind of completely
link |
alien ideas, right?
link |
That, you know, you could try things like this and it wouldn't be, you know, we wouldn't
link |
have to invent entirely new science to do it because if we're all already embedded in
link |
some metric space and there's a notion of distance between you and me and every other,
link |
every piece of content, then, you know, we know exactly, you know, the same model that
link |
tells, you know, dictates how to make me really happy also tells how to make me as unhappy
link |
as possible as well.
link |
The focus in your book and algorithmic fairness research today in general is on machine learning,
link |
like we said, is data, but, and just even the entire AI field right now is captivated
link |
with machine learning, with deep learning.
link |
Do you think ideas in symbolic AI or totally other kinds of approaches are interesting,
link |
useful in the space, have some promising ideas in terms of fairness?
link |
I haven't thought about that question specifically in the context of fairness.
link |
I definitely would agree with that statement in the large, right?
link |
I mean, I am, you know, one of many machine learning researchers who do believe that the
link |
great successes that have been shown in machine learning recently are great successes, but
link |
they're on a pretty narrow set of tasks.
link |
I mean, I don't, I don't think we're kind of notably closer to general artificial intelligence
link |
now than we were when I started my career.
link |
I mean, there's been progress and I do think that we are kind of as a community, maybe
link |
looking a bit where the light is, but the light is shining pretty bright there right
link |
now and we're finding a lot of stuff.
link |
So I don't want to like argue with the progress that's been made in areas like deep learning,
link |
This touches another sort of related thing that you mentioned and that people might misinterpret
link |
from the title of your book, ethical algorithm.
link |
Is it possible for the algorithm to automate some of those decisions?
link |
Sort of a higher level decisions of what kind of, like what, what should be fair, what should
link |
The more you know about a field, the more aware you are of its limitations.
link |
And so I'm a, I'm pretty leery of sort of trying, you know, there's, there's so much
link |
we don't all, we already don't know in fairness, even when we're the ones picking the fairness
link |
definitions and, you know, comparing alternatives and thinking about the tensions between different
link |
definitions that the idea of kind of letting the algorithm start exploring as well.
link |
I definitely think, you know, this is a much narrower statement.
link |
I definitely think that kind of algorithmic auditing for different types of unfairness,
link |
So like in this gerrymandering example where I might want to prevent not just discrimination
link |
against very broad categories, but against combinations of broad categories.
link |
You know, you quickly get to a point where there's a lot of, a lot of categories.
link |
There's a lot of combinations of end features and, you know, you can use algorithmic techniques
link |
to sort of try to find the subgroups on which you're discriminating the most and try to
link |
That's actually kind of the form of one of the algorithms we developed for this fairness
link |
gerrymandering problem.
link |
But I'm, I'm, you know, partly because of our technological, you know, our sort of our
link |
scientific ignorance on these topics right now.
link |
And also partly just because these topics are so loaded emotionally for people that
link |
I just don't see the value.
link |
I mean, again, never say never, but I just don't think we're at a moment where it's
link |
a great time for computer scientists to be rolling out the idea like, hey, you know,
link |
you know, not only have we kind of figured fairness out, but, you know, we think the
link |
algorithm should start deciding what's fair or giving input on that decision.
link |
I just don't, it's like the cost benefit analysis to the field of kind of going there
link |
right now just doesn't seem worth it to me.
link |
That said, I should say that I think computer scientists should be more philosophically,
link |
like should enrich their thinking about these kinds of things.
link |
I think it's been too often used as an excuse for roboticists working on autonomous vehicles,
link |
for example, to not think about the human factor or psychology or safety in the same
link |
way like computer science design algorithms that have been sort of using it as an excuse.
link |
And I think it's time for basically everybody to become a computer scientist.
link |
I was about to agree with everything you said except that last point.
link |
I think that the other way of looking at it is that I think computer scientists, you know,
link |
and many of us are, but we need to weigh it out into the world more, right?
link |
I mean, just the influence that computer science and therefore computer scientists have had
link |
on society at large just like has exponentially magnified in the last 10 or 20 years or so.
link |
And you know, before when we were just tinkering around amongst ourselves and it didn't matter
link |
that much, there was no need for sort of computer scientists to be citizens of the world more
link |
And I think those days need to be over very, very fast.
link |
And I'm not saying everybody needs to do it, but to me, like the right way of doing it
link |
is to not to sort of think that everybody else is going to become a computer scientist.
link |
But you know, I think people are becoming more sophisticated about computer science,
link |
You know, I think one of the reasons we decided to write this book is we thought 10 years
link |
ago I wouldn't have tried this just because I just didn't think that sort of people's
link |
awareness of algorithms and machine learning, you know, the general population would have
link |
I mean, you would have had to first, you know, write one of the many books kind of just explicating
link |
that topic to a lay audience first.
link |
Now I think we're at the point where like lots of people without any technical training
link |
at all know enough about algorithms and machine learning that you can start getting to these
link |
nuances of things like ethical algorithms.
link |
I think we agree that there needs to be much more mixing, but I think a lot of the onus
link |
of that mixing needs to be on the computer science community.
link |
So just to linger on the disagreement, because I do disagree with you on the point that I
link |
think if you're a biologist, if you're a chemist, if you're an MBA business person, all of those
link |
things you can, like if you learned a program, and not only program, if you learned to do
link |
machine learning, if you learned to do data science, you immediately become much more
link |
powerful in the kinds of things you can do.
link |
And therefore literature, like library sciences, like, so you were speaking, I think, I think
link |
it holds true what you're saying for the next few years.
link |
But long term, if you're interested to me, if you're interested in philosophy, you should
link |
learn a program, because then you can scrape data and study what people are thinking about
link |
on Twitter, and then start making philosophical conclusions about the meaning of life.
link |
I just feel like the access to data, the digitization of whatever problem you're trying to solve,
link |
will fundamentally change what it means to be a computer scientist.
link |
I mean, a computer scientist in 20, 30 years will go back to being Donald Knuth style theoretical
link |
computer science, and everybody would be doing basically, exploring the kinds of ideas that
link |
you explore in your book.
link |
It won't be a computer science major.
link |
Yeah, I mean, I don't think I disagree enough, but I think that that trend of more and more
link |
people in more and more disciplines adopting ideas from computer science, learning how
link |
to code, I think that that trend seems firmly underway.
link |
I mean, you know, like an interesting digressive question along these lines is maybe in 50
link |
years, there won't be computer science departments anymore, because the field will just sort
link |
of be ambient in all of the different disciplines.
link |
And people will look back and having a computer science department will look like having an
link |
electricity department or something that's like, you know, everybody uses this, it's
link |
I mean, I do think there will always be that kind of Knuth style core to it, but it's not
link |
an implausible path that we kind of get to the point where the academic discipline of
link |
computer science becomes somewhat marginalized because of its very success in kind of infiltrating
link |
all of science and society and the humanities, etcetera.
link |
What is differential privacy, or more broadly, algorithmic privacy?
link |
Algorithmic privacy more broadly is just the study or the notion of privacy definitions
link |
or norms being encoded inside of algorithms.
link |
And so, you know, I think we count among this body of work just, you know, the literature
link |
and practice of things like data anonymization, which we kind of at the beginning of our discussion
link |
of privacy say like, okay, this is sort of a notion of algorithmic privacy.
link |
It kind of tells you, you know, something to go do with data, but, you know, our view
link |
is that it's, and I think this is now, you know, quite widespread, that it's, you know,
link |
despite the fact that those notions of anonymization kind of redacting and coarsening are the most
link |
widely adopted technical solutions for data privacy, they are like deeply fundamentally
link |
And so, you know, to your first question, what is differential privacy?
link |
Differential privacy seems to be a much, much better notion of privacy that kind of avoids
link |
a lot of the weaknesses of anonymization notions while still letting us do useful stuff with
link |
What is anonymization of data?
link |
So by anonymization, I'm, you know, kind of referring to techniques like I have a database.
link |
The rows of that database are, let's say, individual people's medical records, okay?
link |
And I want to let people use that data.
link |
Maybe I want to let researchers access that data to build predictive models for some disease,
link |
but I'm worried that that will leak, you know, sensitive information about specific people's
link |
So anonymization broadly refers to the set of techniques where I say like, okay, I'm
link |
first going to like, I'm going to delete the column with people's names.
link |
I'm going to not put, you know, so that would be like a redaction, right?
link |
I'm just redacting that information.
link |
I am going to take ages and I'm not going to like say your exact age.
link |
I'm going to say whether you're, you know, zero to 10, 10 to 20, 20 to 30, I might put
link |
the first three digits of your zip code, but not the last two, et cetera, et cetera.
link |
And so the idea is that through some series of operations like this on the data, I anonymize
link |
You know, another term of art that's used is removing personally identifiable information.
link |
And you know, this is basically the most common way of providing data privacy, but that it's
link |
in a way that still lets people access the, some variant form of the data.
link |
So at a slightly broader picture, as you talk about what does anonymization mean when you
link |
have multiple database, like with a Netflix prize, when you can start combining stuff
link |
So this is exactly the problem with these notions, right?
link |
Is that notions of a anonymization, removing personally identifiable information, the kind
link |
of fundamental conceptual flaw is that, you know, these definitions kind of pretend as
link |
if the data set in question is the only data set that exists in the world or that ever
link |
will exist in the future.
link |
And of course, things like the Netflix prize and many, many other examples since the Netflix
link |
prize, I think that was one of the earliest ones though, you know, you can reidentify
link |
people that were, you know, that were anonymized in the data set by taking that anonymized
link |
data set and combining it with other allegedly anonymized data sets and maybe publicly available
link |
information about you.
link |
for people who don't know the Netflix prize was, was being publicly released this data.
link |
So the names from those rows were removed, but what was released is the preference or
link |
the ratings of what movies you like and you don't like.
link |
And from that combined with other things, I think forum posts and so on, you can start
link |
I guess it was specifically the internet movie database where, where lots of Netflix users
link |
publicly rate their movie, you know, their movie preferences.
link |
And so the anonymized data and Netflix, when it's just this phenomenon, I think that we've
link |
all come to realize in the last decade or so is that just knowing a few apparently irrelevant
link |
innocuous things about you can often act as a fingerprint.
link |
Like if I know, you know, what, what rating you gave to these 10 movies and the date on
link |
which you entered these movies, this is almost like a fingerprint for you in the sea of all
link |
There were just another paper on this in science or nature of about a month ago that, you know,
link |
kind of 18 attributes.
link |
I mean, my favorite example of this is, was actually a paper from several years ago now
link |
where it was shown that just from your likes on Facebook, just from the time, you know,
link |
the things on which you clicked on the thumbs up button on the platform, not using any information,
link |
demographic information, nothing about who your friends are, just knowing the content
link |
that you had liked was enough to, you know, in the aggregate accurately predict things
link |
like sexual orientation, drug and alcohol use, whether you were the child of divorced parents.
link |
So we live in this era where, you know, even the apparently irrelevant data that we offer
link |
about ourselves on public platforms and forums often unbeknownst to us, more or less acts
link |
as signature or, you know, fingerprint.
link |
And that if you can kind of, you know, do a join between that kind of data and allegedly
link |
anonymized data, you have real trouble.
link |
So is there hope for any kind of privacy in a world where a few likes can identify you?
link |
So there is differential privacy, right?
link |
What is differential privacy?
link |
Yeah, so differential privacy basically is a kind of alternate, much stronger notion
link |
of privacy than these anonymization ideas.
link |
And, you know, it's a technical definition, but like the spirit of it is we compare two
link |
alternate worlds, okay?
link |
So let's suppose I'm a researcher and I want to do, you know, there's a database of medical
link |
records and one of them is yours, and I want to use that database of medical records to
link |
build a predictive model for some disease.
link |
So based on people's symptoms and test results and the like, I want to, you know, build a
link |
probably model predicting the probability that people have disease.
link |
So, you know, this is the type of scientific research that we would like to be allowed
link |
And in differential privacy, you ask a very particular counterfactual question.
link |
We basically compare two alternatives.
link |
One is when I do this, I build this model on the database of medical records, including
link |
your medical record.
link |
And the other one is where I do the same exercise with the same database with just your medical
link |
So basically, you know, it's two databases, one with N records in it and one with N minus
link |
one records in it.
link |
The N minus one records are the same, and the only one that's missing in the second
link |
case is your medical record.
link |
So differential privacy basically says that any harms that might come to you from the
link |
analysis in which your data was included are essentially nearly identical to the harms
link |
that would have come to you if the same analysis had been done without your medical record
link |
So in other words, this doesn't say that bad things cannot happen to you as a result of
link |
It just says that these bad things were going to happen to you already, even if your data
link |
And to give a very concrete example, right, you know, like we discussed at some length,
link |
the study that, you know, in the 50s that was done that established the link between
link |
smoking and lung cancer.
link |
And we make the point that, like, well, if your data was used in that analysis and, you
link |
know, the world kind of knew that you were a smoker because, you know, there was no stigma
link |
associated with smoking before those findings, real harm might have come to you as a result
link |
of that study that your data was included in.
link |
In particular, your insurer now might have a higher posterior belief that you might have
link |
lung cancer and raise your premium.
link |
So you've suffered economic damage.
link |
But the point is, is that if the same analysis has been done with all the other N minus one
link |
medical records and just yours missing, the outcome would have been the same.
link |
Or your data wasn't idiosyncratically crucial to establishing the link between smoking and
link |
lung cancer because the link between smoking and lung cancer is like a fact about the world
link |
that can be discovered with any sufficiently large database of medical records.
link |
But that's a very low value of harm.
link |
So that's showing that very little harm is done.
link |
But how what is the mechanism of differential privacy?
link |
So that's the kind of beautiful statement of it.
link |
It's the mechanism by which privacy is preserved.
link |
So it's basically by adding noise to computations, right?
link |
So the basic idea is that every differentially private algorithm, first of all, or every
link |
good differentially private algorithm, every useful one, is a probabilistic algorithm.
link |
So it doesn't, on a given input, if you gave the algorithm the same input multiple times,
link |
it would give different outputs each time from some distribution.
link |
And the way you achieve differential privacy algorithmically is by kind of carefully and
link |
tastefully adding noise to a computation in the right places.
link |
And to give a very concrete example, if I wanna compute the average of a set of numbers,
link |
the non private way of doing that is to take those numbers and average them and release
link |
like a numerically precise value for the average.
link |
In differential privacy, you wouldn't do that.
link |
You would first compute that average to numerical precisions, and then you'd add some noise
link |
You'd add some kind of zero mean, Gaussian or exponential noise to it so that the actual
link |
value you output is not the exact mean, but it'll be close to the mean, but it'll be close...
link |
The noise that you add will sort of prove that nobody can kind of reverse engineer any
link |
particular value that went into the average.
link |
So noise is a savior.
link |
How many algorithms can be aided by adding noise?
link |
Yeah, so I'm a relatively recent member of the differential privacy community.
link |
My co author, Aaron Roth is really one of the founders of the field and has done a great
link |
deal of work and I've learned a tremendous amount working with him on it.
link |
It's a pretty grown up field already.
link |
Yeah, but now it's pretty mature.
link |
But I must admit, the first time I saw the definition of differential privacy, my reaction
link |
was like, wow, that is a clever definition and it's really making very strong promises.
link |
And I first saw the definition in much earlier days and my first reaction was like, well,
link |
my worry about this definition would be that it's a great definition of privacy, but that
link |
it'll be so restrictive that we won't really be able to use it.
link |
We won't be able to compute many things in a differentially private way.
link |
So that's one of the great successes of the field, I think, is in showing that the opposite
link |
is true and that most things that we know how to compute, absent any privacy considerations,
link |
can be computed in a differentially private way.
link |
So for example, pretty much all of statistics and machine learning can be done differentially
link |
So pick your favorite machine learning algorithm, back propagation and neural networks, cart
link |
for decision trees, support vector machines, boosting, you name it, as well as classic
link |
hypothesis testing and the like in statistics.
link |
None of those algorithms are differentially private in their original form.
link |
All of them have modifications that add noise to the computation in different places in
link |
different ways that achieve differential privacy.
link |
So this really means that to the extent that we've become a scientific community very dependent
link |
on the use of machine learning and statistical modeling and data analysis, we really do have
link |
a path to provide privacy guarantees to those methods and so we can still enjoy the benefits
link |
of the data science era while providing rather robust privacy guarantees to individuals.
link |
So perhaps a slightly crazy question, but if we take the ideas of differential privacy
link |
and take it to the nature of truth that's being explored currently.
link |
So what's your most favorite and least favorite food?
link |
I'm not a real foodie, so I'm a big fan of spaghetti.
link |
What do you really don't like?
link |
I really don't like cauliflower.
link |
Wow, I love cauliflower.
link |
Is there one way to protect your preference for spaghetti by having an information campaign
link |
bloggers and so on of bots saying that you like cauliflower?
link |
So like this kind of the same kind of noise ideas, I mean if you think of in our politics
link |
today there's this idea of Russia hacking our elections.
link |
What's meant there I believe is bots spreading different kinds of information.
link |
Is that a kind of privacy or is that too much of a stretch?
link |
No it's not a stretch.
link |
I've not seen those ideas, you know, that is not a technique that to my knowledge will
link |
provide differential privacy, but to give an example like one very specific example
link |
about what you're discussing is there was a very interesting project at NYU I think
link |
led by Helen Nissenbaum there in which they basically built a browser plugin that tried
link |
to essentially obfuscate your Google searches.
link |
So to the extent that you're worried that Google is using your searches to build, you
link |
know, predictive models about you to decide what ads to show you which they might very
link |
reasonably want to do, but if you object to that they built this widget you could plug
link |
in and basically whenever you put in a query into Google it would send that query to Google,
link |
but in the background all of the time from your browser it would just be sending this
link |
torrent of irrelevant queries to the search engine.
link |
So you know it's like a weed and chaff thing so you know out of every thousand queries
link |
let's say that Google was receiving from your browser one of them was one that you put in
link |
but the other 999 were not okay so it's the same kind of idea kind of you know privacy
link |
So I think that's an interesting idea, doesn't give you differential privacy.
link |
It's also I was actually talking to somebody at one of the large tech companies recently
link |
about the fact that you know just this kind of thing that there are some times when the
link |
response to my data needs to be very specific to my data right like I type mountain biking
link |
into Google, I want results on mountain biking and I really want Google to know that I typed
link |
in mountain biking, I don't want noise added to that.
link |
And so I think there's sort of maybe even interesting technical questions around notions
link |
of privacy that are appropriate where you know it's not that my data is part of some
link |
aggregate like medical records and that we're trying to discover important correlations
link |
and facts about the world at large but rather you know there's a service that I really want
link |
to you know pay attention to my specific data yet I still want some kind of privacy guarantee
link |
and I think these kind of obfuscation ideas are sort of one way of getting at that but
link |
maybe there are others as well.
link |
So where do you think we'll land in this algorithm driven society in terms of privacy?
link |
So sort of China like Kai Fuli describes you know it's collecting a lot of data on its
link |
citizens but in the best form it's actually able to provide a lot of sort of protect human
link |
rights and provide a lot of amazing services and it's worst forms that can violate those
link |
human rights and limit services.
link |
So where do you think we'll land because algorithms are powerful when they use data.
link |
So as a society do you think we'll give over more data?
link |
Is it possible to protect the privacy of that data?
link |
So I'm optimistic about the possibility of you know balancing the desire for individual
link |
privacy and individual control of privacy with kind of societally and commercially beneficial
link |
uses of data not unrelated to differential privacy or suggestions that say like well
link |
individuals should have control of their data.
link |
They should be able to limit the uses of that data.
link |
They should even you know there's you know fledgling discussions going on in research
link |
circles about allowing people selective use of their data and being compensated for it.
link |
And then you get to sort of very interesting economic questions like pricing right.
link |
And one interesting idea is that maybe differential privacy would also you know be a conceptual
link |
framework in which you could talk about the relative value of different people's data
link |
like you know to demystify this a little bit.
link |
If I'm trying to build a predictive model for some rare disease and I'm trying to use
link |
machine learning to do it, it's easy to get negative examples because the disease is rare
link |
But I really want to have lots of people with the disease in my data set okay.
link |
And so somehow those people's data with respect to this application is much more valuable
link |
to me than just like the background population.
link |
And so maybe they should be compensated more for it.
link |
And so you know I think these are kind of very, very fledgling conceptual questions
link |
that maybe we'll have kind of technical thought on them sometime in the coming years.
link |
But I do think we'll you know to kind of get more directly answer your question.
link |
I think I'm optimistic at this point from what I've seen that we will land at some you
link |
know better compromise than we're at right now where again you know privacy guarantees
link |
are few far between and weak and users have very, very little control.
link |
And I'm optimistic that we'll land in something that you know provides better privacy overall
link |
and more individual control of data and privacy.
link |
But you know I think to get there it's again just like fairness it's not going to be enough
link |
to propose algorithmic solutions.
link |
There's going to have to be a whole kind of regulatory legal process that prods companies
link |
and other parties to kind of adopt solutions.
link |
And I think you've mentioned the word control a lot and I think giving people control that's
link |
something that people don't quite have in a lot of these algorithms and that's a really
link |
interesting idea of giving them control.
link |
Some of that is actually literally an interface design question sort of just enabling because
link |
I think it's good for everybody to give users control.
link |
It's almost not a trade off except that you have to hire people that are good at interface
link |
I mean the other thing that has to be said right is that you know it's a cliche but you
link |
know we as the users of many systems platforms and apps you know we are the product.
link |
We are not the customer.
link |
The customer are advertisers and our data is the product.
link |
So it's one thing to kind of suggest more individual control of data and privacy and
link |
uses but this you know if this happens in sufficient degree it will upend the entire
link |
economic model that has supported the internet to date.
link |
And so some other economic model will have to be you know we'll have to replace it.
link |
So the idea of markets you mentioned by exposing the economic model to the people they will
link |
then become a market.
link |
They could be participants in it.
link |
And you know this isn't you know this is not a weird idea right because there are markets
link |
It's just that consumers are not participants and there's like you know there's sort of
link |
you know publishers and content providers on one side that have inventory and then their
link |
advertisers on the others and you know you know Google and Facebook are running you know
link |
they're pretty much their entire revenue stream is by running two sided markets between those
link |
And so it's not a crazy idea that there would be like a three sided market or that you know
link |
that on one side of the market or the other we would have proxies representing our interest.
link |
It's not you know it's not a crazy idea but it would it's not a crazy technical idea but
link |
it would have pretty extreme economic consequences.
link |
Speaking of markets a lot of fascinating aspects of this world arise not from individual human
link |
beings but from the interaction of human beings.
link |
You've done a lot of work in game theory.
link |
First can you say what is game theory and how does it help us model and study?
link |
Yeah game theory of course let us give credit where it's due.
link |
You know it comes from the economist first and foremost but as I've mentioned before
link |
like you know computer scientists never hesitate to wander into other people's turf and so
link |
there is now this 20 year old field called algorithmic game theory.
link |
But you know game theory first and foremost is a mathematical framework for reasoning
link |
about collective outcomes in systems of interacting individuals.
link |
You know so you need at least two people to get started in game theory and many people
link |
are probably familiar with Prisoner's Dilemma as kind of a classic example of game theory
link |
and a classic example where everybody looking out for their own individual interests leads
link |
to a collective outcome that's kind of worse for everybody than what might be possible
link |
if they cooperated for example.
link |
But cooperation is not an equilibrium in Prisoner's Dilemma.
link |
And so my work in the field of algorithmic game theory more generally in these areas
link |
kind of looks at settings in which the number of actors is potentially extraordinarily large
link |
and their incentives might be quite complicated and kind of hard to model directly but you
link |
still want kind of algorithmic ways of kind of predicting what will happen or influencing
link |
what will happen in the design of platforms.
link |
So what to you is the most beautiful idea that you've encountered in game theory?
link |
There's a lot of them.
link |
I'm a big fan of the field.
link |
I mean you know I mean technical answers to that of course would include Nash's work just
link |
establishing that you know there is a competitive equilibrium under very very general circumstances
link |
which in many ways kind of put the field on a firm conceptual footing because if you don't
link |
have equilibrium it's kind of hard to ever reason about what might happen since you know
link |
there's just no stability.
link |
So just the idea that stability can emerge when there's multiple.
link |
Not that it will necessarily emerge just that it's possible right.
link |
Like the existence of equilibrium doesn't mean that sort of natural iterative behavior
link |
will necessarily lead to it.
link |
In the real world.
link |
Maybe answering a slightly less personally than you asked the question I think within
link |
the field of algorithmic game theory perhaps the single most important kind of technical
link |
contribution that's been made is the realization between close connections between machine
link |
learning and game theory and in particular between game theory and the branch of machine
link |
learning that's known as no regret learning and this sort of provides a very general framework
link |
in which a bunch of players interacting in a game or a system each one kind of doing
link |
something that's in their self interest will actually kind of reach an equilibrium and
link |
actually reach an equilibrium in a you know a pretty you know a rather you know short
link |
So you kind of mentioned acting greedily can somehow end up pretty good for everybody.
link |
It will end up stable.
link |
And and you know stability or equilibrium by itself is neither is not necessarily either
link |
a good thing or a bad thing.
link |
So what's the connection between machine learning and the ideas.
link |
Well I think we kind of talked about these ideas already in kind of a non technical way
link |
which is maybe the more interesting way of understanding them first which is you know
link |
we have many systems platforms and apps these days that work really hard to use our data
link |
and the data of everybody else on the platform to selfishly optimize on behalf of each user.
link |
So you know let me let me give I think the cleanest example which is just driving apps
link |
navigation apps like you know Google Maps and Waze where you know miraculously compared
link |
to when I was growing up at least you know the objective would be the same when you wanted
link |
to drive from point A to point B spend the least time driving not necessarily minimize
link |
the distance but minimize the time.
link |
And when I was growing up like the only resources you had to do that were like maps in the car
link |
which literally just told you what roads were available and then you might have like half
link |
hourly traffic reports just about the major freeways but not about side roads.
link |
So you were pretty much on your own.
link |
And now we've got these apps you pull it out and you say I want to go from point A to point
link |
B and in response kind of to what everybody else is doing if you like what all the other
link |
players in this game are doing right now here's the you know the route that minimizes your
link |
So it is really kind of computing a selfish best response for each of us in response to
link |
what all of the rest of us are doing at any given moment.
link |
And so you know I think it's quite fair to think of these apps as driving or nudging
link |
us all towards the competitive or Nash equilibrium of that game.
link |
Now you might ask like well that sounds great why is that a bad thing.
link |
Well you know it's known both in theory and with some limited studies from actual like
link |
traffic data that all of us being in this competitive equilibrium might cause our collective
link |
driving time to be higher maybe significantly higher than it would be under other solutions.
link |
And then you have to talk about what those other solutions might be and what the algorithms
link |
to implement them are which we do discuss in the kind of game theory chapter of the
link |
But similarly you know on social media platforms or on Amazon you know all these algorithms
link |
that are essentially trying to optimize our behalf they're driving us in a colloquial
link |
sense towards some kind of competitive equilibrium and you know one of the most important lessons
link |
of game theory is that just because we're at equilibrium doesn't mean that there's not
link |
a solution in which some or maybe even all of us might be better off.
link |
And then the connection to machine learning of course is that in all these platforms I've
link |
mentioned the optimization that they're doing on our behalf is driven by machine learning
link |
you know like predicting where the traffic will be predicting what products I'm going
link |
to like predicting what would make me happy in my newsfeed.
link |
Now in terms of the stability and the promise of that I have to ask just out of curiosity
link |
how stable are these mechanisms that you game theory is just the economist came up with
link |
and we all know that economists don't live in the real world just kidding sort of what's
link |
do you think when we look at the fact that we haven't blown ourselves up from the from
link |
a game theoretic concept of mutually shared destruction what are the odds that we destroy
link |
ourselves with nuclear weapons as one example of a stable game theoretic system?
link |
Just to prime your viewers a little bit I mean I think you're referring to the fact
link |
that game theory was taken quite seriously back in the 60s as a tool for reasoning about
link |
kind of Soviet US nuclear armament disarmament detente things like that.
link |
I'll be honest as huge of a fan as I am of game theory and its kind of rich history it
link |
still surprises me that you know you had people at the RAND Corporation back in those days
link |
kind of drawing up you know two by two tables and one the row player is you know the US
link |
and the column player is Russia and that they were taking seriously you know I'm sure if
link |
I was there maybe it wouldn't have seemed as naive as it does at the time you know.
link |
Seems to have worked which is why it seems naive.
link |
Well we're still here.
link |
We're still here in that sense.
link |
Yeah even though I kind of laugh at those efforts they were more sensible then than
link |
they would be now right because there were sort of only two nuclear powers at the time
link |
and you didn't have to worry about deterring new entrants and who was developing the capacity
link |
and so we have many you know it's definitely a game with more players now and more potential
link |
I'm not in general somebody who advocates using kind of simple mathematical models when
link |
the stakes are as high as things like that and the complexities are very political and
link |
social but we are still here.
link |
So you've worn many hats one of which the one that first caused me to become a big fan
link |
of your work many years ago is algorithmic trading.
link |
So I have to just ask a question about this because you have so much fascinating work
link |
there in the 21st century what role do you think algorithms have in space of trading
link |
investment in the financial sector?
link |
Yeah it's a good question I mean in the time I've spent on Wall Street and in finance you
link |
know I've seen a clear progression and I think it's a progression that kind of models the
link |
use of algorithms and automation more generally in society which is you know the things that
link |
kind of get taken over by the algos first are sort of the things that computers are
link |
obviously better at than people right so you know so first of all there needed to be this
link |
era of automation right where just you know financial exchanges became largely electronic
link |
which then enabled the possibility of you know trading becoming more algorithmic because
link |
once you know that exchanges are electronic an algorithm can submit an order through an
link |
API just as well as a human can do at a monitor quickly can read all the data so yeah and
link |
so you know I think the places where algorithmic trading have had the greatest inroads and
link |
had the first inroads were in kind of execution problems kind of optimized execution problems
link |
so what I mean by that is at a large brokerage firm for example one of the lines of business
link |
might be on behalf of large institutional clients taking you know what we might consider
link |
difficult trade so it's not like a mom and pop investor saying I want to buy a hundred
link |
shares of Microsoft it's a large hedge fund saying you know I want to buy a very very
link |
large stake in Apple and I want to do it over the span of a day and it's such a large volume
link |
that if you're not clever about how you break that trade up not just over time but over
link |
perhaps multiple different electronic exchanges that all let you trade Apple on their platform
link |
you know you will you will move you'll push prices around in a way that hurts your your
link |
execution so you know this is the kind of you know this is an optimization problem this
link |
is a control problem right and so machines are better we we know how to design algorithms
link |
you know that are better at that kind of thing than a person is going to be able to do because
link |
we can take volumes of historical and real time data to kind of optimize the schedule
link |
with which we trade and you know similarly high frequency trading you know which is closely
link |
related but not the same as optimized execution where you're just trying to spot very very
link |
temporary you know mispricings between exchanges or within an asset itself or just predict
link |
directional movement of a stock because of the kind of very very low level granular buying
link |
and selling data in the in the exchange machines are good at this kind of stuff it's kind of
link |
like the mechanics of trading what about the can machines do long terms of prediction yeah
link |
so I think we are in an era where you know clearly there have been some very successful
link |
you know quant hedge funds that are you know in what we would traditionally call you know
link |
still in this the stat arb regime like so you know what's that stat arb referring to
link |
statistical arbitrage but but for the purposes of this conversation what it really means
link |
is making directional predictions in asset price movement or returns your prediction
link |
about that directional movement is good for you know you you have a view that it's valid
link |
for some period of time between a few seconds and a few days and that's the amount of time
link |
that you're going to kind of get into the position hold it and then hopefully be right
link |
about the directional movement and you know buy low and sell high as the cliche goes.
link |
So that is a you know kind of a sweet spot I think for quant trading and investing right
link |
now and has been for some time when you really get to kind of more Warren Buffett style timescales
link |
right like you know my cartoon of Warren Buffett is that you know Warren Buffett sits and thinks
link |
what the long term value of Apple really should be and he doesn't even look at what Apple
link |
is doing today he just decides you know you know I think that this is what its long term
link |
value is and it's far from that right now and so I'm going to buy some Apple or you
link |
know short some Apple and I'm going to I'm going to sit on that for 10 or 20 years okay.
link |
So when you're at that kind of timescale or even more than just a few days all kinds of
link |
other sources of risk and information you know so now you're talking about holding things
link |
through recessions and economic cycles, wars can break out.
link |
So there you have to understand human nature at a level that.
link |
Yeah and you need to just be able to ingest many many more sources of data that are on
link |
wildly different timescales right.
link |
So if I'm an HFT I'm a high frequency trader like I don't I don't I really my main source
link |
of data is just the data from the exchanges themselves about the activity in the exchanges
link |
right and maybe I need to pay you know I need to keep an eye on the news right because you
link |
know that can cause sudden you know the CEO gets caught in a scandal or you know gets
link |
run over by a bus or something that can cause very sudden changes but you know I don't need
link |
to understand economic cycles I don't need to understand recessions I don't need to worry
link |
about the political situation or war breaking out in this part of the world because you
link |
know all I need to know is as long as that's not going to happen in the next 500 milliseconds
link |
then you know my model is good.
link |
When you get to these longer timescales you really have to worry about that kind of stuff
link |
and people in the machine learning community are starting to think about this.
link |
We held a we jointly sponsored a workshop at Penn with the Federal Reserve Bank of Philadelphia
link |
a little more than a year ago on you know I think the title is something like machine
link |
learning for macroeconomic prediction.
link |
You know macroeconomic referring specifically to these longer timescales and you know it
link |
was an interesting conference but it you know my it left me with greater confidence that
link |
we have a long way to go to you know and so I think that people that you know in the grand
link |
scheme of things you know if somebody asked me like well whose job on Wall Street is safe
link |
from the bots I think people that are at that longer you know timescale and have that appetite
link |
for all the risks involved in long term investing and that really need kind of not just algorithms
link |
that can optimize from data but they need views on stuff they need views on the political
link |
landscape economic cycles and the like and I think you know they're they're they're pretty
link |
safe for a while as far as I can tell.
link |
So Warren Buffett's job is not seeing you know a robo Warren Buffett anytime soon.
link |
If you could go back to if there's a day in your life you could relive because it made
link |
Maybe you outside family what otherwise you know what what what day would it be.
link |
But can you look back you remember just being profoundly transformed in some way or blissful.
link |
I'll answer a slightly different question which is like what's a day in my my life or
link |
my career that was kind of a watershed moment.
link |
I went straight from undergrad to doctoral studies and you know that's not at all atypical
link |
and I'm also from an academic family like my my dad was a professor my uncle on his
link |
side as a professor both my grandfathers were professors.
link |
All kinds of majors to philosophy.
link |
Yeah they're kind of all over the map yeah and I was a grad student here just up the
link |
river at Harvard and came to study with Les Valiant which was a wonderful experience.
link |
But you know I remember my first year of graduate school I was generally pretty unhappy and
link |
I was unhappy because you know at Berkeley as an undergraduate you know yeah I studied
link |
a lot of math and computer science but it was a huge school first of all and I took
link |
a lot of other courses as we've discussed I started as an English major and took history
link |
courses and art history classes and had friends you know that did all kinds of different things.
link |
And you know Harvard's a much smaller institution than Berkeley and its computer science department
link |
especially at that time was was a much smaller place than it is now.
link |
And I suddenly just felt very you know like I'd gone from this very big world to this
link |
highly specialized world and now all of the classes I was taking were computer science
link |
classes and I was only in classes with math and computer science people.
link |
And so I was you know I thought often in that first year of grad school about whether I
link |
really wanted to stick with it or not and you know I thought like oh I could you know
link |
stop with a master's I could go back to the Bay Area and to California and you know this
link |
was in one of the early periods where there was you know like you could definitely get
link |
a relatively good job paying job at one of the one of the tech companies back you know
link |
that were the big tech companies back then.
link |
And so I distinctly remember like kind of a late spring day when I was kind of you know
link |
sitting in Boston Common and kind of really just kind of chewing over what I wanted to
link |
do with my life and I realized like okay and I think this is where my academic background
link |
helped me a great deal.
link |
I sort of realized you know yeah you're not having a great time right now this feels really
link |
narrowing but you know that you're here for research eventually and to do something original
link |
and to try to you know carve out a career where you kind of you know choose what you
link |
want to think about you know and have a great deal of independence.
link |
And so you know at that point I really didn't have any real research experience yet I mean
link |
it was trying to think about some problems with very little success but I knew that like
link |
I hadn't really tried to do the thing that I knew I'd come to do and so I thought you
link |
know I'm going to stick through it for the summer and you know and that was very formative
link |
because I went from kind of contemplating quitting to you know a year later it being
link |
very clear to me I was going to finish because I still had a ways to go but I kind of started
link |
doing research it was going well it was really interesting and it was sort of a complete
link |
transformation you know it's just that transition that I think every doctoral student makes
link |
at some point which is to sort of go from being like a student of what's been done before
link |
to doing you know your own thing and figure out what makes you interested in what your
link |
strengths and weaknesses are as a researcher and once you know I kind of made that decision
link |
on that particular day at that particular moment in Boston Common you know I'm glad
link |
I made that decision.
link |
And also just accepting the painful nature of that journey.
link |
Yeah exactly exactly.
link |
In that moment said I'm gonna I'm gonna stick it out yeah I'm gonna stick around for a while.
link |
Well Michael I've looked off do you work for a long time it's really nice to talk to you
link |
thank you so much.
link |
It's great to get back in touch with you too and see how great you're doing as well.