back to indexPo-Shen Loh: Mathematics, Math Olympiad, Combinatorics & Contact Tracing | Lex Fridman Podcast #183
link |
The following is a conversation with Po Shen Lou,
link |
a professor of mathematics at Carnegie Mellon University,
link |
national coach of the USA International Math Olympia team,
link |
and founder of XP that does online education
link |
of basic math and science.
link |
He's also the founder of Novid,
link |
an app that takes a really interesting approach
link |
to contact tracing,
link |
making sure you stay completely anonymous
link |
and it gives you statistical information about COVID cases
link |
in your physical network of interactions.
link |
So you can maintain privacy, very important,
link |
and make informed decisions.
link |
we desperately needed solutions like this in early 2020.
link |
And unfortunately, I think,
link |
we will again need it for the next pandemic.
link |
To me, solutions that require large scale,
link |
distributed coordination of human beings
link |
need ideas that emphasize freedom and knowledge.
link |
Quick mention of our sponsors,
link |
Jordan Harbinger Show, Onnit, BetterHelp,
link |
Aidsleep, and Element.
link |
Check them out in the description to support this podcast.
link |
As a side note, let me say that Po and I
link |
filmed a few short videos
link |
about simple, beautiful math concepts
link |
that I will release soon.
link |
It was really fun.
link |
I really enjoyed Po sharing his passion for math with me
link |
I'm hoping to do a few more short videos
link |
in the coming months that are educational in nature
link |
on AI, robotics, math, science, philosophy,
link |
or if all else fails,
link |
just fun snippets into my life on music, books, martial arts,
link |
and other random things,
link |
if that's of interest to anyone at all.
link |
This is the Lex Friedman Podcast,
link |
and here's my conversation with Po Shenlow.
link |
You know, you mentioned you really enjoy flying
link |
and experiencing different people in different places.
link |
There's something about flying for me,
link |
I don't know if you have the same experience,
link |
that every time I get on an airplane,
link |
it's incredible to me that human beings
link |
have actually been able to achieve this.
link |
And when I look at like what's happening now
link |
with humans traveling out into space,
link |
I see it as all the same thing.
link |
It's incredible that humans are able to get into a box
link |
and fly in the air and safely and land
link |
in the same, it seems like,
link |
and everybody's taking it for granted.
link |
So when I observe them, it's quite fascinating
link |
because I see that cleanly mapping to the world
link |
where we're now in rockets and traveling to the moon,
link |
traveling to Mars, and at the same kind of way,
link |
I can already see the future
link |
where we will all take it for granted.
link |
So I don't know if you have,
link |
you personally, when you fly,
link |
have the same kind of magical experience
link |
of like how the heck did humans actually accomplish this?
link |
So I do, especially when there's turbulence,
link |
which is like on the way here, there was turbulence
link |
and the plane jiggled, even the flight attendant
link |
had to hold onto the side.
link |
And I was just thinking to myself,
link |
it's amazing that this happens all the time
link |
and the wings don't fall off,
link |
given how many planes are flying.
link |
But then I often think about it and I'm like,
link |
a long time ago, I think people didn't trust elevators
link |
in a 40 story building in New York City.
link |
And now we just take it completely for granted
link |
that you can step into this shaft,
link |
which is 40 floors up and down, and it will just not fail.
link |
Yeah, again, I'm the same way with elevators,
link |
but also buildings, when I'll stand on the 40th floor
link |
and wonder how the heck are we not falling right now?
link |
Like how amazing it is with the high winds,
link |
like structurally, just the earthquakes and the vibrations,
link |
I mean, natural vibrations in the ground.
link |
Like how is this, how are all of these,
link |
you go to like New York City, all of these buildings standing.
link |
I mean, to me, one of the most beautiful things,
link |
actually mathematically too, is bridges.
link |
I used to build bridges in high school from like toothpicks,
link |
just like out of the pure joy of like physics,
link |
making some structure really strong.
link |
Understanding like from a civil engineering perspective,
link |
what kind of structure will be stronger
link |
than another kind of structure, like suspension bridges.
link |
And then you see that at scale,
link |
humans being able to span a body of water
link |
with a giant bridge.
link |
And it's, I don't know, it's so humbling.
link |
It makes you realize how dependent we are on each other.
link |
Sort of, I talk about love a lot,
link |
but there's a certain element in which we little ants
link |
have just a small amount of knowledge
link |
about our particular thing.
link |
And then we're depending on a network of knowledge
link |
that other experts hold.
link |
And then most of our lives,
link |
most of the quality of life we have
link |
has to do with the richness of that network of knowledge,
link |
of that collaboration,
link |
and then sort of the ability to build on top of it,
link |
levels of abstractions.
link |
You start from like bits in a computer,
link |
then you can have assembly, then you can have C++,
link |
or you have an operating system,
link |
then you can have C++ and Python, finally,
link |
some machine learning on top.
link |
All of these are abstractions.
link |
And eventually we'll have AI that runs all of us humans.
link |
But anyway, but speaking of abstractions and programming,
link |
in high school, you wrote some impressive games
link |
I got a chance to, in browser somehow,
link |
it's magic, I got a chance to play them.
link |
Alien Attack 1, 2, 3, and 4.
link |
What's the hardest part about programming those games?
link |
And maybe can you tell the story about building those games?
link |
I actually tried to do those in high school
link |
because I was just curious if I could.
link |
That's a good starting point for anything, right?
link |
Yeah, yeah, yeah, it's like, could you?
link |
But the appealing thing was also,
link |
it was a soup to nuts kind of thing.
link |
So something that has always attracted me is,
link |
I like beautiful ideas, I like seeing beautiful ideas,
link |
but I actually also like seeing execution of an idea
link |
all the way from beginning to end in something that works.
link |
So for example, in high school,
link |
I was lucky enough to grow up in the late 90s
link |
when even a high school student could hope
link |
to make something sort of comparable
link |
to the shareware games that were out there.
link |
I say the word sort of, like still quite far away,
link |
but at least I didn't need to hire a 3D CG artist.
link |
There weren't enough pixels to draw anyway,
link |
even I can draw, right?
link |
Bad art, of course.
link |
But the point is, I wanted to know,
link |
is it possible for me to try to do those things
link |
where back in those days,
link |
you didn't even have an easy way
link |
to draw letters on the screen in a particular font.
link |
You couldn't just say import a font, it wasn't like Python.
link |
So for example, back then,
link |
if you played those games in the web browser,
link |
which is emulating the old school computer,
link |
those, even the letters you see,
link |
those are made by individual calls
link |
to draw pixels on the screen.
link |
So you built that from scratch,
link |
almost building a computer graphics library from scratch?
link |
Yes, the primitive that I got to use
link |
was some code I copied off of a book in assembly
link |
of how to put a pixel on a screen in a particular color.
link |
And the programming language was Pascal?
link |
Ah, yeah, the first one was in Pascal,
link |
but then the other ones were in C++ after that.
link |
How's the emulation in the browser work, by the way?
link |
Because it's pretty cool, you get to play these games
link |
that have a very much 90s feeling to them.
link |
Ah, so it's literally making an MSDOS environment,
link |
which is literally running the old.exe file.
link |
Wow, in the browser.
link |
This is, that could be more amazing than the airplane.
link |
So it wasn't so much about the video games,
link |
it was more about,
link |
can you build something really cool from scratch?
link |
And you did a bunch of programming competitions.
link |
What was your interest, your love for programming?
link |
What did you learn through that experience?
link |
Especially now that as much of your work
link |
has taken a long journey through mathematics.
link |
I think I always was amazed
link |
by how computers could do things fast.
link |
If I wanted to make it an abstract analysis
link |
of why it is that I saw some power in the computer.
link |
Because if the computer can do things
link |
so many times faster than humans,
link |
where the hard part is telling the computer
link |
what to do and how to do it,
link |
if you can master that asking the computer what to do,
link |
then you could conceivably achieve more things.
link |
And those contests I was in,
link |
those were the opposite in some sense
link |
of making a complete product, like a game is a product.
link |
Those contests were effectively write a function
link |
to do something extremely efficiently.
link |
And if you are able to do that,
link |
then you can unlock more of the power of the computer.
link |
But also doing it quickly.
link |
There's a time element from the human perspective
link |
to be able to program quickly.
link |
There's something nice.
link |
So there's almost like an athletics component
link |
to where you're almost like an athlete
link |
seeking optimal performance as a human being
link |
trying to write these programs.
link |
And at the same time, it's kind of art
link |
because the best way to write a program quickly
link |
is to write a simple program.
link |
You just have a damn good solution.
link |
So it's not necessarily you have to type fast.
link |
You have to think through a really clean,
link |
beautiful solution.
link |
I mean, what do you think is the use
link |
of those programming competitions?
link |
Do you think they're ultimately something
link |
you would recommend for students,
link |
for people interested in programming,
link |
or people interested in building stuff?
link |
Yes, I think so because especially with the work
link |
that I've been doing nowadays,
link |
even trying to control COVID,
link |
something that was very helpful from day one
link |
was understanding that the kinds of computations
link |
we would want to do,
link |
we could conceivably do on like a four core cloud machine
link |
on Amazon Web Services out to a population
link |
which might have hundreds of thousands
link |
or millions of people.
link |
The reason why that was important
link |
to have that back of the envelope calculation
link |
with efficient algorithms
link |
is because if we couldn't do that,
link |
then we would bankrupt ourselves
link |
before we could get to a big enough scale.
link |
If you think about how you grow anything from small to big,
link |
if in order to grow it from small to big,
link |
you also already need 10,000 cloud servers,
link |
you'll never get to big.
link |
And also the nice thing about programming competitions
link |
is that you actually build a thing that works.
link |
So you finish it, there's a completion thing,
link |
and you realize, I think there's a magic to it,
link |
where you realize that it's not so hard
link |
to build something that works.
link |
To have a system that successfully takes in inputs
link |
and produces outputs and solves a difficult problem,
link |
and that directly transfers to building a startup essentially
link |
that can help some aspect of this world
link |
as long as it's mostly based on software engineering.
link |
Things get really tricky
link |
when you have to manufacture stuff.
link |
That's why people like Elon Musk are so impressive
link |
that it's not just software.
link |
Tesla Autopilot is not just software.
link |
It's like you have to actually have factories
link |
that build cars, and there's like a million components
link |
involved in the machinery required
link |
to assemble those cars and so on.
link |
But in software, one person can change the world,
link |
which is incredible.
link |
But on the mathematics side,
link |
what, if you look back, or maybe today,
link |
what made you fall in love with mathematics?
link |
For me, I think I've always been very attracted
link |
to challenge, as I already indicated
link |
with writing the program.
link |
I guess if I see something that's hard
link |
or supposed to be impossible,
link |
sometimes I say, maybe I want to see if I can pull that off.
link |
And with the mathematics, the math competitions
link |
presented problems that were hard,
link |
that I didn't know how to start,
link |
but for which I could conceivably try to learn
link |
how to solve them.
link |
So, I mean, there are other things that are hard
link |
called like get something to Mars, get people to Mars.
link |
And I didn't, and I still don't think
link |
that I am able to solve that problem.
link |
On the other hand, the math problems struck me
link |
as things which are hard
link |
and with significant amount of extra work,
link |
I could figure it out.
link |
And maybe they would actually even be useful,
link |
like that mathematical skill is the core
link |
of lots of other things.
link |
That's really interesting.
link |
Maybe you could speak to that
link |
because a lot of people say that math is hard
link |
as a kind of negative statement.
link |
It always seemed to me a little bit like
link |
that's kind of a positive statement
link |
that all things that are worth having in this world,
link |
I mean, everything that people think about
link |
that they would love to do, whether it's sports,
link |
whether it's art, music, and all the sciences,
link |
they're going to be hard
link |
if you want to do something special.
link |
So is there something you could say to that idea
link |
that math is hard?
link |
Should it be made easy or should it be hard?
link |
Ah, so I think maybe I want to dig in a little bit
link |
onto this hard part and say,
link |
I think the interesting thing about the math
link |
is that you can see a question
link |
that you didn't know how to start doing it before.
link |
And over a course of thinking about it,
link |
you can come up with a way to solve it.
link |
And so you can move from a state
link |
of not being able to do something
link |
to a state of being able to do something
link |
where you help to take yourself through that
link |
instead of somebody else spoon feeding you that technique.
link |
So actually here, I'm already digging into
link |
maybe part of my teaching philosophy also,
link |
which is that I actually don't want to ever
link |
just tell somebody, here's how you do something.
link |
I actually prefer to say, here's an interesting question.
link |
I know you don't quite know how to do it.
link |
Do you have any ideas?
link |
I'm actually explaining another way
link |
that you could try to do teaching.
link |
And I'm contrasting this to a method of watch me do this,
link |
now practice it 20 times.
link |
I'm trying to say a lot of people consider math to be hard
link |
because maybe they can't remember
link |
all of the methods that were taught.
link |
But for me, I look at the hardness
link |
and I don't think of it as a memory hardness.
link |
I think of it as a, can you invent something hardness?
link |
And I think that if we can teach more people
link |
how to do that art of invention in a pure cognitive way,
link |
not as hard as the actual hardware stuff, right?
link |
But like in terms of the concepts
link |
and the thoughts and the mathematics,
link |
teaching people how to invent,
link |
then suddenly actually they might not even find math
link |
to be that tiresomeness hard anymore,
link |
but that rewardingness hard of I have the capability
link |
of looking at something which I don't know what to do
link |
and coming up with how to do it.
link |
I actually think we should be doing that,
link |
giving people that capability.
link |
So hard in the same way that invention is hard,
link |
that is ultimately rewarding.
link |
So maybe you can dig in that a little bit longer,
link |
which is do you see basically the way to teach math
link |
is to present a problem and to give a person a chance
link |
to try to invent a solution
link |
with minimal amount of information first?
link |
Is that basically,
link |
how do you build that muscle of invention in a student?
link |
Yes, so the way that,
link |
I guess I have two different sort of ways
link |
that I try to teach.
link |
Actually, one of them is, in fact, this semester,
link |
because all my classes were remotely delivered,
link |
I even threw them all onto my YouTube channel.
link |
So you can see how I teach at Carnegie Mellon,
link |
but I'd often say, hey, everyone, let's try to do this.
link |
And that actually changes my role as a professor
link |
from a person who shows up for class
link |
with a script of what I wanna talk through.
link |
I actually, I don't have a script.
link |
The way I show up for classes,
link |
there's something that we want to learn how to do,
link |
and we're gonna do it by improv.
link |
I'm talking about the same method as improv comedy,
link |
which is where you tell me some ideas,
link |
and I'll try to yes and them.
link |
You know what I mean?
link |
And then together,
link |
we're gonna come up with a proof of this concept
link |
where you were deeply involved in creating the proof.
link |
Actually, every time I teach the class,
link |
we do every proof slightly differently
link |
because it's based on how the students came up with it.
link |
And that's how I do it when I'm in person.
link |
I also have another line of courses that we make
link |
that is delivered online.
link |
Those things are where I can't do it live,
link |
but the teaching method became also similar.
link |
It was just, here's an interesting question.
link |
I know it's out of reach.
link |
Why don't you think about it?
link |
And then automatic hints.
link |
We feed automatically hints through the internet
link |
to go and let the person try to invent.
link |
So that's like a more rigorous prodding of invention.
link |
But you did mention disease and COVID,
link |
and you've been doing some very interesting stuff
link |
from a mathematical, but also software engineering angle
link |
of coming up with ideas.
link |
It's back to the, I see a problem.
link |
I think I can help.
link |
So you stepped into this world.
link |
Can you tell me about your work there
link |
under the flag of Novid
link |
and both the software and the technical details
link |
of how the thing works?
link |
So first I want to make sure that I say,
link |
this is actually team effort.
link |
I happen to be the one speaking,
link |
but there's no way this would exist
link |
without an incredible team of people
link |
who inspire me every day to work on this.
link |
But I'll speak on behalf of them.
link |
So the idea was indeed that we stepped forward
link |
in March of last year, when the world started to become,
link |
our part of the world started to become,
link |
our part meaning the United States
link |
started to become paralyzed by COVID.
link |
The shutdown started to happen.
link |
And at that time it started as a figment of an idea,
link |
which was network theory,
link |
which is the area of math that I work in,
link |
could potentially be combined with smartphones
link |
and some kind of health information anonymized.
link |
We didn't know yet.
link |
We tried to crystallize it.
link |
And many months into this work,
link |
we ended up accidentally discovering a new way
link |
to control diseases,
link |
which is now what is the main impetus of all of this work
link |
is to take this idea and polish it
link |
and hopefully have it be useful not only now,
link |
but for future pandemics.
link |
The idea is really simple to describe.
link |
Actually, my main thing in the world
link |
is I come up with obvious observations.
link |
That's that, so I'll explain it now.
link |
Einstein did the same thing
link |
and he wrote a few short papers.
link |
But so the idea is like this.
link |
If we describe how usually people control disease
link |
for a lot of history,
link |
it was that you'd find out who was sick,
link |
you'd find out who they've been around
link |
and you try to remove all of those people from society
link |
against their will.
link |
Now that's the problem.
link |
The against their will part
link |
gives you the wrong kind of a feedback loop,
link |
which makes it hard to control the disease
link |
because then the people you're trying to control
link |
keep getting other people sick.
link |
You can see already how I'm thinking
link |
and talking about this feedback loops.
link |
This is actually related to something you said earlier
link |
about even like how skyscrapers stay in the air.
link |
The whole point is control theory.
link |
You actually want to, or even how an airplane stays,
link |
you need to have control loops
link |
which are feedbacking in the right way.
link |
And what we observed was that the feedback control loop
link |
for controlling disease by asking people
link |
to be removed from society against their will
link |
It was running against human incentives
link |
and you suddenly are trying to control
link |
seven billion, eight billion people
link |
in ways that they don't individually want
link |
to necessarily do.
link |
So here's the idea.
link |
And this is inspired by the fact
link |
that at the core of our team
link |
were user experience designers.
link |
That's actually, in fact, the first thing I knew
link |
we needed when we started
link |
was to bring user experience at the core.
link |
But so the idea was suppose hypothetically
link |
there was a pandemic.
link |
What would you want?
link |
You would want a way to be able to live your life
link |
as much as possible and avoid getting sick.
link |
Can we make an app to help you avoid getting sick?
link |
Notice how I've just articulated the problem.
link |
It is not, can we make an app
link |
so that after you are around somebody who's sick
link |
you can be removed from society.
link |
It's can we make an app so that you can avoid getting sick.
link |
That would run a positive feed.
link |
I don't know if I want to call it positive or negative
link |
but it would run a good feedback loop.
link |
So then how would you do this?
link |
The only problem is that you don't know who's sick
link |
because especially with this disease
link |
if I see somebody who looks perfectly healthy
link |
the disease spreads two days before you have any symptoms.
link |
And so it's actually not possible.
link |
That's where the network theory comes in.
link |
You caught it from someone.
link |
What if we changed the paradigm
link |
and we said, whenever there's a sickness
link |
tell everybody how many physical relationships
link |
separate them from the sickness.
link |
That is the trivial idea we added.
link |
The trivial idea was the distance between you and a disease
link |
is not measured in feet or seconds.
link |
It's measured in terms of how many
link |
close physical relationships separate you
link |
like these six degrees of separation like LinkedIn.
link |
What if we told everyone that?
link |
It turns out that actually unlocks
link |
some interesting behavioral feedback loops
link |
which for example, let me now jump to a non COVID example
link |
to show why this maybe could be useful.
link |
Actually we think it could be quite useful.
link |
Imagine there was Ebola or some hemorrhagic fever.
link |
Imagine it spread through contact through the air.
link |
In fact, pretend, pretend.
link |
That's a disastrous disease.
link |
It has high fatality rate.
link |
And as you die, you're bleeding out of every orifice.
link |
Yeah, not pleasant.
link |
So the question is, suppose that such a disease broke
link |
who would want to install an app that would tell them
link |
how many relationships away from them
link |
this disease had struck?
link |
Like a lot of people.
link |
In fact, almost, I don't want to say almost everyone.
link |
That's a very strong statement
link |
but a very large number of people.
link |
That's fascinating framing.
link |
Like the more deadly and transmissible the disease
link |
the stronger the incentive to install it in a positive sense
link |
the, in the good feedback loop sense.
link |
That's a really good example.
link |
It's a really good way to frame it.
link |
Cause with COVID, it was not as deadly
link |
as potential pandemics could have been
link |
viruses could have been.
link |
So it's sometimes muddled with how we think about it
link |
but yeah, this is a really good framing.
link |
If the virus was a lot more deadly
link |
you want to create a system that has a set of incentives
link |
that it quickly spreads to the population
link |
where everybody is using it
link |
and it's contributing in a positive way to the system.
link |
And actually that point you just made
link |
I don't take credit for that observation.
link |
There was another person I talked to
link |
who pointed out that it's very interesting
link |
that this feedback loop is even more effective
link |
when the disease is worse.
link |
And that's actually not a bad characteristic to have
link |
in your feedback loop
link |
if you're trying to help civilization keep running.
link |
Yeah, it's a really, it's in this dynamic
link |
like people figure out, they dynamically figure out
link |
how bad the disease is.
link |
The more it spreads and the deadlier it is
link |
as the people observe it
link |
as long as the spread of information
link |
like semantic information, natural language information
link |
is closely aligned with the reality of the disease
link |
which is a whole nother conversation, right?
link |
We, that's, we might, maybe we'll chat about that
link |
how we sort of make sure there's not misinformation
link |
while there's accurate information
link |
but that aside, okay, so this is a really nice property.
link |
Right, and just going on on that
link |
actually just talking more about what that could do
link |
and why we're so excited about it.
link |
It's that not only would people want to install it
link |
but what would they do if you start to see
link |
that this disease is getting closer and closer?
link |
We surveyed informally people
link |
but they said, as we saw it getting closer, we would hide.
link |
We would try to not have contacts.
link |
But now you notice what this has just achieved.
link |
The whole goal on this whole exercise was
link |
you got the people who might be sick
link |
and you got everyone else, set A and set B.
link |
Set A is the people who might be sick,
link |
set B is everyone else.
link |
And for the entirety of the past
link |
contact tracing approaches, you tried to get set A
link |
to do things that might not be to their liking or their will
link |
because that's removing them from society.
link |
We found out that there's two ways
link |
to separate set A from set B.
link |
You can also let the people at set B
link |
at the fringe of set A
link |
attempt to remove themselves from this interface.
link |
It's the symmetry of A and B separation.
link |
Everyone was looking at A, we look at B
link |
and suddenly B is in their incentive to do so.
link |
So there's a virus that jumps from human to human.
link |
So there's a network sometimes called graph
link |
of the spread of a virus.
link |
It hops from person to person to person to person.
link |
And each one of us individuals are sitting
link |
or plopped into that network.
link |
We have close friends and relations and so on.
link |
It's kind of fascinating
link |
to actually think about this network
link |
and we can maybe talk about the shapes
link |
of this kind of network.
link |
Because I was trying to think exactly this,
link |
like how many people do I,
link |
well, I'm kind of an introvert, not kind of,
link |
I'm very much an introvert.
link |
But so can I be explicit about the kind of people
link |
I meet in regular life?
link |
Say when it was completely opened up, there's no pandemic.
link |
There is a kind of network and there's maybe
link |
in the graph theoretic sense, there's some weights
link |
or something about how close that relationship is
link |
in terms of the frequency of visits,
link |
the duration of visits and all of those kinds of things.
link |
So you're saying we might want to be,
link |
to create on top of that network,
link |
a spread of information to let you know
link |
as the virus travels through this network,
link |
how close is it getting to you?
link |
And the number of hops away it is on that network
link |
is really powerful information
link |
that creates a positive feedback loop
link |
where you can act essentially anonymously
link |
Like nobody's telling you what to do,
link |
which is really important, is decentralized
link |
and not whatever the opposite of authoritarian is.
link |
But you get to sort of the American way.
link |
You get to choose to do it yourself.
link |
You have the freedom to do it yourself
link |
and you're incentivized to do it.
link |
And you're most likely going to do it
link |
to protect yourself against you getting the disease
link |
as the closer it gets to you
link |
based on the information that you have.
link |
But can you maybe elaborate, first of all, brilliant.
link |
Whenever I saw the thing you're working on,
link |
so forget for COVID, this is of course,
link |
really relevant for COVID, but it's also probably relevant
link |
for future diseases as well.
link |
So that was the thing I'm nervous about.
link |
I was like, if this whole,
link |
if our society shut down because of COVID,
link |
like what the heck is gonna happen
link |
when there's a much deadlier disease?
link |
Like this, this is disappointing.
link |
The whole time, 2020, the whole time
link |
I'm just sitting like this,
link |
like is the incompetence of everybody
link |
except the people developing vaccines.
link |
The biologists are the only ones
link |
that got their stuff together.
link |
But in terms of institutions and all that kind of stuff,
link |
it's just been terrible.
link |
But this is exactly the power of information
link |
and the power of information
link |
that doesn't limit personal freedom.
link |
So your idea is brilliant.
link |
Okay, mathematically, can you maybe elaborate
link |
what are we talking about?
link |
Like how do you actually make that work?
link |
Sure, first I'm gonna reply to something you said
link |
about the freedom inside this,
link |
because actually that was the idea.
link |
The idea is this is game theory, right?
link |
And effectively what we did is analogous
link |
to free market economy, as opposed to central planning.
link |
If you just line up the set of incentives correctly
link |
so that people have in their purely selfish behavior
link |
are contributing to the optimization of the global function,
link |
And the point of what we do, I guess in mathematics
link |
is we try to explore the search space
link |
to go and find out as many possibilities as there are.
link |
And in this case, it's an applied search space.
link |
That's why the inputs from design,
link |
user experience design and actual people are important.
link |
But you asked about, I guess, the mathematical
link |
or the technical things underpinning it.
link |
So I think the first thing I'll say is
link |
we wanted to make this thing
link |
not require your personal information.
link |
And so in order to do that,
link |
what gave me the confidence to, I guess,
link |
lead our team to run at the beginning
link |
is we saw that this could be done without using GPS information.
link |
So technically what's going on is if two smartphones,
link |
it's a smartphone app.
link |
If two smartphones have this thing installed,
link |
they just communicate with each other by Bluetooth
link |
to go and find out how far,
link |
they can detect nearby things by Bluetooth.
link |
And then they can find out that these two phones
link |
were approximately such and such distance apart.
link |
And that kind of relative proximity information
link |
is enough to construct this big network.
link |
Okay, so the physical network is constructed
link |
based on proximity that's through Bluetooth
link |
and you don't have to specify your exact location,
link |
it's the proximity.
link |
I'm not using the Pythagorean theorem basically.
link |
I mean, if I just knew the GPS coordinates,
link |
we could use the Pythagorean theorem too.
link |
Sorry, that's just how I call it.
link |
Distance formula, whatever you want to call it.
link |
Yeah, so we're not doing
link |
the old Pythagorean based violation of privacy.
link |
But so is that enough to form,
link |
to give you enough information about physical connection
link |
to another human being?
link |
Is there a time element there?
link |
Is there, so, okay.
link |
That sounds like a really strong, like low hanging fruit.
link |
Like if you have that,
link |
you could probably go really, really far.
link |
My natural question is,
link |
is there extra information you can add on top of that?
link |
Like the duration of the physical proximity?
link |
So first of all, we actually do estimate the duration,
link |
but the way we estimate the duration
link |
is like how a movie is filmed,
link |
in the sense that every so often, every few minutes,
link |
we check what's nearby.
link |
It's like how a movie is filmed.
link |
You take lots of snapshots.
link |
So there's no way in a battery efficient way
link |
to really keep track of that proximity.
link |
However, fortunately, we're using probability.
link |
The fact is the paradigm that we're using
link |
is it's not super important
link |
if you run into that person only for 10 minutes
link |
at the grocery store.
link |
If that's a stranger that you run into 10 minutes
link |
in this grocery store,
link |
that's not gonna be relevant for our paradigm
link |
because our paradigm is not telling you
link |
who were you around before
link |
and might therefore have gotten infected by already.
link |
Ours is about predicting the future.
link |
We change from, I mean, the standard paradigm was
link |
what already happened, quick damage control.
link |
Ours is predict the future.
link |
If you run into that person once in the grocery store today
link |
and never see them again,
link |
it's irrelevant for predicting the future.
link |
And therefore, for ours, what really matters
link |
is the many hours around the other person,
link |
at which point, if you're scanning every five
link |
That's going to come out in the problem,
link |
like statistically speaking,
link |
it's going to come out as a strong relationship
link |
and a person in the grocery store is going to wash out
link |
is not an important physical relationship.
link |
I mean, this is brilliant.
link |
How difficult is it to make work?
link |
So you said, one, there's a mathematical component
link |
that we just kind of talked about,
link |
and then there's the user experience component.
link |
So how difficult does it to go,
link |
just like you built the video game, Alien Attack,
link |
from zero to completion, what's involved?
link |
How difficult is it?
link |
So I'm going to answer that question
link |
in terms of building the product,
link |
but then I'm also going to acknowledge
link |
that just having an app doesn't make it useful
link |
because that's actually maybe the easy part.
link |
If you know what I mean,
link |
there's like all of this stuff
link |
about rollout adoption and awareness,
link |
but let's focus on the app part first.
link |
So that's again, why I said the team is incredible.
link |
So we have a bunch of people who,
link |
let's just say that the technology that we use to make it
link |
is not the standard way you make an app.
link |
If you think about a standard iOS app or Android app,
link |
those are a user interface that contacts a web server
link |
and sends some information back and forth.
link |
We're doing some stuff that has to hook
link |
into the operating system of saying,
link |
let's go use Bluetooth for something
link |
it wasn't really meant for, right?
link |
So there's that part.
link |
By the way, what is the app called?
link |
Oh, it's called Novid, COVID with an N.
link |
So you have to hook into Bluetooth.
link |
You're saying you have to do that beyond the permissions
link |
that are like at the very surface level
link |
provided on the phone?
link |
Well, I don't want to call them permissions.
link |
I just want to say,
link |
that's not what you usually do with Bluetooth.
link |
Usually with Bluetooth, you say,
link |
do I have headphones nearby?
link |
You don't go and say, do I have headphones nearby?
link |
Or do I have another phone nearby, which is doing something?
link |
And then keep asking that same question.
link |
Keep asking the question.
link |
So it's actually not easy.
link |
And I mean, there were some parts of it,
link |
which actually a lot of people had tried unsuccessfully.
link |
Actually, it's known that, for example,
link |
the UK was trying to do something similar.
link |
And the problem they ran into was,
link |
when you program things on iOS,
link |
iOS is very good at making it hard
link |
to do things in the background.
link |
And so there was quite a lot of effort required
link |
to go and make this thing work.
link |
So the whole point, this thing would run in the background
link |
and iOS, I mean, most Android probably as well, right?
link |
But yeah, iOS certainly makes it difficult
link |
for something to run in the background,
link |
especially when it's eating up your battery, right?
link |
Well, we wanted to make sure we didn't eat up the battery.
link |
So that one we can,
link |
we actually are very proud of the fact
link |
that ours uses very little battery.
link |
Actually, even if compared to Apple's own system, so.
link |
So what else is required to make this thing work?
link |
Right, so the key was that you had to do
link |
a significant amount of work on the actual
link |
mobile app development,
link |
which fortunately the team that we brought
link |
was this kind of general thinkers
link |
where we would dig in deep into the operating system
link |
documentation and the API libraries.
link |
So we got that working.
link |
But there's another angle, which is,
link |
you also need the servers to be able to compute fast enough,
link |
which is tying back to this old school
link |
computer programming competitions and math Olympiads.
link |
In fact, our team that was working on the algorithm
link |
and backend side included several people
link |
who had been in these competitions from before,
link |
which I happen to know because I do coach the team
link |
And so we were able to bring people in to build servers,
link |
a server infrastructure in C++ actually,
link |
so that we could support significant numbers of people
link |
without needing tons of servers.
link |
Is there some distributed algorithms working here
link |
or you basically have to keep in the same place
link |
the entire graph as it builds?
link |
Cause especially the more and more people use it,
link |
the bigger, the bigger the graph gets.
link |
I mean, this is very difficult scaling problem, right?
link |
Ah, so that's actually why this computer algorithm
link |
competition stuff was handy.
link |
It's because there are only about seven to eight
link |
giga people in the world.
link |
That's not that many.
link |
So if you can make your algorithms linear time
link |
or almost linear time, a computer operates in gigahertz.
link |
I only need to do one run, one recalculation every hour
link |
in terms of telling people how far away these dangers are.
link |
So I suddenly have 3,600 seconds
link |
and my CPU cores are running in gigahertz.
link |
And at most they're eight giga people.
link |
Well, you skipping over the fact that there's N squared
link |
potential connections between people.
link |
So how do you get around the fact that, you know,
link |
that we, you know, the potential set of relationship
link |
any one of us could have is a billion.
link |
So it's a billion times squared.
link |
That's the potential amount of data you have to be storing
link |
and computing over and constantly updating.
link |
So the way we dealt with that is we actually expect
link |
that the typical network is very sparse.
link |
The technical term sparse would mean that the average degree
link |
or the average number of connections that a person has
link |
is going to be at most like a hundred strong connections
link |
that you care about.
link |
If you think of it almost in terms of the heavy hitters,
link |
actually in most people's lives,
link |
a hundred, if we just kept track
link |
of their top hundred interactions,
link |
that's probably most of the signal.
link |
I'm saddened to think that I might not be even
link |
in a double digits, but.
link |
Oh, I was intentionally giving a crazy number
link |
to account for college students.
link |
You call, oh, those are the,
link |
who you call on the heavy hitters,
link |
the people who are like the social butterflies.
link |
I'd love to know that information about myself,
link |
by the way, that, do you expose the graph,
link |
like how many, like about yourself,
link |
how many connections you have?
link |
We do expose to each person
link |
how many direct connections they have.
link |
But for privacy purposes,
link |
we don't tell anybody who their connections,
link |
like how their connections are interconnected.
link |
But at the same time, we do expose also to everyone
link |
an interesting chart that says,
link |
here's how many people you have
link |
that you're connected to directly.
link |
Here's how many at distance two,
link |
meaning via people.
link |
And then here's how many at distance three.
link |
And the reason we do that,
link |
is that actually ends up being a dynamic
link |
that also boosts adoption.
link |
It drives another feedback loop.
link |
The reason is because we saw, actually,
link |
when we deployed this in some universities,
link |
that when people see on their app
link |
that they are indirectly connected to hundreds
link |
or thousands of other people,
link |
they get excited and they tell other people,
link |
hey, let's download this app.
link |
But you know, we also saw in those examples,
link |
especially looking at the screenshots people gave,
link |
that is hit as soon as the typical person
link |
has two or three other direct connections on the system.
link |
Because that means that our app
link |
has reached a virality or not of two to three.
link |
The key is we were making a viral app to fight a virus
link |
spreading on the same network that the virus spreads on.
link |
So you're trying to out virus the virus.
link |
That's exactly right.
link |
What have you learned from this whole experience
link |
in terms of, let's say for COVID,
link |
but for future pandemics as well,
link |
is it possible to use the power information here
link |
of networked information as a virus spreads and travels
link |
in order to basically keep the society open?
link |
Is it possible for people to protect themselves
link |
with this information?
link |
Or do you still have to have most,
link |
like in this overarching policy
link |
of everybody should stay at home, that kind of thing?
link |
We are trying to answer that question right now.
link |
So the answer is we don't know yet,
link |
but that's actually why we're very happy
link |
that now the idea has started to become more widely known.
link |
And we're already starting to collaborate
link |
with epidemiologists.
link |
Again, I'm just a mathematician, right?
link |
And a mathematician should not be the person
link |
who is telling everybody, this will definitely work.
link |
But because of the potential power of this approach,
link |
especially the potential power
link |
of this being an end game for COVID,
link |
we have gotten the interest of real researchers.
link |
And we're now working together
link |
to try to actually understand the answer to that question.
link |
Because you see, there's a theory.
link |
So what I can share is the mathematics of,
link |
here's why there's some hope that this would work.
link |
And that's because I'm talking about end game now.
link |
End game means you have very few cases.
link |
But everywhere, we're always thinking,
link |
once there's few cases, then does that mean we now open up?
link |
Once you open up in the past, then the cases go up again
link |
until you have to lock down again.
link |
And now when we talk about the dynamic process that makes,
link |
it's guaranteeing you always have cases
link |
until you have the great vaccines,
link |
which is, we both got vaccinated, this is good.
link |
But at the same time, why I'm thinking
link |
this is still important is because we know
link |
that many vaccine makers have said
link |
they're preparing for the next dose next year.
link |
And if we have a perpetual thing
link |
where you just always need a new vaccine every year,
link |
it could actually be beneficial to make sure
link |
we have as many other techniques as possible
link |
for parts of the world that can't afford,
link |
for example, that kind of distribution.
link |
Yeah, so actually, no matter how deadly the virus is,
link |
no matter how many things,
link |
whether you have a vaccine or not,
link |
it's still useful to be having this information.
link |
Because to stay home or not, depending on how risk,
link |
I'm a big fan, just like you said, of having the freedom
link |
for you to decide how risk averse you wanna be, right?
link |
Depending on your own conditions,
link |
but also on the state of like what you,
link |
just how dangerously you like to live.
link |
So I think that actually makes a lot of sense.
link |
And I also think that since we're,
link |
when you think of disease spreading,
link |
it spreads in aggregate in the sense that
link |
if there are some people who maybe are more risk tolerant
link |
because of other things in their life,
link |
well, there might also be other people
link |
who are less risk tolerance.
link |
And then those people decide to isolate.
link |
But what matters is in the aggregate
link |
that this R naught of the infection spreading
link |
And so the key is if you can empower people
link |
with that power to make that decision,
link |
you might actually still be able to drive
link |
that R naught down below one.
link |
Yeah, and also, this is me talking,
link |
people get a little bit nervous, I think,
link |
with information somehow mapping to privacy violation.
link |
But first of all, in the approach you're describing,
link |
that's respecting anonymity.
link |
But I would love to have information
link |
from the very beginning, from March and April of last year,
link |
almost like a map of like where it's risky
link |
and where it's not to go.
link |
And not map based on sort of the exact location of people,
link |
but where people usually hang out kind of thing.
link |
Just, and maybe not necessarily about actual location,
link |
but just maybe activities,
link |
like just to have information about what is good to do
link |
and not, in terms of like safety,
link |
is it okay to run outside and not,
link |
is it okay to go to a restaurant and not,
link |
I just feel like we're operating in the blind.
link |
And then what you had is a very imperfect signal,
link |
which is like basically politicians desperately trying
link |
to make statements about what is safe and not.
link |
They don't know what the heck they're doing.
link |
They have a bunch of smart scientists telling them stuff.
link |
And the scientists themselves also, very important,
link |
don't always know what they're doing.
link |
Epidemiology is not, is as much an art as a science.
link |
You're desperately trying to predict the future,
link |
which nobody can do.
link |
And then you're trying to speak with some level of authority.
link |
I mean, if I were to criticize scientists,
link |
they spoke with too much authority.
link |
It's okay to say, I'm not sure.
link |
But then they think like, if I say, I'm not sure,
link |
then there's going to be a distrust.
link |
What they realize is when you're wrong and you say,
link |
I'm sure, it's going to lead to more distrust.
link |
So there's this imperfect, like just chaotic,
link |
messy system of people trying to figure out
link |
with very little information.
link |
And what you're proposing is just a huge amount
link |
of information, and information is power.
link |
Is there challenges with adoption that you see
link |
in the future here?
link |
So there's, maybe we could speak to,
link |
there's approaches, I guess, from Google.
link |
There's different people that have tried
link |
similar kind of ideas.
link |
Not, you have quite a novel idea, actually.
link |
But speaking, the umbrella idea of contact tracing,
link |
is there something you can comment about
link |
why their approaches haven't been fully adopted?
link |
Is there challenges there?
link |
Is there reasons why Novid might be a better idea
link |
moving forward, in general, just about adoption?
link |
Yeah, so first of all, I want to say,
link |
I always have respect for the methods that other people use.
link |
And so it's good to see the other people I've been trying.
link |
But what we have noticed is that the difference
link |
between our value proposition to the user
link |
and the value proposition to the user delivered
link |
by everything that was made before is that,
link |
unfortunately, the action of installing
link |
a standard contact tracing app will then tell you
link |
after you have already been exposed to the disease
link |
so that you can protect other people from you.
link |
And what that does to your own direct probability
link |
of getting sick, if you think about it,
link |
suppose you were making the decision,
link |
should I or should I not install one of those apps?
link |
What does that do to your own probability of getting sick?
link |
It's close to zero.
link |
This is the sad thing you're speaking to, not sad.
link |
I suppose it's the way the world is.
link |
The only incentive there is to just help other people,
link |
I suppose, but a much stronger incentive
link |
is anything that allows you to help yourself.
link |
Yes, so what I'm saying is that,
link |
let's just say free market capitalism
link |
was not based on altruism, I think it's based on,
link |
if you make a system of incentives
link |
so that everybody trying to maximize their own situation
link |
somehow contributes to the whole,
link |
that's a game theoretic solution to a very hard problem.
link |
And so this is actually basically mechanism design,
link |
that we've basically come up with a different mechanism,
link |
different set of incentives,
link |
which incentivizes the adoption,
link |
because actually whenever we've been rolling it out,
link |
usually the first question we ask people,
link |
like say in a university is,
link |
do you know what Novid does?
link |
And most of them have read about the other apps
link |
and they say, Oh, Novid will tell you
link |
after you've been around someone so you can quarantine.
link |
And we have to explain to them,
link |
actually, Novid never wants to ask you to quarantine.
link |
That's not the principle.
link |
Our principle isn't based on that at all.
link |
We just want to let you know if something is coming close
link |
so that you can protect yourself.
link |
If you want, if you want, if you want.
link |
And then the quarantine is like, yes,
link |
in that case, if you're quarantining,
link |
it's because you're shutting the door from the inside,
link |
if that makes sense.
link |
I mean, this is brilliant.
link |
So what do you think the future looks like
link |
for future pandemics?
link |
What's your plan with Novid?
link |
What's your plan with these set of ideas?
link |
I am actually still an academic and a researcher.
link |
So the biggest work I'm working on right now
link |
is to try to build as many collaborations
link |
with other public health researchers at other universities
link |
to actually work on pilot deployments together
link |
in various places.
link |
That's actually ongoing work right now.
link |
And so, for example, if anyone's watching this
link |
and you happen to be a public health researcher
link |
and you want to be involved in something like this,
link |
I'm just gonna say, I'm still incentive thinking.
link |
There's something in it for the researchers too.
link |
This could open up an entire new way
link |
of controlling disease.
link |
I mean, it might actually be true.
link |
And people who are involved in figuring out
link |
how to make this work,
link |
well, it could actually be good for their careers too.
link |
I always have to think like,
link |
if a researcher was getting involved,
link |
what are they getting out of it?
link |
Oh, so you mean like from a research perspective,
link |
you can like publications and sets of ideas
link |
about how to, from a sort of network theory perspective,
link |
understand how we control the spread of a pandemic.
link |
Yes, and what I'm doing right now
link |
is this is basically interdisciplinary research
link |
where maybe our side is bringing the technology
link |
and the network theory,
link |
and the missing parts are epidemiology
link |
and public health expertise.
link |
And if the two things start to join,
link |
also because everywhere that you deploy,
link |
let's just say that the world is different
link |
in the Philippines as it is in the United States.
link |
And just the natures of the locality
link |
would mean that someone like me
link |
should not be trying to figure out how to do that.
link |
But if we can work with the researchers
link |
who are based there,
link |
now suddenly we might come up with a solution
link |
that will help scale in parts of the world
link |
where they aren't all getting the Moderna and Pfizer vaccines
link |
which cost like $20 a pop in the US.
link |
So if they want to participate,
link |
who do they reach out to?
link |
Oh, that would just be us.
link |
I mean, the novid.org website has...
link |
It has a feedback reach out form.
link |
And actually we are, I mean, again,
link |
this is the DNA of being a researcher.
link |
I am actually very excited by the idea
link |
that this could contribute knowledge
link |
that will outlast all of our generations,
link |
like all of our lifetimes.
link |
Reach out to novid.org.
link |
What about individual people?
link |
Should they install the app and try it out?
link |
Or is this really geographically restricted?
link |
Oh, yeah, I didn't come on here to tell everyone
link |
to install the app.
link |
I did not come to tell everyone to install the app
link |
because it works best
link |
if your local health authority is working with us.
link |
It's because, this is back to the game theory.
link |
If anyone could just say, I'm positive,
link |
the high school senior prank would be to say that
link |
we have a massive outbreak on finals week.
link |
Let's not have final exams.
link |
So the way that our system works,
link |
it actually borrows some ideas, not borrows,
link |
we came up with them independently.
link |
But this idea is similar to what Google and Apple do,
link |
which is that if the local health authority
link |
is working with this, they can,
link |
for everyone who's positive,
link |
give them a passcode that expires in a short time.
link |
So for ours, if you're on the app and saying, I'm positive,
link |
you can either just say that,
link |
and that's called unverified,
link |
or you can enter in one of these codes
link |
that you got from the local health authority.
link |
So basically, for anyone who's watching this,
link |
it's not that you should just go and download it
link |
unless you want to go and look at it.
link |
But if you, on the other hand,
link |
if you happen to know anyone at the local health authority,
link |
which is trying to figure out how to handle COVID,
link |
well then, I mean, we'd be very happy
link |
to also work with you.
link |
So the verified there is really important
link |
because you're maintaining anonymity.
link |
And because of that,
link |
you have to have some source of verification
link |
in order to make sure that it's not possible to manipulate
link |
because it's ultimately about trust and information.
link |
So it could be, verification is really important there.
link |
So basically, individual people should
link |
ask their local health authorities
link |
to sign up to contact you.
link |
I hope this spreads.
link |
I hope this spreads for future pandemics
link |
because I'm really, it's the amount,
link |
the millions of people who are hurt by this,
link |
I think our response to the virus,
link |
economically speaking,
link |
the number of people who lost their dream,
link |
lost their jobs, but also lost their dream.
link |
Entrepreneurs, jobs often give meaning.
link |
There's people who financially and psychologically
link |
are suffering because of our,
link |
I'll say, incompetent response to the virus
link |
across the world, but certainly the United States,
link |
that should be the beacon of entrepreneurial hope
link |
So I hope that we'll be able to respond
link |
to these kinds of events much better in the future.
link |
And this is exactly the right kind of idea.
link |
And now is the time to do the investment.
link |
Let's step back to the beauty of mathematics.
link |
Maybe ask the big, silly question first,
link |
which is, what do you find beautiful about mathematics?
link |
I think that being able to look at a complicated problem,
link |
which looks unsolvable,
link |
and then to be able to change the perspective
link |
to come from a different angle
link |
and suddenly see that there's a nice solution.
link |
I don't mean that every problem in math
link |
is supposed to be this way,
link |
but I think that these reframings
link |
and changing of perspectives
link |
that cause difficult things to get simplified
link |
and crystallized and factored in certain ways is beautiful.
link |
Actually, that's related to what we were just talking about
link |
with even this fighting pandemics.
link |
The crystal idea was just quantify proximity
link |
by the number of relationships in the physical network,
link |
instead of just by the feet and meters, right?
link |
It's just that if you change that perspective,
link |
now all of these things follow.
link |
And so mathematics to me is beautiful
link |
in the pure sense just for that.
link |
Yeah, it's quite interesting to see a human civilization
link |
as a network, as a graph,
link |
and our relationships as kind of edges in that graph.
link |
And to then do, outside of just pandemic,
link |
do interesting inferences based on that.
link |
This is true for like Twitter, social networks and so on,
link |
how we expand the kind of things we talk about,
link |
think about sort of politically,
link |
if you have this little bubble, quote unquote,
link |
of ideas that you play with,
link |
it's nice from a recommender system perspective,
link |
how do you jump out of those bubbles?
link |
It's really fascinating.
link |
YouTube was working on that, Twitter's working on that,
link |
but not always so successfully,
link |
but there's a lot of interesting work
link |
from a mathematical and a psychological,
link |
sociological perspective there within those graphs.
link |
But if we look at the cleanest formulation of that,
link |
of looking at a problem from a different perspective,
link |
you're also involved
link |
with the International Mathematics Olympiad,
link |
which takes small, clean problems that are really hard,
link |
but once you look at them differently, can become easy.
link |
But that little jump of innovation is the entire trick.
link |
So maybe at the high level,
link |
can you say what is the International Mathematical Olympiad?
link |
Sure, so this is the competition
link |
for people who aren't yet in college, math competition,
link |
which is the most prestigious one in the entire world.
link |
It's the Olympics of mathematics,
link |
but only for people who aren't yet in college.
link |
Now, the kinds of questions that they ask you to do
link |
are not computational.
link |
Usually you're not supposed to find that the answer is 42.
link |
Instead, you're supposed to explain why something is true.
link |
And the problem is that at the beginning,
link |
when you look at each of the questions,
link |
first of all, you have four and a half hours
link |
to solve three questions, and this is one day,
link |
and then you have a second day,
link |
which is four and a half hours, three questions.
link |
But when you look at the questions,
link |
they're all asking you,
link |
explain why the following thing is true,
link |
which you've never seen before.
link |
And by the way, even though there are six questions,
link |
if you solve any one of them, you're a genius
link |
and you get an honorable mention.
link |
So this is hard to solve.
link |
So what about, is it one person, is it a team?
link |
Ah, so each country can send six people
link |
and the score of the country is actually unofficial.
link |
There's not an official country versus country system,
link |
although everyone just adds up the point scores
link |
of the six people and they say,
link |
well, now which country stacked up where?
link |
Yeah, so maybe as a side comment,
link |
I should say that there's a bunch of countries,
link |
including the former Soviet Union and Russia,
link |
where I grew up, where this is one of the
link |
most important competitions that the country participates in.
link |
It was a source of pride for a lot of the country.
link |
You look at the Olympic sports,
link |
like wrestling, weightlifting,
link |
there's certain sports and hockey
link |
that Russia and the Soviet Union truly took pride in.
link |
And actually the Mathematical Olympiad,
link |
it was one of them for many years.
link |
It's still one of them.
link |
And that's kind of fascinating.
link |
We don't think about it this way in the United States.
link |
Maybe you can correct me if I'm wrong,
link |
but it's not nearly as popular in the United States
link |
in terms of its integration into the culture,
link |
into just basic conversation, into the pride.
link |
Like, if you won an Olympic gold medal
link |
or if you win the Super Bowl, you can walk around proud.
link |
I think that was the case
link |
with the Mathematical Olympiad in Russia.
link |
Not as much the case in the United States, I think.
link |
So I just wanna give that a little aside
link |
because beating anybody from Russia,
link |
from the Eastern Republic or from China
link |
is very, very difficult.
link |
Like, if I remember correctly,
link |
there's people, this was a multiyear training process.
link |
And this is everything that they're focused on.
link |
My dad was a participant in this.
link |
And it's, I mean, it's as serious as Olympic sports.
link |
You think about like gymnastics,
link |
like young athletes participating in gymnastics.
link |
This is as serious as that, if not more serious.
link |
So I just wanna give that a little bit of context
link |
because we're talking about serious high level math,
link |
athletics almost here.
link |
Yeah, and actually I also think that it made sense
link |
from the Soviet Union's perspective
link |
because if you look at what these people do eventually,
link |
even though, let's look at the USSR's
link |
International Math Olympiad record.
link |
Even though they, I say, even though they won
link |
a lot of awards at the high school thing,
link |
many of them went on to do incredible things
link |
in research mathematics or research other things.
link |
And that's showing the generalization,
link |
generalizability of what they were working on.
link |
Because ultimately we're just playing with ideas
link |
of how to prove things.
link |
And if you get pretty good at inventing creative ways
link |
to turn problems apart, split them apart,
link |
observe neat ways to turn messy things into simple crystals.
link |
Well, if you're gonna try to solve any real problem
link |
in the real world, that could be a really handy tool too.
link |
So I don't think it was a bad investment.
link |
I think it clearly worked well for Soviet Union.
link |
Yeah, so this is interesting.
link |
People sometimes ask me, you know,
link |
you go up and under communism, you know,
link |
was there anything good about communism?
link |
And it's difficult for me to talk about it
link |
because it's not, communism is one of those things
link |
that's looked down on like without,
link |
in absolutist terms currently.
link |
But you could still, in my perspective,
link |
talk about the actual, forget communism
link |
or whatever the actual term is,
link |
but you know, certain ways that the society functioned
link |
that we can learn lessons from.
link |
And one of the things in the Soviet Union
link |
that was highly prized is knowledge,
link |
not even knowledge, it's wisdom
link |
and the skill of invention, of innovation at a young age.
link |
So we're not talking about a selection process
link |
where you pick the best students in the school
link |
to do the mathematics or to read literature.
link |
It's like, everybody did it.
link |
Everybody, it was almost treated
link |
as if anyone could be the next Einstein,
link |
anybody could be the next, I don't know,
link |
Hemingway, James Joyce.
link |
And so you're forcing an education on the populace
link |
and a rigorous deep education,
link |
like as opposed to kind of like,
link |
oh, we wanna make sure we teach
link |
to the weakest student in the class,
link |
which American systems can sometimes do
link |
because we don't wanna leave anyone behind.
link |
The Russian system was anyone can be the strongest student
link |
and we're gonna teach you the strongest student
link |
and we're going to pretend or force everybody,
link |
even the weakest student to be strong.
link |
And what that results in, it's obviously,
link |
this is what people talk about,
link |
is a huge amount of pressure.
link |
Like it's psychologically very difficult.
link |
This is why people struggle when they go to MIT,
link |
this very competitive environment.
link |
It can be very psychologically difficult,
link |
but at the same time,
link |
it's bringing out the best out of people.
link |
And that mathematics was certainly one of those things.
link |
And exactly what you're saying,
link |
which kind of clicked with me just now,
link |
as opposed to kind of a spelling bee in the United States,
link |
which I guess you spell, I'm horrible at this,
link |
but it's a competition about spelling,
link |
which I'm not sure, but you could argue
link |
it doesn't generalize well to the future skills.
link |
Mathematics, especially this kind of mathematics
link |
is essentially formalized competition of invention,
link |
of creating new ideas.
link |
And that generalizes really, really well.
link |
So that's quite brilliantly put.
link |
I didn't really think about that.
link |
So this is not just about the competition.
link |
This is about developing minds
link |
that will come to do some incredible stuff in the future.
link |
Yeah, actually, I want to respond
link |
to a couple of things there.
link |
The first one, this one, which is this notion
link |
of whether or not that is possible
link |
in a non authoritarian regime.
link |
And that's actually why I spent some of my efforts
link |
before the COVID thing,
link |
actually trying to work towards there.
link |
The reason is because if you think about it,
link |
let's say in America,
link |
lots of people are pretty serious
link |
about training very hard for football,
link |
or baseball, or basketball.
link |
Basketball is very, very accessible,
link |
but lots of people are doing that.
link |
Well, actually, I think that what was going on
link |
with the authoritarian thing was at least the message
link |
that was universally sent was being a good thinker
link |
and a creator of ideas is a good thing.
link |
There's no reason why that message can't be sent everywhere.
link |
And I think it actually should be.
link |
So that's the first thing.
link |
The second thing is what you commented about this thing
link |
about the generalizable skill
link |
and what could people do with Olympiads afterwards.
link |
So that's actually my interest in the whole thing.
link |
I don't just coach students how to do problems.
link |
In fact, I'm not even the best person for that.
link |
I'm not the best at solving these problems.
link |
There are other people who are much better
link |
at making problems and teaching people how to solve problems.
link |
In fact, when the Mathematical Association of America,
link |
which is the group which is in charge
link |
of the US participation in these Olympiads,
link |
when they were deciding whether or not to put me in
link |
back in 2013 as the head coach,
link |
I had a conversation with their executive director
link |
where I commented that we might do worse
link |
because my position was I don't,
link |
I mean, I actually didn't want to focus on winning.
link |
I said, if you're going to let me work
link |
with 60 very strong minds as picked through this system,
link |
because the coach works with these,
link |
gets to run a camp for these students.
link |
I said, I'm actually not going to define my success
link |
in terms of winning this contest.
link |
I said, I wanted to maximize the number of the students
link |
that I read about in the New York Times in 20 years.
link |
And the executive director
link |
of the Mathematical Association of America
link |
was fully in support of this
link |
because that's also how their philosophy is.
link |
So in America, the way we run this
link |
is we're actually not just training to win,
link |
even though the students are very good
link |
and they can win anyway.
link |
One reason, for example, I went and even did the COVID thing
link |
involving quite a few of them
link |
is so that hopefully some of them get ideas
link |
because in 20, 30 years, I won't have the energy
link |
or the insight to solve problems.
link |
We'll have another catastrophe.
link |
And hopefully some of these people will step up and do it.
link |
And ultimately have that longterm impact.
link |
I wonder if this is scalable to,
link |
because that's such a great metric for education,
link |
not how to get an A on the test, but how to have,
link |
how to be on the cover of New York Times
link |
for inventing something new.
link |
And do you think that's generalizable to education
link |
beyond just this particular Olympia?
link |
Like, even you saying this feels like a rare statement,
link |
almost like a radical statement as a goal for education.
link |
So actually the way I teach my classes at Carnegie Mellon,
link |
which I will admit right away is not equivalent
link |
to the average in the world,
link |
but it's already not just the top 60 in the country
link |
as picked by something.
link |
Let me just explain.
link |
I have exams in my class, which are 90% of the grade.
link |
So the exams are the whole thing,
link |
or most of the whole thing.
link |
And the way that I let students prepare for the exams
link |
is I show them all the problems I've ever given
link |
on the previous exams.
link |
And the exam that they will take is open notes.
link |
They can take all the notes they want
link |
on the previous problems.
link |
And the guarantee is that the exam problems this time
link |
will have no overlap with anything
link |
you have seen me give in the past,
link |
as well as no overlap with anything I taught in the class.
link |
So the entire exam is invention.
link |
But that's how I go, right?
link |
My point is I have explained to people when I teach you,
link |
I don't want you to have remembered a method I showed you.
link |
I want you to have learned enough about this area
link |
that if you face a new question,
link |
which I came up with the night before
link |
by thinking about like,
link |
what could I ask that I have never asked before?
link |
That's what the answer is.
link |
Aha, that's an exam problem.
link |
That's exactly what I do before the exam.
link |
And then that's what I want them to learn.
link |
And the first exam, usually people have a rough time
link |
because it's like, what kind of crazy class is this?
link |
The professor doesn't teach you anything for the exam.
link |
But then by the second or third,
link |
and by the time they finished the class,
link |
they have learned how to solve anything in the area.
link |
How to invent in that area, yeah.
link |
Can we walk back to the Mathematical Olympiad?
link |
What's the scoring and format like?
link |
And also what does it take to win?
link |
So the way it works is that each of the six students
link |
do the problems and there are six problems.
link |
All the problems are equally weighted.
link |
So each one's worth seven points.
link |
That means that your maximum score
link |
is six problems times seven points,
link |
which is the nice number of 42.
link |
And now the way that they're scored by the way
link |
is there's partial credit.
link |
So the question is asking you,
link |
explain why this weird fact is true.
link |
Okay, if you explain why you get seven points.
link |
If you make minor mistake, maybe you get six points.
link |
But if you don't succeed in explaining why,
link |
but you explain some other true fact,
link |
which is along the way of proving it,
link |
then you get partial credit.
link |
And actually now this is tricky
link |
because how do you score such a thing?
link |
It's not like the answer was 72
link |
and you wrote 71 and it's close, right?
link |
The answer is 72 and you wrote 36.
link |
Oh, but that's pretty close
link |
because maybe you're just off by it.
link |
By the way, they're not numerical anyway,
link |
but I'm just giving some numerical analog
link |
to the way the scoring might work.
link |
They're all essays.
link |
And that's where I guess I have some role
link |
as well as some other people
link |
who helped me in the US delegation for coaches.
link |
We actually debate with the country which is organizing it.
link |
The country which is organizing the Olympiad
link |
brings about 50 people to help judge the written solutions.
link |
And you schedule these half hour appointments
link |
where the delegation from one country
link |
sits down at a table like this.
link |
Opposite side is two or three people from the host country.
link |
And they're just looking over these exam papers
link |
saying, well, how many points is this worth
link |
based on some rubric that has been designed?
link |
And this is a negotiation process
link |
where we're not trying to bargain
link |
and get the best score we can.
link |
In fact, sometimes we go to this table
link |
and we will say, we think we want less than what you gave us.
link |
This is how our, these are our principles.
link |
If you give us too much, we say, no, you gave us too much.
link |
However, the reason why this is an interesting process
link |
is because if you can imagine every country
link |
which is participating has its own language.
link |
And so if you're trying to grade the Mongolian scripts
link |
and they're written in Mongolian,
link |
if you don't read Mongolian, which most people don't,
link |
then the coaches are explaining to you,
link |
this is what the student has written.
link |
It's actually quite interesting process.
link |
So it's almost like a jury.
link |
You have, in the American legal system,
link |
you have a jury that where they're deliberating,
link |
but unlike a jury, there's the members of the jury
link |
speaking different languages sometimes.
link |
Yes. That's fascinating.
link |
But I mean, it's hard to know what to do
link |
because it's probably really, really competitive.
link |
But your sense is that ultimately people,
link |
like how do you prevent manipulation here, right?
link |
Well, we just hope that it's not happening.
link |
So we write in English.
link |
Therefore, everything that the US does,
link |
everyone can look at.
link |
So it's very hard for me.
link |
It's very hard for you to manipulate.
link |
We don't manipulate.
link |
We only hope that other people aren't.
link |
But at the same time, as you see, our philosophy was,
link |
we want to use this as a way to develop general talent.
link |
And although we do this for the six people who go
link |
to the International Math Olympiad,
link |
we really want that everyone at any,
link |
touched at any stage of this process
link |
get some skills that can help to contribute more later.
link |
So I don't know if you can say something insightful
link |
but what do you think makes a really hard math problem
link |
on this Olympiad, maybe in the courses you teach
link |
What makes for a hard problem?
link |
You've seen, I'm sure, a lot of really difficult problems.
link |
What makes a hard problem?
link |
So I could quantify it by the number of leaps of insight
link |
of changes of perspective that are along the way.
link |
This is like a very theoretical computer science
link |
way of looking at it, okay?
link |
It's that each reframing of the problem
link |
and using of some tool,
link |
I actually call that a leap of insight.
link |
When you say, oh, wow, now I see,
link |
I should kind of put these plugs into those sockets
link |
like so, and suddenly I get to use that machine.
link |
Oh, but I'm not done yet.
link |
Now I need to do it again.
link |
Each such step is a large possible,
link |
large fan out in the search space.
link |
The number of these tells you the exponent.
link |
The base of the exponent is like how big,
link |
how many different possibilities you could try.
link |
And that's actually why,
link |
like if you have a three insight problem,
link |
that is not three times as hard as a one insight problem,
link |
because after you've made the one insight,
link |
it's not clear that that was the right track necessarily.
link |
Well, unless you're very into it.
link |
There's still a branching of possibility.
link |
You're saying there's problems like on the math Olympia
link |
that requires more than one insight?
link |
Those are the hard ones.
link |
And also I can tell you how you can tell.
link |
So this is how I also taught myself math
link |
when I was in college.
link |
So if you are taking a, not taught myself,
link |
I was taking classes, of course,
link |
but I was trying to read the textbook
link |
and I found out I was very bad at reading math textbooks.
link |
A math textbook has a long page of stuff that is all true,
link |
which after you read the page,
link |
you have no idea what you just read.
link |
This is just a good summary of a math textbook.
link |
Yeah, because it's not clear why anything was done that way.
link |
And yes, everything is true,
link |
but how the heck did anyone think of that?
link |
So the way that I taught myself math eventually was,
link |
the way I read a math textbook
link |
is I would look at the theorem statement.
link |
I would look at the length of the proof
link |
and then I would close the book
link |
and attempt to reproof it myself.
link |
The length of the proof is telling you
link |
the number of insights,
link |
because the length of the proof is linear
link |
in the number of insights.
link |
Each insight takes space.
link |
And if I know that it's a short proof,
link |
I know that there's only one insight.
link |
So when I'm doing my own way of solving the problem,
link |
like finding the proof,
link |
I quit if I have to do too many plugins.
link |
It's equivalent to a math contest.
link |
In a math contest I look,
link |
is it problem one, two, or three?
link |
That tells me how many insights there are.
link |
This is exactly what I did.
link |
Linear in the number.
link |
I think it's possible that that's true.
link |
Approximately, approximately.
link |
Approximately, yeah.
link |
I don't know if somebody out there
link |
is gonna try to formally prove this.
link |
Oh no, I mean, you're right.
link |
There are cases where maybe it's not quite linear,
link |
Well, some of it's notation too,
link |
and some of it is style and all those kinds of things,
link |
but within a textbook.
link |
Within the same book.
link |
Within the same book with the same.
link |
Within the same book on the same subject.
link |
This is what I was using.
link |
Because you know, if it's a two page proof,
link |
you just know this is gonna be insane, right?
link |
That's the scary thing about insights.
link |
You look like Andrew Wiles
link |
working on the Fermat's Last Theorem,
link |
is you don't know.
link |
Something seems like a good idea,
link |
and you have that idea,
link |
and it feels like this is a leap,
link |
like a totally new way to see it,
link |
but you have no idea if it's at all useful.
link |
Even if you think it's correct,
link |
you have no idea if this is like going to go down a path
link |
that's completely counterproductive
link |
or not productive at all.
link |
That's the crappy thing about invention,
link |
is like I have, I'm sure you do.
link |
I have a lot of really good ideas every single day,
link |
but like, and I'll go inside my head along them,
link |
along that little trajectory,
link |
but it could be just a total waste.
link |
And it's, you know what that feels like?
link |
It just feels like patience is required,
link |
not to get excited at any one thing.
link |
So I think this is interesting
link |
because you raised Andrew Wiles.
link |
He spent seven years attacking the same thing, right?
link |
And so I think that what attracts
link |
professional researchers to this
link |
is because even though it's very painful
link |
that you keep fighting with something,
link |
when you finally find the right insights
link |
and string them together,
link |
it feels really good, so.
link |
Well, there's also like short term,
link |
it feels good to, whether it's real or not,
link |
to pretend like you've solved something
link |
in the sense like you have an insight
link |
and there's a sense like this might be the insight
link |
So at least for me, I just enjoy that rush of positivity
link |
even though I know statistically speaking
link |
is probably going to be a dead end.
link |
I'm the same way, I'm the same way.
link |
In fact, that's how I know whether
link |
I might want to keep thinking about this general problem.
link |
It's like, if I still see that I'm getting some insights,
link |
I'm not at a dead end yet.
link |
But that's also where I learned something
link |
from my PhD advisor.
link |
Actually, he was a real big inspiration on my life.
link |
His name is Benny Sudakov.
link |
In fact, he grew up in the former Soviet Union.
link |
He was from Georgia, but he's an incredible person.
link |
But one thing I learned was choose the problems to work on
link |
that might matter if you succeed.
link |
Because that's why, for example, we dug into COVID.
link |
It was just, well, suppose we succeed
link |
in finding some interesting insight here.
link |
Well, it actually matters.
link |
That is worth a laugh.
link |
Yeah, and I think COVID, the way you're approaching COVID
link |
has two interesting possibilities.
link |
One, it might help with COVID or another pandemic,
link |
but two, I mean, just this whole network theory space,
link |
you might unlock some deep understanding
link |
about the interaction with human beings.
link |
That might have nothing to do with the pandemic.
link |
There's a space of possible impacts
link |
that may be direct or indirect.
link |
And the same thing is with Andrew Wiles's proof.
link |
I don't understand, but apparently the pieces of it
link |
are really impactful for mathematics,
link |
even if the main theorem is not.
link |
So along the way, the insights you have
link |
might be really powerful for unexpected reasons.
link |
So I like what you said.
link |
This is something that I learned from another friend of mine.
link |
He's a very famous researcher.
link |
All these people are more famous than I am.
link |
His name is Jacob Fox.
link |
He's Jacob Fox at Stanford.
link |
Also a very big inspiration for me.
link |
We were both grad students together at the same time.
link |
Well, most importantly,
link |
you're good at selecting good friends.
link |
Ah, yeah, well, that's the key.
link |
You gotta find good people to learn things from.
link |
But his thing was, he often said,
link |
if you solve a math problem and have this math proof,
link |
math problem for him is like a proof, right?
link |
So suppose you came up with this proof.
link |
He always asks, what have we learned from this
link |
that we could potentially use for something else?
link |
It's not just, did you solve the problem
link |
that was supposed to be famous?
link |
And is there something new in the course of solving this
link |
that you had to invent
link |
that we could now use as a tool elsewhere?
link |
Yeah, there's this funny effect
link |
where just looking at different fields
link |
where people discover parallels.
link |
They'll prove something, it'll be a totally new result.
link |
And then somebody later realizes
link |
this was already done 30 years ago
link |
in another discipline, in another way.
link |
And it's really interesting.
link |
Now, we did this offline
link |
in another illustration he showed to me.
link |
It's interesting to see the different perspectives
link |
It kind of points like there's just like
link |
very few novel ideas that everything else,
link |
that most of us are just looking at different perspective
link |
And it makes you wonder this old silly question
link |
that I have to ask you is,
link |
do you think mathematics is discovered or invented?
link |
Do you think we're creating new idea?
link |
Are we building a set of knowledge
link |
that's distinct from reality?
link |
Or are we actually like,
link |
is math almost like a shovel
link |
where we're digging to like this core set of truths
link |
that were always there all along?
link |
So I personally feel like it's discovered.
link |
But that's also because I guess the way that
link |
I like to choose what questions to work on
link |
are questions that maybe we'll get to learn something about
link |
I mean, I'm often attracted to questions
link |
that look simple, but are hard, right?
link |
And what could you possibly learn from that?
link |
Sort of like probably the attraction
link |
of Fermat's last theorem, as you mentioned,
link |
simple statement, why is it so hard?
link |
So I'm more on the discovered side.
link |
And I also feel like if we ever ran into
link |
an intelligent other species in the universe,
link |
probably if we compared notes,
link |
there might be some similarities between both of us
link |
realizing that pi is important.
link |
Because you might say, why, why humans,
link |
do humans like circles more than others?
link |
I think stars also like circles.
link |
I think planets like circles.
link |
They're not perfect circles,
link |
but nevertheless, the concept of a circle
link |
is just point and constant distance.
link |
Doesn't get any simpler than that.
link |
It's possible that like an alien species
link |
will have, depending on different cognitive capabilities
link |
and different perception systems,
link |
will be able to see things
link |
that are much different than circles.
link |
And so if it's discovered,
link |
it will still be pointing at a lot of same
link |
geometrical concepts, mathematical concepts,
link |
but it's interesting to think of how many things
link |
we would have to still align,
link |
not just based on notation, but based on understanding,
link |
like just like some basic mathematical concepts,
link |
like how much work is there going to be
link |
in trying to find a common language?
link |
I mean, this is, I think Stephen Wolfram and his son
link |
helped with the movie Arrival,
link |
like the developing an alien language,
link |
like how would aliens communicate with humans?
link |
because like math seems to be the most promising thing,
link |
but even like math,
link |
like how do you visualize mathematical ideas?
link |
It feels like there has to be an interactive component,
link |
just like we have a conversation.
link |
There has to be, this is something we don't,
link |
I think, think about often, which is like,
link |
with somebody who doesn't know anything about math,
link |
doesn't know anything about English
link |
or any other natural language,
link |
how would we describe,
link |
we talked offline about visual proofs.
link |
How would we, through visual proofs, have a conversation
link |
where we say something, here's the concept,
link |
the way we see it, does that make sense to you?
link |
And like, can you mess with that concept
link |
to make it sense for you?
link |
And then go back and forth in this kind of way.
link |
So purely through mathematics,
link |
I'm sure it's possible to have those kinds of experiments
link |
with like tribes on earth that don't,
link |
there's no common language.
link |
Through math, like draw a circle
link |
and see what they do with it, right?
link |
Do some of these visual proofs,
link |
like the summation of the odds and adds up to the squares.
link |
Yes, I wonder how difficult that is
link |
before one or the other species murders themselves.
link |
That's a good question.
link |
I hope that the curiosity for knowledge
link |
will overpower the greedy,
link |
this is back to our game theory thing,
link |
that the curiosity of like discovering math together
link |
will overpower the desire for resources
link |
and ultimately like willing to commit violence
link |
in order to gain those resources.
link |
I think as we progress,
link |
become more and more intelligent as a species,
link |
I'm hoping we would value more and more of the knowledge
link |
because we'll come up with clever ways
link |
to gain more resources so we won't be so resource starved.
link |
That's a hopeful message for when we finally meet aliens.
link |
The cool thing about the Math Olympiad,
link |
I don't know if you know work from Francois Chollet
link |
from Google, he came up with this kind of IQ test slash,
link |
it kind of has similar aspects to it
link |
that also the Math Olympiad does for AI.
link |
So he came up with these tests
link |
where they're very simple for humans,
link |
but very difficult for AI to illustrate exactly
link |
why we're just not good at seeing a totally new problem.
link |
Sorry, AI systems are not good at looking at a new problem
link |
that requires you to detect
link |
that there's a symmetry of some kind,
link |
or there's a pattern that hasn't seen before.
link |
The pattern is like obvious to us humans,
link |
but it's not so obvious to find that kind of,
link |
you're inventing a pattern that's there
link |
in order to then find a solution.
link |
I don't know if you can comment on that.
link |
If you can comment on, but from an AI perspective
link |
and from a math problem perspective,
link |
what do you think is intelligence?
link |
What do you think is the thing
link |
that allows us to solve that problem?
link |
And how hard is it to build a machine to do that?
link |
Asking for a friend.
link |
So I guess, you see,
link |
because if I just think of the raw search space, it's huge.
link |
That's why you can't do it.
link |
And if I think about what makes somebody
link |
good at doing these things, they have this heuristic sense.
link |
It's almost like a good chess player of saying,
link |
let's not keep analyzing down this way
link |
because there's some heuristic reason
link |
why that's a bad way to go.
link |
Where did they get that heuristic from?
link |
Now, that's a good question.
link |
Because that, if you asked them to explain to you,
link |
they could probably say something in words
link |
that sounds like it makes sense,
link |
but I'm guessing that's only a part
link |
of what's really going on in their brain
link |
of evaluating that position.
link |
You know what I mean?
link |
If you ask Gary Kasparov, what is good,
link |
or why is this position good, he will say something,
link |
but probably not approximating everything
link |
that's going on inside.
link |
So there's basically a function being computed,
link |
but it's hard to articulate what that function is.
link |
Now, the question is, could a computer get as good
link |
at computing these kinds of heuristic functions?
link |
I'm not enough of an expert to understand,
link |
but one bit of me has always been a little bit curious
link |
of whether or not the human brain has a particular tendency
link |
due to its wiring to come up with certain kinds of things,
link |
which is just natural due to the way
link |
that the topology of the neurons and whatever is there,
link |
for which if you tried to just build from scratch
link |
a computer to do it,
link |
would it naturally have different tendencies?
link |
This is just me being completely ignorant
link |
and just saying a few ideas.
link |
Well, this is a good thing that mathematics shows
link |
is we don't have to be,
link |
so math and physics or mathematical physics
link |
operates in a world that's different
link |
than our descendants of eight brains operate in.
link |
So it allows us to have multiple, many, many dimensions.
link |
It allows us to work on weird surfaces.
link |
I would like topology as a discipline is just weird to me.
link |
It's really complicated,
link |
but it allows us to work in that space,
link |
the differential geometry and all those kinds of things
link |
where it's totally outside of our natural day to day
link |
four dimensional experience,
link |
3D dimensional with time experience.
link |
So math gives me hope that we can discover
link |
the processes of intelligence outside the limited nature
link |
of our own human experiences.
link |
But you said that you're not an expert.
link |
It's kind of funny.
link |
I find that we know so little about intelligence
link |
that I honestly think like almost children are more expert
link |
at creating artificial intelligence systems than adults.
link |
I feel like we know so little,
link |
we really need to think outside the box.
link |
I found people should check out
link |
Francois Chollet's little exams,
link |
but even just solving math problems,
link |
I don't know if you've ever done this for yourself,
link |
but when you solve a math problem,
link |
you kind of then trace back and try to figure out
link |
where did that idea come from?
link |
Like what was I visualizing in my head?
link |
How did I start visualizing it that way?
link |
Why did I start rotating that cube in my head in that way?
link |
Like what is that?
link |
If I were to try to build a program that does that,
link |
where did that come from?
link |
So this is interesting.
link |
So I try to do this to teach middle school students
link |
how to learn how to create and think and invent.
link |
And the way I do it
link |
is there are these math competition problems
link |
and I'm working in collaboration
link |
with the people who run those.
link |
And I will turn on my YouTube live
link |
and for the first time,
link |
look at those questions and live solve them.
link |
The reason I do this is to let the middle school students
link |
and the high school students and the adults
link |
whoever wants to watch,
link |
just see what exactly goes on through someone's head
link |
as they go and attempt to invent what they need to do
link |
to solve the question.
link |
So I've actually thought about that.
link |
I think that, first of all, as a teacher,
link |
I think about that because whenever I want to explain
link |
to a student how to do something,
link |
I want to explain how it made sense,
link |
why it's intuitive to do the following things
link |
and why the wrong things are wrong.
link |
Not just why this one short fast way,
link |
well, why this is the right way, if that makes sense.
link |
So my point is I'm actually always thinking about that.
link |
Like how would you think about these things?
link |
And then I eventually decided the easiest way
link |
to expose this would just be to go live on YouTube
link |
and just say, I've never seen any of these questions before.
link |
Don't you get, that's anxiety inducing for me.
link |
Don't you get trapped in a kind of like little dead ends
link |
of confusion, even on middle school problems?
link |
Yes, that's what the comments are for.
link |
The live comments come in and students say, try this.
link |
It's actually pretty good.
link |
And I'll never get stuck.
link |
I mean, I'm willing to go on camera and say,
link |
guess what, Potion Dough can't do this.
link |
But then what ends up happening is you will then see
link |
how maybe somebody saying something and I look at the chat
link |
and I say, aha, that actually looks useful.
link |
Now that also shows how not all ideas,
link |
not all suggestions are the same power, if that makes sense.
link |
Because if I actually do get stuck,
link |
I'll go fishing through the chat, if you've got any ideas.
link |
I don't know if you can speak to this,
link |
but is there a moment for the middle school students,
link |
maybe high school as well,
link |
where there's like a turning point for them
link |
where they maybe fall in love with mathematics
link |
Is there something to be said about like discovering
link |
that moment and trying to grab them to get them
link |
to understand that mathematics is something,
link |
no matter what they wanna do in life
link |
could be part of their life?
link |
I actually do think that the middle school
link |
is exactly the right time
link |
because that's the place
link |
where your mathematical understanding
link |
gets just sophisticated enough
link |
that you can start doing interesting things.
link |
Because if you're early on and counting,
link |
I'm honestly not very good at teaching you new insights.
link |
My wife is pretty good at that.
link |
But somehow once you get to this part
link |
where you know what a fraction is
link |
and when you know how to add and how to multiply
link |
and what the area of a triangle is,
link |
at that point to me, the whole world opens up
link |
and you can start observing
link |
there are really nifty coincidences,
link |
the things that made the Greek mathematicians
link |
and the ancient mathematicians excited.
link |
Actually back then it was exciting
link |
to discover the Pythagorean theorem.
link |
It wasn't just homework.
link |
which discipline do you think
link |
has the most exciting coincidences?
link |
So is it geometry?
link |
Or is it calculus?
link |
Well, you see, you're asking me
link |
and I'm the guy who gets the most excited
link |
when the combinatorics shows up in the geometry.
link |
So it's the combinatorics in the geometry.
link |
So first of all, the nice thing about geometry,
link |
this is the same nice thing about computer vision
link |
So geometry, you can draw circles and triangles and stuff.
link |
So it naturally presents itself
link |
to the visual proof, right?
link |
But also the nice thing about geometry,
link |
I think for me is the earliest class,
link |
the earliest discipline where there's,
link |
that's most amenable to the exploration,
link |
the invention through proofs.
link |
The idea of proofs I think is most easily shown in geometry
link |
because it's so visual, I guess.
link |
So that to me is like,
link |
if I were to think about
link |
when I first fell in love with math, it would be geometry.
link |
And sadly enough, that's not used.
link |
Geometry only has a little,
link |
appears briefly in the journey of a student.
link |
And it kind of disappears.
link |
And not until much later,
link |
which there may be like differential geometry,
link |
I don't know where else it shows up.
link |
For me in computer science,
link |
like you could start to think about
link |
like computational geometry or even graph theory
link |
as a kind of geometry.
link |
You could start to think about it visually,
link |
although it's pretty tricky.
link |
But yeah, it was always,
link |
that was the most beautiful one.
link |
Everything else, I guess calculus can be kind of visual too.
link |
That can be pretty beautiful.
link |
But is there something you try to look for in the student
link |
to see like, how can I inspire them at this moment?
link |
Or is this like individual student to student?
link |
Is there something you could say there?
link |
I really think that every student
link |
can pick up all of this skill.
link |
I really do think so.
link |
I don't think it's something only for a few.
link |
And so if I'm looking for a student,
link |
actually oftentimes what I'm,
link |
if I'm looking at a particular student,
link |
how can we help you feel like
link |
you have the power to invent also?
link |
Because I think a lot of people
link |
are used to thinking about math
link |
as something where the teacher will show you what to do
link |
and then you will do it.
link |
So I think that the key is to show that they have some,
link |
let them see that they have some power to invent.
link |
And at that point,
link |
it's often starting by trying to give a question
link |
that they don't know how to do.
link |
You want to find these questions
link |
that they don't know how to do,
link |
that they can think about,
link |
and then they can solve.
link |
And then suddenly they say,
link |
my gosh, I've had a situation,
link |
I've had an experience where I didn't know what to do.
link |
And after a while, I did.
link |
Is there advice you can give on how to learn math
link |
for people, whether it's a middle school,
link |
whether it's somebody as an adult
link |
kind of gave up on math maybe early on?
link |
I actually think that these math competition problems,
link |
middle school and high school are really good.
link |
They're actually very hard.
link |
So if you haven't had this kind of experience before
link |
and you grab a middle school math competition problem
link |
from the state level,
link |
which is used to decide who represents the state
link |
in the country, in the United States, for example,
link |
those are pretty tricky.
link |
And even if you are a professional,
link |
maybe not doing mathematical things
link |
and you're not a middle school student, you'll struggle.
link |
So I find that these things really do teach you things
link |
by trying to work on these questions.
link |
Is there a Googleable term that you could use
link |
for the organization, for the state competitions?
link |
So there are a number of different ones
link |
that are quite popular.
link |
One of them is called Math Counts, M A T H C O U N T S.
link |
And that's a big tournament,
link |
which actually has a state level.
link |
There's also a mathleague.org,
link |
mathleague, L E A G U E dot org,
link |
also has this kind of tiered tournament structure.
link |
There's also the American math competitions, AMC 8.
link |
AMC also has AMC 10, that's for 10th grade and below
link |
These are all run by the Mathematical Association
link |
And these are all ways to find old questions.
link |
What about the daily challenges that you run?
link |
What are those about?
link |
But I mean, the difference was ours isn't,
link |
that one's not free.
link |
So I should actually probably be careful.
link |
The things that I've just mentioned are also not free.
link |
Not all of those things I mentioned just now
link |
Well, people can figure out what is free and what's not,
link |
but this is really nice to know what's out there.
link |
But can you speak a little bit to the daily challenges?
link |
So that's actually what we did when,
link |
I guess I was thinking about,
link |
how would I try to develop that skill in people
link |
if we had the power to architect the entire system ourselves?
link |
So that's called the daily challenge with Po Shan Luo.
link |
It's not free because that's actually how I pay
link |
for everything else I do.
link |
So that was the idea.
link |
But the concept was, aha, now let's invent from scratch.
link |
So if we're gonna go from scratch
link |
and we're gonna use technology,
link |
what if we made every single lesson
link |
something where first I say,
link |
hey, here's an interesting question.
link |
Recorded, of course, not live.
link |
But it's like, I say,
link |
hey, here's an interesting question.
link |
Why don't we think about this?
link |
But I know you don't know how to do it.
link |
and a minute later a hint pops on the screen.
link |
But you still think.
link |
And a minute later a big hint pops on the screen.
link |
And then finally, after the three minutes,
link |
hopefully you got some ideas you tried to answer.
link |
And then suddenly there's like this
link |
pretty extended explanation of,
link |
oh yeah, so here's like multiple different ways
link |
that you can do the question.
link |
And by accident, you also just learned this other concept.
link |
That's what we did.
link |
Is this targeted towards middle school students,
link |
high school students?
link |
It's targeted towards middle school students
link |
with competitions.
link |
But there's a lot of high school students
link |
who didn't do competitions in middle school
link |
where they would also learn how to think.
link |
If you can see the whole concept was,
link |
can we teach people how to think?
link |
How would you do that?
link |
You need to give people the chance to,
link |
on their own, invent without that kid in the front row
link |
answering every question in two seconds.
link |
And people can find it, I think, what daily.
link |
It's daily.potionlo.com.
link |
But if you go to find my website,
link |
you'll be able to find it.
link |
Can we zoom out a little bit in the,
link |
so day to day, week to week, month to month,
link |
year to year, what does the lifelong educational process
link |
look like, do you think?
link |
For yourself, but for me,
link |
what would you recommend in the world of mathematics
link |
or sort of as opposed to studying for a test,
link |
but just like lifelong expanding of knowledge
link |
in that skill for invention?
link |
I think I often articulate this as,
link |
can you always try to do more than you could do in the past?
link |
But that comes in many ways.
link |
And I will say it's great
link |
if one wants to build that with mathematics,
link |
but it's also great to use that philosophy
link |
with all other things.
link |
In fact, if I just think of myself, I just think,
link |
what do I know now that I didn't know a year ago
link |
or a month ago or a week ago?
link |
And not just know,
link |
but what do I have the capability of doing?
link |
And if you just have that attitude, it brings more.
link |
See, the thing is, there's also a habit,
link |
like it is a skill, like I've been using Anki,
link |
it's an app for helps you memorize things.
link |
And I've actually, a few months ago,
link |
started doing this daily of setting aside time
link |
to think about an idea that's outside of my work.
link |
Like, let's say, it's all over the place, by the way,
link |
but let's say politics, like gun control.
link |
Is it good to have a lot of guns or not in society?
link |
And just, I've set aside time every day,
link |
I do at least 10 minutes, but I try to do 30,
link |
where I think about a problem.
link |
And I kind of outline it for myself from scratch,
link |
from not looking anything up,
link |
just thinking about it, using common sense.
link |
And I think the practice of that is really important.
link |
It's the daily routine of it, it's the discipline of it.
link |
It's not just that I figured something out
link |
from thinking about gun control,
link |
it's more that that muscle is built too,
link |
it's that thinking muscle.
link |
So I'm kind of interested in, you know, math has,
link |
because especially because I've gotten specialized
link |
into machine learning,
link |
and because I love programming so much,
link |
I've lost touch with math a little bit
link |
to where I feel quite sad about it,
link |
and I want to fix that.
link |
Even just not math, like pure knowledge math,
link |
but math, like these middle school problems,
link |
the challenges, right?
link |
Is that something you see a person
link |
be able to do every single day,
link |
kind of just practice every single day for years?
link |
So I can give an answer to that,
link |
that gives a practical way you could do it,
link |
assuming you have kids.
link |
So, no, you can do it yourself.
link |
Step one, get kids.
link |
No, no, I'm just saying this
link |
because I'm just thinking out loud right now,
link |
what could I do to suggest?
link |
Because what I have noticed is that, for example,
link |
if you do have kids who are in elementary school
link |
or middle school, if you yourself go and look
link |
at those middle school math problems
link |
to think about interesting ways
link |
that you can teach your elementary school
link |
or middle school kid, it works.
link |
That's what my wife did.
link |
She never did any of those contests before,
link |
but now she knows quite a lot about them.
link |
And I didn't teach her anything.
link |
She just was messing around with them
link |
and taught herself all of that stuff.
link |
And that had the automatic daily.
link |
I'm always thinking, how do you make it practical, right?
link |
And the way to make it practical
link |
is if the timer on the automatically daily
link |
is that you are going to automatically daily
link |
do something with your own kid.
link |
Now it feeds back.
link |
And that includes the whole lesson
link |
that if you wanna learn something, you should teach it.
link |
Oh, I strongly believe that.
link |
I strongly believe that.
link |
So I currently don't have kids.
link |
So that's, maybe I should just get kids
link |
to help me with the math thing.
link |
But outside of that,
link |
I do want to do great math into daily practice.
link |
So I'll definitely check out the daily challenges
link |
and see, because what is it?
link |
Grant Sanderson, we talked about offline,
link |
the three blue and brown.
link |
He speaks to this as well,
link |
that his videos aren't necessarily,
link |
they don't speak to the thing that I'm referring to,
link |
which is the daily practice.
link |
They're more almost tools of inspiration.
link |
They kind of show you the beauty
link |
of a particular problem in mathematics,
link |
but they're not a daily ritual.
link |
And I'm in search of that daily ritual mathematics.
link |
It's not trivial to find,
link |
but I hope to find that
link |
because I think math gives you a perspective on the world
link |
that enriches everything else.
link |
So I like what you said about the daily also,
link |
because that's also one reason
link |
why I put my Carnegie Mellon class online.
link |
It's not every day.
link |
It's every other day.
link |
Semester is almost over.
link |
But the idea was, I guess my philosophy was,
link |
if I'm already doing the class,
link |
let's just put it there, right?
link |
But I do know that there are people
link |
who have been following it,
link |
who are not in my class at all,
link |
who have just been following it because,
link |
yes, it's combinatorics.
link |
And the value of that is you could,
link |
you don't really need to know calculus to follow it,
link |
if that makes sense.
link |
So it's actually something that people could follow.
link |
So again, and that one's free.
link |
So that one's just there on YouTube.
link |
Well, speaking of combinatorics,
link |
what is it, what do you find interesting,
link |
what do you find beautiful about combinatorics?
link |
So combinatorics to me is the study of things
link |
where they might be more finite and more discreet.
link |
What I mean is like, if I look at a network,
link |
actually a lot of times the combinatorics
link |
will boil down to something,
link |
and the combinatorics I think about
link |
might be something related to graphs or networks.
link |
And they're very discreet because if you have a node,
link |
it's not that you have 0.7 of a node
link |
and 0.3 of a node over there.
link |
It's that you've got one node,
link |
and then you jump one step to go to the next node.
link |
So that notion is different from say, calculus,
link |
which is very continuous,
link |
where you go and say, I have this speed,
link |
which is changing over time.
link |
And now what's the distance I've traveled?
link |
That's the notion of an integral,
link |
where you have to think of subdividing time
link |
into very, very small pieces.
link |
So the kinds of things that you do
link |
when you reason about these finite discreet structures
link |
often might be iterative, algorithmic, inductive.
link |
These are ideas where I go from one step to the next step
link |
and so on and make progress.
link |
I guess I actually personally like all kinds of math.
link |
My area of research just ended up in here
link |
because I met a really interesting PhD advisor,
link |
potential, that's honestly the reason
link |
I went into that direction.
link |
I met a really interesting guy.
link |
He seemed like he did good stuff, interesting stuff,
link |
and he looked like he cared about students.
link |
And I said, let me just go and learn whatever you do,
link |
even though my prior practice and preparation
link |
before my PhD was not combinatorics,
link |
but analysis, the continuous stuff.
link |
So the annoying thing about combinatorics
link |
and discreet stuff is it's often really difficult to solve
link |
from a sort of running time complexity perspective.
link |
Could you speak to the idea of complexity analysis
link |
of problems, do you find it useful, do you find it interesting?
link |
Do you find that lens of studying the difficulty
link |
of how difficult the computer science problem is
link |
a useful lens onto the world?
link |
Because if you want to make something practical
link |
which has large numbers of people using it,
link |
the computational complexity to me is almost question one.
link |
And that's, again, that's at the origin
link |
of when we started doing this stuff with disease control.
link |
From the very beginning, the deep questions
link |
that were running through my mind were,
link |
would we be able to support a large population
link |
with only one server?
link |
And if the answer is no, we can't start
link |
because I don't have enough money.
link |
Yeah, and there the question is very much
link |
linear time versus anything slower than linear time.
link |
As a very specific thing, you have a bunch
link |
of really interesting papers.
link |
If I could ask, maybe we could pull out some cool insights
link |
at the high level.
link |
Can you describe the data structure of a voting tree
link |
and what are some interesting results on it?
link |
You have a paper that I noticed on it.
link |
Yeah, so this is an example of, I guess,
link |
how in math we might say here's an interesting
link |
kind of a question that we just can't seem
link |
to understand enough about.
link |
Maybe there's something else going on here.
link |
And the way to describe this is you could imagine
link |
trying to hold elections where if you have
link |
only two candidates, that's kind of easy.
link |
You just run them against each other
link |
and see who gets more votes.
link |
But as you know, once you have more candidates,
link |
it's very difficult to decide who wins the election.
link |
And there's an entire voting theory around this.
link |
So a theoretical question became,
link |
what if you made like a system of runoffs,
link |
like a system of head to head contests,
link |
which is structured like a tree,
link |
almost looking like a circuit.
link |
I'm using that way of thinking because it's sort of like
link |
in electrical engineering or computer science,
link |
you might imagine having a bunch of leads
link |
that carry signal, which are going through AND gates
link |
and OR gates and whatnot.
link |
And you've managed to compute beautiful things.
link |
This is just from a purely abstract point of view.
link |
What if the inputs are candidates?
link |
And for every two candidates, it is known
link |
which of the candidates is more popular than the other.
link |
Now can you build some kind of a circuit board
link |
which says, first candidate number four
link |
will play against five and see who wins and so on.
link |
Okay, so now what would be a nice outcome, right?
link |
This is a general question of,
link |
could I make a big circuit board to feed an election into?
link |
Like maybe one nice outcome would be whoever wins
link |
at least is preferred over a lot of people.
link |
So for example, if you ran in 1,024 candidates,
link |
ideally we would like a guarantee that says
link |
that the winner beats a lot of people.
link |
Actually in any system where there are 1,024 candidates,
link |
there's always a candidate who beats
link |
at least 512 of the others.
link |
This is a mathematical fact
link |
that there's actually always a person who beats
link |
at least half of the other people.
link |
I'm trying to make sense of that mathematical fact.
link |
Is this supposed to be obvious?
link |
No, but I can explain it.
link |
The way it works is that, think of it this way.
link |
Every time I think, imagine I have all these candidates
link |
and everyone is competing,
link |
everyone is like compared with everyone else at some point.
link |
Well, think of it this way.
link |
Whenever there's a comparison, somebody gets a point.
link |
That's the one who is better than the other one.
link |
My claim is there's somebody whose score
link |
is at least half of how many other people there are.
link |
Yeah, I'm just trying to,
link |
like my intuition is very close to that being true,
link |
but it's beautiful.
link |
I didn't at first, that's not an obvious fact.
link |
And it feels like a beautiful fact.
link |
Well, let me explain it this way.
link |
Imagine that for every match,
link |
you didn't give one point, but you gave two points.
link |
You gave one point to each person.
link |
Now that's not what we're really doing.
link |
We really want to give one point to the winner of the match,
link |
but instead we'll just give two.
link |
If you gave two points to everyone on every matchup,
link |
actually everyone has the same number of points.
link |
And the number of points they get
link |
is how many other people there are.
link |
Does that sort of make sense?
link |
I'm just like saying.
link |
No, no, everything you're saying makes perfect sense.
link |
So the point is if for every comparison between two people,
link |
which I'm doing for every two people,
link |
I gave one point to each person,
link |
your score, everyone's score is the same.
link |
It's how many other people there are.
link |
Now we only make one change.
link |
For each matchup, you give one point only to the winner.
link |
So we're awarding half the points.
link |
So now the deal is if in the original situation,
link |
everyone's score was equal,
link |
which is how many other people there are.
link |
Now there's only half the number of points to go around.
link |
So what ends up happening is that
link |
there's always going to be,
link |
like the average number of points per person
link |
is going to be half of how many other people there are.
link |
And somebody is gonna be above average.
link |
Somebody is going to be above that.
link |
Yeah, this is this notion of expected value,
link |
that if I have a random variable,
link |
which has an expected value,
link |
there's going to be some possibility
link |
in the probability space
link |
where you're at least as big as the expected value.
link |
Yeah, when you describe it like that, it's obvious.
link |
But when you're first saying in this little circuit
link |
that there's going to be one candidate better than half,
link |
that's not obvious.
link |
Yeah, it's not obvious.
link |
Math, this is nice.
link |
Okay, so you have this,
link |
but ultimately you're trying to with a voting tree,
link |
I don't know if you're trying this,
link |
but to have a circuit that's like, that's small.
link |
Well, you'd like it to be small.
link |
That achieves the same kind of,
link |
I mean, the smaller it is,
link |
if we look at practically speaking,
link |
the lower the cost of running the election,
link |
of running through, of computing the circuit.
link |
But actually at this point,
link |
the reason the question was interesting
link |
is because there was no good guarantee
link |
that the winner of that circuit
link |
would have like have beaten a lot of people.
link |
Let me give an example.
link |
The best known circuit,
link |
when we started thinking about this,
link |
was the circuit called candidate one
link |
plays against candidate two,
link |
candidate three plays against four,
link |
and then the winners play against each other.
link |
And then by the way, five plays against six,
link |
seven against eight, the winners play against each other.
link |
You understand, it's like a giant binary tree.
link |
Yeah, it's a binary, like a balanced binary tree.
link |
It's a balanced binary tree.
link |
One, two, three, four, up to 1,024,
link |
everyone going up to find the winner.
link |
Well, you know what?
link |
There's a system in the world
link |
where it could just be
link |
that there's a candidate called number one,
link |
that just beats like 10 other people,
link |
just the 10 that they need to be on their way up
link |
and they lose to everyone else.
link |
But somehow they would get all the way up.
link |
My point is it is possible to outsmart that circuit
link |
in one weird way of the world,
link |
which makes that circuit a bad one
link |
because you want to say,
link |
I will use this circuit for all elections.
link |
And you might have a system of inputs that go in there
link |
where the winner only beat 10 other people,
link |
which is the people they had to beat on the way up.
link |
So you want to have a circuit where there's as many,
link |
like the final result is as strong as possible.
link |
And so what ideas do you have for that?
link |
So we actually only managed to improve it
link |
to square root of N.
link |
So if N is number of vertices,
link |
N over two would be the ideal.
link |
We got it to square root of N.
link |
Versus log base two.
link |
Well, that is halfway.
link |
It could be a lot.
link |
Could be a big improvement.
link |
So that's a, okay, cool.
link |
Is there something you can say with words
link |
about what kind of circuit, what that looks like?
link |
I can give an idea of one of the tools inside,
link |
but the actual execution ends up being more complicated.
link |
But one of the widgets inside this
link |
is building a system where you have like a candidate
link |
who plays, like one part of the whole huge, huge tree
link |
is that that same candidate, let's call them seven.
link |
Seven plays against somebody,
link |
let's make up some numbers.
link |
Let's call the others like letters.
link |
So seven plays against A.
link |
Seven's also gonna play against B separately.
link |
And the winners of each of those will play each other.
link |
By the way, seven's also gonna play C.
link |
Seven's gonna play D.
link |
And the winners are gonna play each other.
link |
And the winners are gonna play each other.
link |
We call this seven against all.
link |
Well, seven against like everyone from a bunch of.
link |
So there's some nice overlap between the matchups
link |
that somehow has a nice feature to it.
link |
Yes, and I can tell you the nice feature
link |
because if at the base of this giant tree,
link |
at the base of this giant circuit,
link |
like this is a widget.
link |
We build the things out of widgets.
link |
So I'm just describing one widget.
link |
But in the base of this widget,
link |
you have lots of things which are seven against someone,
link |
seven against someone, seven against someone.
link |
In fact, every matchup at the bottom
link |
is seven against someone.
link |
What that means is
link |
if seven actually beat everyone they were matched up against,
link |
well, seven would rise to the top.
link |
So one possibility is if you see a seven
link |
emerge from the top,
link |
you know that seven actually beat everyone
link |
they were against.
link |
On the other hand, if anyone else is on top,
link |
If F is on top, how did F get there?
link |
Well, F beat seven on the way at the beginning.
link |
So the point is the outcome of this circuit
link |
has a certain property.
link |
If you see a seven,
link |
you know that the seven actually beat a person
link |
but the seven actually beat a bazillion people.
link |
If you see anyone else,
link |
at least you know they beat seven.
link |
Yeah, then you can prove that it has a nice property.
link |
That's really interesting.
link |
Is there something you can say,
link |
perhaps going completely outside
link |
of what we're talking about,
link |
have mathematical ideas
link |
of improving the electoral process?
link |
No, I can't give you that one.
link |
I mean, is there, like, do you ever see it as,
link |
do you see as there being a lot of opportunities
link |
for improving how we vote?
link |
Like from your, I don't know if you saw parallels,
link |
but, you know, it seems like if,
link |
this actually kind of maps to your sort of COVID work,
link |
which is there's a network effect, right?
link |
It seems like we should be able to apply similar kind
link |
of effects of how we decide other things in our lives.
link |
And one of the big decisions we'll make
link |
is who represents us in government.
link |
Do you ever think about like mathematically
link |
about those kinds of systems?
link |
I think a little bit about those,
link |
because where I went to college,
link |
the way we voted for student government
link |
was based on this, is it called ranked choice?
link |
Where you eliminate the bottom
link |
and there was runoff elections.
link |
So that was the first time I ever saw that.
link |
And I thought that made sense.
link |
The only problem is it doesn't seem so easy
link |
to get something that makes sense adopted
link |
as the new voting system.
link |
That's a whole nother, that's not a math solution.
link |
That's a, well, it's math in the sense that it's game theory.
link |
So you have to come up with incentive,
link |
it's mechanism design.
link |
You have to figure out how to trick us
link |
despite our basic human nature
link |
to adopt solutions that are better.
link |
That's a whole nother conversation, I think.
link |
Can you just, cause it sounded really cool,
link |
talk a little bit about stochastic coalescence
link |
and you have a paper on showing that,
link |
so you could describe what it is,
link |
but I guess it's a super linear, super logarithmic time
link |
and you came up with some kind of trick
link |
that make it faster.
link |
Can you just talk about it a little bit?
link |
Yeah, so this was something which came up
link |
when I was at Microsoft Research for a summer.
link |
And I'm putting that context because that shows
link |
that it has some practical motivation at some point.
link |
Actually, I think it's still.
link |
It doesn't need to.
link |
It doesn't need to.
link |
It can be beautiful and it's all right.
link |
Yeah, so the easiest way to describe this is
link |
suppose you got like a big crowd of people
link |
and everybody knows how many hours of sleep
link |
they got last night.
link |
And you wanna know how many total hours of sleep
link |
were gotten by this big crowd of people.
link |
At the beginning, you might say,
link |
that sounds like a linear time algorithm
link |
of saying, hey, how many hours you got?
link |
But there's a way to do this
link |
if you remember that there are people
link |
and they presumably know how to add.
link |
You could make a distributed algorithm
link |
to make this happen.
link |
For example, while we're thinking of these trees,
link |
imagine you had 1,024 people.
link |
If you could just say, hey, person number one
link |
and person number two, you will add your hours of sleep.
link |
Person number two will go away
link |
and person number one is gonna remember the sum.
link |
Person three and four add up
link |
and person three takes charge of remembering it.
link |
Person four goes away.
link |
Now this like person one knows the sum of these two.
link |
Person three knows the sum of those two.
link |
You see what I mean?
link |
You're going up this tree,
link |
same tree that we talked about earlier.
link |
Built up a tree from the bottom up.
link |
Yeah, build up a tree from the bottom up.
link |
And the beautiful thing is
link |
since everyone's doing stuff in parallel,
link |
the amount of time it takes to get the total sum
link |
is actually just the number of layers in the tree,
link |
So now that's logarithmic time
link |
to add up the number of hours that people slept today.
link |
There's only one problem.
link |
How do you decide who's person number one
link |
and person number two?
link |
So if, for example, you just went out into the downtown
link |
and said, hey, get these thousand people, go.
link |
Well, if you're gonna go and say,
link |
and by the way, you're one and you're two and you're three,
link |
that's linear time.
link |
So now the question is how to do this
link |
in a distributed way.
link |
And there were some people who proposed
link |
a very elegant algorithm and they wanted to analyze it.
link |
So I came in onto the analyze side,
link |
but the elegant algorithm was like this.
link |
It was like, well, we don't actually know
link |
what this big tree is.
link |
There isn't any big tree.
link |
So what's gonna happen is first,
link |
everyone is going to decide right now.
link |
Oh, one important thing.
link |
Everyone is going to,
link |
at the very beginning of the whole game,
link |
they will have delegated responsibility to themselves
link |
as the one who knows the sum so far.
link |
So the point is there's gonna be,
link |
people are all gonna have like a pointer which says,
link |
you are the one who knows my,
link |
you've taken care of my ticket, my number.
link |
You're the representative for this particular piece
link |
And at the very beginning, you're your own representative.
link |
The thing has to start simple, right?
link |
So at the beginning, you're your own representative.
link |
You're pointing to yourself, got it.
link |
And now the way this works is that at every time step,
link |
someone blares a ding dong on the town clock or whatever.
link |
And each person flips a coin themselves to decide,
link |
am I going to hunt for somebody to give my number to
link |
and let them represent me?
link |
Or am I going to sit here and wait for someone to come?
link |
Well, they flipped their coin.
link |
Some of the people start asking other people saying,
link |
hey, I would like you to be my representative.
link |
Here is my number.
link |
But the problem is that there's limited bandwidth
link |
of the people who are getting asked.
link |
It's like, you can't get,
link |
you can't go out to prom with five people.
link |
But this is not what we're doing.
link |
We're adding numbers, okay?
link |
But you can only add one number.
link |
So the person who has suddenly gotten asked
link |
by all these people,
link |
well, they'll have to decide who they're going
link |
And they randomly just choose one.
link |
When they randomly choose one,
link |
all the others are rejected
link |
and they don't get to delegate anything in that round.
link |
But now if this person has absorbed this one who said,
link |
okay, here, you take charge of my number.
link |
This person now updates their pointer.
link |
And this person adds the two numbers.
link |
That was the first round.
link |
In the next round, when they do the coin flipping,
link |
this person doesn't flip anymore
link |
because they're just delegating.
link |
It's that anyone who has the pointers themselves,
link |
that's like a person who is in charge
link |
of some number of informations,
link |
they flip the coin to decide,
link |
should I find other people who are agents?
link |
Or should I wait for people to ask me?
link |
This is somebody else's idea.
link |
And then now the idea is, okay,
link |
if you just keep doing this process,
link |
what ends up happening?
link |
Oh yeah, and also by the way,
link |
if you decide that you want to go reach out
link |
to other people, here's the catch.
link |
When you're one of these agents saying,
link |
okay, I'm going to go look for someone.
link |
You have no idea who in this crowd is an agent
link |
or somebody who delegated it to someone else.
link |
You just pick a random person.
link |
When you pick the random person,
link |
if it lands on someone and the person says,
link |
oh, I actually delegated it to someone,
link |
then you follow the point.
link |
You walk up the delegation chain.
link |
Walk up the delegation chain.
link |
And you can do like path compression in the algorithm
link |
to make it so you don't consistently
link |
do lots of walking up.
link |
But the bottom line is that what ends up happening
link |
is that you end up reaching out.
link |
Whenever you're one of the ones reaching out,
link |
you can think of it as each agent is responsible
link |
for some number of people.
link |
It's almost like they're the leader of a bunch.
link |
As the process is evolving, you have these lumps.
link |
Each lump has an agent.
link |
And when the agent reaches out,
link |
they reach out to another lump
link |
where the probability of them hitting that lump
link |
is proportional to the size of the lump.
link |
That is the one funny thing about this process.
link |
This is not that they can reach out
link |
to a uniformly random lump
link |
where every lump has the same chance
link |
of getting reached out to.
link |
The bigger the lump is,
link |
the more likely it is that you end up reaching that lump.
link |
Which is a problem?
link |
Let me explain why that's a problem.
link |
Because you see, you're hoping
link |
that this has a small number of steps,
link |
but here's a bad situation that could happen.
link |
Imagine if you had like,
link |
there are n people that you're adding up.
link |
Imagine that you have exactly square root of n lumps left,
link |
of which almost all of them are just one person
link |
who's still their own boss, their own manager.
link |
Except one giant one.
link |
Now what's gonna happen?
link |
It's gonna be a huge bottleneck
link |
because every round the giant one
link |
can only absorb one of the others.
link |
And now you suddenly have time
link |
which is about square root of n.
link |
The square root of n is chosen
link |
because that is one where the lumps are such
link |
that you really are limited by this large one
link |
slowly sucking up the rest of them.
link |
So the heart of the question became,
link |
well, but is that just so unusual
link |
that it doesn't usually happen?
link |
Because remember you start with everyone
link |
just being independent.
link |
It's like a lot of lumps of size one.
link |
How naturally do the big lumps emerge?
link |
And so what that heart of the proof was,
link |
was showing that that was a joint work with Eyal Lubezki.
link |
That one was showing that actually in that thing
link |
the lumps do kind of get out of whack.
link |
And so it's not the purely logarithmic number of steps.
link |
But if you make one very slight change,
link |
which is if you are one of the agents
link |
and you have just been propositioned,
link |
possibly relayed along by a couple of different people.
link |
If you just say, don't take a random one,
link |
but accept the smallest lump.
link |
That actually does enough to even the whole economy.
link |
Distributes the lump size.
link |
I mean, yeah, it's fascinating how
link |
with the distributed algorithms,
link |
a little adjustment can make all the difference
link |
Actually, by the way, this does,
link |
back to our voting conversation,
link |
this makes me think of like,
link |
these networking systems are so fascinating to study.
link |
They immediately spring to mind ideas
link |
of how to have representation.
link |
Like maybe as opposed to me voting for a president,
link |
I want to vote for like,
link |
for you, Paul, to represent me,
link |
maybe on a particular issue.
link |
And then you will delegate that further.
link |
And then we naturally construct those kinds of networks
link |
because that feels like I can have a good conversation
link |
with you and figure out that you know what you're doing
link |
and I can delegate it to you.
link |
And in that way, construct a representative government,
link |
a representative decision maker.
link |
That feels really nice as opposed to like us,
link |
like a tree of height one or something,
link |
where it's like everybody's just,
link |
it feels like there's a lot of room for layers
link |
of representation to form organically from the bottom up.
link |
I wonder if there are systems like that.
link |
This is the cool thing about the internet
link |
and the digital space where we're so well connected,
link |
just like with the Novid app to distribute information
link |
about the spread of the disease.
link |
We can in the same way, in a distributed sense,
link |
form anything like any kind of knowledge bases
link |
that are formed in a decentralized way
link |
and in a hierarchical way,
link |
as opposed to sort of old way
link |
where there is no mechanism for large scale,
link |
fast distributed transactional information.
link |
This is really interesting.
link |
This is where almost like network graph theory,
link |
becomes practical.
link |
Most of that exciting work was done in the 20th century,
link |
but most of the application will be in the 21st,
link |
which is cool to think about.
link |
Let me ask the most ridiculous question.
link |
You think P equals NP?
link |
I mean, I would say,
link |
I know there are enough people who have very strong interest
link |
in trying to show that it is.
link |
I'm talking about government agencies.
link |
For security purposes.
link |
For security purposes.
link |
And most computer scientists,
link |
we should say believe that P equals NP.
link |
My question almost like,
link |
this is back to our aliens discussion.
link |
You want to think outside the box,
link |
the low probability event,
link |
what is the world,
link |
what kind of discoveries would lead us to prove
link |
that P does not equal to NP?
link |
Like there could be giant misunderstandings
link |
or gaps in our knowledge about computer science,
link |
about theoretical computer science, about computation,
link |
which allow us to think like flatten all problems.
link |
Yeah, so I don't know the answer to this question.
link |
I think it's very interesting, but I actually,
link |
I know, let's put it this way.
link |
By being at Carnegie Mellon
link |
and being around the theoretical computer scientists,
link |
I know enough about what I don't know to say.
link |
I'm the wrong person to answer this question.
link |
Well, Scott Aaronson, who's now here at UT Austin,
link |
he used to be at MIT,
link |
puts the probability of P not equals to NP at 3%.
link |
I always love it when you ask,
link |
it's very rare in science and academics
link |
because most folks are humble
link |
in the face of the mystery,
link |
the uncertainty of everything around us.
link |
To have both the humor and the guts to say like,
link |
what are the chance that there's aliens in our galaxy,
link |
intelligent alien civilizations?
link |
As opposed to saying, I don't know, it could be zero.
link |
It could be, depending on the fact, you're saying it's 2.5%.
link |
There's something very pleasant about just having,
link |
it's the number thing.
link |
It's powered to the number.
link |
It's just like 42.
link |
It's like, why 42?
link |
I don't know, but it's a powerful number.
link |
And then everything,
link |
this is the power of human psychology
link |
is once you have the number 42,
link |
it's not that the number has meaning,
link |
but because it's placed in a book with humor around it,
link |
it has the meme effect of actually creating reality.
link |
I mean, you could say that 42 has a strong contribution
link |
of helping us colonize Mars
link |
because it created,
link |
it gave the whatever existential crisis to many of us,
link |
including Elon Musk when he was young,
link |
reading a book like that.
link |
And then now 42 is now part of his humor
link |
that he doesn't shut up about,
link |
it's constantly joking about.
link |
And that humor is spreading through our minds
link |
and somehow this like silly number just had an effect.
link |
In that same way, after Scott told me like the 3% chance,
link |
it's stuck in my head.
link |
And I think it's been having a ripple effect
link |
in everybody else.
link |
The believing that P is not equal to NP,
link |
Scott almost as a joke saying it's 3%
link |
is actually motivating a large number of researchers