back to index

Po-Shen Loh: Mathematics, Math Olympiad, Combinatorics & Contact Tracing | Lex Fridman Podcast #183


small model | large model

link |
00:00:00.000
The following is a conversation with Po Shen Lou,
link |
00:00:02.720
a professor of mathematics at Carnegie Mellon University,
link |
00:00:06.280
national coach of the USA International Math Olympia team,
link |
00:00:09.920
and founder of XP that does online education
link |
00:00:13.480
of basic math and science.
link |
00:00:15.240
He's also the founder of Novid,
link |
00:00:17.040
an app that takes a really interesting approach
link |
00:00:19.560
to contact tracing,
link |
00:00:20.760
making sure you stay completely anonymous
link |
00:00:23.080
and it gives you statistical information about COVID cases
link |
00:00:25.840
in your physical network of interactions.
link |
00:00:28.880
So you can maintain privacy, very important,
link |
00:00:31.600
and make informed decisions.
link |
00:00:34.160
In my opinion,
link |
00:00:35.000
we desperately needed solutions like this in early 2020.
link |
00:00:38.720
And unfortunately, I think,
link |
00:00:41.440
we will again need it for the next pandemic.
link |
00:00:44.680
To me, solutions that require large scale,
link |
00:00:46.920
distributed coordination of human beings
link |
00:00:49.640
need ideas that emphasize freedom and knowledge.
link |
00:00:53.400
Quick mention of our sponsors,
link |
00:00:55.120
Jordan Harbinger Show, Onnit, BetterHelp,
link |
00:00:58.720
Aidsleep, and Element.
link |
00:01:01.200
Check them out in the description to support this podcast.
link |
00:01:04.320
As a side note, let me say that Po and I
link |
00:01:07.160
filmed a few short videos
link |
00:01:08.400
about simple, beautiful math concepts
link |
00:01:10.640
that I will release soon.
link |
00:01:12.960
It was really fun.
link |
00:01:14.080
I really enjoyed Po sharing his passion for math with me
link |
00:01:16.640
in those videos.
link |
00:01:17.840
I'm hoping to do a few more short videos
link |
00:01:19.960
in the coming months that are educational in nature
link |
00:01:23.160
on AI, robotics, math, science, philosophy,
link |
00:01:26.520
or if all else fails,
link |
00:01:28.840
just fun snippets into my life on music, books, martial arts,
link |
00:01:32.920
and other random things,
link |
00:01:34.720
if that's of interest to anyone at all.
link |
00:01:38.240
This is the Lex Friedman Podcast,
link |
00:01:40.200
and here's my conversation with Po Shenlow.
link |
00:01:43.840
You know, you mentioned you really enjoy flying
link |
00:01:46.520
and experiencing different people in different places.
link |
00:01:49.720
There's something about flying for me,
link |
00:01:51.160
I don't know if you have the same experience,
link |
00:01:53.080
that every time I get on an airplane,
link |
00:01:55.680
it's incredible to me that human beings
link |
00:01:58.040
have actually been able to achieve this.
link |
00:02:00.240
And when I look at like what's happening now
link |
00:02:03.720
with humans traveling out into space,
link |
00:02:06.400
I see it as all the same thing.
link |
00:02:08.080
It's incredible that humans are able to get into a box
link |
00:02:11.240
and fly in the air and safely and land
link |
00:02:16.320
in the same, it seems like,
link |
00:02:18.040
and everybody's taking it for granted.
link |
00:02:19.920
So when I observe them, it's quite fascinating
link |
00:02:22.480
because I see that cleanly mapping to the world
link |
00:02:25.880
where we're now in rockets and traveling to the moon,
link |
00:02:31.520
traveling to Mars, and at the same kind of way,
link |
00:02:34.360
I can already see the future
link |
00:02:36.480
where we will all take it for granted.
link |
00:02:40.040
So I don't know if you have,
link |
00:02:42.680
you personally, when you fly,
link |
00:02:44.320
have the same kind of magical experience
link |
00:02:46.440
of like how the heck did humans actually accomplish this?
link |
00:02:49.080
So I do, especially when there's turbulence,
link |
00:02:52.560
which is like on the way here, there was turbulence
link |
00:02:56.080
and the plane jiggled, even the flight attendant
link |
00:02:58.880
had to hold onto the side.
link |
00:03:00.200
And I was just thinking to myself,
link |
00:03:01.360
it's amazing that this happens all the time
link |
00:03:03.200
and the wings don't fall off,
link |
00:03:04.960
given how many planes are flying.
link |
00:03:06.600
But then I often think about it and I'm like,
link |
00:03:09.160
a long time ago, I think people didn't trust elevators
link |
00:03:12.160
in a 40 story building in New York City.
link |
00:03:14.400
And now we just take it completely for granted
link |
00:03:17.000
that you can step into this shaft,
link |
00:03:18.600
which is 40 floors up and down, and it will just not fail.
link |
00:03:24.440
Yeah, again, I'm the same way with elevators,
link |
00:03:26.760
but also buildings, when I'll stand on the 40th floor
link |
00:03:30.360
and wonder how the heck are we not falling right now?
link |
00:03:35.320
Like how amazing it is with the high winds,
link |
00:03:39.120
like structurally, just the earthquakes and the vibrations,
link |
00:03:42.200
I mean, natural vibrations in the ground.
link |
00:03:44.520
Like how is this, how are all of these,
link |
00:03:46.960
you go to like New York City, all of these buildings standing.
link |
00:03:49.920
I mean, to me, one of the most beautiful things,
link |
00:03:52.120
actually mathematically too, is bridges.
link |
00:03:54.760
I used to build bridges in high school from like toothpicks,
link |
00:03:57.560
just like out of the pure joy of like physics,
link |
00:04:02.320
making some structure really strong.
link |
00:04:05.800
Understanding like from a civil engineering perspective,
link |
00:04:09.600
what kind of structure will be stronger
link |
00:04:12.040
than another kind of structure, like suspension bridges.
link |
00:04:14.720
And then you see that at scale,
link |
00:04:17.160
humans being able to span a body of water
link |
00:04:20.400
with a giant bridge.
link |
00:04:22.240
And it's, I don't know, it's so humbling.
link |
00:04:26.480
It makes you realize how dependent we are on each other.
link |
00:04:31.920
Sort of, I talk about love a lot,
link |
00:04:33.360
but there's a certain element in which we little ants
link |
00:04:37.960
have just a small amount of knowledge
link |
00:04:39.640
about our particular thing.
link |
00:04:41.880
And then we're depending on a network of knowledge
link |
00:04:45.360
that other experts hold.
link |
00:04:47.360
And then most of our lives,
link |
00:04:49.200
most of the quality of life we have
link |
00:04:51.120
has to do with the richness of that network of knowledge,
link |
00:04:56.600
of that collaboration,
link |
00:04:58.280
and then sort of the ability to build on top of it,
link |
00:05:02.000
levels of abstractions.
link |
00:05:03.200
You start from like bits in a computer,
link |
00:05:05.760
then you can have assembly, then you can have C++,
link |
00:05:08.800
or you have an operating system,
link |
00:05:09.920
then you can have C++ and Python, finally,
link |
00:05:12.040
some machine learning on top.
link |
00:05:13.440
All of these are abstractions.
link |
00:05:14.960
And eventually we'll have AI that runs all of us humans.
link |
00:05:17.120
But anyway, but speaking of abstractions and programming,
link |
00:05:21.280
in high school, you wrote some impressive games
link |
00:05:24.720
for MS DOS.
link |
00:05:26.080
I got a chance to, in browser somehow,
link |
00:05:28.080
it's magic, I got a chance to play them.
link |
00:05:31.720
Alien Attack 1, 2, 3, and 4.
link |
00:05:34.560
What's the hardest part about programming those games?
link |
00:05:37.240
And maybe can you tell the story about building those games?
link |
00:05:41.080
Sure.
link |
00:05:41.920
I actually tried to do those in high school
link |
00:05:44.240
because I was just curious if I could.
link |
00:05:46.480
That's a good starting point for anything, right?
link |
00:05:49.280
Yeah, yeah, yeah, it's like, could you?
link |
00:05:50.480
But the appealing thing was also,
link |
00:05:52.000
it was a soup to nuts kind of thing.
link |
00:05:54.160
So something that has always attracted me is,
link |
00:05:56.560
I like beautiful ideas, I like seeing beautiful ideas,
link |
00:05:59.920
but I actually also like seeing execution of an idea
link |
00:06:03.760
all the way from beginning to end in something that works.
link |
00:06:06.360
So for example, in high school,
link |
00:06:08.000
I was lucky enough to grow up in the late 90s
link |
00:06:10.880
when even a high school student could hope
link |
00:06:13.720
to make something sort of comparable
link |
00:06:16.200
to the shareware games that were out there.
link |
00:06:18.640
I say the word sort of, like still quite far away,
link |
00:06:21.240
but at least I didn't need to hire a 3D CG artist.
link |
00:06:25.000
There weren't enough pixels to draw anyway,
link |
00:06:27.200
even I can draw, right?
link |
00:06:29.360
Bad art, of course.
link |
00:06:30.280
But the point is, I wanted to know,
link |
00:06:31.640
is it possible for me to try to do those things
link |
00:06:34.640
where back in those days,
link |
00:06:36.600
you didn't even have an easy way
link |
00:06:38.440
to draw letters on the screen in a particular font.
link |
00:06:41.200
You couldn't just say import a font, it wasn't like Python.
link |
00:06:44.080
So for example, back then,
link |
00:06:45.440
if you played those games in the web browser,
link |
00:06:47.560
which is emulating the old school computer,
link |
00:06:51.840
those, even the letters you see,
link |
00:06:53.520
those are made by individual calls
link |
00:06:55.120
to draw pixels on the screen.
link |
00:06:56.800
So you built that from scratch,
link |
00:06:58.240
almost building a computer graphics library from scratch?
link |
00:07:01.000
Yes, the primitive that I got to use
link |
00:07:03.120
was some code I copied off of a book in assembly
link |
00:07:05.600
of how to put a pixel on a screen in a particular color.
link |
00:07:08.720
And the programming language was Pascal?
link |
00:07:11.760
Ah, yeah, the first one was in Pascal,
link |
00:07:14.400
but then the other ones were in C++ after that.
link |
00:07:18.120
How's the emulation in the browser work, by the way?
link |
00:07:20.640
Is that trivial?
link |
00:07:21.640
Because it's pretty cool, you get to play these games
link |
00:07:23.520
that have a very much 90s feeling to them.
link |
00:07:26.480
Ah, so it's literally making an MSDOS environment,
link |
00:07:29.320
which is literally running the old.exe file.
link |
00:07:32.240
Wow, in the browser.
link |
00:07:33.520
This is, that could be more amazing than the airplane.
link |
00:07:37.800
So it wasn't so much about the video games,
link |
00:07:40.240
it was more about,
link |
00:07:41.400
can you build something really cool from scratch?
link |
00:07:44.120
Yes.
link |
00:07:45.440
And you did a bunch of programming competitions.
link |
00:07:50.280
What was your interest, your love for programming?
link |
00:07:54.680
What did you learn through that experience?
link |
00:07:56.880
Especially now that as much of your work
link |
00:07:59.720
has taken a long journey through mathematics.
link |
00:08:03.160
I think I always was amazed
link |
00:08:04.920
by how computers could do things fast.
link |
00:08:08.440
If I wanted to make it an abstract analysis
link |
00:08:11.560
of why it is that I saw some power in the computer.
link |
00:08:14.640
Because if the computer can do things
link |
00:08:16.360
so many times faster than humans,
link |
00:08:18.280
where the hard part is telling the computer
link |
00:08:20.280
what to do and how to do it,
link |
00:08:21.840
if you can master that asking the computer what to do,
link |
00:08:25.600
then you could conceivably achieve more things.
link |
00:08:28.200
And those contests I was in,
link |
00:08:29.640
those were the opposite in some sense
link |
00:08:31.840
of making a complete product, like a game is a product.
link |
00:08:35.840
Those contests were effectively write a function
link |
00:08:38.520
to do something extremely efficiently.
link |
00:08:40.760
And if you are able to do that,
link |
00:08:42.440
then you can unlock more of the power of the computer.
link |
00:08:45.800
But also doing it quickly.
link |
00:08:47.440
There's a time element from the human perspective
link |
00:08:49.640
to be able to program quickly.
link |
00:08:53.360
There's something nice.
link |
00:08:54.840
So there's almost like an athletics component
link |
00:08:57.920
to where you're almost like an athlete
link |
00:09:01.240
seeking optimal performance as a human being
link |
00:09:03.920
trying to write these programs.
link |
00:09:05.680
And at the same time, it's kind of art
link |
00:09:07.440
because the best way to write a program quickly
link |
00:09:11.080
is to write a simple program.
link |
00:09:13.440
You just have a damn good solution.
link |
00:09:14.840
So it's not necessarily you have to type fast.
link |
00:09:17.040
You have to think through a really clean,
link |
00:09:19.520
beautiful solution.
link |
00:09:22.760
I mean, what do you think is the use
link |
00:09:26.080
of those programming competitions?
link |
00:09:27.760
Do you think they're ultimately something
link |
00:09:29.160
you would recommend for students,
link |
00:09:30.600
for people interested in programming,
link |
00:09:32.040
or people interested in building stuff?
link |
00:09:34.000
Yes, I think so because especially with the work
link |
00:09:37.080
that I've been doing nowadays,
link |
00:09:38.120
even trying to control COVID,
link |
00:09:40.200
something that was very helpful from day one
link |
00:09:42.560
was understanding that the kinds of computations
link |
00:09:45.560
we would want to do,
link |
00:09:47.080
we could conceivably do on like a four core cloud machine
link |
00:09:50.440
on Amazon Web Services out to a population
link |
00:09:53.760
which might have hundreds of thousands
link |
00:09:55.280
or millions of people.
link |
00:09:56.480
The reason why that was important
link |
00:09:57.840
to have that back of the envelope calculation
link |
00:10:00.760
with efficient algorithms
link |
00:10:02.720
is because if we couldn't do that,
link |
00:10:04.960
then we would bankrupt ourselves
link |
00:10:06.280
before we could get to a big enough scale.
link |
00:10:08.160
If you think about how you grow anything from small to big,
link |
00:10:11.480
if in order to grow it from small to big,
link |
00:10:13.600
you also already need 10,000 cloud servers,
link |
00:10:16.480
you'll never get to big.
link |
00:10:19.400
And also the nice thing about programming competitions
link |
00:10:22.320
is that you actually build a thing that works.
link |
00:10:26.440
So you finish it, there's a completion thing,
link |
00:10:29.040
and you realize, I think there's a magic to it,
link |
00:10:32.520
where you realize that it's not so hard
link |
00:10:35.520
to build something that works.
link |
00:10:37.520
To have a system that successfully takes in inputs
link |
00:10:40.560
and produces outputs and solves a difficult problem,
link |
00:10:43.360
and that directly transfers to building a startup essentially
link |
00:10:47.040
that can help some aspect of this world
link |
00:10:50.000
as long as it's mostly based on software engineering.
link |
00:10:53.840
Things get really tricky
link |
00:10:54.960
when you have to manufacture stuff.
link |
00:10:58.480
That's why people like Elon Musk are so impressive
link |
00:11:00.720
that it's not just software.
link |
00:11:02.840
Tesla Autopilot is not just software.
link |
00:11:05.640
It's like you have to actually have factories
link |
00:11:07.640
that build cars, and there's like a million components
link |
00:11:11.640
involved in the machinery required
link |
00:11:14.280
to assemble those cars and so on.
link |
00:11:16.200
But in software, one person can change the world,
link |
00:11:19.000
which is incredible.
link |
00:11:21.440
But on the mathematics side,
link |
00:11:23.240
what, if you look back, or maybe today,
link |
00:11:26.760
what made you fall in love with mathematics?
link |
00:11:29.680
For me, I think I've always been very attracted
link |
00:11:33.320
to challenge, as I already indicated
link |
00:11:35.720
with writing the program.
link |
00:11:37.320
I guess if I see something that's hard
link |
00:11:40.440
or supposed to be impossible,
link |
00:11:44.200
sometimes I say, maybe I want to see if I can pull that off.
link |
00:11:47.120
And with the mathematics, the math competitions
link |
00:11:49.920
presented problems that were hard,
link |
00:11:53.560
that I didn't know how to start,
link |
00:11:55.040
but for which I could conceivably try to learn
link |
00:11:57.640
how to solve them.
link |
00:11:59.120
So, I mean, there are other things that are hard
link |
00:12:00.480
called like get something to Mars, get people to Mars.
link |
00:12:03.360
And I didn't, and I still don't think
link |
00:12:05.800
that I am able to solve that problem.
link |
00:12:08.160
On the other hand, the math problems struck me
link |
00:12:10.000
as things which are hard
link |
00:12:11.440
and with significant amount of extra work,
link |
00:12:13.560
I could figure it out.
link |
00:12:14.640
And maybe they would actually even be useful,
link |
00:12:16.560
like that mathematical skill is the core
link |
00:12:18.920
of lots of other things.
link |
00:12:22.680
That's really interesting.
link |
00:12:23.720
Maybe you could speak to that
link |
00:12:25.120
because a lot of people say that math is hard
link |
00:12:29.440
as a kind of negative statement.
link |
00:12:32.880
It always seemed to me a little bit like
link |
00:12:35.120
that's kind of a positive statement
link |
00:12:37.360
that all things that are worth having in this world,
link |
00:12:40.360
they're hard.
link |
00:12:41.440
I mean, everything that people think about
link |
00:12:44.480
that they would love to do, whether it's sports,
link |
00:12:47.800
whether it's art, music, and all the sciences,
link |
00:12:52.920
they're going to be hard
link |
00:12:54.000
if you want to do something special.
link |
00:12:56.520
So is there something you could say to that idea
link |
00:12:59.000
that math is hard?
link |
00:13:00.160
Should it be made easy or should it be hard?
link |
00:13:04.200
Ah, so I think maybe I want to dig in a little bit
link |
00:13:07.120
onto this hard part and say,
link |
00:13:09.360
I think the interesting thing about the math
link |
00:13:12.120
is that you can see a question
link |
00:13:14.800
that you didn't know how to start doing it before.
link |
00:13:18.240
And over a course of thinking about it,
link |
00:13:20.920
you can come up with a way to solve it.
link |
00:13:24.400
And so you can move from a state
link |
00:13:25.960
of not being able to do something
link |
00:13:28.160
to a state of being able to do something
link |
00:13:30.280
where you help to take yourself through that
link |
00:13:33.280
instead of somebody else spoon feeding you that technique.
link |
00:13:37.600
So actually here, I'm already digging into
link |
00:13:39.480
maybe part of my teaching philosophy also,
link |
00:13:42.080
which is that I actually don't want to ever
link |
00:13:45.400
just tell somebody, here's how you do something.
link |
00:13:48.840
I actually prefer to say, here's an interesting question.
link |
00:13:52.160
I know you don't quite know how to do it.
link |
00:13:54.240
Do you have any ideas?
link |
00:13:55.920
I'm actually explaining another way
link |
00:13:58.920
that you could try to do teaching.
link |
00:14:00.400
And I'm contrasting this to a method of watch me do this,
link |
00:14:04.160
now practice it 20 times.
link |
00:14:06.320
I'm trying to say a lot of people consider math to be hard
link |
00:14:09.080
because maybe they can't remember
link |
00:14:10.800
all of the methods that were taught.
link |
00:14:12.800
But for me, I look at the hardness
link |
00:14:15.560
and I don't think of it as a memory hardness.
link |
00:14:17.600
I think of it as a, can you invent something hardness?
link |
00:14:21.920
And I think that if we can teach more people
link |
00:14:24.360
how to do that art of invention in a pure cognitive way,
link |
00:14:28.640
not as hard as the actual hardware stuff, right?
link |
00:14:31.240
But like in terms of the concepts
link |
00:14:32.640
and the thoughts and the mathematics,
link |
00:14:34.000
teaching people how to invent,
link |
00:14:36.360
then suddenly actually they might not even find math
link |
00:14:38.840
to be that tiresomeness hard anymore,
link |
00:14:42.680
but that rewardingness hard of I have the capability
link |
00:14:47.480
of looking at something which I don't know what to do
link |
00:14:49.920
and coming up with how to do it.
link |
00:14:51.400
I actually think we should be doing that,
link |
00:14:52.840
giving people that capability.
link |
00:14:55.040
So hard in the same way that invention is hard,
link |
00:14:58.160
that is ultimately rewarding.
link |
00:14:59.520
So maybe you can dig in that a little bit longer,
link |
00:15:03.640
which is do you see basically the way to teach math
link |
00:15:11.200
is to present a problem and to give a person a chance
link |
00:15:14.840
to try to invent a solution
link |
00:15:18.480
with minimal amount of information first?
link |
00:15:21.080
Is that basically,
link |
00:15:22.840
how do you build that muscle of invention in a student?
link |
00:15:26.400
Yes, so the way that,
link |
00:15:27.840
I guess I have two different sort of ways
link |
00:15:30.120
that I try to teach.
link |
00:15:30.960
Actually, one of them is, in fact, this semester,
link |
00:15:32.960
because all my classes were remotely delivered,
link |
00:15:35.520
I even threw them all onto my YouTube channel.
link |
00:15:37.200
So you can see how I teach at Carnegie Mellon,
link |
00:15:39.960
but I'd often say, hey, everyone, let's try to do this.
link |
00:15:43.280
Any ideas?
link |
00:15:44.680
And that actually changes my role as a professor
link |
00:15:47.760
from a person who shows up for class
link |
00:15:50.040
with a script of what I wanna talk through.
link |
00:15:52.480
I actually, I don't have a script.
link |
00:15:54.000
The way I show up for classes,
link |
00:15:55.680
there's something that we want to learn how to do,
link |
00:15:58.080
and we're gonna do it by improv.
link |
00:16:00.120
I'm talking about the same method as improv comedy,
link |
00:16:02.760
which is where you tell me some ideas,
link |
00:16:05.240
and I'll try to yes and them.
link |
00:16:07.040
You know what I mean?
link |
00:16:09.320
And then together,
link |
00:16:10.400
we're gonna come up with a proof of this concept
link |
00:16:13.000
where you were deeply involved in creating the proof.
link |
00:16:16.360
Actually, every time I teach the class,
link |
00:16:18.040
we do every proof slightly differently
link |
00:16:19.800
because it's based on how the students came up with it.
link |
00:16:23.400
And that's how I do it when I'm in person.
link |
00:16:25.760
I also have another line of courses that we make
link |
00:16:27.880
that is delivered online.
link |
00:16:29.200
Those things are where I can't do it live,
link |
00:16:31.640
but the teaching method became also similar.
link |
00:16:34.560
It was just, here's an interesting question.
link |
00:16:37.160
I know it's out of reach.
link |
00:16:38.240
Why don't you think about it?
link |
00:16:39.400
And then automatic hints.
link |
00:16:40.640
We feed automatically hints through the internet
link |
00:16:44.120
to go and let the person try to invent.
link |
00:16:47.560
So that's like a more rigorous prodding of invention.
link |
00:16:52.240
But you did mention disease and COVID,
link |
00:16:56.000
and you've been doing some very interesting stuff
link |
00:16:58.160
from a mathematical, but also software engineering angle
link |
00:17:01.920
of coming up with ideas.
link |
00:17:04.520
It's back to the, I see a problem.
link |
00:17:07.440
I think I can help.
link |
00:17:09.640
So you stepped into this world.
link |
00:17:11.120
Can you tell me about your work there
link |
00:17:13.880
under the flag of Novid
link |
00:17:16.120
and both the software and the technical details
link |
00:17:20.560
of how the thing works?
link |
00:17:21.680
Sure, sure.
link |
00:17:22.520
So first I want to make sure that I say,
link |
00:17:24.520
this is actually team effort.
link |
00:17:26.000
I happen to be the one speaking,
link |
00:17:27.480
but there's no way this would exist
link |
00:17:29.040
without an incredible team of people
link |
00:17:30.560
who inspire me every day to work on this.
link |
00:17:33.120
But I'll speak on behalf of them.
link |
00:17:34.600
So the idea was indeed that we stepped forward
link |
00:17:40.200
in March of last year, when the world started to become,
link |
00:17:43.320
our part of the world started to become,
link |
00:17:44.800
our part meaning the United States
link |
00:17:46.360
started to become paralyzed by COVID.
link |
00:17:48.920
The shutdown started to happen.
link |
00:17:50.760
And at that time it started as a figment of an idea,
link |
00:17:54.240
which was network theory,
link |
00:17:57.160
which is the area of math that I work in,
link |
00:17:59.480
could potentially be combined with smartphones
link |
00:18:02.440
and some kind of health information anonymized.
link |
00:18:06.040
Exactly how?
link |
00:18:07.080
We didn't know yet.
link |
00:18:07.960
We tried to crystallize it.
link |
00:18:09.400
And many months into this work,
link |
00:18:11.400
we ended up accidentally discovering a new way
link |
00:18:15.200
to control diseases,
link |
00:18:17.000
which is now what is the main impetus of all of this work
link |
00:18:20.840
is to take this idea and polish it
link |
00:18:23.440
and hopefully have it be useful not only now,
link |
00:18:25.960
but for future pandemics.
link |
00:18:27.400
The idea is really simple to describe.
link |
00:18:29.680
Actually, my main thing in the world
link |
00:18:31.080
is I come up with obvious observations.
link |
00:18:33.720
That's that, so I'll explain it now.
link |
00:18:35.080
Einstein did the same thing
link |
00:18:36.880
and he wrote a few short papers.
link |
00:18:39.920
But so the idea is like this.
link |
00:18:41.800
If we describe how usually people control disease
link |
00:18:46.440
for a lot of history,
link |
00:18:47.920
it was that you'd find out who was sick,
link |
00:18:51.000
you'd find out who they've been around
link |
00:18:53.280
and you try to remove all of those people from society
link |
00:18:56.120
against their will.
link |
00:18:57.840
Now that's the problem.
link |
00:18:58.960
The against their will part
link |
00:19:00.880
gives you the wrong kind of a feedback loop,
link |
00:19:03.560
which makes it hard to control the disease
link |
00:19:05.520
because then the people you're trying to control
link |
00:19:07.000
keep getting other people sick.
link |
00:19:08.760
You can see already how I'm thinking
link |
00:19:10.120
and talking about this feedback loops.
link |
00:19:11.920
This is actually related to something you said earlier
link |
00:19:14.240
about even like how skyscrapers stay in the air.
link |
00:19:17.120
The whole point is control theory.
link |
00:19:19.360
You actually want to, or even how an airplane stays,
link |
00:19:22.600
you need to have control loops
link |
00:19:24.880
which are feedbacking in the right way.
link |
00:19:27.040
And what we observed was that the feedback control loop
link |
00:19:29.880
for controlling disease by asking people
link |
00:19:32.160
to be removed from society against their will
link |
00:19:34.480
was not working.
link |
00:19:35.600
It was running against human incentives
link |
00:19:37.640
and you suddenly are trying to control
link |
00:19:39.160
seven billion, eight billion people
link |
00:19:41.440
in ways that they don't individually want
link |
00:19:43.600
to necessarily do.
link |
00:19:45.160
So here's the idea.
link |
00:19:47.080
And this is inspired by the fact
link |
00:19:48.320
that at the core of our team
link |
00:19:49.480
were user experience designers.
link |
00:19:51.280
That's actually, in fact, the first thing I knew
link |
00:19:53.480
we needed when we started
link |
00:19:54.560
was to bring user experience at the core.
link |
00:19:57.160
Okay.
link |
00:19:58.160
But so the idea was suppose hypothetically
link |
00:20:03.600
there was a pandemic.
link |
00:20:05.320
What would you want?
link |
00:20:08.000
You would want a way to be able to live your life
link |
00:20:10.320
as much as possible and avoid getting sick.
link |
00:20:13.680
Can we make an app to help you avoid getting sick?
link |
00:20:17.400
Notice how I've just articulated the problem.
link |
00:20:19.680
It is not, can we make an app
link |
00:20:21.840
so that after you are around somebody who's sick
link |
00:20:24.720
you can be removed from society.
link |
00:20:27.080
It's can we make an app so that you can avoid getting sick.
link |
00:20:30.520
That would run a positive feed.
link |
00:20:33.520
I don't know if I want to call it positive or negative
link |
00:20:35.320
but it would run a good feedback loop.
link |
00:20:36.840
Okay.
link |
00:20:37.680
So then how would you do this?
link |
00:20:38.840
The only problem is that you don't know who's sick
link |
00:20:41.480
because especially with this disease
link |
00:20:44.120
if I see somebody who looks perfectly healthy
link |
00:20:46.960
the disease spreads two days before you have any symptoms.
link |
00:20:50.000
And so it's actually not possible.
link |
00:20:52.360
That's where the network theory comes in.
link |
00:20:54.760
You caught it from someone.
link |
00:20:56.560
What if we changed the paradigm
link |
00:20:59.560
and we said, whenever there's a sickness
link |
00:21:02.440
tell everybody how many physical relationships
link |
00:21:06.200
separate them from the sickness.
link |
00:21:08.000
That is the trivial idea we added.
link |
00:21:09.840
The trivial idea was the distance between you and a disease
link |
00:21:13.360
is not measured in feet or seconds.
link |
00:21:16.600
It's measured in terms of how many
link |
00:21:19.040
close physical relationships separate you
link |
00:21:22.040
like these six degrees of separation like LinkedIn.
link |
00:21:25.400
Simple idea.
link |
00:21:26.480
What if we told everyone that?
link |
00:21:28.080
It turns out that actually unlocks
link |
00:21:30.200
some interesting behavioral feedback loops
link |
00:21:33.040
which for example, let me now jump to a non COVID example
link |
00:21:37.040
to show why this maybe could be useful.
link |
00:21:39.240
Actually we think it could be quite useful.
link |
00:21:40.920
Imagine there was Ebola or some hemorrhagic fever.
link |
00:21:44.080
Imagine it spread through contact through the air.
link |
00:21:46.400
In fact, pretend, pretend.
link |
00:21:50.520
That's a disastrous disease.
link |
00:21:52.800
It has high fatality rate.
link |
00:21:54.720
And as you die, you're bleeding out of every orifice.
link |
00:22:00.640
Okay.
link |
00:22:01.480
So.
link |
00:22:02.320
Yeah, not pleasant.
link |
00:22:03.160
Not pleasant.
link |
00:22:04.000
So the question is, suppose that such a disease broke
link |
00:22:07.680
who would want to install an app that would tell them
link |
00:22:10.440
how many relationships away from them
link |
00:22:12.640
this disease had struck?
link |
00:22:14.240
Like a lot of people.
link |
00:22:15.600
A lot of people.
link |
00:22:16.440
In fact, almost, I don't want to say almost everyone.
link |
00:22:20.160
That's a very strong statement
link |
00:22:21.000
but a very large number of people.
link |
00:22:22.840
That's fascinating framing.
link |
00:22:24.200
Like the more deadly and transmissible the disease
link |
00:22:28.040
the stronger the incentive to install it in a positive sense
link |
00:22:32.080
the, in the good feedback loop sense.
link |
00:22:36.000
That's a really good example.
link |
00:22:37.200
It's a really good way to frame it.
link |
00:22:38.520
Cause with COVID, it was not as deadly
link |
00:22:42.240
as potential pandemics could have been
link |
00:22:45.440
viruses could have been.
link |
00:22:46.280
So it's sometimes muddled with how we think about it
link |
00:22:49.160
but yeah, this is a really good framing.
link |
00:22:51.040
If the virus was a lot more deadly
link |
00:22:53.720
you want to create a system that has a set of incentives
link |
00:22:56.280
that it quickly spreads to the population
link |
00:22:59.400
where everybody is using it
link |
00:23:00.960
and it's contributing in a positive way to the system.
link |
00:23:04.200
Exactly.
link |
00:23:05.040
And actually that point you just made
link |
00:23:06.080
I don't take credit for that observation.
link |
00:23:07.760
There was another person I talked to
link |
00:23:09.000
who pointed out that it's very interesting
link |
00:23:11.360
that this feedback loop is even more effective
link |
00:23:14.600
when the disease is worse.
link |
00:23:16.960
And that's actually not a bad characteristic to have
link |
00:23:19.720
in your feedback loop
link |
00:23:20.640
if you're trying to help civilization keep running.
link |
00:23:24.320
Yeah, it's a really, it's in this dynamic
link |
00:23:27.320
like people figure out, they dynamically figure out
link |
00:23:30.440
how bad the disease is.
link |
00:23:31.920
The more it spreads and the deadlier it is
link |
00:23:35.080
as the people observe it
link |
00:23:37.200
as long as the spread of information
link |
00:23:39.640
like semantic information, natural language information
link |
00:23:43.440
is closely aligned with the reality of the disease
link |
00:23:46.040
which is a whole nother conversation, right?
link |
00:23:48.120
We, that's, we might, maybe we'll chat about that
link |
00:23:51.000
how we sort of make sure there's not misinformation
link |
00:23:53.800
while there's accurate information
link |
00:23:54.960
but that aside, okay, so this is a really nice property.
link |
00:23:58.840
Right, and just going on on that
link |
00:24:00.880
actually just talking more about what that could do
link |
00:24:02.840
and why we're so excited about it.
link |
00:24:04.440
It's that not only would people want to install it
link |
00:24:07.320
but what would they do if you start to see
link |
00:24:10.800
that this disease is getting closer and closer?
link |
00:24:13.200
We surveyed informally people
link |
00:24:15.080
but they said, as we saw it getting closer, we would hide.
link |
00:24:18.600
We would try to not have contacts.
link |
00:24:21.800
But now you notice what this has just achieved.
link |
00:24:24.080
The whole goal on this whole exercise was
link |
00:24:27.400
you got the people who might be sick
link |
00:24:29.720
and you got everyone else, set A and set B.
link |
00:24:32.000
Set A is the people who might be sick,
link |
00:24:33.360
set B is everyone else.
link |
00:24:34.840
And for the entirety of the past
link |
00:24:37.680
contact tracing approaches, you tried to get set A
link |
00:24:41.680
to do things that might not be to their liking or their will
link |
00:24:44.920
because that's removing them from society.
link |
00:24:47.680
We found out that there's two ways
link |
00:24:49.040
to separate set A from set B.
link |
00:24:51.120
You can also let the people at set B
link |
00:24:53.400
at the fringe of set A
link |
00:24:55.720
attempt to remove themselves from this interface.
link |
00:24:58.920
It's the symmetry of A and B separation.
link |
00:25:01.560
Everyone was looking at A, we look at B
link |
00:25:04.520
and suddenly B is in their incentive to do so.
link |
00:25:07.960
Beautiful.
link |
00:25:08.960
So there's a virus that jumps from human to human.
link |
00:25:11.960
So there's a network sometimes called graph
link |
00:25:16.240
of the spread of a virus.
link |
00:25:18.400
It hops from person to person to person to person.
link |
00:25:21.640
And each one of us individuals are sitting
link |
00:25:25.760
or plopped into that network.
link |
00:25:28.200
We have close friends and relations and so on.
link |
00:25:31.680
It's kind of fascinating
link |
00:25:32.520
to actually think about this network
link |
00:25:33.880
and we can maybe talk about the shapes
link |
00:25:35.520
of this kind of network.
link |
00:25:37.680
Because I was trying to think exactly this,
link |
00:25:39.920
like how many people do I,
link |
00:25:41.400
well, I'm kind of an introvert, not kind of,
link |
00:25:43.480
I'm very much an introvert.
link |
00:25:45.160
But so can I be explicit about the kind of people
link |
00:25:48.200
I meet in regular life?
link |
00:25:50.360
Say when it was completely opened up, there's no pandemic.
link |
00:25:54.840
There is a kind of network and there's maybe
link |
00:25:59.440
in the graph theoretic sense, there's some weights
link |
00:26:02.280
or something about how close that relationship is
link |
00:26:06.920
in terms of the frequency of visits,
link |
00:26:08.800
the duration of visits and all of those kinds of things.
link |
00:26:11.560
So you're saying we might want to be,
link |
00:26:14.840
to create on top of that network,
link |
00:26:18.040
a spread of information to let you know
link |
00:26:22.240
as the virus travels through this network,
link |
00:26:24.400
how close is it getting to you?
link |
00:26:26.240
And the number of hops away it is on that network
link |
00:26:29.240
is really powerful information
link |
00:26:31.320
that creates a positive feedback loop
link |
00:26:33.960
where you can act essentially anonymously
link |
00:26:39.080
and on your own.
link |
00:26:41.200
Like nobody's telling you what to do,
link |
00:26:43.960
which is really important, is decentralized
link |
00:26:46.920
and not whatever the opposite of authoritarian is.
link |
00:26:52.200
But you get to sort of the American way.
link |
00:26:54.640
You get to choose to do it yourself.
link |
00:26:56.320
You have the freedom to do it yourself
link |
00:26:58.440
and you're incentivized to do it.
link |
00:27:00.160
And you're most likely going to do it
link |
00:27:01.920
to protect yourself against you getting the disease
link |
00:27:08.400
as the closer it gets to you
link |
00:27:10.000
based on the information that you have.
link |
00:27:12.000
But can you maybe elaborate, first of all, brilliant.
link |
00:27:17.880
Whenever I saw the thing you're working on,
link |
00:27:20.320
so forget for COVID, this is of course,
link |
00:27:23.360
really relevant for COVID, but it's also probably relevant
link |
00:27:26.800
for future diseases as well.
link |
00:27:28.160
So that was the thing I'm nervous about.
link |
00:27:30.400
I was like, if this whole,
link |
00:27:31.840
if our society shut down because of COVID,
link |
00:27:34.720
like what the heck is gonna happen
link |
00:27:38.040
when there's a much deadlier disease?
link |
00:27:40.480
Like this, this is disappointing.
link |
00:27:41.920
The whole time, 2020, the whole time
link |
00:27:44.760
I'm just sitting like this,
link |
00:27:45.960
like is the incompetence of everybody
link |
00:27:49.840
except the people developing vaccines.
link |
00:27:53.000
The biologists are the only ones
link |
00:27:54.400
that got their stuff together.
link |
00:27:56.040
But in terms of institutions and all that kind of stuff,
link |
00:27:58.560
it's just been terrible.
link |
00:28:00.720
But this is exactly the power of information
link |
00:28:04.080
and the power of information
link |
00:28:05.960
that doesn't limit personal freedom.
link |
00:28:08.080
So your idea is brilliant.
link |
00:28:09.400
Okay, mathematically, can you maybe elaborate
link |
00:28:12.680
what are we talking about?
link |
00:28:14.080
Like how do you actually make that work?
link |
00:28:16.480
What's involved?
link |
00:28:17.680
Sure, first I'm gonna reply to something you said
link |
00:28:19.720
about the freedom inside this,
link |
00:28:22.400
because actually that was the idea.
link |
00:28:24.480
The idea is this is game theory, right?
link |
00:28:27.320
And effectively what we did is analogous
link |
00:28:29.600
to free market economy, as opposed to central planning.
link |
00:28:34.840
If you just line up the set of incentives correctly
link |
00:28:38.120
so that people have in their purely selfish behavior
link |
00:28:43.240
are contributing to the optimization of the global function,
link |
00:28:47.120
that's it.
link |
00:28:47.960
And the point of what we do, I guess in mathematics
link |
00:28:50.400
is we try to explore the search space
link |
00:28:52.800
to go and find out as many possibilities as there are.
link |
00:28:54.960
And in this case, it's an applied search space.
link |
00:28:58.080
That's why the inputs from design,
link |
00:29:00.120
user experience design and actual people are important.
link |
00:29:02.720
But you asked about, I guess, the mathematical
link |
00:29:05.760
or the technical things underpinning it.
link |
00:29:07.640
So I think the first thing I'll say is
link |
00:29:09.800
we wanted to make this thing
link |
00:29:12.600
not require your personal information.
link |
00:29:14.760
And so in order to do that,
link |
00:29:16.360
what gave me the confidence to, I guess,
link |
00:29:18.800
lead our team to run at the beginning
link |
00:29:20.560
is we saw that this could be done without using GPS information.
link |
00:29:24.960
So technically what's going on is if two smartphones,
link |
00:29:28.160
it's a smartphone app.
link |
00:29:29.040
If two smartphones have this thing installed,
link |
00:29:31.480
they just communicate with each other by Bluetooth
link |
00:29:35.440
to go and find out how far,
link |
00:29:38.200
they can detect nearby things by Bluetooth.
link |
00:29:40.080
And then they can find out that these two phones
link |
00:29:42.040
were approximately such and such distance apart.
link |
00:29:44.880
And that kind of relative proximity information
link |
00:29:47.680
is enough to construct this big network.
link |
00:29:50.640
Okay, so the physical network is constructed
link |
00:29:53.160
based on proximity that's through Bluetooth
link |
00:29:56.040
and you don't have to specify your exact location,
link |
00:29:59.320
it's the proximity.
link |
00:30:01.200
I'm not using the Pythagorean theorem basically.
link |
00:30:03.320
I mean, if I just knew the GPS coordinates,
link |
00:30:05.520
we could use the Pythagorean theorem too.
link |
00:30:07.320
Sorry, that's just how I call it.
link |
00:30:08.520
Distance formula, whatever you want to call it.
link |
00:30:10.520
Yeah, so we're not doing
link |
00:30:14.800
the old Pythagorean based violation of privacy.
link |
00:30:18.280
Okay.
link |
00:30:21.240
But so is that enough to form,
link |
00:30:27.280
to give you enough information about physical connection
link |
00:30:31.160
to another human being?
link |
00:30:32.880
Is there a time element there?
link |
00:30:35.240
Is there, so, okay.
link |
00:30:37.400
That sounds like a really strong, like low hanging fruit.
link |
00:30:41.560
Like if you have that,
link |
00:30:42.480
you could probably go really, really far.
link |
00:30:44.920
My natural question is,
link |
00:30:46.880
is there extra information you can add on top of that?
link |
00:30:49.760
Like the duration of the physical proximity?
link |
00:30:53.520
So first of all, we actually do estimate the duration,
link |
00:30:56.640
but the way we estimate the duration
link |
00:30:58.360
is like how a movie is filmed,
link |
00:31:00.440
in the sense that every so often, every few minutes,
link |
00:31:03.200
we check what's nearby.
link |
00:31:04.640
It's like how a movie is filmed.
link |
00:31:06.240
You take lots of snapshots.
link |
00:31:07.960
So there's no way in a battery efficient way
link |
00:31:11.280
to really keep track of that proximity.
link |
00:31:14.680
However, fortunately, we're using probability.
link |
00:31:17.120
The fact is the paradigm that we're using
link |
00:31:20.120
is it's not super important
link |
00:31:22.000
if you run into that person only for 10 minutes
link |
00:31:24.080
at the grocery store.
link |
00:31:25.680
If that's a stranger that you run into 10 minutes
link |
00:31:27.640
in this grocery store,
link |
00:31:28.920
that's not gonna be relevant for our paradigm
link |
00:31:30.880
because our paradigm is not telling you
link |
00:31:33.080
who were you around before
link |
00:31:35.000
and might therefore have gotten infected by already.
link |
00:31:38.600
Ours is about predicting the future.
link |
00:31:40.160
We change from, I mean, the standard paradigm was
link |
00:31:42.680
what already happened, quick damage control.
link |
00:31:45.200
Ours is predict the future.
link |
00:31:46.560
If you run into that person once in the grocery store today
link |
00:31:49.280
and never see them again,
link |
00:31:50.360
it's irrelevant for predicting the future.
link |
00:31:52.520
And therefore, for ours, what really matters
link |
00:31:54.800
is the many hours around the other person,
link |
00:31:57.960
at which point, if you're scanning every five
link |
00:31:59.600
to eight minutes.
link |
00:32:00.640
That's going to come out in the problem,
link |
00:32:02.040
like statistically speaking,
link |
00:32:03.240
it's going to come out as a strong relationship
link |
00:32:05.400
and a person in the grocery store is going to wash out
link |
00:32:08.760
is not an important physical relationship.
link |
00:32:11.360
I mean, this is brilliant.
link |
00:32:14.400
How difficult is it to make work?
link |
00:32:15.880
So you said, one, there's a mathematical component
link |
00:32:19.000
that we just kind of talked about,
link |
00:32:21.280
and then there's the user experience component.
link |
00:32:24.000
So how difficult does it to go,
link |
00:32:26.080
just like you built the video game, Alien Attack,
link |
00:32:29.920
from zero to completion, what's involved?
link |
00:32:33.800
How difficult is it?
link |
00:32:34.920
So I'm going to answer that question
link |
00:32:36.720
in terms of building the product,
link |
00:32:39.360
but then I'm also going to acknowledge
link |
00:32:40.880
that just having an app doesn't make it useful
link |
00:32:44.760
because that's actually maybe the easy part.
link |
00:32:48.560
If you know what I mean,
link |
00:32:49.400
there's like all of this stuff
link |
00:32:50.320
about rollout adoption and awareness,
link |
00:32:52.040
but let's focus on the app part first.
link |
00:32:53.760
So that's again, why I said the team is incredible.
link |
00:32:56.320
So we have a bunch of people who,
link |
00:32:59.800
let's just say that the technology that we use to make it
link |
00:33:02.720
is not the standard way you make an app.
link |
00:33:04.960
If you think about a standard iOS app or Android app,
link |
00:33:08.680
those are a user interface that contacts a web server
link |
00:33:12.120
and sends some information back and forth.
link |
00:33:14.080
We're doing some stuff that has to hook
link |
00:33:16.000
into the operating system of saying,
link |
00:33:17.880
let's go use Bluetooth for something
link |
00:33:19.280
it wasn't really meant for, right?
link |
00:33:21.600
So there's that part.
link |
00:33:22.960
By the way, what is the app called?
link |
00:33:24.520
Oh, it's called Novid, COVID with an N.
link |
00:33:28.760
Very nice.
link |
00:33:29.600
So you have to hook into Bluetooth.
link |
00:33:31.360
You're saying you have to do that beyond the permissions
link |
00:33:36.240
that are like at the very surface level
link |
00:33:39.560
provided on the phone?
link |
00:33:40.680
Well, I don't want to call them permissions.
link |
00:33:42.440
I just want to say,
link |
00:33:43.280
that's not what you usually do with Bluetooth.
link |
00:33:45.440
Gotcha.
link |
00:33:46.280
Usually with Bluetooth, you say,
link |
00:33:47.680
do I have headphones nearby?
link |
00:33:49.080
Yes.
link |
00:33:49.920
Okay, I'm done.
link |
00:33:50.760
You don't go and say, do I have headphones nearby?
link |
00:33:53.160
Or do I have another phone nearby, which is doing something?
link |
00:33:55.520
And then keep asking that same question.
link |
00:33:56.760
Keep asking the question.
link |
00:33:58.320
Right?
link |
00:33:59.160
So it's actually not easy.
link |
00:34:00.240
And I mean, there were some parts of it,
link |
00:34:02.160
which actually a lot of people had tried unsuccessfully.
link |
00:34:05.840
Actually, it's known that, for example,
link |
00:34:07.960
the UK was trying to do something similar.
link |
00:34:11.600
And the problem they ran into was,
link |
00:34:13.640
when you program things on iOS,
link |
00:34:16.360
iOS is very good at making it hard
link |
00:34:19.320
to do things in the background.
link |
00:34:21.760
And so there was quite a lot of effort required
link |
00:34:23.800
to go and make this thing work.
link |
00:34:25.440
So the whole point, this thing would run in the background
link |
00:34:28.160
and iOS, I mean, most Android probably as well, right?
link |
00:34:33.720
But yeah, iOS certainly makes it difficult
link |
00:34:35.480
for something to run in the background,
link |
00:34:36.880
especially when it's eating up your battery, right?
link |
00:34:40.400
Well, we wanted to make sure we didn't eat up the battery.
link |
00:34:42.240
So that one we can,
link |
00:34:43.480
we actually are very proud of the fact
link |
00:34:44.920
that ours uses very little battery.
link |
00:34:47.760
Actually, even if compared to Apple's own system, so.
link |
00:34:51.560
Beautiful.
link |
00:34:52.400
So what else is required to make this thing work?
link |
00:34:54.480
Right, so the key was that you had to do
link |
00:34:56.840
a significant amount of work on the actual
link |
00:34:58.800
mobile app development,
link |
00:35:00.160
which fortunately the team that we brought
link |
00:35:02.120
was this kind of general thinkers
link |
00:35:04.000
where we would dig in deep into the operating system
link |
00:35:07.120
documentation and the API libraries.
link |
00:35:09.400
So we got that working.
link |
00:35:10.600
But there's another angle, which is,
link |
00:35:12.280
you also need the servers to be able to compute fast enough,
link |
00:35:15.000
which is tying back to this old school
link |
00:35:17.360
computer programming competitions and math Olympiads.
link |
00:35:20.040
In fact, our team that was working on the algorithm
link |
00:35:23.000
and backend side included several people
link |
00:35:25.440
who had been in these competitions from before,
link |
00:35:29.320
which I happen to know because I do coach the team
link |
00:35:32.360
for the math.
link |
00:35:33.360
And so we were able to bring people in to build servers,
link |
00:35:37.240
a server infrastructure in C++ actually,
link |
00:35:40.120
so that we could support significant numbers of people
link |
00:35:43.080
without needing tons of servers.
link |
00:35:45.360
Is there some distributed algorithms working here
link |
00:35:48.000
or you basically have to keep in the same place
link |
00:35:51.560
the entire graph as it builds?
link |
00:35:53.720
Cause especially the more and more people use it,
link |
00:35:56.280
the bigger, the bigger the graph gets.
link |
00:35:58.240
I mean, this is very difficult scaling problem, right?
link |
00:36:02.280
Ah, so that's actually why this computer algorithm
link |
00:36:05.600
competition stuff was handy.
link |
00:36:07.120
It's because there are only about seven to eight
link |
00:36:11.200
giga people in the world.
link |
00:36:12.920
Yeah.
link |
00:36:14.000
That's not that many.
link |
00:36:15.160
So if you can make your algorithms linear time
link |
00:36:17.520
or almost linear time, a computer operates in gigahertz.
link |
00:36:22.320
I only need to do one run, one recalculation every hour
link |
00:36:26.000
in terms of telling people how far away these dangers are.
link |
00:36:29.080
Yes.
link |
00:36:29.920
So I suddenly have 3,600 seconds
link |
00:36:33.520
and my CPU cores are running in gigahertz.
link |
00:36:36.520
And at most they're eight giga people.
link |
00:36:39.520
Well, you skipping over the fact that there's N squared
link |
00:36:44.000
potential connections between people.
link |
00:36:46.840
So how do you get around the fact that, you know,
link |
00:36:51.400
that we, you know, the potential set of relationship
link |
00:36:54.440
any one of us could have is a billion.
link |
00:36:56.160
So it's a billion times squared.
link |
00:36:59.200
That's the potential amount of data you have to be storing
link |
00:37:02.720
and computing over and constantly updating.
link |
00:37:05.280
So the way we dealt with that is we actually expect
link |
00:37:08.120
that the typical network is very sparse.
link |
00:37:10.840
The technical term sparse would mean that the average degree
link |
00:37:14.840
or the average number of connections that a person has
link |
00:37:17.640
is going to be at most like a hundred strong connections
link |
00:37:21.160
that you care about.
link |
00:37:22.520
If you think of it almost in terms of the heavy hitters,
link |
00:37:25.520
actually in most people's lives,
link |
00:37:28.200
a hundred, if we just kept track
link |
00:37:29.880
of their top hundred interactions,
link |
00:37:32.440
that's probably most of the signal.
link |
00:37:35.480
Yeah, yeah.
link |
00:37:37.680
I'm saddened to think that I might not be even
link |
00:37:40.400
in a double digits, but.
link |
00:37:42.040
Oh, I was intentionally giving a crazy number
link |
00:37:44.720
to account for college students.
link |
00:37:46.160
You call, oh, those are the,
link |
00:37:48.960
who you call on the heavy hitters,
link |
00:37:50.000
the people who are like the social butterflies.
link |
00:37:52.040
Yeah, I need to,
link |
00:37:54.840
I'd love to know that information about myself,
link |
00:37:56.920
by the way, that, do you expose the graph,
link |
00:38:01.960
like how many, like about yourself,
link |
00:38:04.480
how many connections you have?
link |
00:38:06.080
We do expose to each person
link |
00:38:07.920
how many direct connections they have.
link |
00:38:09.600
That's great.
link |
00:38:10.440
But for privacy purposes,
link |
00:38:11.600
we don't tell anybody who their connections,
link |
00:38:14.120
like how their connections are interconnected.
link |
00:38:16.000
Yes, gotcha.
link |
00:38:16.880
But at the same time, we do expose also to everyone
link |
00:38:19.400
an interesting chart that says,
link |
00:38:21.000
here's how many people you have
link |
00:38:22.840
that you're connected to directly.
link |
00:38:24.400
Here's how many at distance two,
link |
00:38:26.720
meaning via people.
link |
00:38:27.880
And then here's how many at distance three.
link |
00:38:29.600
And the reason we do that,
link |
00:38:30.920
is that actually ends up being a dynamic
link |
00:38:32.880
that also boosts adoption.
link |
00:38:34.440
It drives another feedback loop.
link |
00:38:36.360
The reason is because we saw, actually,
link |
00:38:38.120
when we deployed this in some universities,
link |
00:38:40.520
that when people see on their app
link |
00:38:42.760
that they are indirectly connected to hundreds
link |
00:38:46.000
or thousands of other people,
link |
00:38:47.600
they get excited and they tell other people,
link |
00:38:49.120
hey, let's download this app.
link |
00:38:50.760
But you know, we also saw in those examples,
link |
00:38:52.960
especially looking at the screenshots people gave,
link |
00:38:55.480
that is hit as soon as the typical person
link |
00:38:58.600
has two or three other direct connections on the system.
link |
00:39:02.600
Because that means that our app
link |
00:39:04.440
has reached a virality or not of two to three.
link |
00:39:07.680
The key is we were making a viral app to fight a virus
link |
00:39:10.880
spreading on the same network that the virus spreads on.
link |
00:39:14.880
So you're trying to out virus the virus.
link |
00:39:17.120
That's right.
link |
00:39:17.960
That's exactly right.
link |
00:39:20.280
Okay, great.
link |
00:39:21.400
What have you learned from this whole experience
link |
00:39:23.640
in terms of, let's say for COVID,
link |
00:39:26.520
but for future pandemics as well,
link |
00:39:29.840
is it possible to use the power information here
link |
00:39:33.760
of networked information as a virus spreads and travels
link |
00:39:38.520
in order to basically keep the society open?
link |
00:39:41.400
Is it possible for people to protect themselves
link |
00:39:44.840
with this information?
link |
00:39:46.200
Or do you still have to have most,
link |
00:39:48.920
like in this overarching policy
link |
00:39:50.680
of everybody should stay at home, that kind of thing?
link |
00:39:53.680
We are trying to answer that question right now.
link |
00:39:55.400
So the answer is we don't know yet,
link |
00:39:57.600
but that's actually why we're very happy
link |
00:39:59.320
that now the idea has started to become more widely known.
link |
00:40:02.720
And we're already starting to collaborate
link |
00:40:04.640
with epidemiologists.
link |
00:40:06.520
Again, I'm just a mathematician, right?
link |
00:40:08.880
And a mathematician should not be the person
link |
00:40:11.040
who is telling everybody, this will definitely work.
link |
00:40:13.720
But because of the potential power of this approach,
link |
00:40:17.680
especially the potential power
link |
00:40:19.160
of this being an end game for COVID,
link |
00:40:22.880
we have gotten the interest of real researchers.
link |
00:40:26.240
And we're now working together
link |
00:40:27.720
to try to actually understand the answer to that question.
link |
00:40:30.240
Because you see, there's a theory.
link |
00:40:31.600
So what I can share is the mathematics of,
link |
00:40:34.160
here's why there's some hope that this would work.
link |
00:40:36.680
And that's because I'm talking about end game now.
link |
00:40:39.440
End game means you have very few cases.
link |
00:40:41.560
But everywhere, we're always thinking,
link |
00:40:43.600
once there's few cases, then does that mean we now open up?
link |
00:40:46.480
Once you open up in the past, then the cases go up again
link |
00:40:49.560
until you have to lock down again.
link |
00:40:51.440
And now when we talk about the dynamic process that makes,
link |
00:40:54.200
it's guaranteeing you always have cases
link |
00:40:55.880
until you have the great vaccines,
link |
00:40:57.160
which is, we both got vaccinated, this is good.
link |
00:41:00.680
But at the same time, why I'm thinking
link |
00:41:02.520
this is still important is because we know
link |
00:41:04.360
that many vaccine makers have said
link |
00:41:06.560
they're preparing for the next dose next year.
link |
00:41:09.760
And if we have a perpetual thing
link |
00:41:11.720
where you just always need a new vaccine every year,
link |
00:41:14.480
it could actually be beneficial to make sure
link |
00:41:16.280
we have as many other techniques as possible
link |
00:41:18.800
for parts of the world that can't afford,
link |
00:41:20.840
for example, that kind of distribution.
link |
00:41:23.160
Yeah, so actually, no matter how deadly the virus is,
link |
00:41:26.360
no matter how many things,
link |
00:41:27.720
whether you have a vaccine or not,
link |
00:41:29.680
it's still useful to be having this information.
link |
00:41:31.920
Yes.
link |
00:41:32.760
Because to stay home or not, depending on how risk,
link |
00:41:35.880
I'm a big fan, just like you said, of having the freedom
link |
00:41:39.160
for you to decide how risk averse you wanna be, right?
link |
00:41:43.240
Depending on your own conditions,
link |
00:41:44.400
but also on the state of like what you,
link |
00:41:47.240
just how dangerously you like to live.
link |
00:41:50.080
So I think that actually makes a lot of sense.
link |
00:41:51.960
And I also think that since we're,
link |
00:41:54.920
when you think of disease spreading,
link |
00:41:56.960
it spreads in aggregate in the sense that
link |
00:42:00.520
if there are some people who maybe are more risk tolerant
link |
00:42:04.680
because of other things in their life,
link |
00:42:06.480
well, there might also be other people
link |
00:42:08.040
who are less risk tolerance.
link |
00:42:09.800
And then those people decide to isolate.
link |
00:42:12.720
But what matters is in the aggregate
link |
00:42:14.520
that this R naught of the infection spreading
link |
00:42:17.680
drops below one.
link |
00:42:19.000
And so the key is if you can empower people
link |
00:42:21.200
with that power to make that decision,
link |
00:42:23.440
you might actually still be able to drive
link |
00:42:25.080
that R naught down below one.
link |
00:42:27.600
Yeah, and also, this is me talking,
link |
00:42:31.480
people get a little bit nervous, I think,
link |
00:42:33.720
with information somehow mapping to privacy violation.
link |
00:42:38.280
But first of all, in the approach you're describing,
link |
00:42:42.240
that's respecting anonymity.
link |
00:42:46.240
But I would love to have information
link |
00:42:49.240
from the very beginning, from March and April of last year,
link |
00:42:54.040
almost like a map of like where it's risky
link |
00:42:59.080
and where it's not to go.
link |
00:43:01.360
And not map based on sort of the exact location of people,
link |
00:43:05.200
but where people usually hang out kind of thing.
link |
00:43:07.560
Just, and maybe not necessarily about actual location,
link |
00:43:13.080
but just maybe activities,
link |
00:43:15.240
like just to have information about what is good to do
link |
00:43:19.680
and not, in terms of like safety,
link |
00:43:23.080
is it okay to run outside and not,
link |
00:43:25.440
is it okay to go to a restaurant and not,
link |
00:43:27.720
I just feel like we're operating in the blind.
link |
00:43:29.640
And then what you had is a very imperfect signal,
link |
00:43:33.880
which is like basically politicians desperately trying
link |
00:43:37.160
to make statements about what is safe and not.
link |
00:43:40.040
They don't know what the heck they're doing.
link |
00:43:41.600
They have a bunch of smart scientists telling them stuff.
link |
00:43:44.120
And the scientists themselves also, very important,
link |
00:43:47.720
don't always know what they're doing.
link |
00:43:49.600
Epidemiology is not, is as much an art as a science.
link |
00:43:54.600
You're desperately trying to predict the future,
link |
00:43:56.400
which nobody can do.
link |
00:43:57.960
And then you're trying to speak with some level of authority.
link |
00:44:01.200
I mean, if I were to criticize scientists,
link |
00:44:02.920
they spoke with too much authority.
link |
00:44:04.440
It's okay to say, I'm not sure.
link |
00:44:06.480
But then they think like, if I say, I'm not sure,
link |
00:44:10.640
then there's going to be a distrust.
link |
00:44:12.400
What they realize is when you're wrong and you say,
link |
00:44:14.480
I'm sure, it's going to lead to more distrust.
link |
00:44:16.840
So there's this imperfect, like just chaotic,
link |
00:44:19.760
messy system of people trying to figure out
link |
00:44:23.560
with very little information.
link |
00:44:25.320
And what you're proposing is just a huge amount
link |
00:44:27.880
of information, and information is power.
link |
00:44:31.080
Is there challenges with adoption that you see
link |
00:44:34.320
in the future here?
link |
00:44:36.240
So there's, maybe we could speak to,
link |
00:44:38.640
there's approaches, I guess, from Google.
link |
00:44:40.520
There's different people that have tried
link |
00:44:42.600
similar kind of ideas.
link |
00:44:44.800
Not, you have quite a novel idea, actually.
link |
00:44:49.600
But speaking, the umbrella idea of contact tracing,
link |
00:44:53.600
is there something you can comment about
link |
00:44:58.800
why their approaches haven't been fully adopted?
link |
00:45:02.040
Is there challenges there?
link |
00:45:03.200
Is there reasons why Novid might be a better idea
link |
00:45:06.400
moving forward, in general, just about adoption?
link |
00:45:09.240
Yeah, so first of all, I want to say,
link |
00:45:10.680
I always have respect for the methods that other people use.
link |
00:45:13.280
And so it's good to see the other people I've been trying.
link |
00:45:16.160
But what we have noticed is that the difference
link |
00:45:19.040
between our value proposition to the user
link |
00:45:22.360
and the value proposition to the user delivered
link |
00:45:24.320
by everything that was made before is that,
link |
00:45:27.800
unfortunately, the action of installing
link |
00:45:30.480
a standard contact tracing app will then tell you
link |
00:45:34.480
after you have already been exposed to the disease
link |
00:45:37.760
so that you can protect other people from you.
link |
00:45:40.640
And what that does to your own direct probability
link |
00:45:43.640
of getting sick, if you think about it,
link |
00:45:45.720
suppose you were making the decision,
link |
00:45:47.120
should I or should I not install one of those apps?
link |
00:45:50.040
What does that do to your own probability of getting sick?
link |
00:45:55.000
It's close to zero.
link |
00:45:56.320
This is the sad thing you're speaking to, not sad.
link |
00:46:00.680
I suppose it's the way the world is.
link |
00:46:03.280
The only incentive there is to just help other people,
link |
00:46:06.080
I suppose, but a much stronger incentive
link |
00:46:09.640
is anything that allows you to help yourself.
link |
00:46:13.160
Yes, so what I'm saying is that,
link |
00:46:15.520
let's just say free market capitalism
link |
00:46:17.160
was not based on altruism, I think it's based on,
link |
00:46:21.680
if you make a system of incentives
link |
00:46:23.280
so that everybody trying to maximize their own situation
link |
00:46:26.960
somehow contributes to the whole,
link |
00:46:28.760
that's a game theoretic solution to a very hard problem.
link |
00:46:31.800
And so this is actually basically mechanism design,
link |
00:46:34.200
that we've basically come up with a different mechanism,
link |
00:46:36.640
different set of incentives,
link |
00:46:38.280
which incentivizes the adoption,
link |
00:46:40.960
because actually whenever we've been rolling it out,
link |
00:46:43.120
usually the first question we ask people,
link |
00:46:45.240
like say in a university is,
link |
00:46:46.800
do you know what Novid does?
link |
00:46:48.160
And most of them have read about the other apps
link |
00:46:50.640
and they say, Oh, Novid will tell you
link |
00:46:51.920
after you've been around someone so you can quarantine.
link |
00:46:54.120
And we have to explain to them,
link |
00:46:55.400
actually, Novid never wants to ask you to quarantine.
link |
00:46:58.440
That's not the principle.
link |
00:46:59.280
Our principle isn't based on that at all.
link |
00:47:01.240
We just want to let you know if something is coming close
link |
00:47:04.560
so that you can protect yourself.
link |
00:47:07.360
If you want.
link |
00:47:08.200
If you want, if you want, if you want.
link |
00:47:09.320
And then the quarantine is like, yes,
link |
00:47:11.320
in that case, if you're quarantining,
link |
00:47:13.440
it's because you're shutting the door from the inside,
link |
00:47:16.240
if that makes sense.
link |
00:47:17.080
Yes, exactly.
link |
00:47:18.120
Exactly.
link |
00:47:18.960
I mean, this is brilliant.
link |
00:47:20.080
So what do you think the future looks like
link |
00:47:23.360
for future pandemics?
link |
00:47:24.560
What's your plan with Novid?
link |
00:47:26.680
What's your plan with these set of ideas?
link |
00:47:28.760
I am actually still an academic and a researcher.
link |
00:47:31.160
So the biggest work I'm working on right now
link |
00:47:33.360
is to try to build as many collaborations
link |
00:47:35.560
with other public health researchers at other universities
link |
00:47:39.120
to actually work on pilot deployments together
link |
00:47:42.160
in various places.
link |
00:47:43.040
That's the goal.
link |
00:47:44.080
That's actually ongoing work right now.
link |
00:47:45.960
And so, for example, if anyone's watching this
link |
00:47:47.800
and you happen to be a public health researcher
link |
00:47:49.760
and you want to be involved in something like this,
link |
00:47:52.360
I'm just gonna say, I'm still incentive thinking.
link |
00:47:55.240
There's something in it for the researchers too.
link |
00:47:57.320
This could open up an entire new way
link |
00:47:59.360
of controlling disease.
link |
00:48:00.360
That's my hope.
link |
00:48:01.920
I mean, it might actually be true.
link |
00:48:03.560
And people who are involved in figuring out
link |
00:48:06.360
how to make this work,
link |
00:48:08.040
well, it could actually be good for their careers too.
link |
00:48:09.960
I always have to think like,
link |
00:48:11.080
if a researcher was getting involved,
link |
00:48:12.680
what are they getting out of it?
link |
00:48:14.240
Oh, so you mean like from a research perspective,
link |
00:48:16.600
you can like publications and sets of ideas
link |
00:48:20.200
about how to, from a sort of network theory perspective,
link |
00:48:27.000
understand how we control the spread of a pandemic.
link |
00:48:30.080
Yes, and what I'm doing right now
link |
00:48:31.680
is this is basically interdisciplinary research
link |
00:48:33.720
where maybe our side is bringing the technology
link |
00:48:35.960
and the network theory,
link |
00:48:37.040
and the missing parts are epidemiology
link |
00:48:39.200
and public health expertise.
link |
00:48:40.960
And if the two things start to join,
link |
00:48:42.840
also because everywhere that you deploy,
link |
00:48:45.320
let's just say that the world is different
link |
00:48:46.800
in the Philippines as it is in the United States.
link |
00:48:49.400
And just the natures of the locality
link |
00:48:52.040
would mean that someone like me
link |
00:48:53.560
should not be trying to figure out how to do that.
link |
00:48:55.280
But if we can work with the researchers
link |
00:48:56.840
who are based there,
link |
00:48:57.840
now suddenly we might come up with a solution
link |
00:48:59.720
that will help scale in parts of the world
link |
00:49:01.960
where they aren't all getting the Moderna and Pfizer vaccines
link |
00:49:04.680
which cost like $20 a pop in the US.
link |
00:49:07.400
So if they want to participate,
link |
00:49:09.240
who do they reach out to?
link |
00:49:10.720
Oh, that would just be us.
link |
00:49:11.640
I mean, the novid.org website has...
link |
00:49:13.600
Novid.org.
link |
00:49:14.440
It has a feedback reach out form.
link |
00:49:16.880
And actually we are, I mean, again,
link |
00:49:18.840
this is the DNA of being a researcher.
link |
00:49:21.040
I am actually very excited by the idea
link |
00:49:23.320
that this could contribute knowledge
link |
00:49:25.560
that will outlast all of our generations,
link |
00:49:28.240
like all of our lifetimes.
link |
00:49:29.960
There you go.
link |
00:49:30.800
Reach out to novid.org.
link |
00:49:34.440
What about individual people?
link |
00:49:36.040
Should they install the app and try it out?
link |
00:49:37.680
Or is this really geographically restricted?
link |
00:49:40.080
Oh, yeah, I didn't come on here to tell everyone
link |
00:49:42.040
to install the app.
link |
00:49:42.880
I did not come to tell everyone to install the app
link |
00:49:44.760
because it works best
link |
00:49:46.440
if your local health authority is working with us.
link |
00:49:49.400
Gotcha.
link |
00:49:50.240
There's a reason.
link |
00:49:51.080
It's because, this is back to the game theory.
link |
00:49:54.720
If anyone could just say, I'm positive,
link |
00:49:58.320
the high school senior prank would be to say that
link |
00:50:01.840
we have a massive outbreak on finals week.
link |
00:50:03.960
Let's not have final exams.
link |
00:50:05.240
So the way that our system works,
link |
00:50:06.640
it actually borrows some ideas, not borrows,
link |
00:50:08.760
we came up with them independently.
link |
00:50:10.280
But this idea is similar to what Google and Apple do,
link |
00:50:13.240
which is that if the local health authority
link |
00:50:14.880
is working with this, they can,
link |
00:50:16.800
for everyone who's positive,
link |
00:50:17.960
give them a passcode that expires in a short time.
link |
00:50:20.680
So for ours, if you're on the app and saying, I'm positive,
link |
00:50:23.600
you can either just say that,
link |
00:50:25.120
and that's called unverified,
link |
00:50:26.800
or you can enter in one of these codes
link |
00:50:28.360
that you got from the local health authority.
link |
00:50:30.280
So basically, for anyone who's watching this,
link |
00:50:32.440
it's not that you should just go and download it
link |
00:50:34.000
unless you want to go and look at it.
link |
00:50:35.280
That's cool.
link |
00:50:36.120
But if you, on the other hand,
link |
00:50:37.480
if you happen to know anyone at the local health authority,
link |
00:50:39.880
which is trying to figure out how to handle COVID,
link |
00:50:42.640
well then, I mean, we'd be very happy
link |
00:50:44.280
to also work with you.
link |
00:50:46.440
Gotcha.
link |
00:50:47.280
So the verified there is really important
link |
00:50:49.120
because you're maintaining anonymity.
link |
00:50:51.320
And because of that,
link |
00:50:52.160
you have to have some source of verification
link |
00:50:54.600
in order to make sure that it's not possible to manipulate
link |
00:50:59.080
because it's ultimately about trust and information.
link |
00:51:02.040
So it could be, verification is really important there.
link |
00:51:06.120
So basically, individual people should
link |
00:51:09.480
ask their local health authorities
link |
00:51:11.280
to sign up to contact you.
link |
00:51:15.040
I hope this spreads.
link |
00:51:16.320
I hope this spreads for future pandemics
link |
00:51:18.680
because I'm really, it's the amount,
link |
00:51:21.440
the millions of people who are hurt by this,
link |
00:51:25.600
I think our response to the virus,
link |
00:51:28.720
economically speaking,
link |
00:51:30.040
the number of people who lost their dream,
link |
00:51:32.520
lost their jobs, but also lost their dream.
link |
00:51:35.120
Entrepreneurs, jobs often give meaning.
link |
00:51:38.400
There's people who financially and psychologically
link |
00:51:41.080
are suffering because of our,
link |
00:51:43.760
I'll say, incompetent response to the virus
link |
00:51:47.440
across the world, but certainly the United States,
link |
00:51:49.760
that should be the beacon of entrepreneurial hope
link |
00:51:53.960
for the world.
link |
00:51:54.800
So I hope that we'll be able to respond
link |
00:52:00.600
to these kinds of events much better in the future.
link |
00:52:02.760
And this is exactly the right kind of idea.
link |
00:52:05.040
And now is the time to do the investment.
link |
00:52:08.280
Let's step back to the beauty of mathematics.
link |
00:52:13.040
Maybe ask the big, silly question first,
link |
00:52:16.040
which is, what do you find beautiful about mathematics?
link |
00:52:20.840
I think that being able to look at a complicated problem,
link |
00:52:26.880
which looks unsolvable,
link |
00:52:28.560
and then to be able to change the perspective
link |
00:52:30.800
to come from a different angle
link |
00:52:32.480
and suddenly see that there's a nice solution.
link |
00:52:36.160
I don't mean that every problem in math
link |
00:52:37.920
is supposed to be this way,
link |
00:52:39.160
but I think that these reframings
link |
00:52:40.920
and changing of perspectives
link |
00:52:42.160
that cause difficult things to get simplified
link |
00:52:44.640
and crystallized and factored in certain ways is beautiful.
link |
00:52:48.600
Actually, that's related to what we were just talking about
link |
00:52:50.920
with even this fighting pandemics.
link |
00:52:52.640
The crystal idea was just quantify proximity
link |
00:52:57.840
by the number of relationships in the physical network,
link |
00:53:01.680
instead of just by the feet and meters, right?
link |
00:53:04.960
It's just that if you change that perspective,
link |
00:53:07.360
now all of these things follow.
link |
00:53:09.240
And so mathematics to me is beautiful
link |
00:53:12.280
in the pure sense just for that.
link |
00:53:15.000
Yeah, it's quite interesting to see a human civilization
link |
00:53:17.560
as a network, as a graph,
link |
00:53:20.040
and our relationships as kind of edges in that graph.
link |
00:53:25.240
And to then do, outside of just pandemic,
link |
00:53:29.360
do interesting inferences based on that.
link |
00:53:33.760
This is true for like Twitter, social networks and so on,
link |
00:53:36.920
how we expand the kind of things we talk about,
link |
00:53:40.080
think about sort of politically,
link |
00:53:42.200
if you have this little bubble, quote unquote,
link |
00:53:44.680
of ideas that you play with,
link |
00:53:46.880
it's nice from a recommender system perspective,
link |
00:53:50.200
how do you jump out of those bubbles?
link |
00:53:52.000
It's really fascinating.
link |
00:53:53.600
YouTube was working on that, Twitter's working on that,
link |
00:53:57.520
but not always so successfully,
link |
00:53:59.680
but there's a lot of interesting work
link |
00:54:02.680
from a mathematical and a psychological,
link |
00:54:05.160
sociological perspective there within those graphs.
link |
00:54:09.520
But if we look at the cleanest formulation of that,
link |
00:54:13.360
of looking at a problem from a different perspective,
link |
00:54:16.280
you're also involved
link |
00:54:17.360
with the International Mathematics Olympiad,
link |
00:54:20.200
which takes small, clean problems that are really hard,
link |
00:54:27.640
but once you look at them differently, can become easy.
link |
00:54:31.120
But that little jump of innovation is the entire trick.
link |
00:54:36.320
So maybe at the high level,
link |
00:54:38.560
can you say what is the International Mathematical Olympiad?
link |
00:54:41.480
Sure, so this is the competition
link |
00:54:44.600
for people who aren't yet in college, math competition,
link |
00:54:47.840
which is the most prestigious one in the entire world.
link |
00:54:50.720
It's the Olympics of mathematics,
link |
00:54:52.720
but only for people who aren't yet in college.
link |
00:54:55.040
Now, the kinds of questions that they ask you to do
link |
00:54:58.000
are not computational.
link |
00:54:59.600
Usually you're not supposed to find that the answer is 42.
link |
00:55:02.400
Right?
link |
00:55:03.760
Instead, you're supposed to explain why something is true.
link |
00:55:07.280
And the problem is that at the beginning,
link |
00:55:09.840
when you look at each of the questions,
link |
00:55:11.560
first of all, you have four and a half hours
link |
00:55:13.560
to solve three questions, and this is one day,
link |
00:55:16.080
and then you have a second day,
link |
00:55:16.920
which is four and a half hours, three questions.
link |
00:55:19.360
But when you look at the questions,
link |
00:55:20.600
they're all asking you,
link |
00:55:21.440
explain why the following thing is true,
link |
00:55:23.240
which you've never seen before.
link |
00:55:25.160
And by the way, even though there are six questions,
link |
00:55:27.360
if you solve any one of them, you're a genius
link |
00:55:29.120
and you get an honorable mention.
link |
00:55:30.320
So this is hard to solve.
link |
00:55:32.560
So what about, is it one person, is it a team?
link |
00:55:35.280
Ah, so each country can send six people
link |
00:55:38.720
and the score of the country is actually unofficial.
link |
00:55:42.320
There's not an official country versus country system,
link |
00:55:45.360
although everyone just adds up the point scores
link |
00:55:47.480
of the six people and they say,
link |
00:55:48.760
well, now which country stacked up where?
link |
00:55:51.440
Yeah, so maybe as a side comment,
link |
00:55:53.400
I should say that there's a bunch of countries,
link |
00:55:56.560
including the former Soviet Union and Russia,
link |
00:55:59.480
where I grew up, where this is one of the
link |
00:56:04.320
most important competitions that the country participates in.
link |
00:56:08.360
It was a source of pride for a lot of the country.
link |
00:56:11.880
You look at the Olympic sports,
link |
00:56:14.400
like wrestling, weightlifting,
link |
00:56:17.080
there's certain sports and hockey
link |
00:56:20.280
that Russia and the Soviet Union truly took pride in.
link |
00:56:24.960
And actually the Mathematical Olympiad,
link |
00:56:28.600
it was one of them for many years.
link |
00:56:30.880
It's still one of them.
link |
00:56:32.560
And that's kind of fascinating.
link |
00:56:33.720
We don't think about it this way in the United States.
link |
00:56:36.680
Maybe you can correct me if I'm wrong,
link |
00:56:38.200
but it's not nearly as popular in the United States
link |
00:56:42.360
in terms of its integration into the culture,
link |
00:56:45.560
into just basic conversation, into the pride.
link |
00:56:49.080
Like, if you won an Olympic gold medal
link |
00:56:52.360
or if you win the Super Bowl, you can walk around proud.
link |
00:56:56.080
I think that was the case
link |
00:56:57.040
with the Mathematical Olympiad in Russia.
link |
00:56:59.200
Not as much the case in the United States, I think.
link |
00:57:03.040
So I just wanna give that a little aside
link |
00:57:04.880
because beating anybody from Russia,
link |
00:57:07.520
from the Eastern Republic or from China
link |
00:57:09.400
is very, very difficult.
link |
00:57:11.840
Like, if I remember correctly,
link |
00:57:14.880
there's people, this was a multiyear training process.
link |
00:57:18.880
They train hard.
link |
00:57:20.680
And this is everything that they're focused on.
link |
00:57:25.040
My dad was a participant in this.
link |
00:57:29.320
And it's, I mean, it's as serious as Olympic sports.
link |
00:57:33.120
You think about like gymnastics,
link |
00:57:34.520
like young athletes participating in gymnastics.
link |
00:57:36.600
This is as serious as that, if not more serious.
link |
00:57:38.920
So I just wanna give that a little bit of context
link |
00:57:41.360
because we're talking about serious high level math,
link |
00:57:44.640
athletics almost here.
link |
00:57:46.360
Yeah, and actually I also think that it made sense
link |
00:57:49.800
from the Soviet Union's perspective
link |
00:57:51.400
because if you look at what these people do eventually,
link |
00:57:55.480
even though, let's look at the USSR's
link |
00:57:58.800
International Math Olympiad record.
link |
00:58:00.600
Even though they, I say, even though they won
link |
00:58:03.080
a lot of awards at the high school thing,
link |
00:58:05.280
many of them went on to do incredible things
link |
00:58:07.800
in research mathematics or research other things.
link |
00:58:10.600
And that's showing the generalization,
link |
00:58:13.000
generalizability of what they were working on.
link |
00:58:15.960
Because ultimately we're just playing with ideas
link |
00:58:20.160
of how to prove things.
link |
00:58:22.360
And if you get pretty good at inventing creative ways
link |
00:58:26.040
to turn problems apart, split them apart,
link |
00:58:29.040
observe neat ways to turn messy things into simple crystals.
link |
00:58:34.080
Well, if you're gonna try to solve any real problem
link |
00:58:36.200
in the real world, that could be a really handy tool too.
link |
00:58:39.240
So I don't think it was a bad investment.
link |
00:58:41.160
I think it clearly worked well for Soviet Union.
link |
00:58:44.920
Yeah, so this is interesting.
link |
00:58:47.120
People sometimes ask me, you know,
link |
00:58:48.600
you go up and under communism, you know,
link |
00:58:52.080
was there anything good about communism?
link |
00:58:55.560
And it's difficult for me to talk about it
link |
00:58:58.120
because it's not, communism is one of those things
link |
00:59:00.800
that's looked down on like without,
link |
00:59:02.920
in absolutist terms currently.
link |
00:59:05.360
But you could still, in my perspective,
link |
00:59:07.360
talk about the actual, forget communism
link |
00:59:09.520
or whatever the actual term is,
link |
00:59:11.760
but you know, certain ways that the society functioned
link |
00:59:16.760
that we can learn lessons from.
link |
00:59:18.120
And one of the things in the Soviet Union
link |
00:59:20.280
that was highly prized is knowledge,
link |
00:59:25.160
not even knowledge, it's wisdom
link |
00:59:27.280
and the skill of invention, of innovation at a young age.
link |
00:59:34.120
So we're not talking about a selection process
link |
00:59:37.240
where you pick the best students in the school
link |
00:59:40.440
to do the mathematics or to read literature.
link |
00:59:44.120
It's like, everybody did it.
link |
00:59:47.000
Everybody, it was almost treated
link |
00:59:49.920
as if anyone could be the next Einstein,
link |
00:59:53.000
anybody could be the next, I don't know,
link |
00:59:55.560
Hemingway, James Joyce.
link |
00:59:56.920
And so you're forcing an education on the populace
link |
01:00:01.200
and a rigorous deep education,
link |
01:00:03.960
like as opposed to kind of like,
link |
01:00:06.440
oh, we wanna make sure we teach
link |
01:00:10.320
to the weakest student in the class,
link |
01:00:14.240
which American systems can sometimes do
link |
01:00:16.960
because we don't wanna leave anyone behind.
link |
01:00:19.360
The Russian system was anyone can be the strongest student
link |
01:00:25.000
and we're gonna teach you the strongest student
link |
01:00:26.840
and we're going to pretend or force everybody,
link |
01:00:30.840
even the weakest student to be strong.
link |
01:00:32.880
And what that results in, it's obviously,
link |
01:00:35.240
this is what people talk about,
link |
01:00:36.480
is a huge amount of pressure.
link |
01:00:38.120
Like it's psychologically very difficult.
link |
01:00:40.640
This is why people struggle when they go to MIT,
link |
01:00:42.640
this very competitive environment.
link |
01:00:44.400
It can be very psychologically difficult,
link |
01:00:46.080
but at the same time,
link |
01:00:47.360
it's bringing out the best out of people.
link |
01:00:49.800
And that mathematics was certainly one of those things.
link |
01:00:53.200
And exactly what you're saying,
link |
01:00:54.640
which kind of clicked with me just now,
link |
01:00:56.360
as opposed to kind of a spelling bee in the United States,
link |
01:01:00.760
which I guess you spell, I'm horrible at this,
link |
01:01:03.360
but it's a competition about spelling,
link |
01:01:04.960
which I'm not sure, but you could argue
link |
01:01:07.160
it doesn't generalize well to the future skills.
link |
01:01:10.000
Mathematics, especially this kind of mathematics
link |
01:01:13.320
is essentially formalized competition of invention,
link |
01:01:17.120
of creating new ideas.
link |
01:01:21.760
And that generalizes really, really well.
link |
01:01:23.960
So that's quite brilliantly put.
link |
01:01:25.800
I didn't really think about that.
link |
01:01:27.320
So this is not just about the competition.
link |
01:01:29.160
This is about developing minds
link |
01:01:31.960
that will come to do some incredible stuff in the future.
link |
01:01:37.280
Yeah, actually, I want to respond
link |
01:01:38.640
to a couple of things there.
link |
01:01:39.600
The first one, this one, which is this notion
link |
01:01:42.000
of whether or not that is possible
link |
01:01:43.800
in a non authoritarian regime.
link |
01:01:46.280
I think it is.
link |
01:01:47.120
And that's actually why I spent some of my efforts
link |
01:01:49.520
before the COVID thing,
link |
01:01:51.040
actually trying to work towards there.
link |
01:01:53.240
The reason is because if you think about it,
link |
01:01:55.680
let's say in America,
link |
01:01:57.360
lots of people are pretty serious
link |
01:01:58.840
about training very hard for football,
link |
01:02:01.160
or baseball, or basketball.
link |
01:02:02.520
Basketball is very, very accessible,
link |
01:02:04.040
but lots of people are doing that.
link |
01:02:05.840
Why?
link |
01:02:06.680
Well, actually, I think that what was going on
link |
01:02:09.880
with the authoritarian thing was at least the message
link |
01:02:13.520
that was universally sent was being a good thinker
link |
01:02:17.880
and a creator of ideas is a good thing.
link |
01:02:21.800
Yes, exactly.
link |
01:02:23.360
There's no reason why that message can't be sent everywhere.
link |
01:02:26.920
And I think it actually should be.
link |
01:02:28.680
So that's the first thing.
link |
01:02:29.720
The second thing is what you commented about this thing
link |
01:02:32.600
about the generalizable skill
link |
01:02:35.720
and what could people do with Olympiads afterwards.
link |
01:02:37.960
So that's actually my interest in the whole thing.
link |
01:02:40.360
I don't just coach students how to do problems.
link |
01:02:45.400
In fact, I'm not even the best person for that.
link |
01:02:47.160
I'm not the best at solving these problems.
link |
01:02:49.480
There are other people who are much better
link |
01:02:50.800
at making problems and teaching people how to solve problems.
link |
01:02:53.320
In fact, when the Mathematical Association of America,
link |
01:02:57.160
which is the group which is in charge
link |
01:02:58.640
of the US participation in these Olympiads,
link |
01:03:01.280
when they were deciding whether or not to put me in
link |
01:03:04.320
back in 2013 as the head coach,
link |
01:03:06.920
I had a conversation with their executive director
link |
01:03:09.240
where I commented that we might do worse
link |
01:03:12.680
because my position was I don't,
link |
01:03:15.080
I mean, I actually didn't want to focus on winning.
link |
01:03:17.680
I said, if you're going to let me work
link |
01:03:19.840
with 60 very strong minds as picked through this system,
link |
01:03:24.400
because the coach works with these,
link |
01:03:26.080
gets to run a camp for these students.
link |
01:03:27.680
I said, I'm actually not going to define my success
link |
01:03:30.400
in terms of winning this contest.
link |
01:03:33.080
I said, I wanted to maximize the number of the students
link |
01:03:36.040
that I read about in the New York Times in 20 years.
link |
01:03:40.240
And the executive director
link |
01:03:41.520
of the Mathematical Association of America
link |
01:03:44.000
was fully in support of this
link |
01:03:45.840
because that's also how their philosophy is.
link |
01:03:48.000
So in America, the way we run this
link |
01:03:49.840
is we're actually not just training to win,
link |
01:03:52.920
even though the students are very good
link |
01:03:54.840
and they can win anyway.
link |
01:03:56.440
One reason, for example, I went and even did the COVID thing
link |
01:03:59.120
involving quite a few of them
link |
01:04:01.560
is so that hopefully some of them get ideas
link |
01:04:04.280
because in 20, 30 years, I won't have the energy
link |
01:04:06.520
or the insight to solve problems.
link |
01:04:08.520
We'll have another catastrophe.
link |
01:04:10.480
And hopefully some of these people will step up and do it.
link |
01:04:13.160
And ultimately have that longterm impact.
link |
01:04:14.960
I wonder if this is scalable to,
link |
01:04:17.280
because that's such a great metric for education,
link |
01:04:20.320
not how to get an A on the test, but how to have,
link |
01:04:28.880
how to be on the cover of New York Times
link |
01:04:31.320
for inventing something new.
link |
01:04:33.720
And do you think that's generalizable to education
link |
01:04:37.360
beyond just this particular Olympia?
link |
01:04:39.080
Like, even you saying this feels like a rare statement,
link |
01:04:42.760
almost like a radical statement as a goal for education.
link |
01:04:45.920
So actually the way I teach my classes at Carnegie Mellon,
link |
01:04:48.800
which I will admit right away is not equivalent
link |
01:04:51.280
to the average in the world,
link |
01:04:52.440
but it's already not just the top 60 in the country
link |
01:04:56.280
as picked by something.
link |
01:04:58.120
Let me just explain.
link |
01:04:58.960
I have exams in my class, which are 90% of the grade.
link |
01:05:01.480
So the exams are the whole thing,
link |
01:05:02.840
or most of the whole thing.
link |
01:05:03.920
And the way that I let students prepare for the exams
link |
01:05:06.640
is I show them all the problems I've ever given
link |
01:05:08.880
on the previous exams.
link |
01:05:10.360
And the exam that they will take is open notes.
link |
01:05:12.680
They can take all the notes they want
link |
01:05:13.760
on the previous problems.
link |
01:05:14.800
And the guarantee is that the exam problems this time
link |
01:05:17.240
will have no overlap with anything
link |
01:05:18.840
you have seen me give in the past,
link |
01:05:21.000
as well as no overlap with anything I taught in the class.
link |
01:05:24.400
So the entire exam is invention.
link |
01:05:27.360
Wow.
link |
01:05:28.600
But that's how I go, right?
link |
01:05:29.760
My point is I have explained to people when I teach you,
link |
01:05:33.240
I don't want you to have remembered a method I showed you.
link |
01:05:36.720
I want you to have learned enough about this area
link |
01:05:39.320
that if you face a new question,
link |
01:05:40.880
which I came up with the night before
link |
01:05:42.480
by thinking about like,
link |
01:05:43.640
what could I ask that I have never asked before?
link |
01:05:46.200
Oh, that's cute.
link |
01:05:47.040
That's what the answer is.
link |
01:05:48.040
Aha, that's an exam problem.
link |
01:05:49.160
That's exactly what I do before the exam.
link |
01:05:51.320
And then that's what I want them to learn.
link |
01:05:53.840
And the first exam, usually people have a rough time
link |
01:05:56.080
because it's like, what kind of crazy class is this?
link |
01:05:58.400
The professor doesn't teach you anything for the exam.
link |
01:06:01.920
But then by the second or third,
link |
01:06:03.440
and by the time they finished the class,
link |
01:06:05.280
they have learned how to solve anything in the area.
link |
01:06:09.520
How to invent.
link |
01:06:10.360
How to invent in that area, yeah.
link |
01:06:12.240
Can we walk back to the Mathematical Olympiad?
link |
01:06:15.520
What's the scoring and format like?
link |
01:06:18.520
And also what does it take to win?
link |
01:06:20.840
So the way it works is that each of the six students
link |
01:06:25.640
do the problems and there are six problems.
link |
01:06:27.800
All the problems are equally weighted.
link |
01:06:29.560
So each one's worth seven points.
link |
01:06:31.480
That means that your maximum score
link |
01:06:33.400
is six problems times seven points,
link |
01:06:35.000
which is the nice number of 42.
link |
01:06:37.480
And now the way that they're scored by the way
link |
01:06:40.360
is there's partial credit.
link |
01:06:41.680
So the question is asking you,
link |
01:06:43.200
explain why this weird fact is true.
link |
01:06:46.360
Okay, if you explain why you get seven points.
link |
01:06:48.720
If you make minor mistake, maybe you get six points.
link |
01:06:51.320
But if you don't succeed in explaining why,
link |
01:06:53.840
but you explain some other true fact,
link |
01:06:57.960
which is along the way of proving it,
link |
01:07:02.200
then you get partial credit.
link |
01:07:03.960
And actually now this is tricky
link |
01:07:05.680
because how do you score such a thing?
link |
01:07:07.640
It's not like the answer was 72
link |
01:07:10.800
and you wrote 71 and it's close, right?
link |
01:07:13.160
The answer is 72 and you wrote 36.
link |
01:07:15.280
Oh, but that's pretty close
link |
01:07:16.200
because maybe you're just off by it.
link |
01:07:18.880
By the way, they're not numerical anyway,
link |
01:07:20.400
but I'm just giving some numerical analog
link |
01:07:22.680
to the way the scoring might work.
link |
01:07:24.560
They're all essays.
link |
01:07:25.840
And that's where I guess I have some role
link |
01:07:28.240
as well as some other people
link |
01:07:29.320
who helped me in the US delegation for coaches.
link |
01:07:32.360
We actually debate with the country which is organizing it.
link |
01:07:37.560
The country which is organizing the Olympiad
link |
01:07:39.480
brings about 50 people to help judge the written solutions.
link |
01:07:45.160
And you schedule these half hour appointments
link |
01:07:48.360
where the delegation from one country
link |
01:07:50.520
sits down at a table like this.
link |
01:07:52.360
Opposite side is two or three people from the host country.
link |
01:07:55.520
And they're just looking over these exam papers
link |
01:07:58.000
saying, well, how many points is this worth
link |
01:08:00.640
based on some rubric that has been designed?
link |
01:08:03.120
And this is a negotiation process
link |
01:08:05.600
where we're not trying to bargain
link |
01:08:07.760
and get the best score we can.
link |
01:08:09.320
In fact, sometimes we go to this table
link |
01:08:10.880
and we will say, we think we want less than what you gave us.
link |
01:08:13.960
This is how our, these are our principles.
link |
01:08:16.280
If you give us too much, we say, no, you gave us too much.
link |
01:08:18.800
We do that.
link |
01:08:19.720
However, the reason why this is an interesting process
link |
01:08:22.280
is because if you can imagine every country
link |
01:08:24.160
which is participating has its own language.
link |
01:08:26.760
And so if you're trying to grade the Mongolian scripts
link |
01:08:28.760
and they're written in Mongolian,
link |
01:08:31.040
if you don't read Mongolian, which most people don't,
link |
01:08:33.520
then the coaches are explaining to you,
link |
01:08:36.680
this is what the student has written.
link |
01:08:38.760
It's actually quite interesting process.
link |
01:08:40.480
So it's almost like a jury.
link |
01:08:43.400
Yes.
link |
01:08:44.240
You have, in the American legal system,
link |
01:08:47.120
you have a jury that where they're deliberating,
link |
01:08:49.880
but unlike a jury, there's the members of the jury
link |
01:08:53.640
speaking different languages sometimes.
link |
01:08:55.800
Yes. That's fascinating.
link |
01:08:57.120
But I mean, it's hard to know what to do
link |
01:09:01.640
because it's probably really, really competitive.
link |
01:09:04.560
But your sense is that ultimately people,
link |
01:09:08.560
like how do you prevent manipulation here, right?
link |
01:09:14.240
Well, we just hope that it's not happening.
link |
01:09:17.000
So we write in English.
link |
01:09:19.240
Therefore, everything that the US does,
link |
01:09:21.400
everyone can look at.
link |
01:09:22.800
So it's very hard for me.
link |
01:09:24.120
It's very hard for you to manipulate.
link |
01:09:25.720
We don't manipulate.
link |
01:09:27.480
We only hope that other people aren't.
link |
01:09:29.360
But at the same time, as you see, our philosophy was,
link |
01:09:32.320
we want to use this as a way to develop general talent.
link |
01:09:35.560
And although we do this for the six people who go
link |
01:09:38.560
to the International Math Olympiad,
link |
01:09:40.640
we really want that everyone at any,
link |
01:09:42.880
touched at any stage of this process
link |
01:09:45.080
get some skills that can help to contribute more later.
link |
01:09:48.120
So I don't know if you can say something insightful
link |
01:09:51.520
to this question,
link |
01:09:52.560
but what do you think makes a really hard math problem
link |
01:09:56.200
on this Olympiad, maybe in the courses you teach
link |
01:09:59.760
or in general?
link |
01:10:01.080
What makes for a hard problem?
link |
01:10:03.520
You've seen, I'm sure, a lot of really difficult problems.
link |
01:10:06.280
What makes a hard problem?
link |
01:10:07.880
So I could quantify it by the number of leaps of insight
link |
01:10:12.160
of changes of perspective that are along the way.
link |
01:10:14.480
And here's why.
link |
01:10:15.800
This is like a very theoretical computer science
link |
01:10:17.560
way of looking at it, okay?
link |
01:10:19.080
It's that each reframing of the problem
link |
01:10:22.520
and using of some tool,
link |
01:10:23.840
I actually call that a leap of insight.
link |
01:10:25.360
When you say, oh, wow, now I see,
link |
01:10:27.400
I should kind of put these plugs into those sockets
link |
01:10:30.480
like so, and suddenly I get to use that machine.
link |
01:10:33.600
Oh, but I'm not done yet.
link |
01:10:34.920
Now I need to do it again.
link |
01:10:36.160
Each such step is a large possible,
link |
01:10:38.960
large fan out in the search space.
link |
01:10:41.120
The number of these tells you the exponent.
link |
01:10:44.040
The base of the exponent is like how big,
link |
01:10:46.600
how many different possibilities you could try.
link |
01:10:49.360
And that's actually why,
link |
01:10:51.200
like if you have a three insight problem,
link |
01:10:54.240
that is not three times as hard as a one insight problem,
link |
01:10:57.560
because after you've made the one insight,
link |
01:10:59.080
it's not clear that that was the right track necessarily.
link |
01:11:03.000
Well, unless you're very into it.
link |
01:11:03.840
There's still a branching of possibility.
link |
01:11:06.400
Yeah.
link |
01:11:07.240
Right.
link |
01:11:09.800
You're saying there's problems like on the math Olympia
link |
01:11:12.280
that requires more than one insight?
link |
01:11:13.680
Yes.
link |
01:11:14.520
Those are the hard ones.
link |
01:11:15.360
And also I can tell you how you can tell.
link |
01:11:17.560
So this is how I also taught myself math
link |
01:11:19.760
when I was in college.
link |
01:11:20.720
So if you are taking a, not taught myself,
link |
01:11:23.520
I was taking classes, of course,
link |
01:11:24.840
but I was trying to read the textbook
link |
01:11:26.720
and I found out I was very bad at reading math textbooks.
link |
01:11:29.480
A math textbook has a long page of stuff that is all true,
link |
01:11:32.760
which after you read the page,
link |
01:11:34.040
you have no idea what you just read.
link |
01:11:35.640
Yeah.
link |
01:11:36.480
This is just a good summary of a math textbook.
link |
01:11:39.120
Okay.
link |
01:11:39.960
Yeah, because it's not clear why anything was done that way.
link |
01:11:44.120
And yes, everything is true,
link |
01:11:45.400
but how the heck did anyone think of that?
link |
01:11:47.360
So the way that I taught myself math eventually was,
link |
01:11:50.840
the way I read a math textbook
link |
01:11:52.960
is I would look at the theorem statement.
link |
01:11:55.640
I would look at the length of the proof
link |
01:11:58.920
and then I would close the book
link |
01:12:00.000
and attempt to reproof it myself.
link |
01:12:01.280
Yeah.
link |
01:12:02.120
That's brilliant.
link |
01:12:03.520
The length of the proof is telling you
link |
01:12:05.520
the number of insights,
link |
01:12:06.800
because the length of the proof is linear
link |
01:12:08.760
in the number of insights.
link |
01:12:10.640
Each insight takes space.
link |
01:12:12.200
Yeah.
link |
01:12:13.040
And if I know that it's a short proof,
link |
01:12:14.640
I know that there's only one insight.
link |
01:12:16.080
So when I'm doing my own way of solving the problem,
link |
01:12:19.000
like finding the proof,
link |
01:12:20.520
I quit if I have to do too many plugins.
link |
01:12:23.080
It's equivalent to a math contest.
link |
01:12:24.880
In a math contest I look,
link |
01:12:25.800
is it problem one, two, or three?
link |
01:12:27.200
That tells me how many insights there are.
link |
01:12:28.920
This is exactly what I did.
link |
01:12:29.960
That's brilliant.
link |
01:12:30.880
Linear in the number.
link |
01:12:32.080
I don't know.
link |
01:12:32.920
I think it's possible that that's true.
link |
01:12:36.320
Approximately, approximately.
link |
01:12:37.160
Approximately, yeah.
link |
01:12:38.480
I don't know if somebody out there
link |
01:12:41.320
is gonna try to formally prove this.
link |
01:12:43.040
Oh no, I mean, you're right.
link |
01:12:44.160
There are cases where maybe it's not quite linear,
link |
01:12:46.040
but in general.
link |
01:12:47.040
Well, some of it's notation too,
link |
01:12:48.360
and some of it is style and all those kinds of things,
link |
01:12:50.920
but within a textbook.
link |
01:12:51.880
Within the same book.
link |
01:12:52.720
Within the same book with the same.
link |
01:12:54.040
Within the same book on the same subject.
link |
01:12:56.160
Yeah.
link |
01:12:57.000
This is what I was using.
link |
01:12:57.840
That's hilarious.
link |
01:12:58.760
Because you know, if it's a two page proof,
link |
01:13:00.360
you just know this is gonna be insane, right?
link |
01:13:02.760
That's the scary thing about insights.
link |
01:13:06.880
You look like Andrew Wiles
link |
01:13:08.200
working on the Fermat's Last Theorem,
link |
01:13:11.480
is you don't know.
link |
01:13:13.560
Something seems like a good idea,
link |
01:13:16.200
and you have that idea,
link |
01:13:17.280
and it feels like this is a leap,
link |
01:13:20.160
like a totally new way to see it,
link |
01:13:22.280
but you have no idea if it's at all useful.
link |
01:13:25.760
Even if you think it's correct,
link |
01:13:27.000
you have no idea if this is like going to go down a path
link |
01:13:30.120
that's completely counterproductive
link |
01:13:32.480
or not productive at all.
link |
01:13:34.520
That's the crappy thing about invention,
link |
01:13:38.400
is like I have, I'm sure you do.
link |
01:13:41.080
I have a lot of really good ideas every single day,
link |
01:13:44.640
but like, and I'll go inside my head along them,
link |
01:13:49.600
along that little trajectory,
link |
01:13:52.040
but it could be just a total waste.
link |
01:13:54.640
And it's, you know what that feels like?
link |
01:13:57.200
It just feels like patience is required,
link |
01:13:59.480
not to get excited at any one thing.
link |
01:14:01.800
So I think this is interesting
link |
01:14:03.360
because you raised Andrew Wiles.
link |
01:14:04.800
He spent seven years attacking the same thing, right?
link |
01:14:08.400
And so I think that what attracts
link |
01:14:10.360
professional researchers to this
link |
01:14:12.320
is because even though it's very painful
link |
01:14:14.880
that you keep fighting with something,
link |
01:14:16.680
when you finally find the right insights
link |
01:14:19.480
and string them together,
link |
01:14:20.680
it feels really good, so.
link |
01:14:23.080
Well, there's also like short term,
link |
01:14:26.040
it feels good to, whether it's real or not,
link |
01:14:31.480
to pretend like you've solved something
link |
01:14:33.560
in the sense like you have an insight
link |
01:14:35.400
and there's a sense like this might be the insight
link |
01:14:37.960
that solves it.
link |
01:14:39.000
So at least for me, I just enjoy that rush of positivity
link |
01:14:44.440
even though I know statistically speaking
link |
01:14:46.440
is probably going to be a dead end.
link |
01:14:48.520
I'm the same way, I'm the same way.
link |
01:14:49.840
In fact, that's how I know whether
link |
01:14:51.560
I might want to keep thinking about this general problem.
link |
01:14:54.320
It's like, if I still see that I'm getting some insights,
link |
01:14:57.280
I'm not at a dead end yet.
link |
01:14:59.000
But that's also where I learned something
link |
01:15:00.680
from my PhD advisor.
link |
01:15:01.920
Actually, he was a real big inspiration on my life.
link |
01:15:04.280
His name is Benny Sudakov.
link |
01:15:05.760
In fact, he grew up in the former Soviet Union.
link |
01:15:08.120
He was from Georgia, but he's an incredible person.
link |
01:15:12.280
But one thing I learned was choose the problems to work on
link |
01:15:16.680
that might matter if you succeed.
link |
01:15:21.040
Because that's why, for example, we dug into COVID.
link |
01:15:23.400
It was just, well, suppose we succeed
link |
01:15:25.800
in finding some interesting insight here.
link |
01:15:27.840
Well, it actually matters.
link |
01:15:29.120
That is worth a laugh.
link |
01:15:30.960
Yeah, and I think COVID, the way you're approaching COVID
link |
01:15:36.480
has two interesting possibilities.
link |
01:15:38.200
One, it might help with COVID or another pandemic,
link |
01:15:41.680
but two, I mean, just this whole network theory space,
link |
01:15:48.360
you might unlock some deep understanding
link |
01:15:51.200
about the interaction with human beings.
link |
01:15:53.080
That might have nothing to do with the pandemic.
link |
01:15:55.880
There's a space of possible impacts
link |
01:15:58.360
that may be direct or indirect.
link |
01:16:00.440
And the same thing is with Andrew Wiles's proof.
link |
01:16:03.280
I don't understand, but apparently the pieces of it
link |
01:16:08.120
are really impactful for mathematics,
link |
01:16:12.040
even if the main theorem is not.
link |
01:16:14.880
So along the way, the insights you have
link |
01:16:18.240
might be really powerful for unexpected reasons.
link |
01:16:22.920
So I like what you said.
link |
01:16:23.760
This is something that I learned from another friend of mine.
link |
01:16:26.840
He's a very famous researcher.
link |
01:16:28.080
All these people are more famous than I am.
link |
01:16:29.960
His name is Jacob Fox.
link |
01:16:30.960
He's Jacob Fox at Stanford.
link |
01:16:32.200
Also a very big inspiration for me.
link |
01:16:33.800
We were both grad students together at the same time.
link |
01:16:36.000
Well, most importantly,
link |
01:16:36.840
you're good at selecting good friends.
link |
01:16:38.320
Ah, yeah, well, that's the key.
link |
01:16:40.000
You gotta find good people to learn things from.
link |
01:16:42.040
But his thing was, he often said,
link |
01:16:44.400
if you solve a math problem and have this math proof,
link |
01:16:46.840
math problem for him is like a proof, right?
link |
01:16:48.920
So suppose you came up with this proof.
link |
01:16:50.440
He always asks, what have we learned from this
link |
01:16:53.440
that we could potentially use for something else?
link |
01:16:56.040
It's not just, did you solve the problem
link |
01:16:58.080
that was supposed to be famous?
link |
01:17:00.080
And is there something new in the course of solving this
link |
01:17:03.360
that you had to invent
link |
01:17:04.600
that we could now use as a tool elsewhere?
link |
01:17:06.720
Yeah, there's this funny effect
link |
01:17:08.880
where just looking at different fields
link |
01:17:12.680
where people discover parallels.
link |
01:17:15.040
They'll prove something, it'll be a totally new result.
link |
01:17:17.520
And then somebody later realizes
link |
01:17:19.040
this was already done 30 years ago
link |
01:17:20.680
in another discipline, in another way.
link |
01:17:23.400
And it's really interesting.
link |
01:17:26.440
Now, we did this offline
link |
01:17:27.800
in another illustration he showed to me.
link |
01:17:30.240
It's interesting to see the different perspectives
link |
01:17:33.920
on a problem.
link |
01:17:35.560
It kind of points like there's just like
link |
01:17:38.760
very few novel ideas that everything else,
link |
01:17:42.320
that most of us are just looking at different perspective
link |
01:17:45.480
on the same idea.
link |
01:17:47.200
And it makes you wonder this old silly question
link |
01:17:51.520
that I have to ask you is,
link |
01:17:53.840
do you think mathematics is discovered or invented?
link |
01:17:58.200
Do you think we're creating new idea?
link |
01:18:02.560
Are we building a set of knowledge
link |
01:18:06.160
that's distinct from reality?
link |
01:18:09.200
Or are we actually like,
link |
01:18:12.000
is math almost like a shovel
link |
01:18:13.480
where we're digging to like this core set of truths
link |
01:18:16.880
that were always there all along?
link |
01:18:20.000
So I personally feel like it's discovered.
link |
01:18:22.800
But that's also because I guess the way that
link |
01:18:25.360
I like to choose what questions to work on
link |
01:18:27.400
are questions that maybe we'll get to learn something about
link |
01:18:31.120
why is this hard?
link |
01:18:32.200
I mean, I'm often attracted to questions
link |
01:18:33.880
that look simple, but are hard, right?
link |
01:18:36.880
And what could you possibly learn from that?
link |
01:18:38.680
Sort of like probably the attraction
link |
01:18:40.280
of Fermat's last theorem, as you mentioned,
link |
01:18:42.680
simple statement, why is it so hard?
link |
01:18:44.960
So I'm more on the discovered side.
link |
01:18:47.520
And I also feel like if we ever ran into
link |
01:18:49.600
an intelligent other species in the universe,
link |
01:18:54.800
probably if we compared notes,
link |
01:18:56.960
there might be some similarities between both of us
link |
01:19:00.120
realizing that pi is important.
link |
01:19:02.280
Because you might say, why, why humans,
link |
01:19:04.480
do humans like circles more than others?
link |
01:19:06.080
I think stars also like circles.
link |
01:19:08.120
I think planets like circles.
link |
01:19:09.880
They're not perfect circles,
link |
01:19:10.920
but nevertheless, the concept of a circle
link |
01:19:13.160
is just point and constant distance.
link |
01:19:15.760
Doesn't get any simpler than that.
link |
01:19:17.400
It's possible that like an alien species
link |
01:19:19.960
will have, depending on different cognitive capabilities
link |
01:19:23.280
and different perception systems,
link |
01:19:25.000
will be able to see things
link |
01:19:28.200
that are much different than circles.
link |
01:19:30.400
And so if it's discovered,
link |
01:19:33.640
it will still be pointing at a lot of same
link |
01:19:35.840
geometrical concepts, mathematical concepts,
link |
01:19:38.560
but it's interesting to think of how many things
link |
01:19:43.520
we would have to still align,
link |
01:19:45.720
not just based on notation, but based on understanding,
link |
01:19:48.520
like just like some basic mathematical concepts,
link |
01:19:53.960
like how much work is there going to be
link |
01:19:56.880
in trying to find a common language?
link |
01:19:59.240
I mean, this is, I think Stephen Wolfram and his son
link |
01:20:02.480
helped with the movie Arrival,
link |
01:20:04.600
like the developing an alien language,
link |
01:20:07.240
like how would aliens communicate with humans?
link |
01:20:10.640
It's fascinating,
link |
01:20:11.800
because like math seems to be the most promising thing,
link |
01:20:14.360
but even like math,
link |
01:20:15.920
like how do you visualize mathematical ideas?
link |
01:20:22.000
It feels like there has to be an interactive component,
link |
01:20:24.640
just like we have a conversation.
link |
01:20:26.560
There has to be, this is something we don't,
link |
01:20:28.440
I think, think about often, which is like,
link |
01:20:31.440
with somebody who doesn't know anything about math,
link |
01:20:33.880
doesn't know anything about English
link |
01:20:35.480
or any other natural language,
link |
01:20:37.440
how would we describe,
link |
01:20:40.120
we talked offline about visual proofs.
link |
01:20:42.400
How would we, through visual proofs, have a conversation
link |
01:20:47.400
where we say something, here's the concept,
link |
01:20:50.160
the way we see it, does that make sense to you?
link |
01:20:53.800
And like, can you mess with that concept
link |
01:20:57.360
to make it sense for you?
link |
01:20:58.760
And then go back and forth in this kind of way.
link |
01:21:01.400
So purely through mathematics,
link |
01:21:03.040
I'm sure it's possible to have those kinds of experiments
link |
01:21:04.960
with like tribes on earth that don't,
link |
01:21:07.040
there's no common language.
link |
01:21:08.400
Through math, like draw a circle
link |
01:21:10.840
and see what they do with it, right?
link |
01:21:13.200
Do some of these visual proofs,
link |
01:21:15.640
like the summation of the odds and adds up to the squares.
link |
01:21:19.720
Yes, I wonder how difficult that is
link |
01:21:21.960
before one or the other species murders themselves.
link |
01:21:24.760
That's a good question.
link |
01:21:27.720
I hope that the curiosity for knowledge
link |
01:21:29.760
will overpower the greedy,
link |
01:21:31.480
this is back to our game theory thing,
link |
01:21:33.520
that the curiosity of like discovering math together
link |
01:21:37.280
will overpower the desire for resources
link |
01:21:40.040
and ultimately like willing to commit violence
link |
01:21:44.440
in order to gain those resources.
link |
01:21:46.440
I think as we progress,
link |
01:21:47.920
become more and more intelligent as a species,
link |
01:21:50.080
I'm hoping we would value more and more of the knowledge
link |
01:21:53.320
because we'll come up with clever ways
link |
01:21:54.880
to gain more resources so we won't be so resource starved.
link |
01:21:58.160
I don't know.
link |
01:21:59.000
That's a hopeful message for when we finally meet aliens.
link |
01:22:01.240
Yeah, yeah.
link |
01:22:02.600
The cool thing about the Math Olympiad,
link |
01:22:07.120
I don't know if you know work from Francois Chollet
link |
01:22:11.400
from Google, he came up with this kind of IQ test slash,
link |
01:22:16.400
it kind of has similar aspects to it
link |
01:22:18.880
that also the Math Olympiad does for AI.
link |
01:22:24.320
So he came up with these tests
link |
01:22:25.920
where they're very simple for humans,
link |
01:22:29.040
but very difficult for AI to illustrate exactly
link |
01:22:32.280
why we're just not good at seeing a totally new problem.
link |
01:22:38.960
Sorry, AI systems are not good at looking at a new problem
link |
01:22:43.960
that requires you to detect
link |
01:22:46.680
that there's a symmetry of some kind,
link |
01:22:48.560
or there's a pattern that hasn't seen before.
link |
01:22:53.880
The pattern is like obvious to us humans,
link |
01:22:56.680
but it's not so obvious to find that kind of,
link |
01:22:59.480
you're inventing a pattern that's there
link |
01:23:03.640
in order to then find a solution.
link |
01:23:09.720
I don't know if you can comment on that.
link |
01:23:12.720
If you can comment on, but from an AI perspective
link |
01:23:16.280
and from a math problem perspective,
link |
01:23:19.440
what do you think is intelligence?
link |
01:23:22.120
What do you think is the thing
link |
01:23:23.640
that allows us to solve that problem?
link |
01:23:25.760
And how hard is it to build a machine to do that?
link |
01:23:29.560
Asking for a friend.
link |
01:23:30.560
Yeah.
link |
01:23:31.400
So I guess, you see,
link |
01:23:33.000
because if I just think of the raw search space, it's huge.
link |
01:23:35.920
That's why you can't do it.
link |
01:23:37.000
And if I think about what makes somebody
link |
01:23:38.640
good at doing these things, they have this heuristic sense.
link |
01:23:42.320
It's almost like a good chess player of saying,
link |
01:23:44.320
let's not keep analyzing down this way
link |
01:23:45.960
because there's some heuristic reason
link |
01:23:47.600
why that's a bad way to go.
link |
01:23:49.240
Where did they get that heuristic from?
link |
01:23:50.520
Now, that's a good question.
link |
01:23:51.760
I don't know.
link |
01:23:53.000
Because that, if you asked them to explain to you,
link |
01:23:56.680
they could probably say something in words
link |
01:23:58.200
that sounds like it makes sense,
link |
01:23:59.640
but I'm guessing that's only a part
link |
01:24:01.240
of what's really going on in their brain
link |
01:24:03.000
of evaluating that position.
link |
01:24:04.800
You know what I mean?
link |
01:24:05.640
If you ask Gary Kasparov, what is good,
link |
01:24:06.800
or why is this position good, he will say something,
link |
01:24:10.240
but probably not approximating everything
link |
01:24:12.520
that's going on inside.
link |
01:24:14.000
So there's basically a function being computed,
link |
01:24:16.920
but it's hard to articulate what that function is.
link |
01:24:19.200
Now, the question is, could a computer get as good
link |
01:24:21.760
at computing these kinds of heuristic functions?
link |
01:24:24.200
Maybe.
link |
01:24:25.040
I'm not enough of an expert to understand,
link |
01:24:27.840
but one bit of me has always been a little bit curious
link |
01:24:30.320
of whether or not the human brain has a particular tendency
link |
01:24:34.200
due to its wiring to come up with certain kinds of things,
link |
01:24:37.480
which is just natural due to the way
link |
01:24:39.520
that the topology of the neurons and whatever is there,
link |
01:24:43.520
for which if you tried to just build from scratch
link |
01:24:46.120
a computer to do it,
link |
01:24:47.280
would it naturally have different tendencies?
link |
01:24:49.640
I don't know.
link |
01:24:50.480
This is just me being completely ignorant
link |
01:24:52.280
and just saying a few ideas.
link |
01:24:53.640
Well, this is a good thing that mathematics shows
link |
01:24:56.160
is we don't have to be,
link |
01:24:57.840
so math and physics or mathematical physics
link |
01:25:00.920
operates in a world that's different
link |
01:25:02.440
than our descendants of eight brains operate in.
link |
01:25:07.440
So it allows us to have multiple, many, many dimensions.
link |
01:25:11.880
It allows us to work on weird surfaces.
link |
01:25:15.440
I would like topology as a discipline is just weird to me.
link |
01:25:19.920
It's really complicated,
link |
01:25:21.120
but it allows us to work in that space,
link |
01:25:23.560
the differential geometry and all those kinds of things
link |
01:25:25.840
where it's totally outside of our natural day to day
link |
01:25:30.560
four dimensional experience,
link |
01:25:33.240
3D dimensional with time experience.
link |
01:25:35.320
So math gives me hope that we can discover
link |
01:25:44.200
the processes of intelligence outside the limited nature
link |
01:25:49.720
of our own human experiences.
link |
01:25:51.960
But you said that you're not an expert.
link |
01:25:55.920
It's kind of funny.
link |
01:25:57.200
I find that we know so little about intelligence
link |
01:26:02.200
that I honestly think like almost children are more expert
link |
01:26:08.760
at creating artificial intelligence systems than adults.
link |
01:26:14.400
I feel like we know so little,
link |
01:26:15.760
we really need to think outside the box.
link |
01:26:18.200
And those little,
link |
01:26:19.840
I found people should check out
link |
01:26:22.120
Francois Chollet's little exams,
link |
01:26:24.640
but even just solving math problems,
link |
01:26:27.960
I don't know if you've ever done this for yourself,
link |
01:26:30.560
but when you solve a math problem,
link |
01:26:33.400
you kind of then trace back and try to figure out
link |
01:26:38.160
where did that idea come from?
link |
01:26:40.440
Like what was I visualizing in my head?
link |
01:26:45.240
How did I start visualizing it that way?
link |
01:26:48.640
Why did I start rotating that cube in my head in that way?
link |
01:26:52.280
Like what is that?
link |
01:26:53.120
If I were to try to build a program that does that,
link |
01:26:55.480
where did that come from?
link |
01:26:56.880
So this is interesting.
link |
01:26:58.960
So I try to do this to teach middle school students
link |
01:27:02.680
how to learn how to create and think and invent.
link |
01:27:05.560
And the way I do it
link |
01:27:06.840
is there are these math competition problems
link |
01:27:08.920
and I'm working in collaboration
link |
01:27:10.560
with the people who run those.
link |
01:27:12.000
And I will turn on my YouTube live
link |
01:27:14.040
and for the first time,
link |
01:27:15.080
look at those questions and live solve them.
link |
01:27:18.480
The reason I do this is to let the middle school students
link |
01:27:21.160
and the high school students and the adults
link |
01:27:22.400
whoever wants to watch,
link |
01:27:23.440
just see what exactly goes on through someone's head
link |
01:27:27.360
as they go and attempt to invent what they need to do
link |
01:27:30.400
to solve the question.
link |
01:27:32.080
So I've actually thought about that.
link |
01:27:34.520
I think that, first of all, as a teacher,
link |
01:27:37.600
I think about that because whenever I want to explain
link |
01:27:39.920
to a student how to do something,
link |
01:27:42.000
I want to explain how it made sense,
link |
01:27:44.080
why it's intuitive to do the following things
link |
01:27:46.160
and why the wrong things are wrong.
link |
01:27:48.720
Not just why this one short fast way,
link |
01:27:51.840
well, why this is the right way, if that makes sense.
link |
01:27:54.600
So my point is I'm actually always thinking about that.
link |
01:27:57.200
Like how would you think about these things?
link |
01:27:58.880
And then I eventually decided the easiest way
link |
01:28:00.560
to expose this would just be to go live on YouTube
link |
01:28:04.440
and just say, I've never seen any of these questions before.
link |
01:28:06.560
Here we go.
link |
01:28:07.760
Don't you get, that's anxiety inducing for me.
link |
01:28:12.680
Don't you get trapped in a kind of like little dead ends
link |
01:28:17.240
of confusion, even on middle school problems?
link |
01:28:20.320
Yes, that's what the comments are for.
link |
01:28:21.960
The live comments come in and students say, try this.
link |
01:28:24.600
Oh wow.
link |
01:28:25.600
It's actually pretty good.
link |
01:28:26.640
And I'll never get stuck.
link |
01:28:27.760
I mean, I'm willing to go on camera and say,
link |
01:28:30.400
guess what, Potion Dough can't do this.
link |
01:28:32.160
That's fine.
link |
01:28:33.120
But then what ends up happening is you will then see
link |
01:28:35.760
how maybe somebody saying something and I look at the chat
link |
01:28:38.080
and I say, aha, that actually looks useful.
link |
01:28:40.600
Now that also shows how not all ideas,
link |
01:28:44.000
not all suggestions are the same power, if that makes sense.
link |
01:28:46.880
Because if I actually do get stuck,
link |
01:28:48.120
I'll go fishing through the chat, if you've got any ideas.
link |
01:28:52.000
I don't know if you can speak to this,
link |
01:28:53.280
but is there a moment for the middle school students,
link |
01:28:57.320
maybe high school as well,
link |
01:28:59.640
where there's like a turning point for them
link |
01:29:04.360
where they maybe fall in love with mathematics
link |
01:29:07.800
or they get it?
link |
01:29:09.720
Is there something to be said about like discovering
link |
01:29:13.360
that moment and trying to grab them to get them
link |
01:29:17.640
to understand that mathematics is something,
link |
01:29:20.200
no matter what they wanna do in life
link |
01:29:21.520
could be part of their life?
link |
01:29:23.040
Yes.
link |
01:29:23.880
I actually do think that the middle school
link |
01:29:25.440
is exactly the right time
link |
01:29:26.800
because that's the place
link |
01:29:27.880
where your mathematical understanding
link |
01:29:30.400
gets just sophisticated enough
link |
01:29:32.440
that you can start doing interesting things.
link |
01:29:34.640
Because if you're early on and counting,
link |
01:29:37.600
I'm honestly not very good at teaching you new insights.
link |
01:29:40.720
My wife is pretty good at that.
link |
01:29:41.880
But somehow once you get to this part
link |
01:29:44.120
where you know what a fraction is
link |
01:29:45.680
and when you know how to add and how to multiply
link |
01:29:49.560
and what the area of a triangle is,
link |
01:29:51.200
at that point to me, the whole world opens up
link |
01:29:54.120
and you can start observing
link |
01:29:55.240
there are really nifty coincidences,
link |
01:29:57.360
the things that made the Greek mathematicians
link |
01:29:59.840
and the ancient mathematicians excited.
link |
01:30:02.120
Actually back then it was exciting
link |
01:30:03.880
to discover the Pythagorean theorem.
link |
01:30:05.680
It wasn't just homework.
link |
01:30:07.840
So is there,
link |
01:30:10.280
which discipline do you think
link |
01:30:11.760
has the most exciting coincidences?
link |
01:30:14.080
So is it geometry?
link |
01:30:16.360
Is it algebra?
link |
01:30:17.920
Or is it calculus?
link |
01:30:21.560
Well, you see, you're asking me
link |
01:30:22.680
and I'm the guy who gets the most excited
link |
01:30:24.520
when the combinatorics shows up in the geometry.
link |
01:30:27.320
Is it, okay.
link |
01:30:30.040
So it's the combinatorics in the geometry.
link |
01:30:33.080
So first of all, the nice thing about geometry,
link |
01:30:35.240
this is the same nice thing about computer vision
link |
01:30:37.920
is it's visual.
link |
01:30:40.040
So geometry, you can draw circles and triangles and stuff.
link |
01:30:42.720
So it naturally presents itself
link |
01:30:46.720
to the visual proof, right?
link |
01:30:49.280
But also the nice thing about geometry,
link |
01:30:51.160
I think for me is the earliest class,
link |
01:30:56.360
the earliest discipline where there's,
link |
01:30:59.680
that's most amenable to the exploration,
link |
01:31:02.600
the invention through proofs.
link |
01:31:04.960
The idea of proofs I think is most easily shown in geometry
link |
01:31:09.640
because it's so visual, I guess.
link |
01:31:12.280
So that to me is like,
link |
01:31:14.400
if I were to think about
link |
01:31:15.440
when I first fell in love with math, it would be geometry.
link |
01:31:18.360
And sadly enough, that's not used.
link |
01:31:21.280
Geometry only has a little,
link |
01:31:23.240
appears briefly in the journey of a student.
link |
01:31:29.000
And it kind of disappears.
link |
01:31:30.600
And not until much later,
link |
01:31:32.320
which there may be like differential geometry,
link |
01:31:36.560
I don't know where else it shows up.
link |
01:31:37.720
For me in computer science,
link |
01:31:38.840
like you could start to think about
link |
01:31:40.960
like computational geometry or even graph theory
link |
01:31:44.520
as a kind of geometry.
link |
01:31:45.600
You could start to think about it visually,
link |
01:31:47.200
although it's pretty tricky.
link |
01:31:49.320
But yeah, it was always,
link |
01:31:51.080
that was the most beautiful one.
link |
01:31:53.000
Everything else, I guess calculus can be kind of visual too.
link |
01:31:56.200
That can be pretty beautiful.
link |
01:31:57.800
But is there something you try to look for in the student
link |
01:32:05.360
to see like, how can I inspire them at this moment?
link |
01:32:09.920
Or is this like individual student to student?
link |
01:32:11.960
Is there something you could say there?
link |
01:32:13.880
So first of all,
link |
01:32:14.720
I really think that every student
link |
01:32:16.440
can pick up all of this skill.
link |
01:32:17.880
I really do think so.
link |
01:32:18.720
I don't think it's something only for a few.
link |
01:32:20.520
And so if I'm looking for a student,
link |
01:32:23.680
actually oftentimes what I'm,
link |
01:32:25.080
if I'm looking at a particular student,
link |
01:32:26.800
the question is,
link |
01:32:28.400
how can we help you feel like
link |
01:32:30.560
you have the power to invent also?
link |
01:32:32.880
Because I think a lot of people
link |
01:32:34.200
are used to thinking about math
link |
01:32:35.600
as something where the teacher will show you what to do
link |
01:32:37.960
and then you will do it.
link |
01:32:39.240
Yes.
link |
01:32:40.080
So I think that the key is to show that they have some,
link |
01:32:42.160
let them see that they have some power to invent.
link |
01:32:44.280
And at that point,
link |
01:32:45.120
it's often starting by trying to give a question
link |
01:32:47.640
that they don't know how to do.
link |
01:32:48.720
You want to find these questions
link |
01:32:49.880
that they don't know how to do,
link |
01:32:51.160
that they can think about,
link |
01:32:53.000
and then they can solve.
link |
01:32:54.480
And then suddenly they say,
link |
01:32:55.600
my gosh, I've had a situation,
link |
01:32:57.800
I've had an experience where I didn't know what to do.
link |
01:33:00.440
And after a while, I did.
link |
01:33:03.320
Is there advice you can give on how to learn math
link |
01:33:08.760
for people, whether it's a middle school,
link |
01:33:10.720
whether it's somebody as an adult
link |
01:33:14.520
kind of gave up on math maybe early on?
link |
01:33:19.360
I actually think that these math competition problems,
link |
01:33:22.080
middle school and high school are really good.
link |
01:33:23.800
They're actually very hard.
link |
01:33:25.080
So if you haven't had this kind of experience before
link |
01:33:29.160
and you grab a middle school math competition problem
link |
01:33:32.960
from the state level,
link |
01:33:33.920
which is used to decide who represents the state
link |
01:33:36.320
in the country, in the United States, for example,
link |
01:33:39.160
those are pretty tricky.
link |
01:33:40.680
And even if you are a professional,
link |
01:33:43.560
maybe not doing mathematical things
link |
01:33:45.480
and you're not a middle school student, you'll struggle.
link |
01:33:48.280
So I find that these things really do teach you things
link |
01:33:51.080
by trying to work on these questions.
link |
01:33:53.080
Is there a Googleable term that you could use
link |
01:33:56.760
for the organization, for the state competitions?
link |
01:33:59.440
Ah, yeah.
link |
01:34:00.280
So there are a number of different ones
link |
01:34:02.280
that are quite popular.
link |
01:34:03.600
One of them is called Math Counts, M A T H C O U N T S.
link |
01:34:07.760
And that's a big tournament,
link |
01:34:08.880
which actually has a state level.
link |
01:34:10.480
There's also a mathleague.org,
link |
01:34:12.680
mathleague, L E A G U E dot org,
link |
01:34:15.360
also has this kind of tiered tournament structure.
link |
01:34:18.680
There's also the American math competitions, AMC 8.
link |
01:34:22.600
AMC also has AMC 10, that's for 10th grade and below
link |
01:34:25.920
and AMC 12.
link |
01:34:27.240
These are all run by the Mathematical Association
link |
01:34:29.240
of America.
link |
01:34:30.280
And these are all ways to find old questions.
link |
01:34:32.880
What about the daily challenges that you run?
link |
01:34:34.840
What are those about?
link |
01:34:36.000
We do that too.
link |
01:34:36.840
But I mean, the difference was ours isn't,
link |
01:34:39.000
that one's not free.
link |
01:34:39.920
So I should actually probably be careful.
link |
01:34:42.080
The things that I've just mentioned are also not free.
link |
01:34:44.280
Not all of those things I mentioned just now
link |
01:34:45.720
are free either.
link |
01:34:46.600
Well, people can figure out what is free and what's not,
link |
01:34:48.760
but this is really nice to know what's out there.
link |
01:34:51.040
But can you speak a little bit to the daily challenges?
link |
01:34:53.720
Sure, sure.
link |
01:34:54.560
So that's actually what we did when,
link |
01:34:56.800
I guess I was thinking about,
link |
01:34:58.480
how would I try to develop that skill in people
link |
01:35:02.040
if we had the power to architect the entire system ourselves?
link |
01:35:05.360
So that's called the daily challenge with Po Shan Luo.
link |
01:35:07.440
It's not free because that's actually how I pay
link |
01:35:09.720
for everything else I do.
link |
01:35:11.240
So that was the idea.
link |
01:35:12.840
But the concept was, aha, now let's invent from scratch.
link |
01:35:16.480
So if we're gonna go from scratch
link |
01:35:17.960
and we're gonna use technology,
link |
01:35:19.720
what if we made every single lesson
link |
01:35:23.120
something where first I say,
link |
01:35:24.680
hey, here's an interesting question.
link |
01:35:25.840
Recorded, of course, not live.
link |
01:35:27.080
But it's like, I say,
link |
01:35:27.920
hey, here's an interesting question.
link |
01:35:28.760
Why don't we think about this?
link |
01:35:29.880
But I know you don't know how to do it.
link |
01:35:32.000
So now you think,
link |
01:35:32.960
and a minute later a hint pops on the screen.
link |
01:35:35.520
But you still think.
link |
01:35:36.360
And a minute later a big hint pops on the screen.
link |
01:35:38.560
You still think.
link |
01:35:39.400
And then finally, after the three minutes,
link |
01:35:41.280
hopefully you got some ideas you tried to answer.
link |
01:35:43.520
And then suddenly there's like this
link |
01:35:45.440
pretty extended explanation of,
link |
01:35:47.720
oh yeah, so here's like multiple different ways
link |
01:35:50.280
that you can do the question.
link |
01:35:51.600
And by accident, you also just learned this other concept.
link |
01:35:54.640
That's what we did.
link |
01:35:55.480
So yeah.
link |
01:35:56.320
Is this targeted towards middle school students,
link |
01:35:58.160
high school students?
link |
01:35:59.480
It's targeted towards middle school students
link |
01:36:01.360
with competitions.
link |
01:36:02.400
But there's a lot of high school students
link |
01:36:04.040
who didn't do competitions in middle school
link |
01:36:06.400
where they would also learn how to think.
link |
01:36:07.840
If you can see the whole concept was,
link |
01:36:09.800
can we teach people how to think?
link |
01:36:11.720
How would you do that?
link |
01:36:12.760
You need to give people the chance to,
link |
01:36:14.440
on their own, invent without that kid in the front row
link |
01:36:17.960
answering every question in two seconds.
link |
01:36:20.320
And people can find it, I think, what daily.
link |
01:36:24.040
It's daily.potionlo.com.
link |
01:36:26.080
But if you go to find my website,
link |
01:36:27.840
you'll be able to find it.
link |
01:36:29.200
Beautiful.
link |
01:36:30.600
Can we zoom out a little bit in the,
link |
01:36:32.880
so day to day, week to week, month to month,
link |
01:36:36.240
year to year, what does the lifelong educational process
link |
01:36:41.480
look like, do you think?
link |
01:36:43.880
For yourself, but for me,
link |
01:36:46.600
what would you recommend in the world of mathematics
link |
01:36:50.280
or sort of as opposed to studying for a test,
link |
01:36:52.600
but just like lifelong expanding of knowledge
link |
01:36:59.240
in that skill for invention?
link |
01:37:02.120
I think I often articulate this as,
link |
01:37:05.040
can you always try to do more than you could do in the past?
link |
01:37:10.040
Yeah.
link |
01:37:11.440
But that comes in many ways.
link |
01:37:13.640
And I will say it's great
link |
01:37:15.280
if one wants to build that with mathematics,
link |
01:37:17.680
but it's also great to use that philosophy
link |
01:37:20.000
with all other things.
link |
01:37:21.080
In fact, if I just think of myself, I just think,
link |
01:37:23.920
what do I know now that I didn't know a year ago
link |
01:37:26.480
or a month ago or a week ago?
link |
01:37:28.520
And not just know,
link |
01:37:29.400
but what do I have the capability of doing?
link |
01:37:32.040
And if you just have that attitude, it brings more.
link |
01:37:35.040
See, the thing is, there's also a habit,
link |
01:37:38.280
like it is a skill, like I've been using Anki,
link |
01:37:43.040
it's an app for helps you memorize things.
link |
01:37:46.640
And I've actually, a few months ago,
link |
01:37:50.720
started doing this daily of setting aside time
link |
01:37:54.960
to think about an idea that's outside of my work.
link |
01:37:59.960
Like, let's say, it's all over the place, by the way,
link |
01:38:04.680
but let's say politics, like gun control.
link |
01:38:08.320
Is it good to have a lot of guns or not in society?
link |
01:38:11.800
And just, I've set aside time every day,
link |
01:38:15.000
I do at least 10 minutes, but I try to do 30,
link |
01:38:17.640
where I think about a problem.
link |
01:38:19.000
And I kind of outline it for myself from scratch,
link |
01:38:20.920
from not looking anything up,
link |
01:38:22.200
just thinking about it, using common sense.
link |
01:38:24.960
And I think the practice of that is really important.
link |
01:38:29.120
It's the daily routine of it, it's the discipline of it.
link |
01:38:32.600
It's not just that I figured something out
link |
01:38:35.880
from thinking about gun control,
link |
01:38:38.800
it's more that that muscle is built too,
link |
01:38:43.080
it's that thinking muscle.
link |
01:38:44.200
So I'm kind of interested in, you know, math has,
link |
01:38:49.600
because especially because I've gotten specialized
link |
01:38:52.080
into machine learning,
link |
01:38:53.040
and because I love programming so much,
link |
01:38:55.360
I've lost touch with math a little bit
link |
01:38:59.760
to where I feel quite sad about it,
link |
01:39:02.400
and I want to fix that.
link |
01:39:05.040
Even just not math, like pure knowledge math,
link |
01:39:07.520
but math, like these middle school problems,
link |
01:39:10.200
the challenges, right?
link |
01:39:13.360
Is that something you see a person
link |
01:39:14.680
be able to do every single day,
link |
01:39:16.160
kind of just practice every single day for years?
link |
01:39:19.600
So I can give an answer to that,
link |
01:39:21.520
that gives a practical way you could do it,
link |
01:39:23.160
assuming you have kids.
link |
01:39:24.480
So, no, you can do it yourself.
link |
01:39:26.920
Step one, get kids.
link |
01:39:28.000
No, no, I'm just saying this
link |
01:39:29.560
because I'm just thinking out loud right now,
link |
01:39:31.720
what could I do to suggest?
link |
01:39:33.720
Because what I have noticed is that, for example,
link |
01:39:35.960
if you do have kids who are in elementary school
link |
01:39:37.760
or middle school, if you yourself go and look
link |
01:39:41.080
at those middle school math problems
link |
01:39:43.080
to think about interesting ways
link |
01:39:44.480
that you can teach your elementary school
link |
01:39:46.160
or middle school kid, it works.
link |
01:39:48.120
That's what my wife did.
link |
01:39:48.960
She never did any of those contests before,
link |
01:39:51.040
but now she knows quite a lot about them.
link |
01:39:53.000
And I didn't teach her anything.
link |
01:39:53.920
I don't do that.
link |
01:39:55.160
She just was messing around with them
link |
01:39:57.320
and taught herself all of that stuff.
link |
01:39:59.600
And that had the automatic daily.
link |
01:40:01.400
I'm always thinking, how do you make it practical, right?
link |
01:40:03.840
And the way to make it practical
link |
01:40:04.880
is if the timer on the automatically daily
link |
01:40:07.680
is that you are going to automatically daily
link |
01:40:09.520
do something with your own kid.
link |
01:40:11.240
Now it feeds back.
link |
01:40:13.360
And that includes the whole lesson
link |
01:40:14.760
that if you wanna learn something, you should teach it.
link |
01:40:16.960
Oh, I strongly believe that.
link |
01:40:19.160
I strongly believe that.
link |
01:40:21.760
So I currently don't have kids.
link |
01:40:23.440
So that's, maybe I should just get kids
link |
01:40:25.600
to help me with the math thing.
link |
01:40:27.040
But outside of that,
link |
01:40:29.320
I do want to do great math into daily practice.
link |
01:40:32.040
So I'll definitely check out the daily challenges
link |
01:40:35.920
and see, because what is it?
link |
01:40:39.200
Grant Sanderson, we talked about offline,
link |
01:40:41.080
the three blue and brown.
link |
01:40:42.720
He speaks to this as well,
link |
01:40:44.920
that his videos aren't necessarily,
link |
01:40:48.200
they don't speak to the thing that I'm referring to,
link |
01:40:50.520
which is the daily practice.
link |
01:40:52.640
They're more almost tools of inspiration.
link |
01:40:56.320
They kind of show you the beauty
link |
01:40:58.520
of a particular problem in mathematics,
link |
01:41:03.640
but they're not a daily ritual.
link |
01:41:05.760
And I'm in search of that daily ritual mathematics.
link |
01:41:09.560
It's not trivial to find,
link |
01:41:14.600
but I hope to find that
link |
01:41:16.880
because I think math gives you a perspective on the world
link |
01:41:20.960
that enriches everything else.
link |
01:41:23.960
So I like what you said about the daily also,
link |
01:41:25.800
because that's also one reason
link |
01:41:27.400
why I put my Carnegie Mellon class online.
link |
01:41:29.960
It's not every day.
link |
01:41:30.800
It's every other day.
link |
01:41:31.960
Semester is almost over.
link |
01:41:33.280
But the idea was, I guess my philosophy was,
link |
01:41:35.840
if I'm already doing the class,
link |
01:41:37.320
let's just put it there, right?
link |
01:41:38.880
But I do know that there are people
link |
01:41:40.880
who have been following it,
link |
01:41:42.280
who are not in my class at all,
link |
01:41:43.880
who have just been following it because,
link |
01:41:45.920
yes, it's combinatorics.
link |
01:41:47.560
And the value of that is you could,
link |
01:41:49.840
you don't really need to know calculus to follow it,
link |
01:41:51.800
if that makes sense.
link |
01:41:52.720
So it's actually something that people could follow.
link |
01:41:54.600
So again, and that one's free.
link |
01:41:56.000
So that one's just there on YouTube.
link |
01:41:58.680
Well, speaking of combinatorics,
link |
01:42:01.440
what is it, what do you find interesting,
link |
01:42:03.800
what do you find beautiful about combinatorics?
link |
01:42:07.240
So combinatorics to me is the study of things
link |
01:42:11.520
where they might be more finite and more discreet.
link |
01:42:17.240
What I mean is like, if I look at a network,
link |
01:42:18.960
actually a lot of times the combinatorics
link |
01:42:20.680
will boil down to something,
link |
01:42:21.880
and the combinatorics I think about
link |
01:42:23.240
might be something related to graphs or networks.
link |
01:42:25.960
And they're very discreet because if you have a node,
link |
01:42:28.840
it's not that you have 0.7 of a node
link |
01:42:32.160
and 0.3 of a node over there.
link |
01:42:33.520
It's that you've got one node,
link |
01:42:34.720
and then you jump one step to go to the next node.
link |
01:42:37.440
So that notion is different from say, calculus,
link |
01:42:39.880
which is very continuous,
link |
01:42:42.080
where you go and say, I have this speed,
link |
01:42:44.360
which is changing over time.
link |
01:42:46.200
And now what's the distance I've traveled?
link |
01:42:47.800
That's the notion of an integral,
link |
01:42:49.040
where you have to think of subdividing time
link |
01:42:50.920
into very, very small pieces.
link |
01:42:52.640
So the kinds of things that you do
link |
01:42:54.400
when you reason about these finite discreet structures
link |
01:42:59.360
often might be iterative, algorithmic, inductive.
link |
01:43:03.280
These are ideas where I go from one step to the next step
link |
01:43:06.320
and so on and make progress.
link |
01:43:08.160
I guess I actually personally like all kinds of math.
link |
01:43:11.280
My area of research just ended up in here
link |
01:43:13.560
because I met a really interesting PhD advisor,
link |
01:43:17.160
potential, that's honestly the reason
link |
01:43:19.000
I went into that direction.
link |
01:43:20.320
I met a really interesting guy.
link |
01:43:22.040
He seemed like he did good stuff, interesting stuff,
link |
01:43:24.880
and he looked like he cared about students.
link |
01:43:26.680
And I said, let me just go and learn whatever you do,
link |
01:43:29.240
even though my prior practice and preparation
link |
01:43:32.240
before my PhD was not combinatorics,
link |
01:43:34.280
but analysis, the continuous stuff.
link |
01:43:36.920
So the annoying thing about combinatorics
link |
01:43:40.600
and discreet stuff is it's often really difficult to solve
link |
01:43:45.600
from a sort of running time complexity perspective.
link |
01:43:53.840
Could you speak to the idea of complexity analysis
link |
01:43:59.640
of problems, do you find it useful, do you find it interesting?
link |
01:44:04.480
Do you find that lens of studying the difficulty
link |
01:44:08.240
of how difficult the computer science problem is
link |
01:44:12.600
a useful lens onto the world?
link |
01:44:15.200
Oh, very much so.
link |
01:44:16.200
Because if you want to make something practical
link |
01:44:20.400
which has large numbers of people using it,
link |
01:44:22.760
the computational complexity to me is almost question one.
link |
01:44:27.240
And that's, again, that's at the origin
link |
01:44:29.080
of when we started doing this stuff with disease control.
link |
01:44:31.760
From the very beginning, the deep questions
link |
01:44:33.480
that were running through my mind were,
link |
01:44:35.400
would we be able to support a large population
link |
01:44:38.480
with only one server?
link |
01:44:41.160
And if the answer is no, we can't start
link |
01:44:43.680
because I don't have enough money.
link |
01:44:48.200
Yeah, and there the question is very much
link |
01:44:51.080
linear time versus anything slower than linear time.
link |
01:44:58.360
As a very specific thing, you have a bunch
link |
01:45:00.280
of really interesting papers.
link |
01:45:01.360
If I could ask, maybe we could pull out some cool insights
link |
01:45:04.240
at the high level.
link |
01:45:06.160
Can you describe the data structure of a voting tree
link |
01:45:08.920
and what are some interesting results on it?
link |
01:45:11.240
You have a paper that I noticed on it.
link |
01:45:13.720
Yeah, so this is an example of, I guess,
link |
01:45:17.400
how in math we might say here's an interesting
link |
01:45:20.960
kind of a question that we just can't seem
link |
01:45:24.400
to understand enough about.
link |
01:45:25.720
Maybe there's something else going on here.
link |
01:45:27.560
And the way to describe this is you could imagine
link |
01:45:30.920
trying to hold elections where if you have
link |
01:45:33.800
only two candidates, that's kind of easy.
link |
01:45:35.920
You just run them against each other
link |
01:45:37.200
and see who gets more votes.
link |
01:45:38.600
But as you know, once you have more candidates,
link |
01:45:40.680
it's very difficult to decide who wins the election.
link |
01:45:43.120
And there's an entire voting theory around this.
link |
01:45:46.280
So a theoretical question became,
link |
01:45:49.400
what if you made like a system of runoffs,
link |
01:45:53.560
like a system of head to head contests,
link |
01:45:57.160
which is structured like a tree,
link |
01:45:58.600
almost looking like a circuit.
link |
01:46:00.200
I'm using that way of thinking because it's sort of like
link |
01:46:03.160
in electrical engineering or computer science,
link |
01:46:05.680
you might imagine having a bunch of leads
link |
01:46:08.120
that carry signal, which are going through AND gates
link |
01:46:10.520
and OR gates and whatnot.
link |
01:46:11.640
And you've managed to compute beautiful things.
link |
01:46:13.640
This is just from a purely abstract point of view.
link |
01:46:16.280
What if the inputs are candidates?
link |
01:46:18.600
And for every two candidates, it is known
link |
01:46:20.840
which of the candidates is more popular than the other.
link |
01:46:23.480
Now can you build some kind of a circuit board
link |
01:46:25.880
which says, first candidate number four
link |
01:46:28.000
will play against five and see who wins and so on.
link |
01:46:31.600
Okay, so now what would be a nice outcome, right?
link |
01:46:34.600
This is a general question of,
link |
01:46:35.840
could I make a big circuit board to feed an election into?
link |
01:46:39.120
Like maybe one nice outcome would be whoever wins
link |
01:46:41.320
at least is preferred over a lot of people.
link |
01:46:44.680
Yes.
link |
01:46:45.520
So for example, if you ran in 1,024 candidates,
link |
01:46:48.480
ideally we would like a guarantee that says
link |
01:46:51.080
that the winner beats a lot of people.
link |
01:46:54.080
Actually in any system where there are 1,024 candidates,
link |
01:46:58.520
there's always a candidate who beats
link |
01:47:00.320
at least 512 of the others.
link |
01:47:02.840
This is a mathematical fact
link |
01:47:04.360
that there's actually always a person who beats
link |
01:47:06.200
at least half of the other people.
link |
01:47:09.680
I'm trying to make sense of that mathematical fact.
link |
01:47:13.400
Is this supposed to be obvious?
link |
01:47:15.000
No, but I can explain it.
link |
01:47:17.000
No, I can't.
link |
01:47:17.840
The way it works is that, think of it this way.
link |
01:47:21.400
Every time I think, imagine I have all these candidates
link |
01:47:24.280
and everyone is competing,
link |
01:47:26.040
everyone is like compared with everyone else at some point.
link |
01:47:29.200
Well, think of it this way.
link |
01:47:30.600
Whenever there's a comparison, somebody gets a point.
link |
01:47:34.200
That's the one who is better than the other one.
link |
01:47:37.000
My claim is there's somebody whose score
link |
01:47:39.440
is at least half of how many other people there are.
link |
01:47:42.880
Yeah, I'm just trying to,
link |
01:47:44.840
like my intuition is very close to that being true,
link |
01:47:47.600
but it's beautiful.
link |
01:47:48.680
I didn't at first, that's not an obvious fact.
link |
01:47:52.240
No, it's not.
link |
01:47:53.080
And it feels like a beautiful fact.
link |
01:47:55.760
Well, let me explain it this way.
link |
01:47:57.120
Imagine that for every match,
link |
01:48:00.520
you didn't give one point, but you gave two points.
link |
01:48:03.840
You gave one point to each person.
link |
01:48:05.920
Now that's not what we're really doing.
link |
01:48:07.040
We really want to give one point to the winner of the match,
link |
01:48:10.840
but instead we'll just give two.
link |
01:48:12.240
If you gave two points to everyone on every matchup,
link |
01:48:15.880
actually everyone has the same number of points.
link |
01:48:18.400
And the number of points they get
link |
01:48:19.680
is how many other people there are.
link |
01:48:22.520
Does that sort of make sense?
link |
01:48:23.560
I'm just like saying.
link |
01:48:24.400
No, no, everything you're saying makes perfect sense.
link |
01:48:26.640
So the point is if for every comparison between two people,
link |
01:48:30.720
which I'm doing for every two people,
link |
01:48:32.480
I gave one point to each person,
link |
01:48:34.400
your score, everyone's score is the same.
link |
01:48:36.640
It's how many other people there are.
link |
01:48:38.560
Now we only make one change.
link |
01:48:40.200
For each matchup, you give one point only to the winner.
link |
01:48:44.320
So we're awarding half the points.
link |
01:48:47.040
So now the deal is if in the original situation,
link |
01:48:50.240
everyone's score was equal,
link |
01:48:52.120
which is how many other people there are.
link |
01:48:54.840
Now there's only half the number of points to go around.
link |
01:48:58.480
So what ends up happening is that
link |
01:49:01.000
there's always going to be,
link |
01:49:02.360
like the average number of points per person
link |
01:49:04.680
is going to be half of how many other people there are.
link |
01:49:07.120
And somebody is gonna be above average.
link |
01:49:08.560
Somebody is going to be above that.
link |
01:49:09.880
At least average.
link |
01:49:10.720
Yeah, this is this notion of expected value,
link |
01:49:13.200
that if I have a random variable,
link |
01:49:14.520
which has an expected value,
link |
01:49:16.200
there's going to be some possibility
link |
01:49:17.800
in the probability space
link |
01:49:19.280
where you're at least as big as the expected value.
link |
01:49:21.560
Yeah, when you describe it like that, it's obvious.
link |
01:49:23.680
But when you're first saying in this little circuit
link |
01:49:26.640
that there's going to be one candidate better than half,
link |
01:49:32.040
that's not obvious.
link |
01:49:33.360
Yeah, it's not obvious.
link |
01:49:34.200
It's funny.
link |
01:49:35.040
It's not obvious.
link |
01:49:35.880
Math, this is nice.
link |
01:49:37.160
Okay, so you have this,
link |
01:49:38.600
but ultimately you're trying to with a voting tree,
link |
01:49:42.800
I don't know if you're trying this,
link |
01:49:43.880
but to have a circuit that's like, that's small.
link |
01:49:48.360
Well, you'd like it to be small.
link |
01:49:49.200
That achieves the same kind of,
link |
01:49:53.960
I mean, the smaller it is,
link |
01:49:56.440
if we look at practically speaking,
link |
01:49:59.080
the lower the cost of running the election,
link |
01:50:01.560
of running through, of computing the circuit.
link |
01:50:03.800
That is true.
link |
01:50:04.640
But actually at this point,
link |
01:50:05.800
the reason the question was interesting
link |
01:50:08.600
is because there was no good guarantee
link |
01:50:12.760
that the winner of that circuit
link |
01:50:15.440
would have like have beaten a lot of people.
link |
01:50:18.440
Let me give an example.
link |
01:50:19.680
The best known circuit,
link |
01:50:20.800
when we started thinking about this,
link |
01:50:22.400
was the circuit called candidate one
link |
01:50:24.760
plays against candidate two,
link |
01:50:26.400
candidate three plays against four,
link |
01:50:28.640
and then the winners play against each other.
link |
01:50:30.480
And then by the way, five plays against six,
link |
01:50:32.560
seven against eight, the winners play against each other.
link |
01:50:34.640
You understand, it's like a giant binary tree.
link |
01:50:36.600
Yeah, it's a binary, like a balanced binary tree.
link |
01:50:39.400
It's a balanced binary tree.
link |
01:50:40.720
One, two, three, four, up to 1,024,
link |
01:50:42.680
everyone going up to find the winner.
link |
01:50:44.520
Well, you know what?
link |
01:50:45.480
There's a system in the world
link |
01:50:47.400
where it could just be
link |
01:50:49.720
that there's a candidate called number one,
link |
01:50:52.280
that just beats like 10 other people,
link |
01:50:56.200
just the 10 that they need to be on their way up
link |
01:50:59.760
and they lose to everyone else.
link |
01:51:02.240
But somehow they would get all the way up.
link |
01:51:04.720
My point is it is possible to outsmart that circuit
link |
01:51:11.040
in one weird way of the world,
link |
01:51:13.720
which makes that circuit a bad one
link |
01:51:15.320
because you want to say,
link |
01:51:16.160
I will use this circuit for all elections.
link |
01:51:18.920
And you might have a system of inputs that go in there
link |
01:51:22.480
where the winner only beat 10 other people,
link |
01:51:24.800
which is the people they had to beat on the way up.
link |
01:51:26.720
So you want to have a circuit where there's as many,
link |
01:51:29.720
like the final result is as strong as possible.
link |
01:51:33.160
Yes.
link |
01:51:34.200
And so what ideas do you have for that?
link |
01:51:37.440
So we actually only managed to improve it
link |
01:51:40.480
to square root of N.
link |
01:51:41.800
So if N is number of vertices,
link |
01:51:43.680
N over two would be the ideal.
link |
01:51:46.080
We got it to square root of N.
link |
01:51:48.320
Versus log base two.
link |
01:51:50.120
Yeah, exactly.
link |
01:51:51.080
Yeah.
link |
01:51:52.400
Which is...
link |
01:51:53.440
Well, that is halfway.
link |
01:51:54.280
It could be a lot.
link |
01:51:55.920
Yeah.
link |
01:51:56.760
Could be a big improvement.
link |
01:51:57.760
So that's a, okay, cool.
link |
01:51:59.000
Is there something you can say with words
link |
01:52:01.600
about what kind of circuit, what that looks like?
link |
01:52:04.600
I can give an idea of one of the tools inside,
link |
01:52:08.040
but the actual execution ends up being more complicated.
link |
01:52:10.560
But one of the widgets inside this
link |
01:52:12.800
is building a system where you have like a candidate
link |
01:52:16.960
who plays, like one part of the whole huge, huge tree
link |
01:52:20.720
is that that same candidate, let's call them seven.
link |
01:52:23.240
Seven plays against somebody,
link |
01:52:25.520
let's make up some numbers.
link |
01:52:26.680
Let's call the others like letters.
link |
01:52:27.920
So seven plays against A.
link |
01:52:30.720
Seven's also gonna play against B separately.
link |
01:52:33.680
And the winners of each of those will play each other.
link |
01:52:36.560
By the way, seven's also gonna play C.
link |
01:52:38.440
Seven's gonna play D.
link |
01:52:39.680
And the winners are gonna play each other.
link |
01:52:41.040
And the winners are gonna play each other.
link |
01:52:42.520
We call this seven against all.
link |
01:52:44.960
Well, seven against like everyone from a bunch of.
link |
01:52:47.720
Got it.
link |
01:52:48.560
So there's some nice overlap between the matchups
link |
01:52:50.840
that somehow has a nice feature to it.
link |
01:52:53.000
Yes, and I can tell you the nice feature
link |
01:52:54.200
because if at the base of this giant tree,
link |
01:52:56.480
at the base of this giant circuit,
link |
01:52:57.880
like this is a widget.
link |
01:52:58.720
We build the things out of widgets.
link |
01:52:59.920
So I'm just describing one widget.
link |
01:53:01.320
But in the base of this widget,
link |
01:53:03.400
you have lots of things which are seven against someone,
link |
01:53:05.560
seven against someone, seven against someone.
link |
01:53:07.360
In fact, every matchup at the bottom
link |
01:53:09.800
is seven against someone.
link |
01:53:11.520
What that means is
link |
01:53:12.840
if seven actually beat everyone they were matched up against,
link |
01:53:16.840
well, seven would rise to the top.
link |
01:53:18.640
So one possibility is if you see a seven
link |
01:53:21.240
emerge from the top,
link |
01:53:22.360
you know that seven actually beat everyone
link |
01:53:24.280
they were against.
link |
01:53:25.600
On the other hand, if anyone else is on top,
link |
01:53:28.360
let's call it F.
link |
01:53:29.600
If F is on top, how did F get there?
link |
01:53:31.920
Well, F beat seven on the way at the beginning.
link |
01:53:34.640
So the point is the outcome of this circuit
link |
01:53:37.000
has a certain property.
link |
01:53:38.400
If you see a seven,
link |
01:53:39.640
you know that the seven actually beat a person
link |
01:53:41.680
but the seven actually beat a bazillion people.
link |
01:53:43.960
If you see anyone else,
link |
01:53:45.240
at least you know they beat seven.
link |
01:53:47.040
Yeah, then you can prove that it has a nice property.
link |
01:53:49.880
That's really interesting.
link |
01:53:50.880
Is there something you can say,
link |
01:53:54.120
perhaps going completely outside
link |
01:53:55.760
of what we're talking about,
link |
01:53:56.720
is how we may
link |
01:54:00.720
have mathematical ideas
link |
01:54:03.040
of improving the electoral process?
link |
01:54:06.440
That one, no.
link |
01:54:07.320
No, I can't give you that one.
link |
01:54:09.120
I mean, is there, like, do you ever see it as,
link |
01:54:14.640
do you see as there being a lot of opportunities
link |
01:54:17.400
for improving how we vote?
link |
01:54:20.480
Like from your, I don't know if you saw parallels,
link |
01:54:23.720
but, you know, it seems like if,
link |
01:54:26.560
this actually kind of maps to your sort of COVID work,
link |
01:54:29.680
which is there's a network effect, right?
link |
01:54:32.240
It seems like we should be able to apply similar kind
link |
01:54:34.920
of effects of how we decide other things in our lives.
link |
01:54:39.400
And one of the big decisions we'll make
link |
01:54:42.160
is who represents us in government.
link |
01:54:44.480
Do you ever think about like mathematically
link |
01:54:46.200
about those kinds of systems?
link |
01:54:48.160
I think a little bit about those,
link |
01:54:49.520
because where I went to college,
link |
01:54:51.480
the way we voted for student government
link |
01:54:53.200
was based on this, is it called ranked choice?
link |
01:54:56.120
Where you eliminate the bottom
link |
01:54:58.600
and there was runoff elections.
link |
01:55:00.840
So that was the first time I ever saw that.
link |
01:55:02.840
And I thought that made sense.
link |
01:55:04.480
The only problem is it doesn't seem so easy
link |
01:55:06.920
to get something that makes sense adopted
link |
01:55:08.520
as the new voting system.
link |
01:55:09.760
That's a whole nother, that's not a math solution.
link |
01:55:12.760
That's a, well, it's math in the sense that it's game theory.
link |
01:55:16.040
So you have to come up with incentive,
link |
01:55:17.280
it's mechanism design.
link |
01:55:18.320
You have to figure out how to trick us
link |
01:55:21.040
despite our basic human nature
link |
01:55:24.280
to adopt solutions that are better.
link |
01:55:27.600
That's a whole nother conversation, I think.
link |
01:55:30.520
Can you just, cause it sounded really cool,
link |
01:55:33.240
talk a little bit about stochastic coalescence
link |
01:55:36.120
and you have a paper on showing that,
link |
01:55:39.400
so you could describe what it is,
link |
01:55:40.760
but I guess it's a super linear, super logarithmic time
link |
01:55:44.760
and you came up with some kind of trick
link |
01:55:46.360
that make it faster.
link |
01:55:47.960
Can you just talk about it a little bit?
link |
01:55:49.280
Yeah, so this was something which came up
link |
01:55:51.680
when I was at Microsoft Research for a summer.
link |
01:55:54.160
And I'm putting that context because that shows
link |
01:55:56.680
that it has some practical motivation at some point.
link |
01:56:01.040
Actually, I think it's still.
link |
01:56:01.920
It doesn't need to.
link |
01:56:02.840
It doesn't need to.
link |
01:56:03.680
It can be beautiful and it's all right.
link |
01:56:05.160
Yeah, so the easiest way to describe this is
link |
01:56:07.360
suppose you got like a big crowd of people
link |
01:56:09.920
and everybody knows how many hours of sleep
link |
01:56:12.120
they got last night.
link |
01:56:13.240
And you wanna know how many total hours of sleep
link |
01:56:15.240
were gotten by this big crowd of people.
link |
01:56:17.560
At the beginning, you might say,
link |
01:56:18.760
that sounds like a linear time algorithm
link |
01:56:20.680
of saying, hey, how many hours you got?
link |
01:56:22.680
How many you got?
link |
01:56:23.520
How many you got?
link |
01:56:24.360
Add, add, add.
link |
01:56:25.560
But there's a way to do this
link |
01:56:26.760
if you remember that there are people
link |
01:56:28.520
and they presumably know how to add.
link |
01:56:30.360
You could make a distributed algorithm
link |
01:56:32.160
to make this happen.
link |
01:56:33.400
For example, while we're thinking of these trees,
link |
01:56:35.840
imagine you had 1,024 people.
link |
01:56:38.560
If you could just say, hey, person number one
link |
01:56:40.720
and person number two, you will add your hours of sleep.
link |
01:56:44.920
Person number two will go away
link |
01:56:46.040
and person number one is gonna remember the sum.
link |
01:56:48.360
Person three and four add up
link |
01:56:50.920
and person three takes charge of remembering it.
link |
01:56:53.760
Person four goes away.
link |
01:56:54.840
Now this like person one knows the sum of these two.
link |
01:56:56.960
Person three knows the sum of those two.
link |
01:56:58.160
They talk.
link |
01:56:59.000
You see what I mean?
link |
01:56:59.840
You're going up this tree,
link |
01:57:02.200
same tree that we talked about earlier.
link |
01:57:03.600
Built up a tree from the bottom up.
link |
01:57:05.920
Yeah, build up a tree from the bottom up.
link |
01:57:07.560
And the beautiful thing is
link |
01:57:09.160
since everyone's doing stuff in parallel,
link |
01:57:11.360
the amount of time it takes to get the total sum
link |
01:57:14.560
is actually just the number of layers in the tree,
link |
01:57:17.360
which is 10.
link |
01:57:18.960
So now that's logarithmic time
link |
01:57:20.360
to add up the number of hours that people slept today.
link |
01:57:23.720
Sounds fantastic.
link |
01:57:25.240
There's only one problem.
link |
01:57:26.400
How do you decide who's person number one
link |
01:57:27.960
and person number two?
link |
01:57:29.720
Yes.
link |
01:57:30.560
So if, for example, you just went out into the downtown
link |
01:57:32.680
and said, hey, get these thousand people, go.
link |
01:57:34.760
Well, if you're gonna go and say,
link |
01:57:35.960
and by the way, you're one and you're two and you're three,
link |
01:57:37.720
that's linear time.
link |
01:57:38.760
Yes.
link |
01:57:39.640
That's cheating.
link |
01:57:40.480
So now the question is how to do this
link |
01:57:41.800
in a distributed way.
link |
01:57:43.000
And there were some people who proposed
link |
01:57:44.840
a very elegant algorithm and they wanted to analyze it.
link |
01:57:48.880
So I came in onto the analyze side,
link |
01:57:50.680
but the elegant algorithm was like this.
link |
01:57:52.800
It was like, well, we don't actually know
link |
01:57:55.760
what this big tree is.
link |
01:57:57.640
There isn't any big tree.
link |
01:57:58.880
So what's gonna happen is first,
link |
01:58:01.040
everyone is going to decide right now.
link |
01:58:04.280
Oh, one important thing.
link |
01:58:05.680
Everyone is going to,
link |
01:58:07.000
at the very beginning of the whole game,
link |
01:58:09.960
they will have delegated responsibility to themselves
link |
01:58:13.400
as the one who knows the sum so far.
link |
01:58:16.400
So the point is there's gonna be,
link |
01:58:18.920
people are all gonna have like a pointer which says,
link |
01:58:22.120
you are the one who knows my,
link |
01:58:24.440
you've taken care of my ticket, my number.
link |
01:58:26.680
Yeah.
link |
01:58:27.520
You're the representative for this particular piece
link |
01:58:30.560
of knowledge.
link |
01:58:31.400
And at the very beginning, you're your own representative.
link |
01:58:33.680
The thing has to start simple, right?
link |
01:58:35.120
So at the beginning, you're your own representative.
link |
01:58:36.440
You're pointing to yourself, got it.
link |
01:58:38.040
Yup, yup.
link |
01:58:38.880
And now the way this works is that at every time step,
link |
01:58:41.560
someone blares a ding dong on the town clock or whatever.
link |
01:58:45.840
And each person flips a coin themselves to decide,
link |
01:58:48.680
am I going to hunt for somebody to give my number to
link |
01:58:53.600
and let them represent me?
link |
01:58:55.120
Or am I going to sit here and wait for someone to come?
link |
01:58:58.920
Okay.
link |
01:58:59.960
Okay.
link |
01:59:00.800
Well, they flipped their coin.
link |
01:59:02.600
Some of the people start asking other people saying,
link |
01:59:04.840
hey, I would like you to be my representative.
link |
01:59:08.560
Here is my number.
link |
01:59:10.200
But the problem is that there's limited bandwidth
link |
01:59:12.040
of the people who are getting asked.
link |
01:59:13.240
It's like, you can't get,
link |
01:59:14.440
you can't go out to prom with five people.
link |
01:59:16.680
But this is not what we're doing.
link |
01:59:17.840
We're adding numbers, okay?
link |
01:59:19.160
But you can only add one number.
link |
01:59:20.720
So the person who has suddenly gotten asked
link |
01:59:22.600
by all these people,
link |
01:59:23.920
well, they'll have to decide who they're going
link |
01:59:25.720
to take it from.
link |
01:59:27.280
And they randomly just choose one.
link |
01:59:29.440
When they randomly choose one,
link |
01:59:30.720
all the others are rejected
link |
01:59:31.880
and they don't get to delegate anything in that round.
link |
01:59:34.920
But now if this person has absorbed this one who said,
link |
01:59:38.360
okay, here, you take charge of my number.
link |
01:59:40.680
This person now updates their pointer.
link |
01:59:42.600
You're in charge.
link |
01:59:44.840
And this person adds the two numbers.
link |
01:59:47.680
That was the first round.
link |
01:59:50.000
In the next round, when they do the coin flipping,
link |
01:59:52.840
this person doesn't flip anymore
link |
01:59:54.280
because they're just delegating.
link |
01:59:56.120
It's that anyone who has the pointers themselves,
link |
01:59:59.000
that's like a person who is in charge
link |
02:00:01.600
of some number of informations,
link |
02:00:03.320
they flip the coin to decide,
link |
02:00:04.680
should I find other people who are agents?
link |
02:00:08.240
Or should I wait for people to ask me?
link |
02:00:10.040
Yes.
link |
02:00:10.880
Brilliant.
link |
02:00:11.720
This is somebody else's idea.
link |
02:00:12.840
And then now the idea is, okay,
link |
02:00:14.320
if you just keep doing this process,
link |
02:00:15.720
what ends up happening?
link |
02:00:16.920
Oh yeah, and also by the way,
link |
02:00:18.680
if you decide that you want to go reach out
link |
02:00:20.320
to other people, here's the catch.
link |
02:00:23.120
When you're one of these agents saying,
link |
02:00:24.680
okay, I'm going to go look for someone.
link |
02:00:27.400
You have no idea who in this crowd is an agent
link |
02:00:30.760
or somebody who delegated it to someone else.
link |
02:00:33.400
You just pick a random person.
link |
02:00:35.640
When you pick the random person,
link |
02:00:37.080
if it lands on someone and the person says,
link |
02:00:38.880
oh, I actually delegated it to someone,
link |
02:00:41.840
then you follow the point.
link |
02:00:43.240
You walk up the delegation chain.
link |
02:00:45.280
Walk up the delegation chain.
link |
02:00:46.120
And you can do like path compression in the algorithm
link |
02:00:49.080
to make it so you don't consistently
link |
02:00:50.560
do lots of walking up.
link |
02:00:52.040
But the bottom line is that what ends up happening
link |
02:00:54.680
is that you end up reaching out.
link |
02:00:57.240
Whenever you're one of the ones reaching out,
link |
02:00:59.080
you can think of it as each agent is responsible
link |
02:01:01.840
for some number of people.
link |
02:01:03.320
It's almost like they're the leader of a bunch.
link |
02:01:05.480
As the process is evolving, you have these lumps.
link |
02:01:09.920
Each lump has an agent.
link |
02:01:11.680
And when the agent reaches out,
link |
02:01:13.360
they reach out to another lump
link |
02:01:15.680
where the probability of them hitting that lump
link |
02:01:18.160
is proportional to the size of the lump.
link |
02:01:21.880
That is the one funny thing about this process.
link |
02:01:25.720
This is not that they can reach out
link |
02:01:27.360
to a uniformly random lump
link |
02:01:29.280
where every lump has the same chance
link |
02:01:30.920
of getting reached out to.
link |
02:01:32.440
The bigger the lump is,
link |
02:01:34.520
the more likely it is that you end up reaching that lump.
link |
02:01:38.760
Which is a problem?
link |
02:01:40.160
Let me explain why that's a problem.
link |
02:01:41.600
Because you see, you're hoping
link |
02:01:43.280
that this has a small number of steps,
link |
02:01:45.400
but here's a bad situation that could happen.
link |
02:01:47.680
Imagine if you had like,
link |
02:01:50.000
there are n people that you're adding up.
link |
02:01:52.040
Imagine that you have exactly square root of n lumps left,
link |
02:01:57.040
of which almost all of them are just one person
link |
02:02:01.440
who's still their own boss, their own manager.
link |
02:02:04.280
Except one giant one.
link |
02:02:06.080
Now what's gonna happen?
link |
02:02:06.920
It's gonna be a huge bottleneck
link |
02:02:08.360
because every round the giant one
link |
02:02:09.760
can only absorb one of the others.
link |
02:02:11.880
And now you suddenly have time
link |
02:02:13.560
which is about square root of n.
link |
02:02:15.680
The square root of n is chosen
link |
02:02:16.840
because that is one where the lumps are such
link |
02:02:20.280
that you really are limited by this large one
link |
02:02:23.560
slowly sucking up the rest of them.
link |
02:02:26.000
So the heart of the question became,
link |
02:02:28.160
well, but is that just so unusual
link |
02:02:30.040
that it doesn't usually happen?
link |
02:02:32.760
Because remember you start with everyone
link |
02:02:34.680
just being independent.
link |
02:02:36.080
It's like a lot of lumps of size one.
link |
02:02:37.640
How naturally do the big lumps emerge?
link |
02:02:39.640
Yes.
link |
02:02:40.480
And so what that heart of the proof was,
link |
02:02:42.120
was showing that that was a joint work with Eyal Lubezki.
link |
02:02:45.120
That one was showing that actually in that thing
link |
02:02:49.040
the lumps do kind of get out of whack.
link |
02:02:50.920
And so it's not the purely logarithmic number of steps.
link |
02:02:54.720
But if you make one very slight change,
link |
02:02:56.720
which is if you are one of the agents
link |
02:03:00.400
and you have just been propositioned,
link |
02:03:02.400
possibly relayed along by a couple of different people.
link |
02:03:05.320
If you just say, don't take a random one,
link |
02:03:07.600
but accept the smallest lump.
link |
02:03:12.600
That actually does enough to even the whole economy.
link |
02:03:14.400
Distributes the lump size.
link |
02:03:16.320
I mean, yeah, it's fascinating how
link |
02:03:17.800
with the distributed algorithms,
link |
02:03:19.000
a little adjustment can make all the difference
link |
02:03:21.520
in the world.
link |
02:03:22.440
Yeah.
link |
02:03:23.280
Actually, by the way, this does,
link |
02:03:25.320
back to our voting conversation,
link |
02:03:26.960
this makes me think of like,
link |
02:03:29.440
these networking systems are so fascinating to study.
link |
02:03:32.240
They immediately spring to mind ideas
link |
02:03:35.120
of how to have representation.
link |
02:03:37.680
Like maybe as opposed to me voting for a president,
link |
02:03:42.240
I want to vote for like,
link |
02:03:45.600
for you, Paul, to represent me,
link |
02:03:48.400
maybe on a particular issue.
link |
02:03:50.600
And then you will delegate that further.
link |
02:03:52.560
And then we naturally construct those kinds of networks
link |
02:03:55.120
because that feels like I can have a good conversation
link |
02:03:58.880
with you and figure out that you know what you're doing
link |
02:04:00.760
and I can delegate it to you.
link |
02:04:01.800
And in that way, construct a representative government,
link |
02:04:05.560
a representative decision maker.
link |
02:04:08.440
That feels really nice as opposed to like us,
link |
02:04:12.480
like a tree of height one or something,
link |
02:04:14.600
where it's like everybody's just,
link |
02:04:18.400
it feels like there's a lot of room for layers
link |
02:04:20.440
of representation to form organically from the bottom up.
link |
02:04:23.840
I wonder if there are systems like that.
link |
02:04:25.400
This is the cool thing about the internet
link |
02:04:27.080
and the digital space where we're so well connected,
link |
02:04:29.560
just like with the Novid app to distribute information
link |
02:04:34.080
about the spread of the disease.
link |
02:04:37.000
We can in the same way, in a distributed sense,
link |
02:04:39.280
form anything like any kind of knowledge bases
link |
02:04:44.880
that are formed in a decentralized way
link |
02:04:48.760
and in a hierarchical way,
link |
02:04:51.480
as opposed to sort of old way
link |
02:04:54.320
where there is no mechanism for large scale,
link |
02:04:56.840
fast distributed transactional information.
link |
02:05:01.760
This is really interesting.
link |
02:05:02.600
This is where almost like network graph theory,
link |
02:05:06.800
becomes practical.
link |
02:05:09.280
Most of that exciting work was done in the 20th century,
link |
02:05:11.840
but most of the application will be in the 21st,
link |
02:05:14.080
which is cool to think about.
link |
02:05:15.960
Let me ask the most ridiculous question.
link |
02:05:17.720
You think P equals NP?
link |
02:05:19.880
Wow.
link |
02:05:21.240
I don't know.
link |
02:05:22.360
I mean, I would say,
link |
02:05:26.640
I know there are enough people who have very strong interest
link |
02:05:29.520
in trying to show that it is.
link |
02:05:32.680
I'm talking about government agencies.
link |
02:05:34.760
For security purposes.
link |
02:05:38.520
For security purposes.
link |
02:05:39.360
And most computer scientists,
link |
02:05:40.560
we should say believe that P equals NP.
link |
02:05:43.800
My question almost like,
link |
02:05:45.280
this is back to our aliens discussion.
link |
02:05:47.000
You want to think outside the box,
link |
02:05:48.440
the low probability event,
link |
02:05:51.800
what is the world,
link |
02:05:54.040
what kind of discoveries would lead us to prove
link |
02:05:58.640
that P does not equal to NP?
link |
02:06:01.560
Like there could be giant misunderstandings
link |
02:06:05.480
or gaps in our knowledge about computer science,
link |
02:06:08.080
about theoretical computer science, about computation,
link |
02:06:11.040
which allow us to think like flatten all problems.
link |
02:06:14.800
Yeah, so I don't know the answer to this question.
link |
02:06:17.080
I think it's very interesting, but I actually,
link |
02:06:19.440
I know, let's put it this way.
link |
02:06:21.280
By being at Carnegie Mellon
link |
02:06:22.520
and being around the theoretical computer scientists,
link |
02:06:24.880
I know enough about what I don't know to say.
link |
02:06:27.920
To be humble.
link |
02:06:28.760
I'm the wrong person to answer this question.
link |
02:06:32.320
It's a great one.
link |
02:06:33.240
Well, Scott Aaronson, who's now here at UT Austin,
link |
02:06:35.920
he used to be at MIT,
link |
02:06:37.800
puts the probability of P not equals to NP at 3%.
link |
02:06:45.920
I always love it when you ask,
link |
02:06:48.040
it's very rare in science and academics
link |
02:06:51.040
because most folks are humble
link |
02:06:54.680
in the face of the mystery,
link |
02:06:56.800
the uncertainty of everything around us.
link |
02:06:59.400
To have both the humor and the guts to say like,
link |
02:07:03.360
what are the chance that there's aliens in our galaxy,
link |
02:07:07.160
intelligent alien civilizations?
link |
02:07:09.440
As opposed to saying, I don't know, it could be zero.
link |
02:07:12.280
It could be, depending on the fact, you're saying it's 2.5%.
link |
02:07:15.920
There's something very pleasant about just having,
link |
02:07:20.280
it's the number thing.
link |
02:07:24.400
It's powered to the number.
link |
02:07:25.480
It's just like 42.
link |
02:07:26.440
It's like, why 42?
link |
02:07:27.280
I don't know, but it's a powerful number.
link |
02:07:29.640
And then everything,
link |
02:07:30.640
this is the power of human psychology
link |
02:07:32.600
is once you have the number 42,
link |
02:07:36.560
it's not that the number has meaning,
link |
02:07:39.240
but because it's placed in a book with humor around it,
link |
02:07:43.960
it has the meme effect of actually creating reality.
link |
02:07:49.400
I mean, you could say that 42 has a strong contribution
link |
02:07:53.520
of helping us colonize Mars
link |
02:07:55.920
because it created,
link |
02:07:57.720
it gave the whatever existential crisis to many of us,
link |
02:08:00.480
including Elon Musk when he was young,
link |
02:08:03.160
reading a book like that.
link |
02:08:04.280
And then now 42 is now part of his humor
link |
02:08:07.040
that he doesn't shut up about,
link |
02:08:08.840
it's constantly joking about.
link |
02:08:09.680
And that humor is spreading through our minds
link |
02:08:12.200
and somehow this like silly number just had an effect.
link |
02:08:15.160
In that same way, after Scott told me like the 3% chance,
link |
02:08:19.360
it's stuck in my head.
link |
02:08:20.680
And I think it's been having a ripple effect
link |
02:08:22.720
in everybody else.
link |
02:08:23.720
The believing that P is not equal to NP,
link |
02:08:29.120
Scott almost as a joke saying it's 3%
link |
02:08:32.200
is actually motivating a large number of researchers
link |
02:08:34.920
to work on it.
link |
02:08:35.760
3% is high.
link |
02:08:37.080
It's very high.
link |
02:08:37.920
Because for the potential impact that that would have.
link |
02:08:39.920
But then 3% is not that high because it's only,
link |
02:08:44.280
you know, like we're not very good.
link |
02:08:46.480
I feel like humans are only able to really think about
link |
02:08:48.840
like 1%, 50%.
link |
02:08:51.280
And we kind of, I think a lot of people around 3%
link |
02:08:55.400
up to 50% like in our minds.
link |
02:08:58.800
Like 3% at this point.
link |
02:09:00.640
It could happen.
link |
02:09:01.480
It could happen.
link |
02:09:02.480
And it could happen and it's like, yeah.
link |
02:09:04.760
Like half the time it'll probably happen.
link |
02:09:07.040
So we're not very good at that.
link |
02:09:08.200
That's the other thing with the pandemic
link |
02:09:10.240
is we're not the exponential growth
link |
02:09:13.560
that we also talked about offline
link |
02:09:15.800
is something that we can't quite intuit.
link |
02:09:20.200
And that's something we probably should
link |
02:09:22.680
if we're to predict the future, to anticipate the future
link |
02:09:25.240
and to understand how to create technologies
link |
02:09:27.920
that let us sort of control the future.
link |
02:09:32.320
Can I ask you for some recommendations
link |
02:09:35.040
maybe for books or movies in your life?
link |
02:09:39.120
Long ago when you were baby Po or today
link |
02:09:45.320
that you found insightful or you learned a lot from
link |
02:09:50.160
what you would recommend to others.
link |
02:09:52.080
Yeah.
link |
02:09:52.920
So I think I don't necessarily have an exact name
link |
02:09:55.760
of these old things, but I was generally inspired
link |
02:09:58.600
by stories, true or fictional of campaigns.
link |
02:10:05.920
For example, like the Lord of the rings, that's a campaign.
link |
02:10:09.080
But the thing that always inspired me was
link |
02:10:12.480
it could be possible for somebody who's crazy enough
link |
02:10:16.480
to go up against adversity after adversity,
link |
02:10:18.680
and it succeeds.
link |
02:10:20.240
I mean, those are false, those are fictitious.
link |
02:10:23.040
But I also spent a lot of time, I guess, reading about,
link |
02:10:25.480
I don't know, I was interested somehow
link |
02:10:26.600
in like World War II history for whatever reason.
link |
02:10:29.520
That's a campaign which is much more brutal.
link |
02:10:31.440
But nevertheless, the idea of difficulty, strategy,
link |
02:10:37.560
fighting even when things,
link |
02:10:39.480
in that case it was really fighting,
link |
02:10:40.800
but just pushing on even when things are difficult.
link |
02:10:43.600
I guess these are the kinds of general stories
link |
02:10:46.840
that made me, I guess, want to work on things
link |
02:10:50.880
that would be hard and where it could be a campaign.
link |
02:10:54.920
It could be that you work on something for a year,
link |
02:10:57.520
multiple years, because that was the point, I guess.
link |
02:11:02.200
Yeah, it starts with a single person.
link |
02:11:04.160
That's the interesting thing.
link |
02:11:05.680
I've obviously been, don't shut up about it recently
link |
02:11:08.880
about World War II, especially on the Hitler side
link |
02:11:11.400
and the Stalin side.
link |
02:11:12.640
Some of that has really affected my own family.
link |
02:11:15.600
The roots of my family very much.
link |
02:11:18.760
But it's interesting to think that it was just an idea
link |
02:11:24.840
and one person decided to do stuff
link |
02:11:26.920
and it just builds and builds and builds.
link |
02:11:29.520
And you can truly have an impact on the world,
link |
02:11:31.840
both horrendous and exceptionally positive and inspiring.
link |
02:11:36.840
So yeah, it's like it's a agency of us individuals.
link |
02:11:47.840
Sometimes we think we're just reacting to the world,
link |
02:11:49.920
but we have the full power to actually change the world.
link |
02:11:53.080
Is there advice you can give to young folks?
link |
02:11:56.040
We talked, we gave a bunch of advice
link |
02:11:58.000
on middle school, high school mathematics.
link |
02:12:00.600
Is there more general advice you would give
link |
02:12:02.480
about how to succeed in life,
link |
02:12:04.800
how to learn for high school students,
link |
02:12:07.400
for college students, career or life in general?
link |
02:12:10.600
So I think the first one would be
link |
02:12:12.280
to make sure that you're learning to invent
link |
02:12:14.680
and to make sure you're not just learning how to mimic.
link |
02:12:19.080
Because a lot of times you learn how to do X
link |
02:12:21.880
by watching somebody do X and then repeating X
link |
02:12:24.280
many times with different inputs.
link |
02:12:26.360
I've just been very generic in explaining this.
link |
02:12:28.520
But I guess this is just my own attitude towards the world.
link |
02:12:31.840
I didn't like ever following anyone's directions exactly.
link |
02:12:34.920
Even if you told me this is the way to do your homework
link |
02:12:37.680
is to write in pencil, I would say,
link |
02:12:39.400
but I think pen is nice, let's try, right?
link |
02:12:42.880
So I've been that kind of a funny person.
link |
02:12:45.640
But I do encourage that if you can learn how to invent
link |
02:12:50.720
as your core skill, then you can do a lot.
link |
02:12:52.760
But then the second piece that comes with that
link |
02:12:54.320
is something I learned from my PhD advisor,
link |
02:12:57.400
which was, well, make sure that what you're working on
link |
02:13:01.040
is big enough.
link |
02:13:02.280
And so in that sense, I usually advise to people
link |
02:13:04.760
once they have learned how to invent,
link |
02:13:07.600
ideally don't just try to settle for something comfortable,
link |
02:13:11.520
try to see if you can aim for something which is hard,
link |
02:13:15.000
which might involve a campaign, which might be important,
link |
02:13:18.000
which might make a difference.
link |
02:13:20.000
And it's more of, I guess, rather than worrying
link |
02:13:23.920
what if you didn't achieve that,
link |
02:13:27.040
there's also the regret of what if I didn't try?
link |
02:13:30.320
See, that's how I operate.
link |
02:13:31.600
I don't operate based on did I succeed or fail?
link |
02:13:33.720
It was hard anyway.
link |
02:13:34.640
If I did this novid thing and the whole thing failed,
link |
02:13:36.640
would I feel terrible?
link |
02:13:37.640
No, it's a very hard problem.
link |
02:13:39.480
But would I have had the regret of not jumping in?
link |
02:13:42.640
Yes.
link |
02:13:44.080
So it's that different mentality of don't worry
link |
02:13:46.000
about the failing part as much of the,
link |
02:13:48.680
make sure you give yourself the shot
link |
02:13:50.760
at those potentially unbounded opportunities.
link |
02:13:55.160
You almost make it sound like there's a meaning to it all.
link |
02:13:58.320
Let me ask the big ridiculous question.
link |
02:13:59.960
What do you think is the meaning of life?
link |
02:14:01.840
Or maybe the easier version of that
link |
02:14:04.200
is what brings your life joy?
link |
02:14:06.120
So I'll just answer that one personally.
link |
02:14:07.880
For me, I'm a little bit weird.
link |
02:14:10.080
I sort of, I guess you can tell by now.
link |
02:14:13.520
See the pen and pencil discussion from earlier, yes.
link |
02:14:15.960
Yeah, yeah.
link |
02:14:16.920
So, I mean, my thing is, I guess I personally
link |
02:14:20.360
just wanted to maximize a certain score,
link |
02:14:24.360
which was for how many person years
link |
02:14:28.160
after I'm no longer here anymore,
link |
02:14:31.000
did what I do mattered?
link |
02:14:33.400
Yeah.
link |
02:14:34.240
And it didn't matter if it's necessarily attributed to me.
link |
02:14:36.480
It's just like, did it matter?
link |
02:14:38.600
And so that's what I wanted.
link |
02:14:41.800
I guess that is very inspired by how scientists work.
link |
02:14:45.620
It's like, why do we keep talking about Newton?
link |
02:14:47.760
It's because Newton discovered some interesting things.
link |
02:14:50.840
And so Newton's score is pretty high.
link |
02:14:53.880
It's going to be infinity, right?
link |
02:14:56.240
Well, let's hope it's infinity, but pretty high.
link |
02:14:59.040
Yes, yes.
link |
02:15:00.120
So you're going for, so person years,
link |
02:15:03.780
you're going for like triple digits.
link |
02:15:05.480
You're going for, so like Newton is like four digits,
link |
02:15:08.940
probably like a thousand years or personal lifetimes.
link |
02:15:13.400
How do you like to think, well, what are we?
link |
02:15:15.340
Sorry, I meant people times years.
link |
02:15:17.480
People times.
link |
02:15:18.320
So then it's like, actually his is huge.
link |
02:15:19.980
His is like going to be billions or trillions, trillions.
link |
02:15:23.240
But I guess for me, I actually changed the metric
link |
02:15:27.680
after a while.
link |
02:15:28.520
And the reason is because you may have seen,
link |
02:15:30.080
I found some simple way to solve quadratic equations
link |
02:15:33.640
that is easier than every textbook.
link |
02:15:35.780
So my score might already be not bad,
link |
02:15:39.040
which is why I decided then let's change it
link |
02:15:40.760
into the number of hours in the lifetimes as well.
link |
02:15:44.640
So the way I was doing it before is that
link |
02:15:48.480
if a person was sort of remembering or using
link |
02:15:53.440
or appreciating what I had done
link |
02:15:56.040
for like 10 years of their life.
link |
02:16:00.280
Oh, I see.
link |
02:16:01.120
That would count as 10.
link |
02:16:01.940
I see.
link |
02:16:02.780
So if there was one person who for 10 years remembered
link |
02:16:05.060
or appreciated something I did,
link |
02:16:06.200
that counts as a score of 10 and we add up overall people.
link |
02:16:09.400
And then, and that was with the hypothesis
link |
02:16:13.480
that the score would be very finite in the sense
link |
02:16:16.680
that if I didn't come up with anything
link |
02:16:19.080
that might potentially help a lot of generations
link |
02:16:21.400
in a forever way, then your score will be finite
link |
02:16:23.840
because at some point it's not,
link |
02:16:25.960
people don't remember that you made like nice bottles
link |
02:16:28.760
or something, right?
link |
02:16:30.160
But then after the quadratic equation thing,
link |
02:16:33.040
it was that there's some chance
link |
02:16:34.980
that that actually might make it into textbooks.
link |
02:16:37.720
And if it makes it in textbooks,
link |
02:16:39.080
the chance that there'll be an easier way discovered
link |
02:16:40.960
is actually quite small.
link |
02:16:42.680
So in that case, then the score might get bigger.
link |
02:16:46.220
I was just saying the score might actually already
link |
02:16:48.040
have been achieved in a non trivial way.
link |
02:16:51.240
I see.
link |
02:16:52.080
Because it's fun to think about,
link |
02:16:53.720
cause it could be different.
link |
02:16:54.620
You can achieve a high score by a small number of people
link |
02:16:58.980
using it for most of their lifetime
link |
02:17:01.240
and then generations and generations.
link |
02:17:03.400
Or you can have, if we do dissipate,
link |
02:17:05.960
if we do split colonize, become multi planetary species,
link |
02:17:10.120
you could have that little,
link |
02:17:12.280
a clever way to solve differential equations,
link |
02:17:17.280
spread through like trillions of people
link |
02:17:19.640
as they spread throughout the galaxy.
link |
02:17:21.600
And they would only use it each one,
link |
02:17:24.800
a few hours in their lifetime,
link |
02:17:26.760
but their kids will use it,
link |
02:17:28.280
the kids of kids will use it, it will spread
link |
02:17:30.080
and you'll have that impact in that kind of way.
link |
02:17:33.240
Yes, so that's why I renormalized it
link |
02:17:34.920
because I was like, well, that's kind of dumb
link |
02:17:36.440
because what's the importance of that?
link |
02:17:37.760
That'll save people 15 minutes.
link |
02:17:39.880
But, so what I meant is I didn't want to count that
link |
02:17:42.380
as the main score.
link |
02:17:46.360
Well, I'm gonna have to try to come up
link |
02:17:47.880
with some kind of device that everyone would want to use,
link |
02:17:50.360
maybe to make coffee,
link |
02:17:51.520
cause coffee seems to be the prevalent
link |
02:17:54.720
performance enhancing chemical that everyone uses.
link |
02:17:57.160
So I'll have to think about those kinds of metrics.
link |
02:17:59.880
Yeah, but you see that's just giving an idea
link |
02:18:02.560
of I guess what I found meaningful in general,
link |
02:18:05.080
like whether or not it's like,
link |
02:18:06.200
whether or not that quadratic thing is important or not.
link |
02:18:08.340
The general idea was I wanted to do things
link |
02:18:10.740
that would outlast me.
link |
02:18:12.080
And that was what inspired me
link |
02:18:13.400
and that's just how I choose what problems to work on.
link |
02:18:15.680
And that's a kind of immortality is ideas
link |
02:18:18.520
that you've invented living on long after you
link |
02:18:23.060
in the minds of others.
link |
02:18:24.920
And humans are ultimately not,
link |
02:18:27.160
are like meat vehicles that carry ideas for brief
link |
02:18:32.400
for just a few years may not be the important thing.
link |
02:18:34.880
It might be the ideas that we carry with us
link |
02:18:37.280
and invent new ones.
link |
02:18:38.680
Like we get a bunch of baby ideas in our head.
link |
02:18:41.640
We borrow them from others
link |
02:18:43.120
and then maybe we invent a new one
link |
02:18:45.060
and that new one might have a life of its own.
link |
02:18:47.920
And it's fun to think about that idea
link |
02:18:50.880
of living for many centuries to come
link |
02:18:53.440
unless we destroy ourselves.
link |
02:18:54.960
But maybe AI will borrow it
link |
02:18:56.960
and we'll remember Po as like that one human
link |
02:19:00.160
that helped us out before we of course killed him
link |
02:19:04.680
and the rest of human civilization.
link |
02:19:06.760
On that note, Po, this is a huge honor.
link |
02:19:09.840
You're one of the great educators
link |
02:19:12.880
I've ever gotten a chance to interact with.
link |
02:19:15.260
So it's truly an honor that you would talk with me today.
link |
02:19:18.520
It means especially a lot that you would travel a lot
link |
02:19:21.160
to Austin to talk to me.
link |
02:19:22.480
It really means a lot.
link |
02:19:23.480
So thank you so much.
link |
02:19:25.160
Keep on inspiring.
link |
02:19:26.440
And I'm one of your many, many students.
link |
02:19:30.320
Thank you so much for talking today.
link |
02:19:32.140
Thank you, thank you.
link |
02:19:32.980
It's actually a real honor for me to talk to you
link |
02:19:34.640
and to get this chance to have this really
link |
02:19:37.120
intellectual conversation through all of these topics.
link |
02:19:39.800
Thanks, Po.
link |
02:19:41.440
Thanks for listening to this conversation with Po Chenlo
link |
02:19:44.260
and thank you to Jordan Harmer, the show,
link |
02:19:47.040
Onnit, BetterHelp, AidSleep and Element.
link |
02:19:51.400
Check them out in the description to support this podcast.
link |
02:19:54.480
And now let me leave you with some words from Isaac Newton.
link |
02:19:58.260
I can calculate the motion of heavenly bodies
link |
02:20:01.280
but not the madness of people.
link |
02:20:03.700
Thank you for listening and hope to see you next time.