back to index## Po-Shen Loh: Mathematics, Math Olympiad, Combinatorics & Contact Tracing | Lex Fridman Podcast #183

link |

The following is a conversation with Po Shen Lou,

link |

a professor of mathematics at Carnegie Mellon University,

link |

national coach of the USA International Math Olympia team,

link |

and founder of XP that does online education

link |

of basic math and science.

link |

He's also the founder of Novid,

link |

an app that takes a really interesting approach

link |

to contact tracing,

link |

making sure you stay completely anonymous

link |

and it gives you statistical information about COVID cases

link |

in your physical network of interactions.

link |

So you can maintain privacy, very important,

link |

and make informed decisions.

link |

we desperately needed solutions like this in early 2020.

link |

And unfortunately, I think,

link |

we will again need it for the next pandemic.

link |

To me, solutions that require large scale,

link |

distributed coordination of human beings

link |

need ideas that emphasize freedom and knowledge.

link |

Quick mention of our sponsors,

link |

Jordan Harbinger Show, Onnit, BetterHelp,

link |

Aidsleep, and Element.

link |

Check them out in the description to support this podcast.

link |

As a side note, let me say that Po and I

link |

filmed a few short videos

link |

about simple, beautiful math concepts

link |

that I will release soon.

link |

It was really fun.

link |

I really enjoyed Po sharing his passion for math with me

link |

I'm hoping to do a few more short videos

link |

in the coming months that are educational in nature

link |

on AI, robotics, math, science, philosophy,

link |

or if all else fails,

link |

just fun snippets into my life on music, books, martial arts,

link |

and other random things,

link |

if that's of interest to anyone at all.

link |

This is the Lex Friedman Podcast,

link |

and here's my conversation with Po Shenlow.

link |

You know, you mentioned you really enjoy flying

link |

and experiencing different people in different places.

link |

There's something about flying for me,

link |

I don't know if you have the same experience,

link |

that every time I get on an airplane,

link |

it's incredible to me that human beings

link |

have actually been able to achieve this.

link |

And when I look at like what's happening now

link |

with humans traveling out into space,

link |

I see it as all the same thing.

link |

It's incredible that humans are able to get into a box

link |

and fly in the air and safely and land

link |

in the same, it seems like,

link |

and everybody's taking it for granted.

link |

So when I observe them, it's quite fascinating

link |

because I see that cleanly mapping to the world

link |

where we're now in rockets and traveling to the moon,

link |

traveling to Mars, and at the same kind of way,

link |

I can already see the future

link |

where we will all take it for granted.

link |

So I don't know if you have,

link |

you personally, when you fly,

link |

have the same kind of magical experience

link |

of like how the heck did humans actually accomplish this?

link |

So I do, especially when there's turbulence,

link |

which is like on the way here, there was turbulence

link |

and the plane jiggled, even the flight attendant

link |

had to hold onto the side.

link |

And I was just thinking to myself,

link |

it's amazing that this happens all the time

link |

and the wings don't fall off,

link |

given how many planes are flying.

link |

But then I often think about it and I'm like,

link |

a long time ago, I think people didn't trust elevators

link |

in a 40 story building in New York City.

link |

And now we just take it completely for granted

link |

that you can step into this shaft,

link |

which is 40 floors up and down, and it will just not fail.

link |

Yeah, again, I'm the same way with elevators,

link |

but also buildings, when I'll stand on the 40th floor

link |

and wonder how the heck are we not falling right now?

link |

Like how amazing it is with the high winds,

link |

like structurally, just the earthquakes and the vibrations,

link |

I mean, natural vibrations in the ground.

link |

Like how is this, how are all of these,

link |

you go to like New York City, all of these buildings standing.

link |

I mean, to me, one of the most beautiful things,

link |

actually mathematically too, is bridges.

link |

I used to build bridges in high school from like toothpicks,

link |

just like out of the pure joy of like physics,

link |

making some structure really strong.

link |

Understanding like from a civil engineering perspective,

link |

what kind of structure will be stronger

link |

than another kind of structure, like suspension bridges.

link |

And then you see that at scale,

link |

humans being able to span a body of water

link |

with a giant bridge.

link |

And it's, I don't know, it's so humbling.

link |

It makes you realize how dependent we are on each other.

link |

Sort of, I talk about love a lot,

link |

but there's a certain element in which we little ants

link |

have just a small amount of knowledge

link |

about our particular thing.

link |

And then we're depending on a network of knowledge

link |

that other experts hold.

link |

And then most of our lives,

link |

most of the quality of life we have

link |

has to do with the richness of that network of knowledge,

link |

of that collaboration,

link |

and then sort of the ability to build on top of it,

link |

levels of abstractions.

link |

You start from like bits in a computer,

link |

then you can have assembly, then you can have C++,

link |

or you have an operating system,

link |

then you can have C++ and Python, finally,

link |

some machine learning on top.

link |

All of these are abstractions.

link |

And eventually we'll have AI that runs all of us humans.

link |

But anyway, but speaking of abstractions and programming,

link |

in high school, you wrote some impressive games

link |

I got a chance to, in browser somehow,

link |

it's magic, I got a chance to play them.

link |

Alien Attack 1, 2, 3, and 4.

link |

What's the hardest part about programming those games?

link |

And maybe can you tell the story about building those games?

link |

I actually tried to do those in high school

link |

because I was just curious if I could.

link |

That's a good starting point for anything, right?

link |

Yeah, yeah, yeah, it's like, could you?

link |

But the appealing thing was also,

link |

it was a soup to nuts kind of thing.

link |

So something that has always attracted me is,

link |

I like beautiful ideas, I like seeing beautiful ideas,

link |

but I actually also like seeing execution of an idea

link |

all the way from beginning to end in something that works.

link |

So for example, in high school,

link |

I was lucky enough to grow up in the late 90s

link |

when even a high school student could hope

link |

to make something sort of comparable

link |

to the shareware games that were out there.

link |

I say the word sort of, like still quite far away,

link |

but at least I didn't need to hire a 3D CG artist.

link |

There weren't enough pixels to draw anyway,

link |

even I can draw, right?

link |

Bad art, of course.

link |

But the point is, I wanted to know,

link |

is it possible for me to try to do those things

link |

where back in those days,

link |

you didn't even have an easy way

link |

to draw letters on the screen in a particular font.

link |

You couldn't just say import a font, it wasn't like Python.

link |

So for example, back then,

link |

if you played those games in the web browser,

link |

which is emulating the old school computer,

link |

those, even the letters you see,

link |

those are made by individual calls

link |

to draw pixels on the screen.

link |

So you built that from scratch,

link |

almost building a computer graphics library from scratch?

link |

Yes, the primitive that I got to use

link |

was some code I copied off of a book in assembly

link |

of how to put a pixel on a screen in a particular color.

link |

And the programming language was Pascal?

link |

Ah, yeah, the first one was in Pascal,

link |

but then the other ones were in C++ after that.

link |

How's the emulation in the browser work, by the way?

link |

Because it's pretty cool, you get to play these games

link |

that have a very much 90s feeling to them.

link |

Ah, so it's literally making an MSDOS environment,

link |

which is literally running the old.exe file.

link |

Wow, in the browser.

link |

This is, that could be more amazing than the airplane.

link |

So it wasn't so much about the video games,

link |

it was more about,

link |

can you build something really cool from scratch?

link |

And you did a bunch of programming competitions.

link |

What was your interest, your love for programming?

link |

What did you learn through that experience?

link |

Especially now that as much of your work

link |

has taken a long journey through mathematics.

link |

I think I always was amazed

link |

by how computers could do things fast.

link |

If I wanted to make it an abstract analysis

link |

of why it is that I saw some power in the computer.

link |

Because if the computer can do things

link |

so many times faster than humans,

link |

where the hard part is telling the computer

link |

what to do and how to do it,

link |

if you can master that asking the computer what to do,

link |

then you could conceivably achieve more things.

link |

And those contests I was in,

link |

those were the opposite in some sense

link |

of making a complete product, like a game is a product.

link |

Those contests were effectively write a function

link |

to do something extremely efficiently.

link |

And if you are able to do that,

link |

then you can unlock more of the power of the computer.

link |

But also doing it quickly.

link |

There's a time element from the human perspective

link |

to be able to program quickly.

link |

There's something nice.

link |

So there's almost like an athletics component

link |

to where you're almost like an athlete

link |

seeking optimal performance as a human being

link |

trying to write these programs.

link |

And at the same time, it's kind of art

link |

because the best way to write a program quickly

link |

is to write a simple program.

link |

You just have a damn good solution.

link |

So it's not necessarily you have to type fast.

link |

You have to think through a really clean,

link |

beautiful solution.

link |

I mean, what do you think is the use

link |

of those programming competitions?

link |

Do you think they're ultimately something

link |

you would recommend for students,

link |

for people interested in programming,

link |

or people interested in building stuff?

link |

Yes, I think so because especially with the work

link |

that I've been doing nowadays,

link |

even trying to control COVID,

link |

something that was very helpful from day one

link |

was understanding that the kinds of computations

link |

we would want to do,

link |

we could conceivably do on like a four core cloud machine

link |

on Amazon Web Services out to a population

link |

which might have hundreds of thousands

link |

or millions of people.

link |

The reason why that was important

link |

to have that back of the envelope calculation

link |

with efficient algorithms

link |

is because if we couldn't do that,

link |

then we would bankrupt ourselves

link |

before we could get to a big enough scale.

link |

If you think about how you grow anything from small to big,

link |

if in order to grow it from small to big,

link |

you also already need 10,000 cloud servers,

link |

you'll never get to big.

link |

And also the nice thing about programming competitions

link |

is that you actually build a thing that works.

link |

So you finish it, there's a completion thing,

link |

and you realize, I think there's a magic to it,

link |

where you realize that it's not so hard

link |

to build something that works.

link |

To have a system that successfully takes in inputs

link |

and produces outputs and solves a difficult problem,

link |

and that directly transfers to building a startup essentially

link |

that can help some aspect of this world

link |

as long as it's mostly based on software engineering.

link |

Things get really tricky

link |

when you have to manufacture stuff.

link |

That's why people like Elon Musk are so impressive

link |

that it's not just software.

link |

Tesla Autopilot is not just software.

link |

It's like you have to actually have factories

link |

that build cars, and there's like a million components

link |

involved in the machinery required

link |

to assemble those cars and so on.

link |

But in software, one person can change the world,

link |

which is incredible.

link |

But on the mathematics side,

link |

what, if you look back, or maybe today,

link |

what made you fall in love with mathematics?

link |

For me, I think I've always been very attracted

link |

to challenge, as I already indicated

link |

with writing the program.

link |

I guess if I see something that's hard

link |

or supposed to be impossible,

link |

sometimes I say, maybe I want to see if I can pull that off.

link |

And with the mathematics, the math competitions

link |

presented problems that were hard,

link |

that I didn't know how to start,

link |

but for which I could conceivably try to learn

link |

how to solve them.

link |

So, I mean, there are other things that are hard

link |

called like get something to Mars, get people to Mars.

link |

And I didn't, and I still don't think

link |

that I am able to solve that problem.

link |

On the other hand, the math problems struck me

link |

as things which are hard

link |

and with significant amount of extra work,

link |

I could figure it out.

link |

And maybe they would actually even be useful,

link |

like that mathematical skill is the core

link |

of lots of other things.

link |

That's really interesting.

link |

Maybe you could speak to that

link |

because a lot of people say that math is hard

link |

as a kind of negative statement.

link |

It always seemed to me a little bit like

link |

that's kind of a positive statement

link |

that all things that are worth having in this world,

link |

I mean, everything that people think about

link |

that they would love to do, whether it's sports,

link |

whether it's art, music, and all the sciences,

link |

they're going to be hard

link |

if you want to do something special.

link |

So is there something you could say to that idea

link |

that math is hard?

link |

Should it be made easy or should it be hard?

link |

Ah, so I think maybe I want to dig in a little bit

link |

onto this hard part and say,

link |

I think the interesting thing about the math

link |

is that you can see a question

link |

that you didn't know how to start doing it before.

link |

And over a course of thinking about it,

link |

you can come up with a way to solve it.

link |

And so you can move from a state

link |

of not being able to do something

link |

to a state of being able to do something

link |

where you help to take yourself through that

link |

instead of somebody else spoon feeding you that technique.

link |

So actually here, I'm already digging into

link |

maybe part of my teaching philosophy also,

link |

which is that I actually don't want to ever

link |

just tell somebody, here's how you do something.

link |

I actually prefer to say, here's an interesting question.

link |

I know you don't quite know how to do it.

link |

Do you have any ideas?

link |

I'm actually explaining another way

link |

that you could try to do teaching.

link |

And I'm contrasting this to a method of watch me do this,

link |

now practice it 20 times.

link |

I'm trying to say a lot of people consider math to be hard

link |

because maybe they can't remember

link |

all of the methods that were taught.

link |

But for me, I look at the hardness

link |

and I don't think of it as a memory hardness.

link |

I think of it as a, can you invent something hardness?

link |

And I think that if we can teach more people

link |

how to do that art of invention in a pure cognitive way,

link |

not as hard as the actual hardware stuff, right?

link |

But like in terms of the concepts

link |

and the thoughts and the mathematics,

link |

teaching people how to invent,

link |

then suddenly actually they might not even find math

link |

to be that tiresomeness hard anymore,

link |

but that rewardingness hard of I have the capability

link |

of looking at something which I don't know what to do

link |

and coming up with how to do it.

link |

I actually think we should be doing that,

link |

giving people that capability.

link |

So hard in the same way that invention is hard,

link |

that is ultimately rewarding.

link |

So maybe you can dig in that a little bit longer,

link |

which is do you see basically the way to teach math

link |

is to present a problem and to give a person a chance

link |

to try to invent a solution

link |

with minimal amount of information first?

link |

Is that basically,

link |

how do you build that muscle of invention in a student?

link |

Yes, so the way that,

link |

I guess I have two different sort of ways

link |

that I try to teach.

link |

Actually, one of them is, in fact, this semester,

link |

because all my classes were remotely delivered,

link |

I even threw them all onto my YouTube channel.

link |

So you can see how I teach at Carnegie Mellon,

link |

but I'd often say, hey, everyone, let's try to do this.

link |

And that actually changes my role as a professor

link |

from a person who shows up for class

link |

with a script of what I wanna talk through.

link |

I actually, I don't have a script.

link |

The way I show up for classes,

link |

there's something that we want to learn how to do,

link |

and we're gonna do it by improv.

link |

I'm talking about the same method as improv comedy,

link |

which is where you tell me some ideas,

link |

and I'll try to yes and them.

link |

You know what I mean?

link |

And then together,

link |

we're gonna come up with a proof of this concept

link |

where you were deeply involved in creating the proof.

link |

Actually, every time I teach the class,

link |

we do every proof slightly differently

link |

because it's based on how the students came up with it.

link |

And that's how I do it when I'm in person.

link |

I also have another line of courses that we make

link |

that is delivered online.

link |

Those things are where I can't do it live,

link |

but the teaching method became also similar.

link |

It was just, here's an interesting question.

link |

I know it's out of reach.

link |

Why don't you think about it?

link |

And then automatic hints.

link |

We feed automatically hints through the internet

link |

to go and let the person try to invent.

link |

So that's like a more rigorous prodding of invention.

link |

But you did mention disease and COVID,

link |

and you've been doing some very interesting stuff

link |

from a mathematical, but also software engineering angle

link |

of coming up with ideas.

link |

It's back to the, I see a problem.

link |

I think I can help.

link |

So you stepped into this world.

link |

Can you tell me about your work there

link |

under the flag of Novid

link |

and both the software and the technical details

link |

of how the thing works?

link |

So first I want to make sure that I say,

link |

this is actually team effort.

link |

I happen to be the one speaking,

link |

but there's no way this would exist

link |

without an incredible team of people

link |

who inspire me every day to work on this.

link |

But I'll speak on behalf of them.

link |

So the idea was indeed that we stepped forward

link |

in March of last year, when the world started to become,

link |

our part of the world started to become,

link |

our part meaning the United States

link |

started to become paralyzed by COVID.

link |

The shutdown started to happen.

link |

And at that time it started as a figment of an idea,

link |

which was network theory,

link |

which is the area of math that I work in,

link |

could potentially be combined with smartphones

link |

and some kind of health information anonymized.

link |

We didn't know yet.

link |

We tried to crystallize it.

link |

And many months into this work,

link |

we ended up accidentally discovering a new way

link |

to control diseases,

link |

which is now what is the main impetus of all of this work

link |

is to take this idea and polish it

link |

and hopefully have it be useful not only now,

link |

but for future pandemics.

link |

The idea is really simple to describe.

link |

Actually, my main thing in the world

link |

is I come up with obvious observations.

link |

That's that, so I'll explain it now.

link |

Einstein did the same thing

link |

and he wrote a few short papers.

link |

But so the idea is like this.

link |

If we describe how usually people control disease

link |

for a lot of history,

link |

it was that you'd find out who was sick,

link |

you'd find out who they've been around

link |

and you try to remove all of those people from society

link |

against their will.

link |

Now that's the problem.

link |

The against their will part

link |

gives you the wrong kind of a feedback loop,

link |

which makes it hard to control the disease

link |

because then the people you're trying to control

link |

keep getting other people sick.

link |

You can see already how I'm thinking

link |

and talking about this feedback loops.

link |

This is actually related to something you said earlier

link |

about even like how skyscrapers stay in the air.

link |

The whole point is control theory.

link |

You actually want to, or even how an airplane stays,

link |

you need to have control loops

link |

which are feedbacking in the right way.

link |

And what we observed was that the feedback control loop

link |

for controlling disease by asking people

link |

to be removed from society against their will

link |

It was running against human incentives

link |

and you suddenly are trying to control

link |

seven billion, eight billion people

link |

in ways that they don't individually want

link |

to necessarily do.

link |

So here's the idea.

link |

And this is inspired by the fact

link |

that at the core of our team

link |

were user experience designers.

link |

That's actually, in fact, the first thing I knew

link |

we needed when we started

link |

was to bring user experience at the core.

link |

But so the idea was suppose hypothetically

link |

there was a pandemic.

link |

What would you want?

link |

You would want a way to be able to live your life

link |

as much as possible and avoid getting sick.

link |

Can we make an app to help you avoid getting sick?

link |

Notice how I've just articulated the problem.

link |

It is not, can we make an app

link |

so that after you are around somebody who's sick

link |

you can be removed from society.

link |

It's can we make an app so that you can avoid getting sick.

link |

That would run a positive feed.

link |

I don't know if I want to call it positive or negative

link |

but it would run a good feedback loop.

link |

So then how would you do this?

link |

The only problem is that you don't know who's sick

link |

because especially with this disease

link |

if I see somebody who looks perfectly healthy

link |

the disease spreads two days before you have any symptoms.

link |

And so it's actually not possible.

link |

That's where the network theory comes in.

link |

You caught it from someone.

link |

What if we changed the paradigm

link |

and we said, whenever there's a sickness

link |

tell everybody how many physical relationships

link |

separate them from the sickness.

link |

That is the trivial idea we added.

link |

The trivial idea was the distance between you and a disease

link |

is not measured in feet or seconds.

link |

It's measured in terms of how many

link |

close physical relationships separate you

link |

like these six degrees of separation like LinkedIn.

link |

What if we told everyone that?

link |

It turns out that actually unlocks

link |

some interesting behavioral feedback loops

link |

which for example, let me now jump to a non COVID example

link |

to show why this maybe could be useful.

link |

Actually we think it could be quite useful.

link |

Imagine there was Ebola or some hemorrhagic fever.

link |

Imagine it spread through contact through the air.

link |

In fact, pretend, pretend.

link |

That's a disastrous disease.

link |

It has high fatality rate.

link |

And as you die, you're bleeding out of every orifice.

link |

Yeah, not pleasant.

link |

So the question is, suppose that such a disease broke

link |

who would want to install an app that would tell them

link |

how many relationships away from them

link |

this disease had struck?

link |

Like a lot of people.

link |

In fact, almost, I don't want to say almost everyone.

link |

That's a very strong statement

link |

but a very large number of people.

link |

That's fascinating framing.

link |

Like the more deadly and transmissible the disease

link |

the stronger the incentive to install it in a positive sense

link |

the, in the good feedback loop sense.

link |

That's a really good example.

link |

It's a really good way to frame it.

link |

Cause with COVID, it was not as deadly

link |

as potential pandemics could have been

link |

viruses could have been.

link |

So it's sometimes muddled with how we think about it

link |

but yeah, this is a really good framing.

link |

If the virus was a lot more deadly

link |

you want to create a system that has a set of incentives

link |

that it quickly spreads to the population

link |

where everybody is using it

link |

and it's contributing in a positive way to the system.

link |

And actually that point you just made

link |

I don't take credit for that observation.

link |

There was another person I talked to

link |

who pointed out that it's very interesting

link |

that this feedback loop is even more effective

link |

when the disease is worse.

link |

And that's actually not a bad characteristic to have

link |

in your feedback loop

link |

if you're trying to help civilization keep running.

link |

Yeah, it's a really, it's in this dynamic

link |

like people figure out, they dynamically figure out

link |

how bad the disease is.

link |

The more it spreads and the deadlier it is

link |

as the people observe it

link |

as long as the spread of information

link |

like semantic information, natural language information

link |

is closely aligned with the reality of the disease

link |

which is a whole nother conversation, right?

link |

We, that's, we might, maybe we'll chat about that

link |

how we sort of make sure there's not misinformation

link |

while there's accurate information

link |

but that aside, okay, so this is a really nice property.

link |

Right, and just going on on that

link |

actually just talking more about what that could do

link |

and why we're so excited about it.

link |

It's that not only would people want to install it

link |

but what would they do if you start to see

link |

that this disease is getting closer and closer?

link |

We surveyed informally people

link |

but they said, as we saw it getting closer, we would hide.

link |

We would try to not have contacts.

link |

But now you notice what this has just achieved.

link |

The whole goal on this whole exercise was

link |

you got the people who might be sick

link |

and you got everyone else, set A and set B.

link |

Set A is the people who might be sick,

link |

set B is everyone else.

link |

And for the entirety of the past

link |

contact tracing approaches, you tried to get set A

link |

to do things that might not be to their liking or their will

link |

because that's removing them from society.

link |

We found out that there's two ways

link |

to separate set A from set B.

link |

You can also let the people at set B

link |

at the fringe of set A

link |

attempt to remove themselves from this interface.

link |

It's the symmetry of A and B separation.

link |

Everyone was looking at A, we look at B

link |

and suddenly B is in their incentive to do so.

link |

So there's a virus that jumps from human to human.

link |

So there's a network sometimes called graph

link |

of the spread of a virus.

link |

It hops from person to person to person to person.

link |

And each one of us individuals are sitting

link |

or plopped into that network.

link |

We have close friends and relations and so on.

link |

It's kind of fascinating

link |

to actually think about this network

link |

and we can maybe talk about the shapes

link |

of this kind of network.

link |

Because I was trying to think exactly this,

link |

like how many people do I,

link |

well, I'm kind of an introvert, not kind of,

link |

I'm very much an introvert.

link |

But so can I be explicit about the kind of people

link |

I meet in regular life?

link |

Say when it was completely opened up, there's no pandemic.

link |

There is a kind of network and there's maybe

link |

in the graph theoretic sense, there's some weights

link |

or something about how close that relationship is

link |

in terms of the frequency of visits,

link |

the duration of visits and all of those kinds of things.

link |

So you're saying we might want to be,

link |

to create on top of that network,

link |

a spread of information to let you know

link |

as the virus travels through this network,

link |

how close is it getting to you?

link |

And the number of hops away it is on that network

link |

is really powerful information

link |

that creates a positive feedback loop

link |

where you can act essentially anonymously

link |

Like nobody's telling you what to do,

link |

which is really important, is decentralized

link |

and not whatever the opposite of authoritarian is.

link |

But you get to sort of the American way.

link |

You get to choose to do it yourself.

link |

You have the freedom to do it yourself

link |

and you're incentivized to do it.

link |

And you're most likely going to do it

link |

to protect yourself against you getting the disease

link |

as the closer it gets to you

link |

based on the information that you have.

link |

But can you maybe elaborate, first of all, brilliant.

link |

Whenever I saw the thing you're working on,

link |

so forget for COVID, this is of course,

link |

really relevant for COVID, but it's also probably relevant

link |

for future diseases as well.

link |

So that was the thing I'm nervous about.

link |

I was like, if this whole,

link |

if our society shut down because of COVID,

link |

like what the heck is gonna happen

link |

when there's a much deadlier disease?

link |

Like this, this is disappointing.

link |

The whole time, 2020, the whole time

link |

I'm just sitting like this,

link |

like is the incompetence of everybody

link |

except the people developing vaccines.

link |

The biologists are the only ones

link |

that got their stuff together.

link |

But in terms of institutions and all that kind of stuff,

link |

it's just been terrible.

link |

But this is exactly the power of information

link |

and the power of information

link |

that doesn't limit personal freedom.

link |

So your idea is brilliant.

link |

Okay, mathematically, can you maybe elaborate

link |

what are we talking about?

link |

Like how do you actually make that work?

link |

Sure, first I'm gonna reply to something you said

link |

about the freedom inside this,

link |

because actually that was the idea.

link |

The idea is this is game theory, right?

link |

And effectively what we did is analogous

link |

to free market economy, as opposed to central planning.

link |

If you just line up the set of incentives correctly

link |

so that people have in their purely selfish behavior

link |

are contributing to the optimization of the global function,

link |

And the point of what we do, I guess in mathematics

link |

is we try to explore the search space

link |

to go and find out as many possibilities as there are.

link |

And in this case, it's an applied search space.

link |

That's why the inputs from design,

link |

user experience design and actual people are important.

link |

But you asked about, I guess, the mathematical

link |

or the technical things underpinning it.

link |

So I think the first thing I'll say is

link |

we wanted to make this thing

link |

not require your personal information.

link |

And so in order to do that,

link |

what gave me the confidence to, I guess,

link |

lead our team to run at the beginning

link |

is we saw that this could be done without using GPS information.

link |

So technically what's going on is if two smartphones,

link |

it's a smartphone app.

link |

If two smartphones have this thing installed,

link |

they just communicate with each other by Bluetooth

link |

to go and find out how far,

link |

they can detect nearby things by Bluetooth.

link |

And then they can find out that these two phones

link |

were approximately such and such distance apart.

link |

And that kind of relative proximity information

link |

is enough to construct this big network.

link |

Okay, so the physical network is constructed

link |

based on proximity that's through Bluetooth

link |

and you don't have to specify your exact location,

link |

it's the proximity.

link |

I'm not using the Pythagorean theorem basically.

link |

I mean, if I just knew the GPS coordinates,

link |

we could use the Pythagorean theorem too.

link |

Sorry, that's just how I call it.

link |

Distance formula, whatever you want to call it.

link |

Yeah, so we're not doing

link |

the old Pythagorean based violation of privacy.

link |

But so is that enough to form,

link |

to give you enough information about physical connection

link |

to another human being?

link |

Is there a time element there?

link |

Is there, so, okay.

link |

That sounds like a really strong, like low hanging fruit.

link |

Like if you have that,

link |

you could probably go really, really far.

link |

My natural question is,

link |

is there extra information you can add on top of that?

link |

Like the duration of the physical proximity?

link |

So first of all, we actually do estimate the duration,

link |

but the way we estimate the duration

link |

is like how a movie is filmed,

link |

in the sense that every so often, every few minutes,

link |

we check what's nearby.

link |

It's like how a movie is filmed.

link |

You take lots of snapshots.

link |

So there's no way in a battery efficient way

link |

to really keep track of that proximity.

link |

However, fortunately, we're using probability.

link |

The fact is the paradigm that we're using

link |

is it's not super important

link |

if you run into that person only for 10 minutes

link |

at the grocery store.

link |

If that's a stranger that you run into 10 minutes

link |

in this grocery store,

link |

that's not gonna be relevant for our paradigm

link |

because our paradigm is not telling you

link |

who were you around before

link |

and might therefore have gotten infected by already.

link |

Ours is about predicting the future.

link |

We change from, I mean, the standard paradigm was

link |

what already happened, quick damage control.

link |

Ours is predict the future.

link |

If you run into that person once in the grocery store today

link |

and never see them again,

link |

it's irrelevant for predicting the future.

link |

And therefore, for ours, what really matters

link |

is the many hours around the other person,

link |

at which point, if you're scanning every five

link |

That's going to come out in the problem,

link |

like statistically speaking,

link |

it's going to come out as a strong relationship

link |

and a person in the grocery store is going to wash out

link |

is not an important physical relationship.

link |

I mean, this is brilliant.

link |

How difficult is it to make work?

link |

So you said, one, there's a mathematical component

link |

that we just kind of talked about,

link |

and then there's the user experience component.

link |

So how difficult does it to go,

link |

just like you built the video game, Alien Attack,

link |

from zero to completion, what's involved?

link |

How difficult is it?

link |

So I'm going to answer that question

link |

in terms of building the product,

link |

but then I'm also going to acknowledge

link |

that just having an app doesn't make it useful

link |

because that's actually maybe the easy part.

link |

If you know what I mean,

link |

there's like all of this stuff

link |

about rollout adoption and awareness,

link |

but let's focus on the app part first.

link |

So that's again, why I said the team is incredible.

link |

So we have a bunch of people who,

link |

let's just say that the technology that we use to make it

link |

is not the standard way you make an app.

link |

If you think about a standard iOS app or Android app,

link |

those are a user interface that contacts a web server

link |

and sends some information back and forth.

link |

We're doing some stuff that has to hook

link |

into the operating system of saying,

link |

let's go use Bluetooth for something

link |

it wasn't really meant for, right?

link |

So there's that part.

link |

By the way, what is the app called?

link |

Oh, it's called Novid, COVID with an N.

link |

So you have to hook into Bluetooth.

link |

You're saying you have to do that beyond the permissions

link |

that are like at the very surface level

link |

provided on the phone?

link |

Well, I don't want to call them permissions.

link |

I just want to say,

link |

that's not what you usually do with Bluetooth.

link |

Usually with Bluetooth, you say,

link |

do I have headphones nearby?

link |

You don't go and say, do I have headphones nearby?

link |

Or do I have another phone nearby, which is doing something?

link |

And then keep asking that same question.

link |

Keep asking the question.

link |

So it's actually not easy.

link |

And I mean, there were some parts of it,

link |

which actually a lot of people had tried unsuccessfully.

link |

Actually, it's known that, for example,

link |

the UK was trying to do something similar.

link |

And the problem they ran into was,

link |

when you program things on iOS,

link |

iOS is very good at making it hard

link |

to do things in the background.

link |

And so there was quite a lot of effort required

link |

to go and make this thing work.

link |

So the whole point, this thing would run in the background

link |

and iOS, I mean, most Android probably as well, right?

link |

But yeah, iOS certainly makes it difficult

link |

for something to run in the background,

link |

especially when it's eating up your battery, right?

link |

Well, we wanted to make sure we didn't eat up the battery.

link |

So that one we can,

link |

we actually are very proud of the fact

link |

that ours uses very little battery.

link |

Actually, even if compared to Apple's own system, so.

link |

So what else is required to make this thing work?

link |

Right, so the key was that you had to do

link |

a significant amount of work on the actual

link |

mobile app development,

link |

which fortunately the team that we brought

link |

was this kind of general thinkers

link |

where we would dig in deep into the operating system

link |

documentation and the API libraries.

link |

So we got that working.

link |

But there's another angle, which is,

link |

you also need the servers to be able to compute fast enough,

link |

which is tying back to this old school

link |

computer programming competitions and math Olympiads.

link |

In fact, our team that was working on the algorithm

link |

and backend side included several people

link |

who had been in these competitions from before,

link |

which I happen to know because I do coach the team

link |

And so we were able to bring people in to build servers,

link |

a server infrastructure in C++ actually,

link |

so that we could support significant numbers of people

link |

without needing tons of servers.

link |

Is there some distributed algorithms working here

link |

or you basically have to keep in the same place

link |

the entire graph as it builds?

link |

Cause especially the more and more people use it,

link |

the bigger, the bigger the graph gets.

link |

I mean, this is very difficult scaling problem, right?

link |

Ah, so that's actually why this computer algorithm

link |

competition stuff was handy.

link |

It's because there are only about seven to eight

link |

giga people in the world.

link |

That's not that many.

link |

So if you can make your algorithms linear time

link |

or almost linear time, a computer operates in gigahertz.

link |

I only need to do one run, one recalculation every hour

link |

in terms of telling people how far away these dangers are.

link |

So I suddenly have 3,600 seconds

link |

and my CPU cores are running in gigahertz.

link |

And at most they're eight giga people.

link |

Well, you skipping over the fact that there's N squared

link |

potential connections between people.

link |

So how do you get around the fact that, you know,

link |

that we, you know, the potential set of relationship

link |

any one of us could have is a billion.

link |

So it's a billion times squared.

link |

That's the potential amount of data you have to be storing

link |

and computing over and constantly updating.

link |

So the way we dealt with that is we actually expect

link |

that the typical network is very sparse.

link |

The technical term sparse would mean that the average degree

link |

or the average number of connections that a person has

link |

is going to be at most like a hundred strong connections

link |

that you care about.

link |

If you think of it almost in terms of the heavy hitters,

link |

actually in most people's lives,

link |

a hundred, if we just kept track

link |

of their top hundred interactions,

link |

that's probably most of the signal.

link |

I'm saddened to think that I might not be even

link |

in a double digits, but.

link |

Oh, I was intentionally giving a crazy number

link |

to account for college students.

link |

You call, oh, those are the,

link |

who you call on the heavy hitters,

link |

the people who are like the social butterflies.

link |

I'd love to know that information about myself,

link |

by the way, that, do you expose the graph,

link |

like how many, like about yourself,

link |

how many connections you have?

link |

We do expose to each person

link |

how many direct connections they have.

link |

But for privacy purposes,

link |

we don't tell anybody who their connections,

link |

like how their connections are interconnected.

link |

But at the same time, we do expose also to everyone

link |

an interesting chart that says,

link |

here's how many people you have

link |

that you're connected to directly.

link |

Here's how many at distance two,

link |

meaning via people.

link |

And then here's how many at distance three.

link |

And the reason we do that,

link |

is that actually ends up being a dynamic

link |

that also boosts adoption.

link |

It drives another feedback loop.

link |

The reason is because we saw, actually,

link |

when we deployed this in some universities,

link |

that when people see on their app

link |

that they are indirectly connected to hundreds

link |

or thousands of other people,

link |

they get excited and they tell other people,

link |

hey, let's download this app.

link |

But you know, we also saw in those examples,

link |

especially looking at the screenshots people gave,

link |

that is hit as soon as the typical person

link |

has two or three other direct connections on the system.

link |

Because that means that our app

link |

has reached a virality or not of two to three.

link |

The key is we were making a viral app to fight a virus

link |

spreading on the same network that the virus spreads on.

link |

So you're trying to out virus the virus.

link |

That's exactly right.

link |

What have you learned from this whole experience

link |

in terms of, let's say for COVID,

link |

but for future pandemics as well,

link |

is it possible to use the power information here

link |

of networked information as a virus spreads and travels

link |

in order to basically keep the society open?

link |

Is it possible for people to protect themselves

link |

with this information?

link |

Or do you still have to have most,

link |

like in this overarching policy

link |

of everybody should stay at home, that kind of thing?

link |

We are trying to answer that question right now.

link |

So the answer is we don't know yet,

link |

but that's actually why we're very happy

link |

that now the idea has started to become more widely known.

link |

And we're already starting to collaborate

link |

with epidemiologists.

link |

Again, I'm just a mathematician, right?

link |

And a mathematician should not be the person

link |

who is telling everybody, this will definitely work.

link |

But because of the potential power of this approach,

link |

especially the potential power

link |

of this being an end game for COVID,

link |

we have gotten the interest of real researchers.

link |

And we're now working together

link |

to try to actually understand the answer to that question.

link |

Because you see, there's a theory.

link |

So what I can share is the mathematics of,

link |

here's why there's some hope that this would work.

link |

And that's because I'm talking about end game now.

link |

End game means you have very few cases.

link |

But everywhere, we're always thinking,

link |

once there's few cases, then does that mean we now open up?

link |

Once you open up in the past, then the cases go up again

link |

until you have to lock down again.

link |

And now when we talk about the dynamic process that makes,

link |

it's guaranteeing you always have cases

link |

until you have the great vaccines,

link |

which is, we both got vaccinated, this is good.

link |

But at the same time, why I'm thinking

link |

this is still important is because we know

link |

that many vaccine makers have said

link |

they're preparing for the next dose next year.

link |

And if we have a perpetual thing

link |

where you just always need a new vaccine every year,

link |

it could actually be beneficial to make sure

link |

we have as many other techniques as possible

link |

for parts of the world that can't afford,

link |

for example, that kind of distribution.

link |

Yeah, so actually, no matter how deadly the virus is,

link |

no matter how many things,

link |

whether you have a vaccine or not,

link |

it's still useful to be having this information.

link |

Because to stay home or not, depending on how risk,

link |

I'm a big fan, just like you said, of having the freedom

link |

for you to decide how risk averse you wanna be, right?

link |

Depending on your own conditions,

link |

but also on the state of like what you,

link |

just how dangerously you like to live.

link |

So I think that actually makes a lot of sense.

link |

And I also think that since we're,

link |

when you think of disease spreading,

link |

it spreads in aggregate in the sense that

link |

if there are some people who maybe are more risk tolerant

link |

because of other things in their life,

link |

well, there might also be other people

link |

who are less risk tolerance.

link |

And then those people decide to isolate.

link |

But what matters is in the aggregate

link |

that this R naught of the infection spreading

link |

And so the key is if you can empower people

link |

with that power to make that decision,

link |

you might actually still be able to drive

link |

that R naught down below one.

link |

Yeah, and also, this is me talking,

link |

people get a little bit nervous, I think,

link |

with information somehow mapping to privacy violation.

link |

But first of all, in the approach you're describing,

link |

that's respecting anonymity.

link |

But I would love to have information

link |

from the very beginning, from March and April of last year,

link |

almost like a map of like where it's risky

link |

and where it's not to go.

link |

And not map based on sort of the exact location of people,

link |

but where people usually hang out kind of thing.

link |

Just, and maybe not necessarily about actual location,

link |

but just maybe activities,

link |

like just to have information about what is good to do

link |

and not, in terms of like safety,

link |

is it okay to run outside and not,

link |

is it okay to go to a restaurant and not,

link |

I just feel like we're operating in the blind.

link |

And then what you had is a very imperfect signal,

link |

which is like basically politicians desperately trying

link |

to make statements about what is safe and not.

link |

They don't know what the heck they're doing.

link |

They have a bunch of smart scientists telling them stuff.

link |

And the scientists themselves also, very important,

link |

don't always know what they're doing.

link |

Epidemiology is not, is as much an art as a science.

link |

You're desperately trying to predict the future,

link |

which nobody can do.

link |

And then you're trying to speak with some level of authority.

link |

I mean, if I were to criticize scientists,

link |

they spoke with too much authority.

link |

It's okay to say, I'm not sure.

link |

But then they think like, if I say, I'm not sure,

link |

then there's going to be a distrust.

link |

What they realize is when you're wrong and you say,

link |

I'm sure, it's going to lead to more distrust.

link |

So there's this imperfect, like just chaotic,

link |

messy system of people trying to figure out

link |

with very little information.

link |

And what you're proposing is just a huge amount

link |

of information, and information is power.

link |

Is there challenges with adoption that you see

link |

in the future here?

link |

So there's, maybe we could speak to,

link |

there's approaches, I guess, from Google.

link |

There's different people that have tried

link |

similar kind of ideas.

link |

Not, you have quite a novel idea, actually.

link |

But speaking, the umbrella idea of contact tracing,

link |

is there something you can comment about

link |

why their approaches haven't been fully adopted?

link |

Is there challenges there?

link |

Is there reasons why Novid might be a better idea

link |

moving forward, in general, just about adoption?

link |

Yeah, so first of all, I want to say,

link |

I always have respect for the methods that other people use.

link |

And so it's good to see the other people I've been trying.

link |

But what we have noticed is that the difference

link |

between our value proposition to the user

link |

and the value proposition to the user delivered

link |

by everything that was made before is that,

link |

unfortunately, the action of installing

link |

a standard contact tracing app will then tell you

link |

after you have already been exposed to the disease

link |

so that you can protect other people from you.

link |

And what that does to your own direct probability

link |

of getting sick, if you think about it,

link |

suppose you were making the decision,

link |

should I or should I not install one of those apps?

link |

What does that do to your own probability of getting sick?

link |

It's close to zero.

link |

This is the sad thing you're speaking to, not sad.

link |

I suppose it's the way the world is.

link |

The only incentive there is to just help other people,

link |

I suppose, but a much stronger incentive

link |

is anything that allows you to help yourself.

link |

Yes, so what I'm saying is that,

link |

let's just say free market capitalism

link |

was not based on altruism, I think it's based on,

link |

if you make a system of incentives

link |

so that everybody trying to maximize their own situation

link |

somehow contributes to the whole,

link |

that's a game theoretic solution to a very hard problem.

link |

And so this is actually basically mechanism design,

link |

that we've basically come up with a different mechanism,

link |

different set of incentives,

link |

which incentivizes the adoption,

link |

because actually whenever we've been rolling it out,

link |

usually the first question we ask people,

link |

like say in a university is,

link |

do you know what Novid does?

link |

And most of them have read about the other apps

link |

and they say, Oh, Novid will tell you

link |

after you've been around someone so you can quarantine.

link |

And we have to explain to them,

link |

actually, Novid never wants to ask you to quarantine.

link |

That's not the principle.

link |

Our principle isn't based on that at all.

link |

We just want to let you know if something is coming close

link |

so that you can protect yourself.

link |

If you want, if you want, if you want.

link |

And then the quarantine is like, yes,

link |

in that case, if you're quarantining,

link |

it's because you're shutting the door from the inside,

link |

if that makes sense.

link |

I mean, this is brilliant.

link |

So what do you think the future looks like

link |

for future pandemics?

link |

What's your plan with Novid?

link |

What's your plan with these set of ideas?

link |

I am actually still an academic and a researcher.

link |

So the biggest work I'm working on right now

link |

is to try to build as many collaborations

link |

with other public health researchers at other universities

link |

to actually work on pilot deployments together

link |

in various places.

link |

That's actually ongoing work right now.

link |

And so, for example, if anyone's watching this

link |

and you happen to be a public health researcher

link |

and you want to be involved in something like this,

link |

I'm just gonna say, I'm still incentive thinking.

link |

There's something in it for the researchers too.

link |

This could open up an entire new way

link |

of controlling disease.

link |

I mean, it might actually be true.

link |

And people who are involved in figuring out

link |

how to make this work,

link |

well, it could actually be good for their careers too.

link |

I always have to think like,

link |

if a researcher was getting involved,

link |

what are they getting out of it?

link |

Oh, so you mean like from a research perspective,

link |

you can like publications and sets of ideas

link |

about how to, from a sort of network theory perspective,

link |

understand how we control the spread of a pandemic.

link |

Yes, and what I'm doing right now

link |

is this is basically interdisciplinary research

link |

where maybe our side is bringing the technology

link |

and the network theory,

link |

and the missing parts are epidemiology

link |

and public health expertise.

link |

And if the two things start to join,

link |

also because everywhere that you deploy,

link |

let's just say that the world is different

link |

in the Philippines as it is in the United States.

link |

And just the natures of the locality

link |

would mean that someone like me

link |

should not be trying to figure out how to do that.

link |

But if we can work with the researchers

link |

who are based there,

link |

now suddenly we might come up with a solution

link |

that will help scale in parts of the world

link |

where they aren't all getting the Moderna and Pfizer vaccines

link |

which cost like $20 a pop in the US.

link |

So if they want to participate,

link |

who do they reach out to?

link |

Oh, that would just be us.

link |

I mean, the novid.org website has...

link |

It has a feedback reach out form.

link |

And actually we are, I mean, again,

link |

this is the DNA of being a researcher.

link |

I am actually very excited by the idea

link |

that this could contribute knowledge

link |

that will outlast all of our generations,

link |

like all of our lifetimes.

link |

Reach out to novid.org.

link |

What about individual people?

link |

Should they install the app and try it out?

link |

Or is this really geographically restricted?

link |

Oh, yeah, I didn't come on here to tell everyone

link |

to install the app.

link |

I did not come to tell everyone to install the app

link |

because it works best

link |

if your local health authority is working with us.

link |

It's because, this is back to the game theory.

link |

If anyone could just say, I'm positive,

link |

the high school senior prank would be to say that

link |

we have a massive outbreak on finals week.

link |

Let's not have final exams.

link |

So the way that our system works,

link |

it actually borrows some ideas, not borrows,

link |

we came up with them independently.

link |

But this idea is similar to what Google and Apple do,

link |

which is that if the local health authority

link |

is working with this, they can,

link |

for everyone who's positive,

link |

give them a passcode that expires in a short time.

link |

So for ours, if you're on the app and saying, I'm positive,

link |

you can either just say that,

link |

and that's called unverified,

link |

or you can enter in one of these codes

link |

that you got from the local health authority.

link |

So basically, for anyone who's watching this,

link |

it's not that you should just go and download it

link |

unless you want to go and look at it.

link |

But if you, on the other hand,

link |

if you happen to know anyone at the local health authority,

link |

which is trying to figure out how to handle COVID,

link |

well then, I mean, we'd be very happy

link |

to also work with you.

link |

So the verified there is really important

link |

because you're maintaining anonymity.

link |

And because of that,

link |

you have to have some source of verification

link |

in order to make sure that it's not possible to manipulate

link |

because it's ultimately about trust and information.

link |

So it could be, verification is really important there.

link |

So basically, individual people should

link |

ask their local health authorities

link |

to sign up to contact you.

link |

I hope this spreads.

link |

I hope this spreads for future pandemics

link |

because I'm really, it's the amount,

link |

the millions of people who are hurt by this,

link |

I think our response to the virus,

link |

economically speaking,

link |

the number of people who lost their dream,

link |

lost their jobs, but also lost their dream.

link |

Entrepreneurs, jobs often give meaning.

link |

There's people who financially and psychologically

link |

are suffering because of our,

link |

I'll say, incompetent response to the virus

link |

across the world, but certainly the United States,

link |

that should be the beacon of entrepreneurial hope

link |

So I hope that we'll be able to respond

link |

to these kinds of events much better in the future.

link |

And this is exactly the right kind of idea.

link |

And now is the time to do the investment.

link |

Let's step back to the beauty of mathematics.

link |

Maybe ask the big, silly question first,

link |

which is, what do you find beautiful about mathematics?

link |

I think that being able to look at a complicated problem,

link |

which looks unsolvable,

link |

and then to be able to change the perspective

link |

to come from a different angle

link |

and suddenly see that there's a nice solution.

link |

I don't mean that every problem in math

link |

is supposed to be this way,

link |

but I think that these reframings

link |

and changing of perspectives

link |

that cause difficult things to get simplified

link |

and crystallized and factored in certain ways is beautiful.

link |

Actually, that's related to what we were just talking about

link |

with even this fighting pandemics.

link |

The crystal idea was just quantify proximity

link |

by the number of relationships in the physical network,

link |

instead of just by the feet and meters, right?

link |

It's just that if you change that perspective,

link |

now all of these things follow.

link |

And so mathematics to me is beautiful

link |

in the pure sense just for that.

link |

Yeah, it's quite interesting to see a human civilization

link |

as a network, as a graph,

link |

and our relationships as kind of edges in that graph.

link |

And to then do, outside of just pandemic,

link |

do interesting inferences based on that.

link |

This is true for like Twitter, social networks and so on,

link |

how we expand the kind of things we talk about,

link |

think about sort of politically,

link |

if you have this little bubble, quote unquote,

link |

of ideas that you play with,

link |

it's nice from a recommender system perspective,

link |

how do you jump out of those bubbles?

link |

It's really fascinating.

link |

YouTube was working on that, Twitter's working on that,

link |

but not always so successfully,

link |

but there's a lot of interesting work

link |

from a mathematical and a psychological,

link |

sociological perspective there within those graphs.

link |

But if we look at the cleanest formulation of that,

link |

of looking at a problem from a different perspective,

link |

you're also involved

link |

with the International Mathematics Olympiad,

link |

which takes small, clean problems that are really hard,

link |

but once you look at them differently, can become easy.

link |

But that little jump of innovation is the entire trick.

link |

So maybe at the high level,

link |

can you say what is the International Mathematical Olympiad?

link |

Sure, so this is the competition

link |

for people who aren't yet in college, math competition,

link |

which is the most prestigious one in the entire world.

link |

It's the Olympics of mathematics,

link |

but only for people who aren't yet in college.

link |

Now, the kinds of questions that they ask you to do

link |

are not computational.

link |

Usually you're not supposed to find that the answer is 42.

link |

Instead, you're supposed to explain why something is true.

link |

And the problem is that at the beginning,

link |

when you look at each of the questions,

link |

first of all, you have four and a half hours

link |

to solve three questions, and this is one day,

link |

and then you have a second day,

link |

which is four and a half hours, three questions.

link |

But when you look at the questions,

link |

they're all asking you,

link |

explain why the following thing is true,

link |

which you've never seen before.

link |

And by the way, even though there are six questions,

link |

if you solve any one of them, you're a genius

link |

and you get an honorable mention.

link |

So this is hard to solve.

link |

So what about, is it one person, is it a team?

link |

Ah, so each country can send six people

link |

and the score of the country is actually unofficial.

link |

There's not an official country versus country system,

link |

although everyone just adds up the point scores

link |

of the six people and they say,

link |

well, now which country stacked up where?

link |

Yeah, so maybe as a side comment,

link |

I should say that there's a bunch of countries,

link |

including the former Soviet Union and Russia,

link |

where I grew up, where this is one of the

link |

most important competitions that the country participates in.

link |

It was a source of pride for a lot of the country.

link |

You look at the Olympic sports,

link |

like wrestling, weightlifting,

link |

there's certain sports and hockey

link |

that Russia and the Soviet Union truly took pride in.

link |

And actually the Mathematical Olympiad,

link |

it was one of them for many years.

link |

It's still one of them.

link |

And that's kind of fascinating.

link |

We don't think about it this way in the United States.

link |

Maybe you can correct me if I'm wrong,

link |

but it's not nearly as popular in the United States

link |

in terms of its integration into the culture,

link |

into just basic conversation, into the pride.

link |

Like, if you won an Olympic gold medal

link |

or if you win the Super Bowl, you can walk around proud.

link |

I think that was the case

link |

with the Mathematical Olympiad in Russia.

link |

Not as much the case in the United States, I think.

link |

So I just wanna give that a little aside

link |

because beating anybody from Russia,

link |

from the Eastern Republic or from China

link |

is very, very difficult.

link |

Like, if I remember correctly,

link |

there's people, this was a multiyear training process.

link |

And this is everything that they're focused on.

link |

My dad was a participant in this.

link |

And it's, I mean, it's as serious as Olympic sports.

link |

You think about like gymnastics,

link |

like young athletes participating in gymnastics.

link |

This is as serious as that, if not more serious.

link |

So I just wanna give that a little bit of context

link |

because we're talking about serious high level math,

link |

athletics almost here.

link |

Yeah, and actually I also think that it made sense

link |

from the Soviet Union's perspective

link |

because if you look at what these people do eventually,

link |

even though, let's look at the USSR's

link |

International Math Olympiad record.

link |

Even though they, I say, even though they won

link |

a lot of awards at the high school thing,

link |

many of them went on to do incredible things

link |

in research mathematics or research other things.

link |

And that's showing the generalization,

link |

generalizability of what they were working on.

link |

Because ultimately we're just playing with ideas

link |

of how to prove things.

link |

And if you get pretty good at inventing creative ways

link |

to turn problems apart, split them apart,

link |

observe neat ways to turn messy things into simple crystals.

link |

Well, if you're gonna try to solve any real problem

link |

in the real world, that could be a really handy tool too.

link |

So I don't think it was a bad investment.

link |

I think it clearly worked well for Soviet Union.

link |

Yeah, so this is interesting.

link |

People sometimes ask me, you know,

link |

you go up and under communism, you know,

link |

was there anything good about communism?

link |

And it's difficult for me to talk about it

link |

because it's not, communism is one of those things

link |

that's looked down on like without,

link |

in absolutist terms currently.

link |

But you could still, in my perspective,

link |

talk about the actual, forget communism

link |

or whatever the actual term is,

link |

but you know, certain ways that the society functioned

link |

that we can learn lessons from.

link |

And one of the things in the Soviet Union

link |

that was highly prized is knowledge,

link |

not even knowledge, it's wisdom

link |

and the skill of invention, of innovation at a young age.

link |

So we're not talking about a selection process

link |

where you pick the best students in the school

link |

to do the mathematics or to read literature.

link |

It's like, everybody did it.

link |

Everybody, it was almost treated

link |

as if anyone could be the next Einstein,

link |

anybody could be the next, I don't know,

link |

Hemingway, James Joyce.

link |

And so you're forcing an education on the populace

link |

and a rigorous deep education,

link |

like as opposed to kind of like,

link |

oh, we wanna make sure we teach

link |

to the weakest student in the class,

link |

which American systems can sometimes do

link |

because we don't wanna leave anyone behind.

link |

The Russian system was anyone can be the strongest student

link |

and we're gonna teach you the strongest student

link |

and we're going to pretend or force everybody,

link |

even the weakest student to be strong.

link |

And what that results in, it's obviously,

link |

this is what people talk about,

link |

is a huge amount of pressure.

link |

Like it's psychologically very difficult.

link |

This is why people struggle when they go to MIT,

link |

this very competitive environment.

link |

It can be very psychologically difficult,

link |

but at the same time,

link |

it's bringing out the best out of people.

link |

And that mathematics was certainly one of those things.

link |

And exactly what you're saying,

link |

which kind of clicked with me just now,

link |

as opposed to kind of a spelling bee in the United States,

link |

which I guess you spell, I'm horrible at this,

link |

but it's a competition about spelling,

link |

which I'm not sure, but you could argue

link |

it doesn't generalize well to the future skills.

link |

Mathematics, especially this kind of mathematics

link |

is essentially formalized competition of invention,

link |

of creating new ideas.

link |

And that generalizes really, really well.

link |

So that's quite brilliantly put.

link |

I didn't really think about that.

link |

So this is not just about the competition.

link |

This is about developing minds

link |

that will come to do some incredible stuff in the future.

link |

Yeah, actually, I want to respond

link |

to a couple of things there.

link |

The first one, this one, which is this notion

link |

of whether or not that is possible

link |

in a non authoritarian regime.

link |

And that's actually why I spent some of my efforts

link |

before the COVID thing,

link |

actually trying to work towards there.

link |

The reason is because if you think about it,

link |

let's say in America,

link |

lots of people are pretty serious

link |

about training very hard for football,

link |

or baseball, or basketball.

link |

Basketball is very, very accessible,

link |

but lots of people are doing that.

link |

Well, actually, I think that what was going on

link |

with the authoritarian thing was at least the message

link |

that was universally sent was being a good thinker

link |

and a creator of ideas is a good thing.

link |

There's no reason why that message can't be sent everywhere.

link |

And I think it actually should be.

link |

So that's the first thing.

link |

The second thing is what you commented about this thing

link |

about the generalizable skill

link |

and what could people do with Olympiads afterwards.

link |

So that's actually my interest in the whole thing.

link |

I don't just coach students how to do problems.

link |

In fact, I'm not even the best person for that.

link |

I'm not the best at solving these problems.

link |

There are other people who are much better

link |

at making problems and teaching people how to solve problems.

link |

In fact, when the Mathematical Association of America,

link |

which is the group which is in charge

link |

of the US participation in these Olympiads,

link |

when they were deciding whether or not to put me in

link |

back in 2013 as the head coach,

link |

I had a conversation with their executive director

link |

where I commented that we might do worse

link |

because my position was I don't,

link |

I mean, I actually didn't want to focus on winning.

link |

I said, if you're going to let me work

link |

with 60 very strong minds as picked through this system,

link |

because the coach works with these,

link |

gets to run a camp for these students.

link |

I said, I'm actually not going to define my success

link |

in terms of winning this contest.

link |

I said, I wanted to maximize the number of the students

link |

that I read about in the New York Times in 20 years.

link |

And the executive director

link |

of the Mathematical Association of America

link |

was fully in support of this

link |

because that's also how their philosophy is.

link |

So in America, the way we run this

link |

is we're actually not just training to win,

link |

even though the students are very good

link |

and they can win anyway.

link |

One reason, for example, I went and even did the COVID thing

link |

involving quite a few of them

link |

is so that hopefully some of them get ideas

link |

because in 20, 30 years, I won't have the energy

link |

or the insight to solve problems.

link |

We'll have another catastrophe.

link |

And hopefully some of these people will step up and do it.

link |

And ultimately have that longterm impact.

link |

I wonder if this is scalable to,

link |

because that's such a great metric for education,

link |

not how to get an A on the test, but how to have,

link |

how to be on the cover of New York Times

link |

for inventing something new.

link |

And do you think that's generalizable to education

link |

beyond just this particular Olympia?

link |

Like, even you saying this feels like a rare statement,

link |

almost like a radical statement as a goal for education.

link |

So actually the way I teach my classes at Carnegie Mellon,

link |

which I will admit right away is not equivalent

link |

to the average in the world,

link |

but it's already not just the top 60 in the country

link |

as picked by something.

link |

Let me just explain.

link |

I have exams in my class, which are 90% of the grade.

link |

So the exams are the whole thing,

link |

or most of the whole thing.

link |

And the way that I let students prepare for the exams

link |

is I show them all the problems I've ever given

link |

on the previous exams.

link |

And the exam that they will take is open notes.

link |

They can take all the notes they want

link |

on the previous problems.

link |

And the guarantee is that the exam problems this time

link |

will have no overlap with anything

link |

you have seen me give in the past,

link |

as well as no overlap with anything I taught in the class.

link |

So the entire exam is invention.

link |

But that's how I go, right?

link |

My point is I have explained to people when I teach you,

link |

I don't want you to have remembered a method I showed you.

link |

I want you to have learned enough about this area

link |

that if you face a new question,

link |

which I came up with the night before

link |

by thinking about like,

link |

what could I ask that I have never asked before?

link |

That's what the answer is.

link |

Aha, that's an exam problem.

link |

That's exactly what I do before the exam.

link |

And then that's what I want them to learn.

link |

And the first exam, usually people have a rough time

link |

because it's like, what kind of crazy class is this?

link |

The professor doesn't teach you anything for the exam.

link |

But then by the second or third,

link |

and by the time they finished the class,

link |

they have learned how to solve anything in the area.

link |

How to invent in that area, yeah.

link |

Can we walk back to the Mathematical Olympiad?

link |

What's the scoring and format like?

link |

And also what does it take to win?

link |

So the way it works is that each of the six students

link |

do the problems and there are six problems.

link |

All the problems are equally weighted.

link |

So each one's worth seven points.

link |

That means that your maximum score

link |

is six problems times seven points,

link |

which is the nice number of 42.

link |

And now the way that they're scored by the way

link |

is there's partial credit.

link |

So the question is asking you,

link |

explain why this weird fact is true.

link |

Okay, if you explain why you get seven points.

link |

If you make minor mistake, maybe you get six points.

link |

But if you don't succeed in explaining why,

link |

but you explain some other true fact,

link |

which is along the way of proving it,

link |

then you get partial credit.

link |

And actually now this is tricky

link |

because how do you score such a thing?

link |

It's not like the answer was 72

link |

and you wrote 71 and it's close, right?

link |

The answer is 72 and you wrote 36.

link |

Oh, but that's pretty close

link |

because maybe you're just off by it.

link |

By the way, they're not numerical anyway,

link |

but I'm just giving some numerical analog

link |

to the way the scoring might work.

link |

They're all essays.

link |

And that's where I guess I have some role

link |

as well as some other people

link |

who helped me in the US delegation for coaches.

link |

We actually debate with the country which is organizing it.

link |

The country which is organizing the Olympiad

link |

brings about 50 people to help judge the written solutions.

link |

And you schedule these half hour appointments

link |

where the delegation from one country

link |

sits down at a table like this.

link |

Opposite side is two or three people from the host country.

link |

And they're just looking over these exam papers

link |

saying, well, how many points is this worth

link |

based on some rubric that has been designed?

link |

And this is a negotiation process

link |

where we're not trying to bargain

link |

and get the best score we can.

link |

In fact, sometimes we go to this table

link |

and we will say, we think we want less than what you gave us.

link |

This is how our, these are our principles.

link |

If you give us too much, we say, no, you gave us too much.

link |

However, the reason why this is an interesting process

link |

is because if you can imagine every country

link |

which is participating has its own language.

link |

And so if you're trying to grade the Mongolian scripts

link |

and they're written in Mongolian,

link |

if you don't read Mongolian, which most people don't,

link |

then the coaches are explaining to you,

link |

this is what the student has written.

link |

It's actually quite interesting process.

link |

So it's almost like a jury.

link |

You have, in the American legal system,

link |

you have a jury that where they're deliberating,

link |

but unlike a jury, there's the members of the jury

link |

speaking different languages sometimes.

link |

Yes. That's fascinating.

link |

But I mean, it's hard to know what to do

link |

because it's probably really, really competitive.

link |

But your sense is that ultimately people,

link |

like how do you prevent manipulation here, right?

link |

Well, we just hope that it's not happening.

link |

So we write in English.

link |

Therefore, everything that the US does,

link |

everyone can look at.

link |

So it's very hard for me.

link |

It's very hard for you to manipulate.

link |

We don't manipulate.

link |

We only hope that other people aren't.

link |

But at the same time, as you see, our philosophy was,

link |

we want to use this as a way to develop general talent.

link |

And although we do this for the six people who go

link |

to the International Math Olympiad,

link |

we really want that everyone at any,

link |

touched at any stage of this process

link |

get some skills that can help to contribute more later.

link |

So I don't know if you can say something insightful

link |

but what do you think makes a really hard math problem

link |

on this Olympiad, maybe in the courses you teach

link |

What makes for a hard problem?

link |

You've seen, I'm sure, a lot of really difficult problems.

link |

What makes a hard problem?

link |

So I could quantify it by the number of leaps of insight

link |

of changes of perspective that are along the way.

link |

This is like a very theoretical computer science

link |

way of looking at it, okay?

link |

It's that each reframing of the problem

link |

and using of some tool,

link |

I actually call that a leap of insight.

link |

When you say, oh, wow, now I see,

link |

I should kind of put these plugs into those sockets

link |

like so, and suddenly I get to use that machine.

link |

Oh, but I'm not done yet.

link |

Now I need to do it again.

link |

Each such step is a large possible,

link |

large fan out in the search space.

link |

The number of these tells you the exponent.

link |

The base of the exponent is like how big,

link |

how many different possibilities you could try.

link |

And that's actually why,

link |

like if you have a three insight problem,

link |

that is not three times as hard as a one insight problem,

link |

because after you've made the one insight,

link |

it's not clear that that was the right track necessarily.

link |

Well, unless you're very into it.

link |

There's still a branching of possibility.

link |

You're saying there's problems like on the math Olympia

link |

that requires more than one insight?

link |

Those are the hard ones.

link |

And also I can tell you how you can tell.

link |

So this is how I also taught myself math

link |

when I was in college.

link |

So if you are taking a, not taught myself,

link |

I was taking classes, of course,

link |

but I was trying to read the textbook

link |

and I found out I was very bad at reading math textbooks.

link |

A math textbook has a long page of stuff that is all true,

link |

which after you read the page,

link |

you have no idea what you just read.

link |

This is just a good summary of a math textbook.

link |

Yeah, because it's not clear why anything was done that way.

link |

And yes, everything is true,

link |

but how the heck did anyone think of that?

link |

So the way that I taught myself math eventually was,

link |

the way I read a math textbook

link |

is I would look at the theorem statement.

link |

I would look at the length of the proof

link |

and then I would close the book

link |

and attempt to reproof it myself.

link |

The length of the proof is telling you

link |

the number of insights,

link |

because the length of the proof is linear

link |

in the number of insights.

link |

Each insight takes space.

link |

And if I know that it's a short proof,

link |

I know that there's only one insight.

link |

So when I'm doing my own way of solving the problem,

link |

like finding the proof,

link |

I quit if I have to do too many plugins.

link |

It's equivalent to a math contest.

link |

In a math contest I look,

link |

is it problem one, two, or three?

link |

That tells me how many insights there are.

link |

This is exactly what I did.

link |

Linear in the number.

link |

I think it's possible that that's true.

link |

Approximately, approximately.

link |

Approximately, yeah.

link |

I don't know if somebody out there

link |

is gonna try to formally prove this.

link |

Oh no, I mean, you're right.

link |

There are cases where maybe it's not quite linear,

link |

Well, some of it's notation too,

link |

and some of it is style and all those kinds of things,

link |

but within a textbook.

link |

Within the same book.

link |

Within the same book with the same.

link |

Within the same book on the same subject.

link |

This is what I was using.

link |

Because you know, if it's a two page proof,

link |

you just know this is gonna be insane, right?

link |

That's the scary thing about insights.

link |

You look like Andrew Wiles

link |

working on the Fermat's Last Theorem,

link |

is you don't know.

link |

Something seems like a good idea,

link |

and you have that idea,

link |

and it feels like this is a leap,

link |

like a totally new way to see it,

link |

but you have no idea if it's at all useful.

link |

Even if you think it's correct,

link |

you have no idea if this is like going to go down a path

link |

that's completely counterproductive

link |

or not productive at all.

link |

That's the crappy thing about invention,

link |

is like I have, I'm sure you do.

link |

I have a lot of really good ideas every single day,

link |

but like, and I'll go inside my head along them,

link |

along that little trajectory,

link |

but it could be just a total waste.

link |

And it's, you know what that feels like?

link |

It just feels like patience is required,

link |

not to get excited at any one thing.

link |

So I think this is interesting

link |

because you raised Andrew Wiles.

link |

He spent seven years attacking the same thing, right?

link |

And so I think that what attracts

link |

professional researchers to this

link |

is because even though it's very painful

link |

that you keep fighting with something,

link |

when you finally find the right insights

link |

and string them together,

link |

it feels really good, so.

link |

Well, there's also like short term,

link |

it feels good to, whether it's real or not,

link |

to pretend like you've solved something

link |

in the sense like you have an insight

link |

and there's a sense like this might be the insight

link |

So at least for me, I just enjoy that rush of positivity

link |

even though I know statistically speaking

link |

is probably going to be a dead end.

link |

I'm the same way, I'm the same way.

link |

In fact, that's how I know whether

link |

I might want to keep thinking about this general problem.

link |

It's like, if I still see that I'm getting some insights,

link |

I'm not at a dead end yet.

link |

But that's also where I learned something

link |

from my PhD advisor.

link |

Actually, he was a real big inspiration on my life.

link |

His name is Benny Sudakov.

link |

In fact, he grew up in the former Soviet Union.

link |

He was from Georgia, but he's an incredible person.

link |

But one thing I learned was choose the problems to work on

link |

that might matter if you succeed.

link |

Because that's why, for example, we dug into COVID.

link |

It was just, well, suppose we succeed

link |

in finding some interesting insight here.

link |

Well, it actually matters.

link |

That is worth a laugh.

link |

Yeah, and I think COVID, the way you're approaching COVID

link |

has two interesting possibilities.

link |

One, it might help with COVID or another pandemic,

link |

but two, I mean, just this whole network theory space,

link |

you might unlock some deep understanding

link |

about the interaction with human beings.

link |

That might have nothing to do with the pandemic.

link |

There's a space of possible impacts

link |

that may be direct or indirect.

link |

And the same thing is with Andrew Wiles's proof.

link |

I don't understand, but apparently the pieces of it

link |

are really impactful for mathematics,

link |

even if the main theorem is not.

link |

So along the way, the insights you have

link |

might be really powerful for unexpected reasons.

link |

So I like what you said.

link |

This is something that I learned from another friend of mine.

link |

He's a very famous researcher.

link |

All these people are more famous than I am.

link |

His name is Jacob Fox.

link |

He's Jacob Fox at Stanford.

link |

Also a very big inspiration for me.

link |

We were both grad students together at the same time.

link |

Well, most importantly,

link |

you're good at selecting good friends.

link |

Ah, yeah, well, that's the key.

link |

You gotta find good people to learn things from.

link |

But his thing was, he often said,

link |

if you solve a math problem and have this math proof,

link |

math problem for him is like a proof, right?

link |

So suppose you came up with this proof.

link |

He always asks, what have we learned from this

link |

that we could potentially use for something else?

link |

It's not just, did you solve the problem

link |

that was supposed to be famous?

link |

And is there something new in the course of solving this

link |

that you had to invent

link |

that we could now use as a tool elsewhere?

link |

Yeah, there's this funny effect

link |

where just looking at different fields

link |

where people discover parallels.

link |

They'll prove something, it'll be a totally new result.

link |

And then somebody later realizes

link |

this was already done 30 years ago

link |

in another discipline, in another way.

link |

And it's really interesting.

link |

Now, we did this offline

link |

in another illustration he showed to me.

link |

It's interesting to see the different perspectives

link |

It kind of points like there's just like

link |

very few novel ideas that everything else,

link |

that most of us are just looking at different perspective

link |

And it makes you wonder this old silly question

link |

that I have to ask you is,

link |

do you think mathematics is discovered or invented?

link |

Do you think we're creating new idea?

link |

Are we building a set of knowledge

link |

that's distinct from reality?

link |

Or are we actually like,

link |

is math almost like a shovel

link |

where we're digging to like this core set of truths

link |

that were always there all along?

link |

So I personally feel like it's discovered.

link |

But that's also because I guess the way that

link |

I like to choose what questions to work on

link |

are questions that maybe we'll get to learn something about

link |

I mean, I'm often attracted to questions

link |

that look simple, but are hard, right?

link |

And what could you possibly learn from that?

link |

Sort of like probably the attraction

link |

of Fermat's last theorem, as you mentioned,

link |

simple statement, why is it so hard?

link |

So I'm more on the discovered side.

link |

And I also feel like if we ever ran into

link |

an intelligent other species in the universe,

link |

probably if we compared notes,

link |

there might be some similarities between both of us

link |

realizing that pi is important.

link |

Because you might say, why, why humans,

link |

do humans like circles more than others?

link |

I think stars also like circles.

link |

I think planets like circles.

link |

They're not perfect circles,

link |

but nevertheless, the concept of a circle

link |

is just point and constant distance.

link |

Doesn't get any simpler than that.

link |

It's possible that like an alien species

link |

will have, depending on different cognitive capabilities

link |

and different perception systems,

link |

will be able to see things

link |

that are much different than circles.

link |

And so if it's discovered,

link |

it will still be pointing at a lot of same

link |

geometrical concepts, mathematical concepts,

link |

but it's interesting to think of how many things

link |

we would have to still align,

link |

not just based on notation, but based on understanding,

link |

like just like some basic mathematical concepts,

link |

like how much work is there going to be

link |

in trying to find a common language?

link |

I mean, this is, I think Stephen Wolfram and his son

link |

helped with the movie Arrival,

link |

like the developing an alien language,

link |

like how would aliens communicate with humans?

link |

because like math seems to be the most promising thing,

link |

but even like math,

link |

like how do you visualize mathematical ideas?

link |

It feels like there has to be an interactive component,

link |

just like we have a conversation.

link |

There has to be, this is something we don't,

link |

I think, think about often, which is like,

link |

with somebody who doesn't know anything about math,

link |

doesn't know anything about English

link |

or any other natural language,

link |

how would we describe,

link |

we talked offline about visual proofs.

link |

How would we, through visual proofs, have a conversation

link |

where we say something, here's the concept,

link |

the way we see it, does that make sense to you?

link |

And like, can you mess with that concept

link |

to make it sense for you?

link |

And then go back and forth in this kind of way.

link |

So purely through mathematics,

link |

I'm sure it's possible to have those kinds of experiments

link |

with like tribes on earth that don't,

link |

there's no common language.

link |

Through math, like draw a circle

link |

and see what they do with it, right?

link |

Do some of these visual proofs,

link |

like the summation of the odds and adds up to the squares.

link |

Yes, I wonder how difficult that is

link |

before one or the other species murders themselves.

link |

That's a good question.

link |

I hope that the curiosity for knowledge

link |

will overpower the greedy,

link |

this is back to our game theory thing,

link |

that the curiosity of like discovering math together

link |

will overpower the desire for resources

link |

and ultimately like willing to commit violence

link |

in order to gain those resources.

link |

I think as we progress,

link |

become more and more intelligent as a species,

link |

I'm hoping we would value more and more of the knowledge

link |

because we'll come up with clever ways

link |

to gain more resources so we won't be so resource starved.

link |

That's a hopeful message for when we finally meet aliens.

link |

The cool thing about the Math Olympiad,

link |

I don't know if you know work from Francois Chollet

link |

from Google, he came up with this kind of IQ test slash,

link |

it kind of has similar aspects to it

link |

that also the Math Olympiad does for AI.

link |

So he came up with these tests

link |

where they're very simple for humans,

link |

but very difficult for AI to illustrate exactly

link |

why we're just not good at seeing a totally new problem.

link |

Sorry, AI systems are not good at looking at a new problem

link |

that requires you to detect

link |

that there's a symmetry of some kind,

link |

or there's a pattern that hasn't seen before.

link |

The pattern is like obvious to us humans,

link |

but it's not so obvious to find that kind of,

link |

you're inventing a pattern that's there

link |

in order to then find a solution.

link |

I don't know if you can comment on that.

link |

If you can comment on, but from an AI perspective

link |

and from a math problem perspective,

link |

what do you think is intelligence?

link |

What do you think is the thing

link |

that allows us to solve that problem?

link |

And how hard is it to build a machine to do that?

link |

Asking for a friend.

link |

So I guess, you see,

link |

because if I just think of the raw search space, it's huge.

link |

That's why you can't do it.

link |

And if I think about what makes somebody

link |

good at doing these things, they have this heuristic sense.

link |

It's almost like a good chess player of saying,

link |

let's not keep analyzing down this way

link |

because there's some heuristic reason

link |

why that's a bad way to go.

link |

Where did they get that heuristic from?

link |

Now, that's a good question.

link |

Because that, if you asked them to explain to you,

link |

they could probably say something in words

link |

that sounds like it makes sense,

link |

but I'm guessing that's only a part

link |

of what's really going on in their brain

link |

of evaluating that position.

link |

You know what I mean?

link |

If you ask Gary Kasparov, what is good,

link |

or why is this position good, he will say something,

link |

but probably not approximating everything

link |

that's going on inside.

link |

So there's basically a function being computed,

link |

but it's hard to articulate what that function is.

link |

Now, the question is, could a computer get as good

link |

at computing these kinds of heuristic functions?

link |

I'm not enough of an expert to understand,

link |

but one bit of me has always been a little bit curious

link |

of whether or not the human brain has a particular tendency

link |

due to its wiring to come up with certain kinds of things,

link |

which is just natural due to the way

link |

that the topology of the neurons and whatever is there,

link |

for which if you tried to just build from scratch

link |

a computer to do it,

link |

would it naturally have different tendencies?

link |

This is just me being completely ignorant

link |

and just saying a few ideas.

link |

Well, this is a good thing that mathematics shows

link |

is we don't have to be,

link |

so math and physics or mathematical physics

link |

operates in a world that's different

link |

than our descendants of eight brains operate in.

link |

So it allows us to have multiple, many, many dimensions.

link |

It allows us to work on weird surfaces.

link |

I would like topology as a discipline is just weird to me.

link |

It's really complicated,

link |

but it allows us to work in that space,

link |

the differential geometry and all those kinds of things

link |

where it's totally outside of our natural day to day

link |

four dimensional experience,

link |

3D dimensional with time experience.

link |

So math gives me hope that we can discover

link |

the processes of intelligence outside the limited nature

link |

of our own human experiences.

link |

But you said that you're not an expert.

link |

It's kind of funny.

link |

I find that we know so little about intelligence

link |

that I honestly think like almost children are more expert

link |

at creating artificial intelligence systems than adults.

link |

I feel like we know so little,

link |

we really need to think outside the box.

link |

I found people should check out

link |

Francois Chollet's little exams,

link |

but even just solving math problems,

link |

I don't know if you've ever done this for yourself,

link |

but when you solve a math problem,

link |

you kind of then trace back and try to figure out

link |

where did that idea come from?

link |

Like what was I visualizing in my head?

link |

How did I start visualizing it that way?

link |

Why did I start rotating that cube in my head in that way?

link |

Like what is that?

link |

If I were to try to build a program that does that,

link |

where did that come from?

link |

So this is interesting.

link |

So I try to do this to teach middle school students

link |

how to learn how to create and think and invent.

link |

And the way I do it

link |

is there are these math competition problems

link |

and I'm working in collaboration

link |

with the people who run those.

link |

And I will turn on my YouTube live

link |

and for the first time,

link |

look at those questions and live solve them.

link |

The reason I do this is to let the middle school students

link |

and the high school students and the adults

link |

whoever wants to watch,

link |

just see what exactly goes on through someone's head

link |

as they go and attempt to invent what they need to do

link |

to solve the question.

link |

So I've actually thought about that.

link |

I think that, first of all, as a teacher,

link |

I think about that because whenever I want to explain

link |

to a student how to do something,

link |

I want to explain how it made sense,

link |

why it's intuitive to do the following things

link |

and why the wrong things are wrong.

link |

Not just why this one short fast way,

link |

well, why this is the right way, if that makes sense.

link |

So my point is I'm actually always thinking about that.

link |

Like how would you think about these things?

link |

And then I eventually decided the easiest way

link |

to expose this would just be to go live on YouTube

link |

and just say, I've never seen any of these questions before.

link |

Don't you get, that's anxiety inducing for me.

link |

Don't you get trapped in a kind of like little dead ends

link |

of confusion, even on middle school problems?

link |

Yes, that's what the comments are for.

link |

The live comments come in and students say, try this.

link |

It's actually pretty good.

link |

And I'll never get stuck.

link |

I mean, I'm willing to go on camera and say,

link |

guess what, Potion Dough can't do this.

link |

But then what ends up happening is you will then see

link |

how maybe somebody saying something and I look at the chat

link |

and I say, aha, that actually looks useful.

link |

Now that also shows how not all ideas,

link |

not all suggestions are the same power, if that makes sense.

link |

Because if I actually do get stuck,

link |

I'll go fishing through the chat, if you've got any ideas.

link |

I don't know if you can speak to this,

link |

but is there a moment for the middle school students,

link |

maybe high school as well,

link |

where there's like a turning point for them

link |

where they maybe fall in love with mathematics

link |

Is there something to be said about like discovering

link |

that moment and trying to grab them to get them

link |

to understand that mathematics is something,

link |

no matter what they wanna do in life

link |

could be part of their life?

link |

I actually do think that the middle school

link |

is exactly the right time

link |

because that's the place

link |

where your mathematical understanding

link |

gets just sophisticated enough

link |

that you can start doing interesting things.

link |

Because if you're early on and counting,

link |

I'm honestly not very good at teaching you new insights.

link |

My wife is pretty good at that.

link |

But somehow once you get to this part

link |

where you know what a fraction is

link |

and when you know how to add and how to multiply

link |

and what the area of a triangle is,

link |

at that point to me, the whole world opens up

link |

and you can start observing

link |

there are really nifty coincidences,

link |

the things that made the Greek mathematicians

link |

and the ancient mathematicians excited.

link |

Actually back then it was exciting

link |

to discover the Pythagorean theorem.

link |

It wasn't just homework.

link |

which discipline do you think

link |

has the most exciting coincidences?

link |

So is it geometry?

link |

Or is it calculus?

link |

Well, you see, you're asking me

link |

and I'm the guy who gets the most excited

link |

when the combinatorics shows up in the geometry.

link |

So it's the combinatorics in the geometry.

link |

So first of all, the nice thing about geometry,

link |

this is the same nice thing about computer vision

link |

So geometry, you can draw circles and triangles and stuff.

link |

So it naturally presents itself

link |

to the visual proof, right?

link |

But also the nice thing about geometry,

link |

I think for me is the earliest class,

link |

the earliest discipline where there's,

link |

that's most amenable to the exploration,

link |

the invention through proofs.

link |

The idea of proofs I think is most easily shown in geometry

link |

because it's so visual, I guess.

link |

So that to me is like,

link |

if I were to think about

link |

when I first fell in love with math, it would be geometry.

link |

And sadly enough, that's not used.

link |

Geometry only has a little,

link |

appears briefly in the journey of a student.

link |

And it kind of disappears.

link |

And not until much later,

link |

which there may be like differential geometry,

link |

I don't know where else it shows up.

link |

For me in computer science,

link |

like you could start to think about

link |

like computational geometry or even graph theory

link |

as a kind of geometry.

link |

You could start to think about it visually,

link |

although it's pretty tricky.

link |

But yeah, it was always,

link |

that was the most beautiful one.

link |

Everything else, I guess calculus can be kind of visual too.

link |

That can be pretty beautiful.

link |

But is there something you try to look for in the student

link |

to see like, how can I inspire them at this moment?

link |

Or is this like individual student to student?

link |

Is there something you could say there?

link |

I really think that every student

link |

can pick up all of this skill.

link |

I really do think so.

link |

I don't think it's something only for a few.

link |

And so if I'm looking for a student,

link |

actually oftentimes what I'm,

link |

if I'm looking at a particular student,

link |

how can we help you feel like

link |

you have the power to invent also?

link |

Because I think a lot of people

link |

are used to thinking about math

link |

as something where the teacher will show you what to do

link |

and then you will do it.

link |

So I think that the key is to show that they have some,

link |

let them see that they have some power to invent.

link |

And at that point,

link |

it's often starting by trying to give a question

link |

that they don't know how to do.

link |

You want to find these questions

link |

that they don't know how to do,

link |

that they can think about,

link |

and then they can solve.

link |

And then suddenly they say,

link |

my gosh, I've had a situation,

link |

I've had an experience where I didn't know what to do.

link |

And after a while, I did.

link |

Is there advice you can give on how to learn math

link |

for people, whether it's a middle school,

link |

whether it's somebody as an adult

link |

kind of gave up on math maybe early on?

link |

I actually think that these math competition problems,

link |

middle school and high school are really good.

link |

They're actually very hard.

link |

So if you haven't had this kind of experience before

link |

and you grab a middle school math competition problem

link |

from the state level,

link |

which is used to decide who represents the state

link |

in the country, in the United States, for example,

link |

those are pretty tricky.

link |

And even if you are a professional,

link |

maybe not doing mathematical things

link |

and you're not a middle school student, you'll struggle.

link |

So I find that these things really do teach you things

link |

by trying to work on these questions.

link |

Is there a Googleable term that you could use

link |

for the organization, for the state competitions?

link |

So there are a number of different ones

link |

that are quite popular.

link |

One of them is called Math Counts, M A T H C O U N T S.

link |

And that's a big tournament,

link |

which actually has a state level.

link |

There's also a mathleague.org,

link |

mathleague, L E A G U E dot org,

link |

also has this kind of tiered tournament structure.

link |

There's also the American math competitions, AMC 8.

link |

AMC also has AMC 10, that's for 10th grade and below

link |

These are all run by the Mathematical Association

link |

And these are all ways to find old questions.

link |

What about the daily challenges that you run?

link |

What are those about?

link |

But I mean, the difference was ours isn't,

link |

that one's not free.

link |

So I should actually probably be careful.

link |

The things that I've just mentioned are also not free.

link |

Not all of those things I mentioned just now

link |

Well, people can figure out what is free and what's not,

link |

but this is really nice to know what's out there.

link |

But can you speak a little bit to the daily challenges?

link |

So that's actually what we did when,

link |

I guess I was thinking about,

link |

how would I try to develop that skill in people

link |

if we had the power to architect the entire system ourselves?

link |

So that's called the daily challenge with Po Shan Luo.

link |

It's not free because that's actually how I pay

link |

for everything else I do.

link |

So that was the idea.

link |

But the concept was, aha, now let's invent from scratch.

link |

So if we're gonna go from scratch

link |

and we're gonna use technology,

link |

what if we made every single lesson

link |

something where first I say,

link |

hey, here's an interesting question.

link |

Recorded, of course, not live.

link |

But it's like, I say,

link |

hey, here's an interesting question.

link |

Why don't we think about this?

link |

But I know you don't know how to do it.

link |

and a minute later a hint pops on the screen.

link |

But you still think.

link |

And a minute later a big hint pops on the screen.

link |

And then finally, after the three minutes,

link |

hopefully you got some ideas you tried to answer.

link |

And then suddenly there's like this

link |

pretty extended explanation of,

link |

oh yeah, so here's like multiple different ways

link |

that you can do the question.

link |

And by accident, you also just learned this other concept.

link |

That's what we did.

link |

Is this targeted towards middle school students,

link |

high school students?

link |

It's targeted towards middle school students

link |

with competitions.

link |

But there's a lot of high school students

link |

who didn't do competitions in middle school

link |

where they would also learn how to think.

link |

If you can see the whole concept was,

link |

can we teach people how to think?

link |

How would you do that?

link |

You need to give people the chance to,

link |

on their own, invent without that kid in the front row

link |

answering every question in two seconds.

link |

And people can find it, I think, what daily.

link |

It's daily.potionlo.com.

link |

But if you go to find my website,

link |

you'll be able to find it.

link |

Can we zoom out a little bit in the,

link |

so day to day, week to week, month to month,

link |

year to year, what does the lifelong educational process

link |

look like, do you think?

link |

For yourself, but for me,

link |

what would you recommend in the world of mathematics

link |

or sort of as opposed to studying for a test,

link |

but just like lifelong expanding of knowledge

link |

in that skill for invention?

link |

I think I often articulate this as,

link |

can you always try to do more than you could do in the past?

link |

But that comes in many ways.

link |

And I will say it's great

link |

if one wants to build that with mathematics,

link |

but it's also great to use that philosophy

link |

with all other things.

link |

In fact, if I just think of myself, I just think,

link |

what do I know now that I didn't know a year ago

link |

or a month ago or a week ago?

link |

And not just know,

link |

but what do I have the capability of doing?

link |

And if you just have that attitude, it brings more.

link |

See, the thing is, there's also a habit,

link |

like it is a skill, like I've been using Anki,

link |

it's an app for helps you memorize things.

link |

And I've actually, a few months ago,

link |

started doing this daily of setting aside time

link |

to think about an idea that's outside of my work.

link |

Like, let's say, it's all over the place, by the way,

link |

but let's say politics, like gun control.

link |

Is it good to have a lot of guns or not in society?

link |

And just, I've set aside time every day,

link |

I do at least 10 minutes, but I try to do 30,

link |

where I think about a problem.

link |

And I kind of outline it for myself from scratch,

link |

from not looking anything up,

link |

just thinking about it, using common sense.

link |

And I think the practice of that is really important.

link |

It's the daily routine of it, it's the discipline of it.

link |

It's not just that I figured something out

link |

from thinking about gun control,

link |

it's more that that muscle is built too,

link |

it's that thinking muscle.

link |

So I'm kind of interested in, you know, math has,

link |

because especially because I've gotten specialized

link |

into machine learning,

link |

and because I love programming so much,

link |

I've lost touch with math a little bit

link |

to where I feel quite sad about it,

link |

and I want to fix that.

link |

Even just not math, like pure knowledge math,

link |

but math, like these middle school problems,

link |

the challenges, right?

link |

Is that something you see a person

link |

be able to do every single day,

link |

kind of just practice every single day for years?

link |

So I can give an answer to that,

link |

that gives a practical way you could do it,

link |

assuming you have kids.

link |

So, no, you can do it yourself.

link |

Step one, get kids.

link |

No, no, I'm just saying this

link |

because I'm just thinking out loud right now,

link |

what could I do to suggest?

link |

Because what I have noticed is that, for example,

link |

if you do have kids who are in elementary school

link |

or middle school, if you yourself go and look

link |

at those middle school math problems

link |

to think about interesting ways

link |

that you can teach your elementary school

link |

or middle school kid, it works.

link |

That's what my wife did.

link |

She never did any of those contests before,

link |

but now she knows quite a lot about them.

link |

And I didn't teach her anything.

link |

She just was messing around with them

link |

and taught herself all of that stuff.

link |

And that had the automatic daily.

link |

I'm always thinking, how do you make it practical, right?

link |

And the way to make it practical

link |

is if the timer on the automatically daily

link |

is that you are going to automatically daily

link |

do something with your own kid.

link |

Now it feeds back.

link |

And that includes the whole lesson

link |

that if you wanna learn something, you should teach it.

link |

Oh, I strongly believe that.

link |

I strongly believe that.

link |

So I currently don't have kids.

link |

So that's, maybe I should just get kids

link |

to help me with the math thing.

link |

But outside of that,

link |

I do want to do great math into daily practice.

link |

So I'll definitely check out the daily challenges

link |

and see, because what is it?

link |

Grant Sanderson, we talked about offline,

link |

the three blue and brown.

link |

He speaks to this as well,

link |

that his videos aren't necessarily,

link |

they don't speak to the thing that I'm referring to,

link |

which is the daily practice.

link |

They're more almost tools of inspiration.

link |

They kind of show you the beauty

link |

of a particular problem in mathematics,

link |

but they're not a daily ritual.

link |

And I'm in search of that daily ritual mathematics.

link |

It's not trivial to find,

link |

but I hope to find that

link |

because I think math gives you a perspective on the world

link |

that enriches everything else.

link |

So I like what you said about the daily also,

link |

because that's also one reason

link |

why I put my Carnegie Mellon class online.

link |

It's not every day.

link |

It's every other day.

link |

Semester is almost over.

link |

But the idea was, I guess my philosophy was,

link |

if I'm already doing the class,

link |

let's just put it there, right?

link |

But I do know that there are people

link |

who have been following it,

link |

who are not in my class at all,

link |

who have just been following it because,

link |

yes, it's combinatorics.

link |

And the value of that is you could,

link |

you don't really need to know calculus to follow it,

link |

if that makes sense.

link |

So it's actually something that people could follow.

link |

So again, and that one's free.

link |

So that one's just there on YouTube.

link |

Well, speaking of combinatorics,

link |

what is it, what do you find interesting,

link |

what do you find beautiful about combinatorics?

link |

So combinatorics to me is the study of things

link |

where they might be more finite and more discreet.

link |

What I mean is like, if I look at a network,

link |

actually a lot of times the combinatorics

link |

will boil down to something,

link |

and the combinatorics I think about

link |

might be something related to graphs or networks.

link |

And they're very discreet because if you have a node,

link |

it's not that you have 0.7 of a node

link |

and 0.3 of a node over there.

link |

It's that you've got one node,

link |

and then you jump one step to go to the next node.

link |

So that notion is different from say, calculus,

link |

which is very continuous,

link |

where you go and say, I have this speed,

link |

which is changing over time.

link |

And now what's the distance I've traveled?

link |

That's the notion of an integral,

link |

where you have to think of subdividing time

link |

into very, very small pieces.

link |

So the kinds of things that you do

link |

when you reason about these finite discreet structures

link |

often might be iterative, algorithmic, inductive.

link |

These are ideas where I go from one step to the next step

link |

and so on and make progress.

link |

I guess I actually personally like all kinds of math.

link |

My area of research just ended up in here

link |

because I met a really interesting PhD advisor,

link |

potential, that's honestly the reason

link |

I went into that direction.

link |

I met a really interesting guy.

link |

He seemed like he did good stuff, interesting stuff,

link |

and he looked like he cared about students.

link |

And I said, let me just go and learn whatever you do,

link |

even though my prior practice and preparation

link |

before my PhD was not combinatorics,

link |

but analysis, the continuous stuff.

link |

So the annoying thing about combinatorics

link |

and discreet stuff is it's often really difficult to solve

link |

from a sort of running time complexity perspective.

link |

Could you speak to the idea of complexity analysis

link |

of problems, do you find it useful, do you find it interesting?

link |

Do you find that lens of studying the difficulty

link |

of how difficult the computer science problem is

link |

a useful lens onto the world?

link |

Because if you want to make something practical

link |

which has large numbers of people using it,

link |

the computational complexity to me is almost question one.

link |

And that's, again, that's at the origin

link |

of when we started doing this stuff with disease control.

link |

From the very beginning, the deep questions

link |

that were running through my mind were,

link |

would we be able to support a large population

link |

with only one server?

link |

And if the answer is no, we can't start

link |

because I don't have enough money.

link |

Yeah, and there the question is very much

link |

linear time versus anything slower than linear time.

link |

As a very specific thing, you have a bunch

link |

of really interesting papers.

link |

If I could ask, maybe we could pull out some cool insights

link |

at the high level.

link |

Can you describe the data structure of a voting tree

link |

and what are some interesting results on it?

link |

You have a paper that I noticed on it.

link |

Yeah, so this is an example of, I guess,

link |

how in math we might say here's an interesting

link |

kind of a question that we just can't seem

link |

to understand enough about.

link |

Maybe there's something else going on here.

link |

And the way to describe this is you could imagine

link |

trying to hold elections where if you have

link |

only two candidates, that's kind of easy.

link |

You just run them against each other

link |

and see who gets more votes.

link |

But as you know, once you have more candidates,

link |

it's very difficult to decide who wins the election.

link |

And there's an entire voting theory around this.

link |

So a theoretical question became,

link |

what if you made like a system of runoffs,

link |

like a system of head to head contests,

link |

which is structured like a tree,

link |

almost looking like a circuit.

link |

I'm using that way of thinking because it's sort of like

link |

in electrical engineering or computer science,

link |

you might imagine having a bunch of leads

link |

that carry signal, which are going through AND gates

link |

and OR gates and whatnot.

link |

And you've managed to compute beautiful things.

link |

This is just from a purely abstract point of view.

link |

What if the inputs are candidates?

link |

And for every two candidates, it is known

link |

which of the candidates is more popular than the other.

link |

Now can you build some kind of a circuit board

link |

which says, first candidate number four

link |

will play against five and see who wins and so on.

link |

Okay, so now what would be a nice outcome, right?

link |

This is a general question of,

link |

could I make a big circuit board to feed an election into?

link |

Like maybe one nice outcome would be whoever wins

link |

at least is preferred over a lot of people.

link |

So for example, if you ran in 1,024 candidates,

link |

ideally we would like a guarantee that says

link |

that the winner beats a lot of people.

link |

Actually in any system where there are 1,024 candidates,

link |

there's always a candidate who beats

link |

at least 512 of the others.

link |

This is a mathematical fact

link |

that there's actually always a person who beats

link |

at least half of the other people.

link |

I'm trying to make sense of that mathematical fact.

link |

Is this supposed to be obvious?

link |

No, but I can explain it.

link |

The way it works is that, think of it this way.

link |

Every time I think, imagine I have all these candidates

link |

and everyone is competing,

link |

everyone is like compared with everyone else at some point.

link |

Well, think of it this way.

link |

Whenever there's a comparison, somebody gets a point.

link |

That's the one who is better than the other one.

link |

My claim is there's somebody whose score

link |

is at least half of how many other people there are.

link |

Yeah, I'm just trying to,

link |

like my intuition is very close to that being true,

link |

but it's beautiful.

link |

I didn't at first, that's not an obvious fact.

link |

And it feels like a beautiful fact.

link |

Well, let me explain it this way.

link |

Imagine that for every match,

link |

you didn't give one point, but you gave two points.

link |

You gave one point to each person.

link |

Now that's not what we're really doing.

link |

We really want to give one point to the winner of the match,

link |

but instead we'll just give two.

link |

If you gave two points to everyone on every matchup,

link |

actually everyone has the same number of points.

link |

And the number of points they get

link |

is how many other people there are.

link |

Does that sort of make sense?

link |

I'm just like saying.

link |

No, no, everything you're saying makes perfect sense.

link |

So the point is if for every comparison between two people,

link |

which I'm doing for every two people,

link |

I gave one point to each person,

link |

your score, everyone's score is the same.

link |

It's how many other people there are.

link |

Now we only make one change.

link |

For each matchup, you give one point only to the winner.

link |

So we're awarding half the points.

link |

So now the deal is if in the original situation,

link |

everyone's score was equal,

link |

which is how many other people there are.

link |

Now there's only half the number of points to go around.

link |

So what ends up happening is that

link |

there's always going to be,

link |

like the average number of points per person

link |

is going to be half of how many other people there are.

link |

And somebody is gonna be above average.

link |

Somebody is going to be above that.

link |

Yeah, this is this notion of expected value,

link |

that if I have a random variable,

link |

which has an expected value,

link |

there's going to be some possibility

link |

in the probability space

link |

where you're at least as big as the expected value.

link |

Yeah, when you describe it like that, it's obvious.

link |

But when you're first saying in this little circuit

link |

that there's going to be one candidate better than half,

link |

that's not obvious.

link |

Yeah, it's not obvious.

link |

Math, this is nice.

link |

Okay, so you have this,

link |

but ultimately you're trying to with a voting tree,

link |

I don't know if you're trying this,

link |

but to have a circuit that's like, that's small.

link |

Well, you'd like it to be small.

link |

That achieves the same kind of,

link |

I mean, the smaller it is,

link |

if we look at practically speaking,

link |

the lower the cost of running the election,

link |

of running through, of computing the circuit.

link |

But actually at this point,

link |

the reason the question was interesting

link |

is because there was no good guarantee

link |

that the winner of that circuit

link |

would have like have beaten a lot of people.

link |

Let me give an example.

link |

The best known circuit,

link |

when we started thinking about this,

link |

was the circuit called candidate one

link |

plays against candidate two,

link |

candidate three plays against four,

link |

and then the winners play against each other.

link |

And then by the way, five plays against six,

link |

seven against eight, the winners play against each other.

link |

You understand, it's like a giant binary tree.

link |

Yeah, it's a binary, like a balanced binary tree.

link |

It's a balanced binary tree.

link |

One, two, three, four, up to 1,024,

link |

everyone going up to find the winner.

link |

Well, you know what?

link |

There's a system in the world

link |

where it could just be

link |

that there's a candidate called number one,

link |

that just beats like 10 other people,

link |

just the 10 that they need to be on their way up

link |

and they lose to everyone else.

link |

But somehow they would get all the way up.

link |

My point is it is possible to outsmart that circuit

link |

in one weird way of the world,

link |

which makes that circuit a bad one

link |

because you want to say,

link |

I will use this circuit for all elections.

link |

And you might have a system of inputs that go in there

link |

where the winner only beat 10 other people,

link |

which is the people they had to beat on the way up.

link |

So you want to have a circuit where there's as many,

link |

like the final result is as strong as possible.

link |

And so what ideas do you have for that?

link |

So we actually only managed to improve it

link |

to square root of N.

link |

So if N is number of vertices,

link |

N over two would be the ideal.

link |

We got it to square root of N.

link |

Versus log base two.

link |

Well, that is halfway.

link |

It could be a lot.

link |

Could be a big improvement.

link |

So that's a, okay, cool.

link |

Is there something you can say with words

link |

about what kind of circuit, what that looks like?

link |

I can give an idea of one of the tools inside,

link |

but the actual execution ends up being more complicated.

link |

But one of the widgets inside this

link |

is building a system where you have like a candidate

link |

who plays, like one part of the whole huge, huge tree

link |

is that that same candidate, let's call them seven.

link |

Seven plays against somebody,

link |

let's make up some numbers.

link |

Let's call the others like letters.

link |

So seven plays against A.

link |

Seven's also gonna play against B separately.

link |

And the winners of each of those will play each other.

link |

By the way, seven's also gonna play C.

link |

Seven's gonna play D.

link |

And the winners are gonna play each other.

link |

And the winners are gonna play each other.

link |

We call this seven against all.

link |

Well, seven against like everyone from a bunch of.

link |

So there's some nice overlap between the matchups

link |

that somehow has a nice feature to it.

link |

Yes, and I can tell you the nice feature

link |

because if at the base of this giant tree,

link |

at the base of this giant circuit,

link |

like this is a widget.

link |

We build the things out of widgets.

link |

So I'm just describing one widget.

link |

But in the base of this widget,

link |

you have lots of things which are seven against someone,

link |

seven against someone, seven against someone.

link |

In fact, every matchup at the bottom

link |

is seven against someone.

link |

What that means is

link |

if seven actually beat everyone they were matched up against,

link |

well, seven would rise to the top.

link |

So one possibility is if you see a seven

link |

emerge from the top,

link |

you know that seven actually beat everyone

link |

they were against.

link |

On the other hand, if anyone else is on top,

link |

If F is on top, how did F get there?

link |

Well, F beat seven on the way at the beginning.

link |

So the point is the outcome of this circuit

link |

has a certain property.

link |

If you see a seven,

link |

you know that the seven actually beat a person

link |

but the seven actually beat a bazillion people.

link |

If you see anyone else,

link |

at least you know they beat seven.

link |

Yeah, then you can prove that it has a nice property.

link |

That's really interesting.

link |

Is there something you can say,

link |

perhaps going completely outside

link |

of what we're talking about,

link |

have mathematical ideas

link |

of improving the electoral process?

link |

No, I can't give you that one.

link |

I mean, is there, like, do you ever see it as,

link |

do you see as there being a lot of opportunities

link |

for improving how we vote?

link |

Like from your, I don't know if you saw parallels,

link |

but, you know, it seems like if,

link |

this actually kind of maps to your sort of COVID work,

link |

which is there's a network effect, right?

link |

It seems like we should be able to apply similar kind

link |

of effects of how we decide other things in our lives.

link |

And one of the big decisions we'll make

link |

is who represents us in government.

link |

Do you ever think about like mathematically

link |

about those kinds of systems?

link |

I think a little bit about those,

link |

because where I went to college,

link |

the way we voted for student government

link |

was based on this, is it called ranked choice?

link |

Where you eliminate the bottom

link |

and there was runoff elections.

link |

So that was the first time I ever saw that.

link |

And I thought that made sense.

link |

The only problem is it doesn't seem so easy

link |

to get something that makes sense adopted

link |

as the new voting system.

link |

That's a whole nother, that's not a math solution.

link |

That's a, well, it's math in the sense that it's game theory.

link |

So you have to come up with incentive,

link |

it's mechanism design.

link |

You have to figure out how to trick us

link |

despite our basic human nature

link |

to adopt solutions that are better.

link |

That's a whole nother conversation, I think.

link |

Can you just, cause it sounded really cool,

link |

talk a little bit about stochastic coalescence

link |

and you have a paper on showing that,

link |

so you could describe what it is,

link |

but I guess it's a super linear, super logarithmic time

link |

and you came up with some kind of trick

link |

that make it faster.

link |

Can you just talk about it a little bit?

link |

Yeah, so this was something which came up

link |

when I was at Microsoft Research for a summer.

link |

And I'm putting that context because that shows

link |

that it has some practical motivation at some point.

link |

Actually, I think it's still.

link |

It doesn't need to.

link |

It doesn't need to.

link |

It can be beautiful and it's all right.

link |

Yeah, so the easiest way to describe this is

link |

suppose you got like a big crowd of people

link |

and everybody knows how many hours of sleep

link |

they got last night.

link |

And you wanna know how many total hours of sleep

link |

were gotten by this big crowd of people.

link |

At the beginning, you might say,

link |

that sounds like a linear time algorithm

link |

of saying, hey, how many hours you got?

link |

But there's a way to do this

link |

if you remember that there are people

link |

and they presumably know how to add.

link |

You could make a distributed algorithm

link |

to make this happen.

link |

For example, while we're thinking of these trees,

link |

imagine you had 1,024 people.

link |

If you could just say, hey, person number one

link |

and person number two, you will add your hours of sleep.

link |

Person number two will go away

link |

and person number one is gonna remember the sum.

link |

Person three and four add up

link |

and person three takes charge of remembering it.

link |

Person four goes away.

link |

Now this like person one knows the sum of these two.

link |

Person three knows the sum of those two.

link |

You see what I mean?

link |

You're going up this tree,

link |

same tree that we talked about earlier.

link |

Built up a tree from the bottom up.

link |

Yeah, build up a tree from the bottom up.

link |

And the beautiful thing is

link |

since everyone's doing stuff in parallel,

link |

the amount of time it takes to get the total sum

link |

is actually just the number of layers in the tree,

link |

So now that's logarithmic time

link |

to add up the number of hours that people slept today.

link |

There's only one problem.

link |

How do you decide who's person number one

link |

and person number two?

link |

So if, for example, you just went out into the downtown

link |

and said, hey, get these thousand people, go.

link |

Well, if you're gonna go and say,

link |

and by the way, you're one and you're two and you're three,

link |

that's linear time.

link |

So now the question is how to do this

link |

in a distributed way.

link |

And there were some people who proposed

link |

a very elegant algorithm and they wanted to analyze it.

link |

So I came in onto the analyze side,

link |

but the elegant algorithm was like this.

link |

It was like, well, we don't actually know

link |

what this big tree is.

link |

There isn't any big tree.

link |

So what's gonna happen is first,

link |

everyone is going to decide right now.

link |

Oh, one important thing.

link |

Everyone is going to,

link |

at the very beginning of the whole game,

link |

they will have delegated responsibility to themselves

link |

as the one who knows the sum so far.

link |

So the point is there's gonna be,

link |

people are all gonna have like a pointer which says,

link |

you are the one who knows my,

link |

you've taken care of my ticket, my number.

link |

You're the representative for this particular piece

link |

And at the very beginning, you're your own representative.

link |

The thing has to start simple, right?

link |

So at the beginning, you're your own representative.

link |

You're pointing to yourself, got it.

link |

And now the way this works is that at every time step,

link |

someone blares a ding dong on the town clock or whatever.

link |

And each person flips a coin themselves to decide,

link |

am I going to hunt for somebody to give my number to

link |

and let them represent me?

link |

Or am I going to sit here and wait for someone to come?

link |

Well, they flipped their coin.

link |

Some of the people start asking other people saying,

link |

hey, I would like you to be my representative.

link |

Here is my number.

link |

But the problem is that there's limited bandwidth

link |

of the people who are getting asked.

link |

It's like, you can't get,

link |

you can't go out to prom with five people.

link |

But this is not what we're doing.

link |

We're adding numbers, okay?

link |

But you can only add one number.

link |

So the person who has suddenly gotten asked

link |

by all these people,

link |

well, they'll have to decide who they're going

link |

And they randomly just choose one.

link |

When they randomly choose one,

link |

all the others are rejected

link |

and they don't get to delegate anything in that round.

link |

But now if this person has absorbed this one who said,

link |

okay, here, you take charge of my number.

link |

This person now updates their pointer.

link |

And this person adds the two numbers.

link |

That was the first round.

link |

In the next round, when they do the coin flipping,

link |

this person doesn't flip anymore

link |

because they're just delegating.

link |

It's that anyone who has the pointers themselves,

link |

that's like a person who is in charge

link |

of some number of informations,

link |

they flip the coin to decide,

link |

should I find other people who are agents?

link |

Or should I wait for people to ask me?

link |

This is somebody else's idea.

link |

And then now the idea is, okay,

link |

if you just keep doing this process,

link |

what ends up happening?

link |

Oh yeah, and also by the way,

link |

if you decide that you want to go reach out

link |

to other people, here's the catch.

link |

When you're one of these agents saying,

link |

okay, I'm going to go look for someone.

link |

You have no idea who in this crowd is an agent

link |

or somebody who delegated it to someone else.

link |

You just pick a random person.

link |

When you pick the random person,

link |

if it lands on someone and the person says,

link |

oh, I actually delegated it to someone,

link |

then you follow the point.

link |

You walk up the delegation chain.

link |

Walk up the delegation chain.

link |

And you can do like path compression in the algorithm

link |

to make it so you don't consistently

link |

do lots of walking up.

link |

But the bottom line is that what ends up happening

link |

is that you end up reaching out.

link |

Whenever you're one of the ones reaching out,

link |

you can think of it as each agent is responsible

link |

for some number of people.

link |

It's almost like they're the leader of a bunch.

link |

As the process is evolving, you have these lumps.

link |

Each lump has an agent.

link |

And when the agent reaches out,

link |

they reach out to another lump

link |

where the probability of them hitting that lump

link |

is proportional to the size of the lump.

link |

That is the one funny thing about this process.

link |

This is not that they can reach out

link |

to a uniformly random lump

link |

where every lump has the same chance

link |

of getting reached out to.

link |

The bigger the lump is,

link |

the more likely it is that you end up reaching that lump.

link |

Which is a problem?

link |

Let me explain why that's a problem.

link |

Because you see, you're hoping

link |

that this has a small number of steps,

link |

but here's a bad situation that could happen.

link |

Imagine if you had like,

link |

there are n people that you're adding up.

link |

Imagine that you have exactly square root of n lumps left,

link |

of which almost all of them are just one person

link |

who's still their own boss, their own manager.

link |

Except one giant one.

link |

Now what's gonna happen?

link |

It's gonna be a huge bottleneck

link |

because every round the giant one

link |

can only absorb one of the others.

link |

And now you suddenly have time

link |

which is about square root of n.

link |

The square root of n is chosen

link |

because that is one where the lumps are such

link |

that you really are limited by this large one

link |

slowly sucking up the rest of them.

link |

So the heart of the question became,

link |

well, but is that just so unusual

link |

that it doesn't usually happen?

link |

Because remember you start with everyone

link |

just being independent.

link |

It's like a lot of lumps of size one.

link |

How naturally do the big lumps emerge?

link |

And so what that heart of the proof was,

link |

was showing that that was a joint work with Eyal Lubezki.

link |

That one was showing that actually in that thing

link |

the lumps do kind of get out of whack.

link |

And so it's not the purely logarithmic number of steps.

link |

But if you make one very slight change,

link |

which is if you are one of the agents

link |

and you have just been propositioned,

link |

possibly relayed along by a couple of different people.

link |

If you just say, don't take a random one,

link |

but accept the smallest lump.

link |

That actually does enough to even the whole economy.

link |

Distributes the lump size.

link |

I mean, yeah, it's fascinating how

link |

with the distributed algorithms,

link |

a little adjustment can make all the difference

link |

Actually, by the way, this does,

link |

back to our voting conversation,

link |

this makes me think of like,

link |

these networking systems are so fascinating to study.

link |

They immediately spring to mind ideas

link |

of how to have representation.

link |

Like maybe as opposed to me voting for a president,

link |

I want to vote for like,

link |

for you, Paul, to represent me,

link |

maybe on a particular issue.

link |

And then you will delegate that further.

link |

And then we naturally construct those kinds of networks

link |

because that feels like I can have a good conversation

link |

with you and figure out that you know what you're doing

link |

and I can delegate it to you.

link |

And in that way, construct a representative government,

link |

a representative decision maker.

link |

That feels really nice as opposed to like us,

link |

like a tree of height one or something,

link |

where it's like everybody's just,

link |

it feels like there's a lot of room for layers

link |

of representation to form organically from the bottom up.

link |

I wonder if there are systems like that.

link |

This is the cool thing about the internet

link |

and the digital space where we're so well connected,

link |

just like with the Novid app to distribute information

link |

about the spread of the disease.

link |

We can in the same way, in a distributed sense,

link |

form anything like any kind of knowledge bases

link |

that are formed in a decentralized way

link |

and in a hierarchical way,

link |

as opposed to sort of old way

link |

where there is no mechanism for large scale,

link |

fast distributed transactional information.

link |

This is really interesting.

link |

This is where almost like network graph theory,

link |

becomes practical.

link |

Most of that exciting work was done in the 20th century,

link |

but most of the application will be in the 21st,

link |

which is cool to think about.

link |

Let me ask the most ridiculous question.

link |

You think P equals NP?

link |

I mean, I would say,

link |

I know there are enough people who have very strong interest

link |

in trying to show that it is.

link |

I'm talking about government agencies.

link |

For security purposes.

link |

For security purposes.

link |

And most computer scientists,

link |

we should say believe that P equals NP.

link |

My question almost like,

link |

this is back to our aliens discussion.

link |

You want to think outside the box,

link |

the low probability event,

link |

what is the world,

link |

what kind of discoveries would lead us to prove

link |

that P does not equal to NP?

link |

Like there could be giant misunderstandings

link |

or gaps in our knowledge about computer science,

link |

about theoretical computer science, about computation,

link |

which allow us to think like flatten all problems.

link |

Yeah, so I don't know the answer to this question.

link |

I think it's very interesting, but I actually,

link |

I know, let's put it this way.

link |

By being at Carnegie Mellon

link |

and being around the theoretical computer scientists,

link |

I know enough about what I don't know to say.

link |

I'm the wrong person to answer this question.

link |

Well, Scott Aaronson, who's now here at UT Austin,

link |

he used to be at MIT,

link |

puts the probability of P not equals to NP at 3%.

link |

I always love it when you ask,

link |

it's very rare in science and academics

link |

because most folks are humble

link |

in the face of the mystery,

link |

the uncertainty of everything around us.

link |

To have both the humor and the guts to say like,

link |

what are the chance that there's aliens in our galaxy,

link |

intelligent alien civilizations?

link |

As opposed to saying, I don't know, it could be zero.

link |

It could be, depending on the fact, you're saying it's 2.5%.

link |

There's something very pleasant about just having,

link |

it's the number thing.

link |

It's powered to the number.

link |

It's just like 42.

link |

It's like, why 42?

link |

I don't know, but it's a powerful number.

link |

And then everything,

link |

this is the power of human psychology

link |

is once you have the number 42,

link |

it's not that the number has meaning,

link |

but because it's placed in a book with humor around it,

link |

it has the meme effect of actually creating reality.

link |

I mean, you could say that 42 has a strong contribution

link |

of helping us colonize Mars

link |

because it created,

link |

it gave the whatever existential crisis to many of us,

link |

including Elon Musk when he was young,

link |

reading a book like that.

link |

And then now 42 is now part of his humor

link |

that he doesn't shut up about,

link |

it's constantly joking about.

link |

And that humor is spreading through our minds

link |

and somehow this like silly number just had an effect.

link |

In that same way, after Scott told me like the 3% chance,

link |

it's stuck in my head.

link |

And I think it's been having a ripple effect

link |

in everybody else.

link |

The believing that P is not equal to NP,

link |

Scott almost as a joke saying it's 3%

link |

is actually motivating a large number of researchers