back to index

Tomaso Poggio: Brains, Minds, and Machines | Lex Fridman Podcast #13


small model | large model

link |
00:00:00.000
The following is a conversation with Tommaso Poggio.
link |
00:00:02.920
He's a professor at MIT and is a director of the Center
link |
00:00:06.200
for Brains, Minds, and Machines.
link |
00:00:08.360
Cited over 100,000 times, his work
link |
00:00:11.640
has had a profound impact on our understanding
link |
00:00:14.560
of the nature of intelligence in both biological and artificial
link |
00:00:18.600
neural networks.
link |
00:00:19.920
He has been an advisor to many highly impactful researchers
link |
00:00:23.840
and entrepreneurs in AI, including
link |
00:00:26.120
Demis Hassabis of DeepMind, Amnon Shashua of Mobileye,
link |
00:00:29.880
and Christoph Koch of the Allen Institute for Brain Science.
link |
00:00:34.120
This conversation is part of the MIT course
link |
00:00:36.400
on artificial general intelligence
link |
00:00:38.120
and the artificial intelligence podcast.
link |
00:00:40.240
If you enjoy it, subscribe on YouTube, iTunes,
link |
00:00:42.760
or simply connect with me on Twitter
link |
00:00:44.640
at Lex Friedman, spelled F R I D.
link |
00:00:48.000
And now, here's my conversation with Tommaso Poggio.
link |
00:00:52.480
You've mentioned that in your childhood,
link |
00:00:54.520
you've developed a fascination with physics, especially
link |
00:00:57.560
the theory of relativity.
link |
00:00:59.720
And that Einstein was also a childhood hero to you.
link |
00:01:04.520
What aspect of Einstein's genius, the nature of his genius,
link |
00:01:09.000
do you think was essential for discovering
link |
00:01:11.280
the theory of relativity?
link |
00:01:12.960
You know, Einstein was a hero to me,
link |
00:01:15.920
and I'm sure to many people, because he
link |
00:01:18.680
was able to make, of course, a major, major contribution
link |
00:01:23.440
to physics with simplifying a bit just a gedanken experiment,
link |
00:01:31.920
a thought experiment, you know, imagining communication
link |
00:01:37.960
with lights between a stationary observer
link |
00:01:41.560
and somebody on a train.
link |
00:01:43.240
And I thought, you know, the fact
link |
00:01:47.960
that just with the force of his thought, of his thinking,
link |
00:01:51.320
of his mind, he could get to something so deep
link |
00:01:55.640
in terms of physical reality, how time
link |
00:01:58.760
depend on space and speed, it was something
link |
00:02:02.680
absolutely fascinating.
link |
00:02:04.120
It was the power of intelligence,
link |
00:02:06.720
the power of the mind.
link |
00:02:08.440
Do you think the ability to imagine,
link |
00:02:11.120
to visualize as he did, as a lot of great physicists do,
link |
00:02:15.200
do you think that's in all of us human beings?
link |
00:02:18.640
Or is there something special to that one particular human
link |
00:02:21.840
being?
link |
00:02:22.880
I think, you know, all of us can learn and have, in principle,
link |
00:02:30.480
similar breakthroughs.
link |
00:02:33.200
There are lessons to be learned from Einstein.
link |
00:02:37.200
He was one of five PhD students at ETA,
link |
00:02:42.720
the Eidgenössische Technische Hochschule in Zurich,
link |
00:02:47.560
in physics.
link |
00:02:48.520
And he was the worst of the five,
link |
00:02:50.800
the only one who did not get an academic position when
link |
00:02:55.360
he graduated, when he finished his PhD.
link |
00:02:57.960
And he went to work, as everybody knows,
link |
00:03:01.080
for the patent office.
link |
00:03:02.480
And so it's not so much that he worked for the patent office,
link |
00:03:05.840
but the fact that obviously he was smart,
link |
00:03:08.720
but he was not a top student, obviously
link |
00:03:11.760
was the anti conformist.
link |
00:03:13.560
He was not thinking in the traditional way that probably
link |
00:03:17.480
his teachers and the other students were doing.
link |
00:03:20.040
So there is a lot to be said about trying
link |
00:03:23.880
to do the opposite or something quite different from what
link |
00:03:29.800
other people are doing.
link |
00:03:31.080
That's certainly true for the stock market.
link |
00:03:32.960
Never buy if everybody's buying.
link |
00:03:36.800
And also true for science.
link |
00:03:38.600
Yes.
link |
00:03:39.680
So you've also mentioned, staying
link |
00:03:42.520
on the theme of physics, that you were excited at a young age
link |
00:03:47.600
by the mysteries of the universe that physics could uncover.
link |
00:03:51.800
Such, as I saw mentioned, the possibility of time travel.
link |
00:03:56.760
So the most out of the box question,
link |
00:03:58.960
I think I'll get to ask today, do you
link |
00:04:00.600
think time travel is possible?
link |
00:04:03.440
Well, it would be nice if it were possible right now.
link |
00:04:07.800
In science, you never say no.
link |
00:04:12.800
But your understanding of the nature of time.
link |
00:04:15.040
Yeah.
link |
00:04:15.920
It's very likely that it's not possible to travel in time.
link |
00:04:22.360
We may be able to travel forward in time
link |
00:04:26.000
if we can, for instance, freeze ourselves or go
link |
00:04:31.880
on some spacecraft traveling close to the speed of light.
link |
00:04:37.640
But in terms of actively traveling, for instance,
link |
00:04:40.440
back in time, I find probably very unlikely.
link |
00:04:45.320
So do you still hold the underlying dream
link |
00:04:49.160
of the engineering intelligence that
link |
00:04:52.320
will build systems that are able to do such huge leaps,
link |
00:04:56.800
like discovering the kind of mechanism that would be
link |
00:05:01.120
required to travel through time?
link |
00:05:02.600
Do you still hold that dream or echoes of it
link |
00:05:05.600
from your childhood?
link |
00:05:07.080
Yeah.
link |
00:05:08.640
I don't think whether there are certain problems that probably
link |
00:05:12.400
cannot be solved, depending what you believe
link |
00:05:16.840
about the physical reality, like maybe totally impossible
link |
00:05:21.440
to create energy from nothing or to travel back in time,
link |
00:05:27.720
but about making machines that can think as well as we do
link |
00:05:35.920
or better, or more likely, especially
link |
00:05:38.680
in the short and midterm, help us think better,
link |
00:05:42.400
which is, in a sense, is happening already
link |
00:05:44.280
with the computers we have.
link |
00:05:46.480
And it will happen more and more.
link |
00:05:48.440
But that I certainly believe.
link |
00:05:50.000
And I don't see, in principle, why computers at some point
link |
00:05:55.480
could not become more intelligent than we are,
link |
00:05:59.440
although the word intelligence is a tricky one
link |
00:06:03.640
and one we should discuss.
link |
00:06:05.840
What I mean with that.
link |
00:06:08.040
Intelligence, consciousness, words like love,
link |
00:06:13.200
all these need to be disentangled.
link |
00:06:16.800
So you've mentioned also that you believe
link |
00:06:18.760
the problem of intelligence is the greatest problem
link |
00:06:22.240
in science, greater than the origin of life
link |
00:06:24.480
and the origin of the universe.
link |
00:06:27.200
You've also, in the talk I've listened to,
link |
00:06:30.760
said that you're open to arguments against you.
link |
00:06:34.880
So what do you think is the most captivating aspect
link |
00:06:40.400
of this problem of understanding the nature of intelligence?
link |
00:06:43.320
Why does it captivate you as it does?
link |
00:06:47.440
Well, originally, I think one of the motivation
link |
00:06:51.600
that I had as, I guess, a teenager when I was infatuated
link |
00:06:56.440
with theory of relativity was really
link |
00:06:59.280
that I found that there was the problem of time and space
link |
00:07:05.960
and general relativity.
link |
00:07:07.960
But there were so many other problems
link |
00:07:10.000
of the same level of difficulty and importance
link |
00:07:13.720
that I could, even if I were Einstein,
link |
00:07:16.640
it was difficult to hope to solve all of them.
link |
00:07:19.600
So what about solving a problem whose solution allowed
link |
00:07:24.800
me to solve all the problems?
link |
00:07:26.640
And this was, what if we could find the key to an intelligence
link |
00:07:33.280
10 times better or faster than Einstein?
link |
00:07:37.040
So that's sort of seeing artificial intelligence
link |
00:07:40.160
as a tool to expand our capabilities.
link |
00:07:43.200
But is there just an inherent curiosity in you
link |
00:07:47.920
in just understanding what it is in here
link |
00:07:52.120
that makes it all work?
link |
00:07:54.360
Yes, absolutely, you're right.
link |
00:07:55.760
So I started saying this was the motivation when
link |
00:07:59.320
I was a teenager.
link |
00:08:00.560
But soon after, I think the problem of human intelligence
link |
00:08:07.160
became a real focus of my science and my research
link |
00:08:15.160
because I think for me, the most interesting problem
link |
00:08:22.520
is really asking who we are.
link |
00:08:28.120
It's asking not only a question about science,
link |
00:08:31.720
but even about the very tool we are using to do science, which
link |
00:08:36.000
is our brain.
link |
00:08:37.920
How does our brain work?
link |
00:08:39.880
From where does it come from?
link |
00:08:42.120
What are its limitations?
link |
00:08:43.720
Can we make it better?
link |
00:08:46.160
And that, in many ways, is the ultimate question
link |
00:08:50.000
that underlies this whole effort of science.
link |
00:08:54.400
So you've made significant contributions
link |
00:08:56.360
in both the science of intelligence
link |
00:08:58.240
and the engineering of intelligence.
link |
00:09:02.160
In a hypothetical way, let me ask,
link |
00:09:05.000
how far do you think we can get in creating intelligence
link |
00:09:07.960
systems without understanding the biological,
link |
00:09:12.080
the understanding how the human brain creates intelligence?
link |
00:09:15.800
Put another way, do you think we can
link |
00:09:17.200
build a strong AI system without really getting at the core
link |
00:09:22.080
understanding the functional nature of the brain?
link |
00:09:25.240
Well, this is a real difficult question.
link |
00:09:29.920
We did solve problems like flying
link |
00:09:35.280
without really using too much our knowledge
link |
00:09:40.680
about how birds fly.
link |
00:09:44.720
It was important, I guess, to know that you could have
link |
00:09:48.680
things heavier than air being able to fly, like birds.
link |
00:09:56.760
But beyond that, probably we did not learn very much, some.
link |
00:10:02.560
The Brothers Wright did learn a lot of observation
link |
00:10:06.800
about birds and designing their aircraft.
link |
00:10:12.120
But you can argue we did not use much of biology
link |
00:10:16.000
in that particular case.
link |
00:10:17.920
Now, in the case of intelligence,
link |
00:10:20.720
I think that it's a bit of a bet right now.
link |
00:10:28.920
If you ask, OK, we all agree we'll get at some point,
link |
00:10:36.280
maybe soon, maybe later, to a machine that
link |
00:10:39.480
is indistinguishable from my secretary,
link |
00:10:42.400
say, in terms of what I can ask the machine to do.
link |
00:10:47.600
I think we'll get there.
link |
00:10:49.000
And now the question is, you can ask people,
link |
00:10:51.960
do you think we'll get there without any knowledge
link |
00:10:54.240
about the human brain?
link |
00:10:57.000
Or that the best way to get there
link |
00:10:59.760
is to understand better the human brain?
link |
00:11:02.560
OK, this is, I think, an educated bet
link |
00:11:05.960
that different people with different backgrounds
link |
00:11:09.080
will decide in different ways.
link |
00:11:11.720
The recent history of the progress
link |
00:11:14.440
in AI in the last, I would say, five years or 10 years
link |
00:11:18.920
has been that the main breakthroughs,
link |
00:11:23.760
the main recent breakthroughs, really start from neuroscience.
link |
00:11:32.160
I can mention reinforcement learning as one.
link |
00:11:35.800
It's one of the algorithms at the core of AlphaGo,
link |
00:11:41.040
which is the system that beat the kind of an official world
link |
00:11:45.200
champion of Go, Lee Sedol, two, three years ago in Seoul.
link |
00:11:52.760
That's one.
link |
00:11:53.760
And that started really with the work of Pavlov in 1900,
link |
00:12:00.920
Marvin Minsky in the 60s, and many other neuroscientists
link |
00:12:05.800
later on.
link |
00:12:07.720
And deep learning started, which is at the core, again,
link |
00:12:12.560
of AlphaGo and systems like autonomous driving
link |
00:12:17.720
systems for cars, like the systems that Mobileye,
link |
00:12:22.520
which is a company started by one of my ex postdocs,
link |
00:12:25.600
Amnon Shashua, did.
link |
00:12:28.480
So that is at the core of those things.
link |
00:12:30.720
And deep learning, really, the initial ideas
link |
00:12:34.520
in terms of the architecture of these layered
link |
00:12:37.120
hierarchical networks started with work of Torsten Wiesel
link |
00:12:43.160
and David Hubel at Harvard up the river in the 60s.
link |
00:12:47.800
So recent history suggests that neuroscience played a big role
link |
00:12:53.240
in these breakthroughs.
link |
00:12:54.320
My personal bet is that there is a good chance they continue
link |
00:12:58.720
to play a big role.
link |
00:12:59.880
Maybe not in all the future breakthroughs,
link |
00:13:01.840
but in some of them.
link |
00:13:03.280
At least in inspiration.
link |
00:13:05.000
At least in inspiration, absolutely, yes.
link |
00:13:07.320
So you studied both artificial and biological neural networks.
link |
00:13:12.160
You said these mechanisms that underlie deep learning
link |
00:13:17.080
and reinforcement learning.
link |
00:13:19.760
But there is nevertheless significant differences
link |
00:13:23.880
between biological and artificial neural networks
link |
00:13:26.080
as they stand now.
link |
00:13:27.280
So between the two, what do you find
link |
00:13:30.800
is the most interesting, mysterious, maybe even
link |
00:13:33.080
beautiful difference as it currently
link |
00:13:35.560
stands in our understanding?
link |
00:13:37.800
I must confess that until recently, I
link |
00:13:41.560
found that the artificial networks, too simplistic
link |
00:13:46.040
relative to real neural networks.
link |
00:13:49.720
But recently, I've been starting to think that, yes,
link |
00:13:53.360
there is a very big simplification of what
link |
00:13:57.440
you find in the brain.
link |
00:13:59.040
But on the other hand, they are much closer
link |
00:14:03.000
in terms of the architecture to the brain
link |
00:14:07.080
than other models that we had, that computer science used
link |
00:14:11.280
as model of thinking, which were mathematical logics, LISP,
link |
00:14:16.560
Prologue, and those kind of things.
link |
00:14:19.480
So in comparison to those, they're
link |
00:14:21.520
much closer to the brain.
link |
00:14:23.320
You have networks of neurons, which
link |
00:14:26.360
is what the brain is about.
link |
00:14:27.840
And the artificial neurons in the models, as I said,
link |
00:14:32.200
caricature of the biological neurons.
link |
00:14:35.480
But they're still neurons, single units communicating
link |
00:14:38.520
with other units, something that is absent
link |
00:14:41.400
in the traditional computer type models of mathematics,
link |
00:14:48.520
reasoning, and so on.
link |
00:14:50.840
So what aspect would you like to see
link |
00:14:53.120
in artificial neural networks added over time
link |
00:14:57.280
as we try to figure out ways to improve them?
link |
00:14:59.920
So one of the main differences and problems
link |
00:15:07.320
in terms of deep learning today, and it's not only
link |
00:15:11.840
deep learning, and the brain, is the need for deep learning
link |
00:15:16.760
techniques to have a lot of labeled examples.
link |
00:15:23.160
For instance, for ImageNet, you have
link |
00:15:24.920
like a training set, which is 1 million images, each one
link |
00:15:29.160
labeled by some human in terms of which object is there.
link |
00:15:34.600
And it's clear that in biology, a baby
link |
00:15:42.720
may be able to see millions of images
link |
00:15:44.840
in the first years of life, but will not
link |
00:15:47.560
have millions of labels given to him or her by parents
link |
00:15:52.720
or caretakers.
link |
00:15:56.360
So how do you solve that?
link |
00:15:59.560
I think there is this interesting challenge
link |
00:16:03.880
that today, deep learning and related techniques
link |
00:16:08.080
are all about big data, big data meaning
link |
00:16:11.200
a lot of examples labeled by humans,
link |
00:16:18.760
whereas in nature, you have this big data
link |
00:16:24.400
is n going to infinity.
link |
00:16:26.280
That's the best, n meaning labeled data.
link |
00:16:30.200
But I think the biological world is more n going to 1.
link |
00:16:34.920
A child can learn from a very small number
link |
00:16:38.920
of labeled examples.
link |
00:16:42.720
Like you tell a child, this is a car.
link |
00:16:44.920
You don't need to say, like in ImageNet, this is a car,
link |
00:16:48.880
this is a car, this is not a car, this is not a car,
link |
00:16:51.120
1 million times.
link |
00:16:54.360
And of course, with AlphaGo, or at least the AlphaZero
link |
00:16:57.720
variants, because the world of Go
link |
00:17:01.360
is so simplistic that you can actually
link |
00:17:05.040
learn by yourself through self play,
link |
00:17:06.720
you can play against each other.
link |
00:17:08.520
In the real world, the visual system
link |
00:17:10.600
that you've studied extensively is a lot more complicated
link |
00:17:14.200
than the game of Go.
link |
00:17:16.720
On the comment about children, which
link |
00:17:18.400
are fascinatingly good at learning new stuff,
link |
00:17:23.000
how much of it do you think is hardware,
link |
00:17:24.680
and how much of it is software?
link |
00:17:26.640
Yeah, that's a good, deep question.
link |
00:17:29.800
In a sense, it's the old question of nurture and nature,
link |
00:17:32.960
how much is in the gene, and how much
link |
00:17:36.440
is in the experience of an individual.
link |
00:17:41.280
Obviously, it's both that play a role.
link |
00:17:44.720
And I believe that the way evolution gives,
link |
00:17:53.040
puts prior information, so to speak, hardwired,
link |
00:17:55.760
is not really hardwired.
link |
00:17:58.400
But that's essentially an hypothesis.
link |
00:18:02.720
I think what's going on is that evolution has almost
link |
00:18:10.000
necessarily, if you believe in Darwin, is very opportunistic.
link |
00:18:14.960
And think about our DNA and the DNA of Drosophila.
link |
00:18:24.320
Our DNA does not have many more genes than Drosophila.
link |
00:18:28.800
The fly.
link |
00:18:29.720
The fly, the fruit fly.
link |
00:18:32.560
Now, we know that the fruit fly does not
link |
00:18:35.520
learn very much during its individual existence.
link |
00:18:39.680
It looks like one of these machinery
link |
00:18:42.360
that it's really mostly, not 100%, but 95%,
link |
00:18:47.240
hardcoded by the genes.
link |
00:18:51.720
But since we don't have many more genes than Drosophila,
link |
00:18:55.040
evolution could encode in as a general learning machinery,
link |
00:19:02.640
and then had to give very weak priors.
link |
00:19:09.840
Like, for instance, let me give a specific example,
link |
00:19:15.000
which is recent work by a member of our Center for Brains,
link |
00:19:18.160
Minds, and Machines.
link |
00:19:20.680
We know because of work of other people in our group
link |
00:19:24.440
and other groups, that there are cells
link |
00:19:26.720
in a part of our brain, neurons, that are tuned to faces.
link |
00:19:31.160
They seem to be involved in face recognition.
link |
00:19:33.840
Now, this face area seems to be present in young children
link |
00:19:41.600
and adults.
link |
00:19:44.600
And one question is, is there from the beginning?
link |
00:19:48.400
Is hardwired by evolution?
link |
00:19:51.760
Or somehow it's learned very quickly.
link |
00:19:55.000
So what's your, by the way, a lot of the questions I'm asking,
link |
00:19:58.960
the answer is we don't really know.
link |
00:20:00.920
But as a person who has contributed
link |
00:20:04.520
some profound ideas in these fields,
link |
00:20:06.440
you're a good person to guess at some of these.
link |
00:20:08.360
So of course, there's a caveat before a lot of the stuff
link |
00:20:11.200
we talk about.
link |
00:20:11.760
But what is your hunch?
link |
00:20:14.680
Is the face, the part of the brain
link |
00:20:16.400
that seems to be concentrated on face recognition,
link |
00:20:20.120
are you born with that?
link |
00:20:21.240
Or you just is designed to learn that quickly,
link |
00:20:25.160
like the face of the mother and so on?
link |
00:20:26.920
My hunch, my bias was the second one, learned very quickly.
link |
00:20:32.280
And it turns out that Marge Livingstone at Harvard
link |
00:20:37.240
has done some amazing experiments in which she raised
link |
00:20:41.480
baby monkeys, depriving them of faces
link |
00:20:45.200
during the first weeks of life.
link |
00:20:48.560
So they see technicians, but the technician have a mask.
link |
00:20:53.000
Yes.
link |
00:20:55.080
And so when they looked at the area
link |
00:21:02.000
in the brain of these monkeys that were usually
link |
00:21:05.720
defined faces, they found no face preference.
link |
00:21:10.840
So my guess is that what evolution does in this case
link |
00:21:16.800
is there is a plastic area, which
link |
00:21:19.760
is plastic, which is kind of predetermined
link |
00:21:22.760
to be imprinted very easily.
link |
00:21:26.520
But the command from the gene is not a detailed circuitry
link |
00:21:30.160
for a face template.
link |
00:21:32.280
Could be, but this will require probably a lot of bits.
link |
00:21:36.280
You had to specify a lot of connection of a lot of neurons.
link |
00:21:39.720
Instead, the command from the gene
link |
00:21:42.320
is something like imprint, memorize what you see most
link |
00:21:47.400
often in the first two weeks of life,
link |
00:21:49.480
especially in connection with food and maybe nipples.
link |
00:21:53.440
I don't know.
link |
00:21:54.640
Well, source of food.
link |
00:21:55.960
And so that area is very plastic at first and then solidifies.
link |
00:22:00.320
It'd be interesting if a variant of that experiment
link |
00:22:03.600
would show a different kind of pattern associated
link |
00:22:06.800
with food than a face pattern, whether that could stick.
link |
00:22:10.200
There are indications that during that experiment,
link |
00:22:14.960
what the monkeys saw quite often were
link |
00:22:19.560
the blue gloves of the technicians that were giving
link |
00:22:23.200
to the baby monkeys the milk.
link |
00:22:25.560
And some of the cells, instead of being face sensitive
link |
00:22:29.400
in that area, are hand sensitive.
link |
00:22:33.680
That's fascinating.
link |
00:22:35.960
Can you talk about what are the different parts of the brain
link |
00:22:40.600
and, in your view, sort of loosely,
link |
00:22:43.920
and how do they contribute to intelligence?
link |
00:22:45.760
Do you see the brain as a bunch of different modules,
link |
00:22:49.520
and they together come in the human brain
link |
00:22:52.440
to create intelligence?
link |
00:22:53.800
Or is it all one mush of the same kind
link |
00:22:59.320
of fundamental architecture?
link |
00:23:04.600
Yeah, that's an important question.
link |
00:23:08.840
And there was a phase in neuroscience back in the 1950
link |
00:23:15.200
or so in which it was believed for a while
link |
00:23:19.360
that the brain was equipotential.
link |
00:23:21.920
This was the term.
link |
00:23:22.960
You could cut out a piece, and nothing special
link |
00:23:28.000
happened apart a little bit less performance.
link |
00:23:32.360
There was a surgeon, Lashley, who
link |
00:23:37.120
did a lot of experiments of this type with mice and rats
link |
00:23:41.800
and concluded that every part of the brain
link |
00:23:45.640
was essentially equivalent to any other one.
link |
00:23:51.360
It turns out that that's really not true.
link |
00:23:56.080
There are very specific modules in the brain, as you said.
link |
00:24:00.480
And people may lose the ability to speak
link |
00:24:05.280
if you have a stroke in a certain region,
link |
00:24:07.520
or may lose control of their legs in another region.
link |
00:24:12.840
So they're very specific.
link |
00:24:14.520
The brain is also quite flexible and redundant,
link |
00:24:17.920
so often it can correct things and take over functions
link |
00:24:27.360
from one part of the brain to the other.
link |
00:24:29.840
But really, there are specific modules.
link |
00:24:33.760
So the answer that we know from this old work, which
link |
00:24:40.000
was basically based on lesions, either on animals,
link |
00:24:44.840
or very often there was a mine of very interesting data
link |
00:24:52.960
coming from the war, from different types of injuries
link |
00:25:00.600
that soldiers had in the brain.
link |
00:25:03.800
And more recently, functional MRI,
link |
00:25:09.120
which allow you to check which part of the brain
link |
00:25:13.840
are active when you are doing different tasks,
link |
00:25:21.640
can replace some of this.
link |
00:25:23.720
You can see that certain parts of the brain are involved,
link |
00:25:27.560
are active in certain tasks.
link |
00:25:29.480
Vision, language, yeah, that's right.
link |
00:25:32.320
But sort of taking a step back to that part of the brain
link |
00:25:36.520
that discovers that specializes in the face
link |
00:25:39.320
and how that might be learned, what's your intuition behind?
link |
00:25:45.320
Is it possible that from a physicist perspective,
link |
00:25:48.880
when you get lower and lower, that it's all the same stuff
link |
00:25:51.920
and it just, when you're born, it's plastic
link |
00:25:54.800
and quickly figures out this part is going to be about vision,
link |
00:25:58.040
this is going to be about language,
link |
00:25:59.440
this is about common sense reasoning.
link |
00:26:02.000
Do you have an intuition that that kind of learning
link |
00:26:05.120
is going on really quickly, or is it really
link |
00:26:07.080
kind of solidified in hardware?
link |
00:26:09.760
That's a great question.
link |
00:26:11.440
So there are parts of the brain like the cerebellum
link |
00:26:16.920
or the hippocampus that are quite different from each other.
link |
00:26:21.560
They clearly have different anatomy,
link |
00:26:23.840
different connectivity.
link |
00:26:26.880
Then there is the cortex, which is the most developed part
link |
00:26:33.400
of the brain in humans.
link |
00:26:36.080
And in the cortex, you have different regions
link |
00:26:39.560
of the cortex that are responsible for vision,
link |
00:26:43.360
for audition, for motor control, for language.
link |
00:26:47.880
Now, one of the big puzzles of this
link |
00:26:50.760
is that in the cortex is the cortex is the cortex.
link |
00:26:55.240
Looks like it is the same in terms of hardware,
link |
00:27:00.920
in terms of type of neurons and connectivity
link |
00:27:05.040
across these different modalities.
link |
00:27:08.360
So for the cortex, I think aside these other parts
link |
00:27:13.680
of the brain like spinal cord, hippocampus,
link |
00:27:15.800
cerebellum, and so on, for the cortex,
link |
00:27:18.840
I think your question about hardware and software
link |
00:27:21.920
and learning and so on, I think is rather open.
link |
00:27:28.400
And I find it very interesting for Risa
link |
00:27:33.720
to think about an architecture, computer architecture, that
link |
00:27:36.960
is good for vision and at the same time is good for language.
link |
00:27:41.360
Seems to be so different problem areas that you have to solve.
link |
00:27:49.320
But the underlying mechanism might be the same.
link |
00:27:51.280
And that's really instructive for artificial neural networks.
link |
00:27:55.200
So we've done a lot of great work in vision,
link |
00:27:58.000
in human vision, computer vision.
link |
00:28:01.640
And you mentioned the problem of human vision
link |
00:28:03.800
is really as difficult as the problem of general intelligence.
link |
00:28:07.440
And maybe that connects to the cortex discussion.
link |
00:28:11.480
Can you describe the human visual cortex
link |
00:28:15.320
and how the humans begin to understand the world
link |
00:28:20.320
through the raw sensory information?
link |
00:28:22.480
What's, for folks who are not familiar,
link |
00:28:27.760
especially on the computer vision side,
link |
00:28:30.120
we don't often actually take a step back except saying
link |
00:28:33.400
with a sentence or two that one is inspired by the other.
link |
00:28:36.560
What is it that we know about the human visual cortex?
link |
00:28:40.000
That's interesting.
link |
00:28:40.760
We know quite a bit.
link |
00:28:41.880
At the same time, we don't know a lot.
link |
00:28:43.440
But the bit we know, in a sense, we know a lot of the details.
link |
00:28:50.080
And many we don't know.
link |
00:28:53.440
And we know a lot of the top level,
link |
00:28:58.520
the answer to the top level question.
link |
00:29:00.080
But we don't know some basic ones,
link |
00:29:02.200
even in terms of general neuroscience, forgetting vision.
link |
00:29:06.200
Why do we sleep?
link |
00:29:08.960
It's such a basic question.
link |
00:29:11.960
And we really don't have an answer to that.
link |
00:29:15.360
So taking a step back on that.
link |
00:29:17.160
So sleep, for example, is fascinating.
link |
00:29:18.760
Do you think that's a neuroscience question?
link |
00:29:22.040
Or if we talk about abstractions, what do you
link |
00:29:25.360
think is an interesting way to study intelligence
link |
00:29:28.160
or most effective on the levels of abstraction?
link |
00:29:30.680
Is it chemical, is it biological,
link |
00:29:33.120
is it electrophysical, mathematical,
link |
00:29:35.560
as you've done a lot of excellent work on that side?
link |
00:29:37.880
Which psychology, at which level of abstraction do you think?
link |
00:29:43.280
Well, in terms of levels of abstraction,
link |
00:29:46.880
I think we need all of them.
link |
00:29:50.160
It's like if you ask me, what does it
link |
00:29:54.360
mean to understand a computer?
link |
00:29:57.560
That's much simpler.
link |
00:29:58.640
But in a computer, I could say, well,
link |
00:30:01.080
I understand how to use PowerPoint.
link |
00:30:04.800
That's my level of understanding a computer.
link |
00:30:08.080
It is reasonable.
link |
00:30:09.400
It gives me some power to produce slides
link |
00:30:11.760
and beautiful slides.
link |
00:30:14.480
Now, you can ask somebody else.
link |
00:30:17.320
He says, well, I know how the transistors work
link |
00:30:19.840
that are inside the computer.
link |
00:30:21.360
I can write the equation for transistor and diodes
link |
00:30:25.920
and circuits, logical circuits.
link |
00:30:29.360
And I can ask this guy, do you know how to operate PowerPoint?
link |
00:30:32.440
No idea.
link |
00:30:34.040
So do you think if we discovered computers walking amongst us
link |
00:30:39.800
full of these transistors that are also operating
link |
00:30:43.400
under windows and have PowerPoint,
link |
00:30:45.560
do you think it's digging in a little bit more?
link |
00:30:49.960
How useful is it to understand the transistor in order
link |
00:30:53.280
to be able to understand PowerPoint
link |
00:30:58.040
and these higher level intelligent processes?
link |
00:31:00.320
So I think in the case of computers,
link |
00:31:03.720
because they were made by engineers, by us,
link |
00:31:06.960
this different level of understanding
link |
00:31:09.280
are rather separate on purpose.
link |
00:31:13.280
They are separate modules so that the engineer that
link |
00:31:17.240
designed the circuit for the chips does not
link |
00:31:19.640
need to know what is inside PowerPoint.
link |
00:31:23.600
And somebody can write the software translating
link |
00:31:27.440
from one to the other.
link |
00:31:30.360
So in that case, I don't think understanding the transistor
link |
00:31:36.960
helps you understand PowerPoint, or very little.
link |
00:31:41.120
If you want to understand the computer, this question,
link |
00:31:43.960
I would say you have to understand it
link |
00:31:45.960
at different levels.
link |
00:31:46.800
If you really want to build one, right?
link |
00:31:51.520
But for the brain, I think these levels of understanding,
link |
00:31:57.320
so the algorithms, which kind of computation,
link |
00:32:00.840
the equivalent of PowerPoint, and the circuits,
link |
00:32:04.640
the transistors, I think they are much more
link |
00:32:07.560
intertwined with each other.
link |
00:32:09.560
There is not a neatly level of the software separate
link |
00:32:14.480
from the hardware.
link |
00:32:15.840
And so that's why I think in the case of the brain,
link |
00:32:20.080
the problem is more difficult and more than for computers
link |
00:32:23.640
requires the interaction, the collaboration
link |
00:32:26.560
between different types of expertise.
link |
00:32:30.080
The brain is a big hierarchical mess.
link |
00:32:32.320
You can't just disentangle levels.
link |
00:32:35.120
I think you can, but it's much more difficult.
link |
00:32:37.880
And it's not completely obvious.
link |
00:32:40.840
And as I said, I think it's one of the, personally,
link |
00:32:44.720
I think is the greatest problem in science.
link |
00:32:47.240
So I think it's fair that it's difficult.
link |
00:32:51.880
That's a difficult one.
link |
00:32:53.320
That said, you do talk about compositionality
link |
00:32:56.920
and why it might be useful.
link |
00:32:58.280
And when you discuss why these neural networks,
link |
00:33:01.720
in artificial or biological sense, learn anything,
link |
00:33:05.200
you talk about compositionality.
link |
00:33:07.560
See, there's a sense that nature can be disentangled.
link |
00:33:13.480
Or, well, all aspects of our cognition
link |
00:33:19.840
could be disentangled to some degree.
link |
00:33:22.640
So why do you think, first of all,
link |
00:33:25.920
how do you see compositionality?
link |
00:33:27.720
And why do you think it exists at all in nature?
link |
00:33:31.640
I spoke about, I use the term compositionality
link |
00:33:39.800
when we looked at deep neural networks, multilayers,
link |
00:33:45.320
and trying to understand when and why they are more powerful
link |
00:33:50.560
than more classical one layer networks,
link |
00:33:54.800
like linear classifier, kernel machines, so called.
link |
00:34:01.600
And what we found is that in terms
link |
00:34:05.360
of approximating or learning or representing
link |
00:34:08.360
a function, a mapping from an input to an output,
link |
00:34:12.200
like from an image to the label in the image,
link |
00:34:16.760
if this function has a particular structure,
link |
00:34:20.840
then deep networks are much more powerful than shallow networks
link |
00:34:26.120
to approximate the underlying function.
link |
00:34:28.880
And the particular structure is a structure of compositionality.
link |
00:34:33.920
If the function is made up of functions of function,
link |
00:34:38.960
so that you need to look on when you are interpreting an image,
link |
00:34:45.800
classifying an image, you don't need
link |
00:34:47.720
to look at all pixels at once.
link |
00:34:51.040
But you can compute something from small groups of pixels.
link |
00:34:57.120
And then you can compute something
link |
00:34:59.920
on the output of this local computation and so on,
link |
00:35:04.760
which is similar to what you do when you read a sentence.
link |
00:35:07.320
You don't need to read the first and the last letter.
link |
00:35:11.360
But you can read syllables, combine them in words,
link |
00:35:16.000
combine the words in sentences.
link |
00:35:18.120
So this is this kind of structure.
link |
00:35:21.040
So that's as part of a discussion
link |
00:35:22.600
of why deep neural networks may be more
link |
00:35:26.120
effective than the shallow methods.
link |
00:35:27.880
And is your sense, for most things
link |
00:35:31.320
we can use neural networks for, those problems
link |
00:35:37.400
are going to be compositional in nature, like language,
link |
00:35:42.440
like vision?
link |
00:35:44.240
How far can we get in this kind of way?
link |
00:35:47.840
So here is almost philosophy.
link |
00:35:51.560
Well, let's go there.
link |
00:35:53.120
Yeah, let's go there.
link |
00:35:54.240
So a friend of mine, Max Tegmark, who is a physicist at MIT.
link |
00:36:00.200
I've talked to him on this thing.
link |
00:36:01.560
Yeah, and he disagrees with you, right?
link |
00:36:03.800
A little bit.
link |
00:36:04.440
Yeah, we agree on most.
link |
00:36:07.040
But the conclusion is a bit different.
link |
00:36:10.160
His conclusion is that for images, for instance,
link |
00:36:14.640
the compositional structure of this function
link |
00:36:19.440
that we have to learn or to solve these problems
link |
00:36:23.360
comes from physics, comes from the fact
link |
00:36:27.760
that you have local interactions in physics
link |
00:36:31.920
between atoms and other atoms, between particle
link |
00:36:37.440
of matter and other particles, between planets
link |
00:36:41.120
and other planets, between stars and other.
link |
00:36:44.400
It's all local.
link |
00:36:48.320
And that's true.
link |
00:36:51.160
But you could push this argument a bit further.
link |
00:36:56.280
Not this argument, actually.
link |
00:36:57.600
You could argue that maybe that's part of the truth.
link |
00:37:02.800
But maybe what happens is kind of the opposite,
link |
00:37:06.800
is that our brain is wired up as a deep network.
link |
00:37:11.840
So it can learn, understand, solve
link |
00:37:18.240
problems that have this compositional structure
link |
00:37:22.800
and it cannot solve problems that don't have
link |
00:37:27.520
this compositional structure.
link |
00:37:29.400
So the problems we are accustomed to, we think about,
link |
00:37:34.920
we test our algorithms on, are this compositional structure
link |
00:37:40.160
because our brain is made up.
link |
00:37:42.600
And that's, in a sense, an evolutionary perspective
link |
00:37:45.400
that we've.
link |
00:37:46.400
So the ones that didn't have, that weren't
link |
00:37:50.120
dealing with the compositional nature of reality died off?
link |
00:37:55.200
Yes, but also could be maybe the reason
link |
00:38:00.320
why we have this local connectivity in the brain,
link |
00:38:05.480
like simple cells in cortex looking
link |
00:38:08.840
only at the small part of the image, each one of them,
link |
00:38:11.920
and then other cells looking at the small number
link |
00:38:14.680
of these simple cells and so on.
link |
00:38:16.360
The reason for this may be purely
link |
00:38:19.960
that it was difficult to grow long range connectivity.
link |
00:38:25.080
So suppose it's for biology.
link |
00:38:28.640
It's possible to grow short range connectivity but not
link |
00:38:34.280
long range also because there is a limited number of long range
link |
00:38:38.560
that you can.
link |
00:38:39.720
And so you have this limitation from the biology.
link |
00:38:45.000
And this means you build a deep convolutional network.
link |
00:38:50.160
This would be something like a deep convolutional network.
link |
00:38:53.600
And this is great for solving certain class of problems.
link |
00:38:57.800
These are the ones we find easy and important for our life.
link |
00:39:02.880
And yes, they were enough for us to survive.
link |
00:39:07.320
And you can start a successful business
link |
00:39:10.800
on solving those problems with Mobileye.
link |
00:39:14.600
Driving is a compositional problem.
link |
00:39:17.360
So on the learning task, we don't
link |
00:39:21.080
know much about how the brain learns
link |
00:39:24.000
in terms of optimization.
link |
00:39:26.320
So the thing that's stochastic gradient descent
link |
00:39:29.040
is what artificial neural networks use for the most part
link |
00:39:33.760
to adjust the parameters in such a way that it's
link |
00:39:37.520
able to deal based on the label data,
link |
00:39:40.640
it's able to solve the problem.
link |
00:39:42.520
So what's your intuition about why it works at all?
link |
00:39:50.040
How hard of a problem it is to optimize
link |
00:39:53.360
a neural network, artificial neural network?
link |
00:39:56.320
Is there other alternatives?
link |
00:39:58.720
Just in general, your intuition is
link |
00:40:01.640
behind this very simplistic algorithm
link |
00:40:03.800
that seems to do pretty good, surprisingly so.
link |
00:40:06.640
Yes.
link |
00:40:07.840
So I find neuroscience, the architecture of cortex,
link |
00:40:13.840
is really similar to the architecture of deep networks.
link |
00:40:17.440
So there is a nice correspondence there
link |
00:40:20.360
between the biology and this kind
link |
00:40:23.160
of local connectivity, hierarchical architecture.
link |
00:40:28.200
The stochastic gradient descent, as you said,
link |
00:40:30.960
is a very simple technique.
link |
00:40:35.760
It seems pretty unlikely that biology could do that
link |
00:40:41.320
from what we know right now about cortex and neurons
link |
00:40:47.360
and synapses.
link |
00:40:50.200
So it's a big question open whether there
link |
00:40:53.080
are other optimization learning algorithms that
link |
00:40:59.040
can replace stochastic gradient descent.
link |
00:41:02.000
And my guess is yes, but nobody has found yet a real answer.
link |
00:41:11.760
I mean, people are trying, still trying,
link |
00:41:13.840
and there are some interesting ideas.
link |
00:41:18.280
The fact that stochastic gradient descent
link |
00:41:22.000
is so successful, this has become clearly not so
link |
00:41:26.160
mysterious.
link |
00:41:27.640
And the reason is that it's an interesting fact.
link |
00:41:33.840
It's a change, in a sense, in how
link |
00:41:36.840
people think about statistics.
link |
00:41:39.280
And this is the following, is that typically when
link |
00:41:45.160
you had data and you had, say, a model with parameters,
link |
00:41:51.800
you are trying to fit the model to the data,
link |
00:41:54.520
to fit the parameter.
link |
00:41:55.960
Typically, the kind of crowd wisdom type idea
link |
00:42:04.520
was you should have at least twice the number of data
link |
00:42:09.720
than the number of parameters.
link |
00:42:12.880
Maybe 10 times is better.
link |
00:42:15.480
Now, the way you train neural networks these days
link |
00:42:19.560
is that they have 10 or 100 times more parameters
link |
00:42:23.480
than data, exactly the opposite.
link |
00:42:26.760
And it has been one of the puzzles about neural networks.
link |
00:42:34.080
How can you get something that really works
link |
00:42:37.120
when you have so much freedom?
link |
00:42:40.640
From that little data, it can generalize somehow.
link |
00:42:43.000
Right, exactly.
link |
00:42:44.200
Do you think the stochastic nature of it
link |
00:42:46.400
is essential, the randomness?
link |
00:42:48.160
So I think we have some initial understanding
link |
00:42:50.640
why this happens.
link |
00:42:52.240
But one nice side effect of having
link |
00:42:56.480
this overparameterization, more parameters than data,
link |
00:43:00.920
is that when you look for the minima of a loss function,
link |
00:43:04.720
like stochastic gradient descent is doing,
link |
00:43:08.240
you find I made some calculations based
link |
00:43:12.120
on some old basic theorem of algebra called the Bezu
link |
00:43:19.040
theorem that gives you an estimate of the number
link |
00:43:23.240
of solution of a system of polynomial equation.
link |
00:43:25.960
Anyway, the bottom line is that there are probably
link |
00:43:30.520
more minima for a typical deep networks
link |
00:43:36.080
than atoms in the universe.
link |
00:43:39.480
Just to say, there are a lot because
link |
00:43:42.120
of the overparameterization.
link |
00:43:44.760
A more global minimum, zero minimum, good minimum.
link |
00:43:50.280
A more global minima.
link |
00:43:51.560
Yeah, a lot of them.
link |
00:43:53.200
So you have a lot of solutions.
link |
00:43:54.560
So it's not so surprising that you can find them
link |
00:43:57.920
relatively easily.
link |
00:44:00.400
And this is because of the overparameterization.
link |
00:44:04.200
The overparameterization sprinkles that entire space
link |
00:44:07.920
with solutions that are pretty good.
link |
00:44:09.720
It's not so surprising, right?
link |
00:44:11.240
It's like if you have a system of linear equation
link |
00:44:14.400
and you have more unknowns than equations, then you have,
link |
00:44:18.520
we know, you have an infinite number of solutions.
link |
00:44:22.040
And the question is to pick one.
link |
00:44:24.480
That's another story.
link |
00:44:25.440
But you have an infinite number of solutions.
link |
00:44:27.520
So there are a lot of value of your unknowns
link |
00:44:31.040
that satisfy the equations.
link |
00:44:33.160
But it's possible that there's a lot of those solutions that
link |
00:44:36.360
aren't very good.
link |
00:44:37.560
What's surprising is that they're pretty good.
link |
00:44:39.160
So that's a good question.
link |
00:44:40.160
Why can you pick one that generalizes well?
link |
00:44:42.840
Yeah.
link |
00:44:44.120
That's a separate question with separate answers.
link |
00:44:47.120
One theorem that people like to talk about that kind of
link |
00:44:51.160
inspires imagination of the power of neural networks
link |
00:44:53.800
is the universality, universal approximation theorem,
link |
00:44:57.840
that you can approximate any computable function
link |
00:45:00.960
with just a finite number of neurons
link |
00:45:02.840
in a single hidden layer.
link |
00:45:04.360
Do you find this theorem one surprising?
link |
00:45:07.680
Do you find it useful, interesting, inspiring?
link |
00:45:12.600
No, this one, I never found it very surprising.
link |
00:45:16.440
It was known since the 80s, since I entered the field,
link |
00:45:22.640
because it's basically the same as Weierstrass theorem, which
link |
00:45:27.560
says that I can approximate any continuous function
link |
00:45:32.000
with a polynomial of sufficiently,
link |
00:45:34.560
with a sufficient number of terms, monomials.
link |
00:45:38.120
So basically the same.
link |
00:45:39.360
And the proofs are very similar.
link |
00:45:41.680
So your intuition was there was never
link |
00:45:43.520
any doubt that neural networks in theory
link |
00:45:45.680
could be very strong approximators.
link |
00:45:48.000
Right.
link |
00:45:48.800
The question, the interesting question,
link |
00:45:50.760
is that if this theorem says you can approximate, fine.
link |
00:45:58.520
But when you ask how many neurons, for instance,
link |
00:46:03.200
or in the case of polynomial, how many monomials,
link |
00:46:06.400
I need to get a good approximation.
link |
00:46:11.360
Then it turns out that that depends
link |
00:46:15.960
on the dimensionality of your function,
link |
00:46:18.080
how many variables you have.
link |
00:46:20.520
But it depends on the dimensionality
link |
00:46:22.120
of your function in a bad way.
link |
00:46:25.080
It's, for instance, suppose you want
link |
00:46:28.000
an error which is no worse than 10% in your approximation.
link |
00:46:35.040
You come up with a network that approximate your function
link |
00:46:38.120
within 10%.
link |
00:46:40.440
Then it turns out that the number of units you need
link |
00:46:44.520
are in the order of 10 to the dimensionality, d,
link |
00:46:48.360
how many variables.
link |
00:46:50.080
So if you have two variables, these two words,
link |
00:46:54.840
you have 100 units and OK.
link |
00:46:57.240
But if you have, say, 200 by 200 pixel images,
link |
00:47:02.920
now this is 40,000, whatever.
link |
00:47:06.840
We again go to the size of the universe pretty quickly.
link |
00:47:09.800
Exactly, 10 to the 40,000 or something.
link |
00:47:14.120
And so this is called the curse of dimensionality,
link |
00:47:18.680
not quite appropriately.
link |
00:47:22.280
And the hope is with the extra layers,
link |
00:47:24.200
you can remove the curse.
link |
00:47:28.040
What we proved is that if you have deep layers,
link |
00:47:32.280
hierarchical architecture with the local connectivity
link |
00:47:36.200
of the type of convolutional deep learning,
link |
00:47:39.960
and if you're dealing with a function that
link |
00:47:42.000
has this kind of hierarchical architecture,
link |
00:47:46.680
then you avoid completely the curse.
link |
00:47:50.680
You've spoken a lot about supervised deep learning.
link |
00:47:54.520
What are your thoughts, hopes, views
link |
00:47:56.480
on the challenges of unsupervised learning
link |
00:47:59.640
with GANs, with Generative Adversarial Networks?
link |
00:48:05.800
Do you see those as distinct?
link |
00:48:08.120
The power of GANs, do you see those
link |
00:48:09.920
as distinct from supervised methods in neural networks,
link |
00:48:13.120
or are they really all in the same representation ballpark?
link |
00:48:16.640
GANs is one way to get estimation of probability
link |
00:48:24.040
densities, which is a somewhat new way that people have not
link |
00:48:28.760
done before.
link |
00:48:30.360
I don't know whether this will really play an important role
link |
00:48:36.080
in intelligence.
link |
00:48:39.000
Or it's interesting.
link |
00:48:43.080
I'm less enthusiastic about it than many people in the field.
link |
00:48:48.600
I have the feeling that many people in the field
link |
00:48:50.880
are really impressed by the ability
link |
00:48:54.320
of producing realistic looking images in this generative way.
link |
00:49:01.160
Which describes the popularity of the methods.
link |
00:49:03.080
But you're saying that while that's exciting and cool
link |
00:49:06.320
to look at, it may not be the tool that's useful for it.
link |
00:49:11.200
So you describe it kind of beautifully.
link |
00:49:13.560
Current supervised methods go n to infinity
link |
00:49:16.320
in terms of number of labeled points.
link |
00:49:18.200
And we really have to figure out how to go to n to 1.
link |
00:49:21.360
And you're thinking GANs might help,
link |
00:49:23.200
but they might not be the right.
link |
00:49:25.080
I don't think for that problem, which I really think
link |
00:49:28.480
is important, I think they may help.
link |
00:49:32.000
They certainly have applications,
link |
00:49:33.680
for instance, in computer graphics.
link |
00:49:35.760
And I did work long ago, which was
link |
00:49:41.560
a little bit similar in terms of saying, OK, I have a network.
link |
00:49:47.000
And I present images.
link |
00:49:49.760
And I can input its images.
link |
00:49:54.040
And output is, for instance, the pose of the image.
link |
00:49:57.520
A face, how much is smiling, is rotated 45 degrees or not.
link |
00:50:02.960
What about having a network that I train with the same data
link |
00:50:07.440
set, but now I invert input and output.
link |
00:50:10.600
Now the input is the pose or the expression, a number,
link |
00:50:15.920
set of numbers.
link |
00:50:16.920
And the output is the image.
link |
00:50:18.280
And I train it.
link |
00:50:20.240
And we did pretty good, interesting results
link |
00:50:22.520
in terms of producing very realistic looking images.
link |
00:50:27.840
It was a less sophisticated mechanism.
link |
00:50:31.920
But the output was pretty less than GANs.
link |
00:50:35.320
But the output was pretty much of the same quality.
link |
00:50:38.960
So I think for a computer graphics type application,
link |
00:50:43.400
yeah, definitely GANs can be quite useful.
link |
00:50:46.240
And not only for that, but for helping,
link |
00:50:52.880
for instance, on this problem of unsupervised example
link |
00:50:58.200
of reducing the number of labeled examples.
link |
00:51:02.400
I think people, it's like they think they can get out
link |
00:51:07.920
more than they put in.
link |
00:51:11.080
There's no free lunch, as you said.
link |
00:51:14.000
What do you think, what's your intuition?
link |
00:51:17.320
How can we slow the growth of N to infinity in supervised,
link |
00:51:22.720
N to infinity in supervised learning?
link |
00:51:25.080
So for example, Mobileye has very successfully,
link |
00:51:29.880
I mean, essentially annotated large amounts of data
link |
00:51:33.000
to be able to drive a car.
link |
00:51:34.680
Now one thought is, so we're trying
link |
00:51:37.440
to teach machines, school of AI.
link |
00:51:41.000
And we're trying to, so how can we become better teachers,
link |
00:51:45.560
maybe?
link |
00:51:46.040
That's one way.
link |
00:51:47.320
No, I like that.
link |
00:51:51.240
Because again, one caricature of the history of computer
link |
00:51:57.680
science, you could say, begins with programmers, expensive.
link |
00:52:05.360
Continuous labelers, cheap.
link |
00:52:09.640
And the future will be schools, like we have for kids.
link |
00:52:14.680
Yeah.
link |
00:52:16.360
Currently, the labeling methods were not
link |
00:52:20.280
selective about which examples we teach networks with.
link |
00:52:25.880
So I think the focus of making networks that learn much faster
link |
00:52:31.320
is often on the architecture side.
link |
00:52:33.680
But how can we pick better examples with which to learn?
link |
00:52:37.960
Do you have intuitions about that?
link |
00:52:39.440
Well, that's part of the problem.
link |
00:52:42.480
But the other one is, if we look at biology,
link |
00:52:50.360
a reasonable assumption, I think,
link |
00:52:52.960
is in the same spirit that I said,
link |
00:52:58.120
evolution is opportunistic and has weak priors.
link |
00:53:03.400
The way I think the intelligence of a child,
link |
00:53:08.280
the baby may develop is by bootstrapping weak priors
link |
00:53:16.240
from evolution.
link |
00:53:17.400
For instance, you can assume that you
link |
00:53:24.720
have in most organisms, including human babies,
link |
00:53:28.960
built in some basic machinery to detect motion
link |
00:53:35.400
and relative motion.
link |
00:53:38.200
And in fact, we know all insects from fruit flies
link |
00:53:42.920
to other animals, they have this,
link |
00:53:49.760
even in the retinas, in the very peripheral part.
link |
00:53:53.120
It's very conserved across species, something
link |
00:53:56.160
that evolution discovered early.
link |
00:53:59.040
It may be the reason why babies tend
link |
00:54:01.480
to look in the first few days to moving objects
link |
00:54:06.160
and not to not moving objects.
link |
00:54:08.320
Now, moving objects means, OK, they're attracted by motion.
link |
00:54:12.200
But motion also means that motion
link |
00:54:15.480
gives automatic segmentation from the background.
link |
00:54:20.560
So because of motion boundaries, either the object
link |
00:54:25.360
is moving or the eye of the baby is tracking the moving object
link |
00:54:30.600
and the background is moving, right?
link |
00:54:32.800
Yeah, so just purely on the visual characteristics
link |
00:54:36.040
of the scene, that seems to be the most useful.
link |
00:54:37.920
Right, so it's like looking at an object without background.
link |
00:54:43.960
It's ideal for learning the object.
link |
00:54:45.760
Otherwise, it's really difficult because you
link |
00:54:48.760
have so much stuff.
link |
00:54:50.440
So suppose you do this at the beginning, first weeks.
link |
00:54:55.120
Then after that, you can recognize object.
link |
00:54:58.560
Now they are imprinted, the number one,
link |
00:55:02.160
even in the background, even without motion.
link |
00:55:05.800
So that's, by the way, I just want
link |
00:55:08.160
to ask on the object recognition problem.
link |
00:55:10.920
So there is this being responsive to movement
link |
00:55:13.960
and doing edge detection, essentially.
link |
00:55:16.760
What's the gap between being effective at visually
link |
00:55:21.600
recognizing stuff, detecting where it is,
link |
00:55:24.560
and understanding the scene?
link |
00:55:27.640
Is this a huge gap in many layers, or is it close?
link |
00:55:32.960
No, I think that's a huge gap.
link |
00:55:35.120
I think present algorithm with all the success that we have
link |
00:55:42.040
and the fact that there are a lot of very useful,
link |
00:55:45.120
I think we are in a golden age for applications
link |
00:55:48.440
of low level vision and low level speech recognition
link |
00:55:53.720
and so on, Alexa and so on.
link |
00:55:56.800
There are many more things of similar level
link |
00:55:58.840
to be done, including medical diagnosis and so on.
link |
00:56:02.040
But we are far from what we call understanding
link |
00:56:05.600
of a scene, of language, of actions, of people.
link |
00:56:11.960
That is, despite the claims, that's, I think, very far.
link |
00:56:18.480
We're a little bit off.
link |
00:56:19.560
So in popular culture and among many researchers,
link |
00:56:23.160
some of which I've spoken with, the Stuart Russell
link |
00:56:25.640
and Elon Musk, in and out of the AI field,
link |
00:56:30.920
there's a concern about the existential threat of AI.
link |
00:56:34.520
And how do you think about this concern?
link |
00:56:40.000
And is it valuable to think about large scale, long term,
link |
00:56:45.560
unintended consequences of intelligent systems
link |
00:56:50.360
we try to build?
link |
00:56:51.440
I always think it's better to worry first, early,
link |
00:56:56.000
rather than late.
link |
00:56:58.640
So worry is good.
link |
00:56:59.640
Yeah.
link |
00:57:00.400
I'm not against worrying at all.
link |
00:57:03.000
Personally, I think that it will take a long time
link |
00:57:09.520
before there is real reason to be worried.
link |
00:57:15.920
But as I said, I think it's good to put in place
link |
00:57:19.440
and think about possible safety against.
link |
00:57:24.360
What I find a bit misleading are things
link |
00:57:27.360
like that have been said by people I know, like Elon Musk,
link |
00:57:31.480
and what is Bostrom in particular,
link |
00:57:35.240
and what is his first name?
link |
00:57:36.800
Nick Bostrom.
link |
00:57:37.400
Nick Bostrom, right.
link |
00:57:40.120
And a couple of other people that, for instance, AI
link |
00:57:44.080
is more dangerous than nuclear weapons.
link |
00:57:46.880
I think that's really wrong.
link |
00:57:50.400
That can be misleading.
link |
00:57:52.680
Because in terms of priority, we should still
link |
00:57:56.440
be more worried about nuclear weapons
link |
00:57:59.480
and what people are doing about it and so on than AI.
link |
00:58:05.600
And you've spoken about Demis Hassabis
link |
00:58:09.920
and yourself saying that you think
link |
00:58:12.840
you'll be about 100 years out before we
link |
00:58:16.440
have a general intelligence system that's
link |
00:58:18.920
on par with a human being.
link |
00:58:20.600
Do you have any updates for those predictions?
link |
00:58:22.520
Well, I think he said.
link |
00:58:24.080
He said 20, I think.
link |
00:58:25.080
He said 20, right.
link |
00:58:26.200
This was a couple of years ago.
link |
00:58:27.680
I have not asked him again.
link |
00:58:29.160
So should I have?
link |
00:58:31.480
Your own prediction, what's your prediction
link |
00:58:36.000
about when you'll be truly surprised?
link |
00:58:38.880
And what's the confidence interval on that?
link |
00:58:43.000
It's so difficult to predict the future and even
link |
00:58:45.760
the present sometimes.
link |
00:58:47.120
It's pretty hard to predict.
link |
00:58:48.480
But I would be, as I said, this is completely,
link |
00:58:53.360
I would be more like Rod Brooks.
link |
00:58:56.960
I think he's about 200 years.
link |
00:58:58.960
200 years.
link |
00:59:01.560
When we have this kind of AGI system,
link |
00:59:04.880
artificial general intelligence system,
link |
00:59:06.920
you're sitting in a room with her, him, it.
link |
00:59:12.840
Do you think the underlying design of such a system
link |
00:59:17.120
is something we'll be able to understand?
link |
00:59:19.080
It will be simple?
link |
00:59:20.480
Do you think it'll be explainable,
link |
00:59:25.800
understandable by us?
link |
00:59:27.560
Your intuition, again, we're in the realm of philosophy
link |
00:59:30.760
a little bit.
link |
00:59:32.080
Well, probably no.
link |
00:59:36.120
But again, it depends what you really
link |
00:59:40.280
mean for understanding.
link |
00:59:42.000
So I think we don't understand how deep networks work.
link |
00:59:53.280
I think we are beginning to have a theory now.
link |
00:59:56.520
But in the case of deep networks,
link |
00:59:59.240
or even in the case of the simpler kernel machines
link |
01:00:04.120
or linear classifier, we really don't understand
link |
01:00:08.120
the individual units or so.
link |
01:00:11.520
But we understand what the computation and the limitations
link |
01:00:17.280
and the properties of it are.
link |
01:00:20.440
It's similar to many things.
link |
01:00:24.040
What does it mean to understand how a fusion bomb works?
link |
01:00:29.600
How many of us understand the basic principle?
link |
01:00:36.360
And some of us may understand deeper details.
link |
01:00:40.600
In that sense, understanding is, as a community,
link |
01:00:43.440
as a civilization, can we build another copy of it?
link |
01:00:47.360
And in that sense, do you think there
link |
01:00:50.880
will need to be some evolutionary component where
link |
01:00:53.960
it runs away from our understanding?
link |
01:00:56.200
Or do you think it could be engineered from the ground up,
link |
01:00:59.440
the same way you go from the transistor to PowerPoint?
link |
01:01:02.640
So many years ago, this was actually 40, 41 years ago,
link |
01:01:09.160
I wrote a paper with David Marr, who
link |
01:01:13.560
was one of the founding fathers of computer vision,
link |
01:01:18.000
computational vision.
link |
01:01:20.440
I wrote a paper about levels of understanding,
link |
01:01:23.840
which is related to the question we discussed earlier
link |
01:01:26.160
about understanding PowerPoint, understanding transistors,
link |
01:01:30.200
and so on.
link |
01:01:31.840
And in that kind of framework, we
link |
01:01:36.560
had the level of the hardware and the top level
link |
01:01:39.760
of the algorithms.
link |
01:01:42.240
We did not have learning.
link |
01:01:45.040
Recently, I updated adding levels.
link |
01:01:48.280
And one level I added to those three was learning.
link |
01:01:55.160
And you can imagine, you could have a good understanding
link |
01:01:59.520
of how you construct a learning machine, like we do.
link |
01:02:04.960
But being unable to describe in detail what the learning
link |
01:02:09.720
machines will discover, right?
link |
01:02:13.680
Now, that would be still a powerful understanding,
link |
01:02:17.120
if I can build a learning machine,
link |
01:02:19.400
even if I don't understand in detail every time it
link |
01:02:24.480
learns something.
link |
01:02:26.160
Just like our children, if they start
link |
01:02:28.440
listening to a certain type of music,
link |
01:02:31.320
I don't know, Miley Cyrus or something,
link |
01:02:33.680
you don't understand why they came
link |
01:02:36.240
to that particular preference.
link |
01:02:37.640
But you understand the learning process.
link |
01:02:39.400
That's very interesting.
link |
01:02:41.440
So on learning for systems to be part of our world,
link |
01:02:50.360
it has a certain, one of the challenging things
link |
01:02:53.480
that you've spoken about is learning ethics, learning
link |
01:02:57.920
morals.
link |
01:02:59.400
And how hard do you think is the problem of, first of all,
link |
01:03:04.560
humans understanding our ethics?
link |
01:03:06.800
What is the origin on the neural on the low level of ethics?
link |
01:03:10.600
What is it at the higher level?
link |
01:03:12.400
Is it something that's learnable from machines
link |
01:03:15.160
in your intuition?
link |
01:03:17.840
I think, yeah, ethics is learnable, very likely.
link |
01:03:23.960
I think it's one of these problems where
link |
01:03:29.720
I think understanding the neuroscience of ethics,
link |
01:03:36.680
people discuss there is an ethics of neuroscience.
link |
01:03:41.480
Yeah, yes.
link |
01:03:42.800
How a neuroscientist should or should not behave.
link |
01:03:46.560
Can you think of a neurosurgeon and the ethics
link |
01:03:50.480
rule he has to be or he, she has to be.
link |
01:03:53.960
But I'm more interested on the neuroscience of ethics.
link |
01:03:57.560
You're blowing my mind right now.
link |
01:03:58.840
The neuroscience of ethics is very meta.
link |
01:04:01.080
Yeah, and I think that would be important to understand also
link |
01:04:05.080
for being able to design machines that
link |
01:04:10.880
are ethical machines in our sense of ethics.
link |
01:04:15.160
And you think there is something in neuroscience,
link |
01:04:18.520
there's patterns, tools in neuroscience
link |
01:04:21.520
that could help us shed some light on ethics?
link |
01:04:25.320
Or is it mostly on the psychologists of sociology
link |
01:04:28.920
in which higher level?
link |
01:04:29.840
No, there is psychology.
link |
01:04:30.960
But there is also, in the meantime,
link |
01:04:35.160
there is evidence, fMRI, of specific areas of the brain
link |
01:04:41.080
that are involved in certain ethical judgment.
link |
01:04:44.520
And not only this, you can stimulate those area
link |
01:04:47.640
with magnetic fields and change the ethical decisions.
link |
01:04:53.920
Yeah, wow.
link |
01:04:56.360
So that's work by a colleague of mine, Rebecca Sachs.
link |
01:05:00.800
And there is other researchers doing similar work.
link |
01:05:05.320
And I think this is the beginning.
link |
01:05:08.280
But ideally, at some point, we'll
link |
01:05:11.680
have an understanding of how this works.
link |
01:05:15.560
And why it evolved, right?
link |
01:05:18.520
The big why question.
link |
01:05:19.720
Yeah, it must have some purpose.
link |
01:05:22.000
Yeah, obviously it has some social purposes, probably.
link |
01:05:30.120
If neuroscience holds the key to at least illuminate
link |
01:05:33.600
some aspect of ethics, that means
link |
01:05:35.240
it could be a learnable problem.
link |
01:05:37.120
Yeah, exactly.
link |
01:05:38.880
And as we're getting into harder and harder questions,
link |
01:05:42.040
let's go to the hard problem of consciousness.
link |
01:05:45.440
Is this an important problem for us
link |
01:05:48.080
to think about and solve on the engineering of intelligence
link |
01:05:52.240
side of your work, of our dream?
link |
01:05:56.240
It's unclear.
link |
01:05:57.440
So again, this is a deep problem,
link |
01:06:02.680
partly because it's very difficult to define
link |
01:06:05.720
consciousness.
link |
01:06:06.760
And there is a debate among neuroscientists
link |
01:06:17.800
about whether consciousness and philosophers, of course,
link |
01:06:23.040
whether consciousness is something that requires
link |
01:06:28.280
flesh and blood, so to speak.
link |
01:06:31.360
Or could be that we could have silicon devices that
link |
01:06:38.680
are conscious, or up to statement
link |
01:06:42.840
like everything has some degree of consciousness
link |
01:06:45.800
and some more than others.
link |
01:06:48.480
This is like Giulio Tonioni and phi.
link |
01:06:53.960
We just recently talked to Christoph Koch.
link |
01:06:56.280
OK.
link |
01:06:57.600
Christoph was my first graduate student.
link |
01:07:00.680
Do you think it's important to illuminate
link |
01:07:04.480
aspects of consciousness in order
link |
01:07:07.480
to engineer intelligence systems?
link |
01:07:10.320
Do you think an intelligent system would ultimately
link |
01:07:13.080
have consciousness?
link |
01:07:14.480
Are they interlinked?
link |
01:07:18.800
Most of the people working in artificial intelligence,
link |
01:07:22.800
I think, would answer, we don't strictly
link |
01:07:25.800
need consciousness to have an intelligent system.
link |
01:07:30.040
That's sort of the easier question,
link |
01:07:31.800
because it's a very engineering answer to the question.
link |
01:07:36.000
Pass the Turing test, we don't need consciousness.
link |
01:07:38.120
But if you were to go, do you think
link |
01:07:41.360
it's possible that we need to have
link |
01:07:46.200
that kind of self awareness?
link |
01:07:48.280
We may, yes.
link |
01:07:49.920
So for instance, I personally think
link |
01:07:53.800
that when test a machine or a person in a Turing test,
link |
01:08:00.440
in an extended Turing test, I think
link |
01:08:05.200
consciousness is part of what we require in that test,
link |
01:08:11.520
implicitly, to say that this is intelligent.
link |
01:08:15.000
Christoph disagrees.
link |
01:08:17.440
Yes, he does.
link |
01:08:20.240
Despite many other romantic notions he holds,
link |
01:08:23.440
he disagrees with that one.
link |
01:08:24.800
Yes, that's right.
link |
01:08:26.520
So we'll see.
link |
01:08:29.880
Do you think, as a quick question,
link |
01:08:34.640
Ernest Becker's fear of death, do you
link |
01:08:38.520
think mortality and those kinds of things
link |
01:08:41.920
are important for consciousness and for intelligence?
link |
01:08:49.200
The finiteness of life, finiteness of existence,
link |
01:08:54.040
or is that just a side effect of evolution,
link |
01:08:56.600
evolutionary side effect that's useful for natural selection?
link |
01:09:01.120
Do you think this kind of thing that this interview is
link |
01:09:03.840
going to run out of time soon, our life
link |
01:09:06.160
will run out of time soon, do you
link |
01:09:08.200
think that's needed to make this conversation good and life
link |
01:09:11.720
good?
link |
01:09:12.240
I never thought about it.
link |
01:09:13.480
It's a very interesting question.
link |
01:09:15.920
I think Steve Jobs, in his commencement speech
link |
01:09:21.200
at Stanford, argued that having a finite life
link |
01:09:26.840
was important for stimulating achievements.
link |
01:09:30.280
So it was different.
link |
01:09:31.640
Yeah, live every day like it's your last, right?
link |
01:09:33.680
Yeah.
link |
01:09:34.840
So rationally, I don't think strictly you need mortality
link |
01:09:41.840
for consciousness.
link |
01:09:43.200
But who knows?
link |
01:09:45.960
They seem to go together in our biological system, right?
link |
01:09:48.760
Yeah, yeah.
link |
01:09:51.320
You've mentioned before, and students are associated with,
link |
01:09:57.880
AlphaGo immobilized the big recent success stories in AI.
link |
01:10:01.280
And I think it's captivated the entire world of what AI can do.
link |
01:10:06.040
So what do you think will be the next breakthrough?
link |
01:10:10.360
And what's your intuition about the next breakthrough?
link |
01:10:13.680
Of course, I don't know where the next breakthrough is.
link |
01:10:16.760
I think that there is a good chance, as I said before,
link |
01:10:21.440
that the next breakthrough will also
link |
01:10:23.200
be inspired by neuroscience.
link |
01:10:27.920
But which one, I don't know.
link |
01:10:32.320
And there's, so MIT has this quest for intelligence.
link |
01:10:35.880
And there's a few moon shots, which in that spirit,
link |
01:10:39.240
which ones are you excited about?
link |
01:10:41.800
Which projects kind of?
link |
01:10:44.080
Well, of course, I'm excited about one
link |
01:10:47.400
of the moon shots, which is our Center for Brains, Minds,
link |
01:10:51.040
and Machines, which is the one which is fully funded by NSF.
link |
01:10:58.560
And it is about visual intelligence.
link |
01:11:02.760
And that one is particularly about understanding.
link |
01:11:06.240
Visual intelligence, so the visual cortex,
link |
01:11:09.240
and visual intelligence in the sense
link |
01:11:13.400
of how we look around ourselves and understand
link |
01:11:20.000
the world around ourselves, meaning what is going on,
link |
01:11:25.440
how we could go from here to there without hitting
link |
01:11:29.880
obstacles, whether there are other agents,
link |
01:11:34.360
people in the environment.
link |
01:11:36.720
These are all things that we perceive very quickly.
link |
01:11:41.160
And it's something actually quite close to being conscious,
link |
01:11:46.920
not quite.
link |
01:11:47.640
But there is this interesting experiment
link |
01:11:50.360
that was run at Google X, which is in a sense
link |
01:11:54.800
is just a virtual reality experiment,
link |
01:11:58.840
but in which they had a subject sitting, say,
link |
01:12:02.760
in a chair with goggles, like Oculus and so on, earphones.
link |
01:12:11.800
And they were seeing through the eyes of a robot
link |
01:12:15.040
nearby to cameras, microphones for receiving.
link |
01:12:19.920
So their sensory system was there.
link |
01:12:23.840
And the impression of all the subject, very strong,
link |
01:12:28.120
they could not shake it off, was that they
link |
01:12:31.520
were where the robot was.
link |
01:12:35.240
They could look at themselves from the robot
link |
01:12:38.640
and still feel they were where the robot is.
link |
01:12:42.880
They were looking at their body.
link |
01:12:46.000
Theirself had moved.
link |
01:12:48.480
So some aspect of scene understanding
link |
01:12:50.440
has to have ability to place yourself,
link |
01:12:54.880
have a self awareness about your position in the world
link |
01:12:57.680
and what the world is.
link |
01:12:59.600
So we may have to solve the hard problem of consciousness
link |
01:13:04.080
to solve it.
link |
01:13:04.840
On their way, yes.
link |
01:13:05.920
It's quite a moonshine.
link |
01:13:07.760
So you've been an advisor to some incredible minds,
link |
01:13:12.440
including Demis Hassabis, Krzysztof Koch, Amna Shashua,
link |
01:13:15.680
like you said.
link |
01:13:17.360
All went on to become seminal figures
link |
01:13:20.120
in their respective fields.
link |
01:13:22.000
From your own success as a researcher
link |
01:13:24.240
and from perspective as a mentor of these researchers,
link |
01:13:29.320
having guided them in the way of advice,
link |
01:13:34.160
what does it take to be successful in science
link |
01:13:36.360
and engineering careers?
link |
01:13:39.800
Whether you're talking to somebody in their teens,
link |
01:13:43.280
20s, and 30s, what does that path look like?
link |
01:13:48.160
It's curiosity and having fun.
link |
01:13:53.200
And I think it's important also having
link |
01:13:57.400
fun with other curious minds.
link |
01:14:02.440
It's the people you surround with too,
link |
01:14:04.520
so fun and curiosity.
link |
01:14:06.640
Is there, you mentioned Steve Jobs,
link |
01:14:09.960
is there also an underlying ambition
link |
01:14:13.160
that's unique that you saw?
link |
01:14:14.720
Or does it really does boil down
link |
01:14:16.440
to insatiable curiosity and fun?
link |
01:14:18.800
Well of course, it's being curious
link |
01:14:22.240
in an active and ambitious way, yes.
link |
01:14:26.080
Definitely.
link |
01:14:29.640
But I think sometime in science,
link |
01:14:33.840
there are friends of mine who are like this.
link |
01:14:39.000
There are some of the scientists
link |
01:14:40.680
like to work by themselves
link |
01:14:44.080
and kind of communicate only when they complete their work
link |
01:14:50.920
or discover something.
link |
01:14:52.840
I think I always found the actual process
link |
01:14:58.720
of discovering something is more fun
link |
01:15:03.720
if it's together with other intelligent
link |
01:15:07.280
and curious and fun people.
link |
01:15:09.240
So if you see the fun in that process,
link |
01:15:11.320
the side effect of that process
link |
01:15:13.200
will be that you'll actually end up
link |
01:15:14.360
discovering some interesting things.
link |
01:15:16.320
So as you've led many incredible efforts here,
link |
01:15:23.320
what's the secret to being a good advisor,
link |
01:15:25.520
mentor, leader in a research setting?
link |
01:15:28.360
Is it a similar spirit?
link |
01:15:30.240
Or yeah, what advice could you give
link |
01:15:32.600
to people, young faculty and so on?
link |
01:15:35.960
It's partly repeating what I said
link |
01:15:38.320
about an environment that should be friendly
link |
01:15:41.280
and fun and ambitious.
link |
01:15:44.440
And I think I learned a lot
link |
01:15:49.280
from some of my advisors and friends
link |
01:15:52.880
and some who are physicists.
link |
01:15:55.280
And there was, for instance,
link |
01:15:57.480
this behavior that was encouraged
link |
01:16:02.800
of when somebody comes with a new idea in the group,
link |
01:16:06.720
you are, unless it's really stupid,
link |
01:16:09.080
but you are always enthusiastic.
link |
01:16:11.880
And then, and you're enthusiastic for a few minutes,
link |
01:16:14.280
for a few hours.
link |
01:16:15.120
Then you start asking critically a few questions,
link |
01:16:21.400
testing this.
link |
01:16:23.040
But this is a process that is,
link |
01:16:26.280
I think it's very good.
link |
01:16:29.360
You have to be enthusiastic.
link |
01:16:30.480
Sometimes people are very critical from the beginning.
link |
01:16:33.680
That's not...
link |
01:16:36.280
Yes, you have to give it a chance
link |
01:16:37.600
for that seed to grow.
link |
01:16:39.400
That said, with some of your ideas,
link |
01:16:41.600
which are quite revolutionary,
link |
01:16:42.800
so there's a witness, especially in the human vision side
link |
01:16:45.840
and neuroscience side,
link |
01:16:47.320
there could be some pretty heated arguments.
link |
01:16:50.000
Do you enjoy these?
link |
01:16:51.160
Is that a part of science and academic pursuits
link |
01:16:54.520
that you enjoy?
link |
01:16:55.360
Yeah.
link |
01:16:56.200
Is that something that happens in your group as well?
link |
01:17:01.040
Yeah, absolutely.
link |
01:17:02.440
I also spent some time in Germany.
link |
01:17:04.360
Again, there is this tradition
link |
01:17:05.880
in which people are more forthright,
link |
01:17:10.880
less kind than here.
link |
01:17:14.160
So in the U.S., when you write a bad letter,
link |
01:17:20.120
you still say, this guy's nice.
link |
01:17:23.080
Yes, yes.
link |
01:17:25.600
So...
link |
01:17:26.440
Yeah, here in America, it's degrees of nice.
link |
01:17:28.840
Yes.
link |
01:17:29.680
It's all just degrees of nice, yeah.
link |
01:17:31.040
Right, right.
link |
01:17:31.880
So as long as this does not become personal,
link |
01:17:36.400
and it's really like a football game
link |
01:17:40.680
with these rules, that's great.
link |
01:17:43.520
That's fun.
link |
01:17:46.600
So if you somehow found yourself in a position
link |
01:17:49.280
to ask one question of an oracle,
link |
01:17:51.840
like a genie, maybe a god,
link |
01:17:55.520
and you're guaranteed to get a clear answer,
link |
01:17:58.760
what kind of question would you ask?
link |
01:18:01.320
What would be the question you would ask?
link |
01:18:04.520
In the spirit of our discussion,
link |
01:18:06.040
it could be, how could I become 10 times more intelligent?
link |
01:18:10.080
And so, but see, you only get a clear short answer.
link |
01:18:16.240
So do you think there's a clear short answer to that?
link |
01:18:18.720
No.
link |
01:18:20.720
And that's the answer you'll get.
link |
01:18:22.760
Okay, so you've mentioned Flowers of Algernon.
link |
01:18:26.920
Oh, yeah.
link |
01:18:27.960
As a story that inspires you in your childhood,
link |
01:18:32.800
as this story of a mouse,
link |
01:18:37.200
human achieving genius level intelligence,
link |
01:18:39.360
and then understanding what was happening
link |
01:18:41.520
while slowly becoming not intelligent again,
link |
01:18:44.200
and this tragedy of gaining intelligence
link |
01:18:46.600
and losing intelligence,
link |
01:18:48.600
do you think in that spirit, in that story,
link |
01:18:51.440
do you think intelligence is a gift or a curse
link |
01:18:55.360
from the perspective of happiness and meaning of life?
link |
01:19:00.160
You try to create an intelligent system
link |
01:19:02.200
that understands the universe,
link |
01:19:03.880
but on an individual level, the meaning of life,
link |
01:19:06.480
do you think intelligence is a gift?
link |
01:19:10.840
It's a good question.
link |
01:19:17.120
I don't know.
link |
01:19:22.840
As one of the, as one people consider
link |
01:19:26.520
the smartest people in the world,
link |
01:19:29.280
in some dimension, at the very least, what do you think?
link |
01:19:33.320
I don't know, it may be invariant to intelligence,
link |
01:19:37.560
that degree of happiness.
link |
01:19:39.640
It would be nice if it were.
link |
01:19:43.680
That's the hope.
link |
01:19:44.680
Yeah.
link |
01:19:46.120
You could be smart and happy and clueless and happy.
link |
01:19:50.160
Yeah.
link |
01:19:51.800
As always, on the discussion of the meaning of life,
link |
01:19:54.480
it's probably a good place to end.
link |
01:19:57.320
Tommaso, thank you so much for talking today.
link |
01:19:59.240
Thank you, this was great.