back to indexGilbert Strang: Linear Algebra, Teaching, and MIT OpenCourseWare | Lex Fridman Podcast #52
link |
The following is a conversation with Gilbert Strang.
link |
He's a professor of mathematics at MIT
link |
and perhaps one of the most famous
link |
and impactful teachers of math in the world.
link |
His MIT OpenCourseWare lectures on linear algebra
link |
have been viewed millions of times.
link |
As an undergraduate student,
link |
I was one of those millions of students.
link |
There's something inspiring about the way he teaches.
link |
There's at once calm, simple, and yet full of passion
link |
for the elegance inherent to mathematics.
link |
I remember doing the exercise in his book,
link |
Introduction to Linear Algebra,
link |
and slowly realizing that the world of matrices,
link |
of vector spaces, of determinants and eigenvalues,
link |
of geometric transformations and matrix decompositions
link |
reveal a set of powerful tools
link |
in the toolbox of artificial intelligence.
link |
From signals to images,
link |
from numerical optimization to robotics,
link |
computer vision, deep learning, computer graphics,
link |
and everywhere outside AI,
link |
including, of course, a quantum mechanical study
link |
This is the Artificial Intelligence Podcast.
link |
If you enjoy it, subscribe on YouTube,
link |
give it five stars on Apple Podcast,
link |
support on Patreon,
link |
or simply connect with me on Twitter
link |
at Lex Friedman, spelled F R I D M A N.
link |
This podcast is supported by ZipRecruiter.
link |
Hiring great people is hard,
link |
and to me is the most important element
link |
of a successful mission driven team.
link |
I've been fortunate to be a part of
link |
and to lead several great engineering teams.
link |
The hiring I've done in the past
link |
was mostly through tools that we built ourselves,
link |
but reinventing the wheel was painful.
link |
ZipRecruiter is a tool that's already available for you.
link |
It seeks to make hiring simple, fast, and smart.
link |
For example, Codable cofounder Gretchen Huebner
link |
used ZipRecruiter to find a new game artist
link |
to join her education tech company.
link |
By using ZipRecruiter's screening questions
link |
to filter candidates,
link |
Gretchen found it easier to focus on the best candidates
link |
and finally hiring the perfect person for the role
link |
in less than two weeks from start to finish.
link |
ZipRecruiter, the smartest way to hire.
link |
See why ZipRecruiter is effective
link |
for businesses of all sizes by signing up,
link |
as I did, for free at ziprecruiter.com slash lexpod.
link |
That's ziprecruiter.com slash lexpod.
link |
This show is presented by Cash App,
link |
the number one finance app in the App Store.
link |
I personally use Cash App to send money to friends,
link |
but you can also use it to buy, sell, and deposit Bitcoin.
link |
Most Bitcoin exchanges take days
link |
for a bank transfer to become investable.
link |
Through Cash App, it takes seconds.
link |
Cash App also has a new investing feature.
link |
You can buy fractions of a stock,
link |
which to me is a really interesting concept.
link |
So you can buy $1 worth no matter what the stock price is.
link |
Brokerage services are provided by Cash App Investing,
link |
a subsidiary of Square, and member SIPC.
link |
I'm excited to be working with Cash App
link |
to support one of my favorite organizations
link |
that many of you may know and have benefited from,
link |
called First, best known for their first robotics
link |
and Lego competitions.
link |
They educate and inspire hundreds of thousands of students
link |
in over 110 countries,
link |
and have a perfect rating on Charity Navigator,
link |
which means the donated money is used
link |
to maximum effectiveness.
link |
When you get Cash App from the App Store or Google Play,
link |
and use code LexPodcast,
link |
you get $10, and Cash App will also donate $10 to First,
link |
which again, is an organization
link |
that I've personally seen inspire girls and boys
link |
to dream of engineering a better world.
link |
And now, here's my conversation with Gilbert Strang.
link |
How does it feel to be one of the modern day rock stars
link |
I don't feel like a rock star.
link |
That's kind of crazy for an old math person.
link |
But it's true that the videos in linear algebra
link |
that I made way back in 2000, I think,
link |
have been watched a lot.
link |
And well, partly the importance of linear algebra,
link |
which I'm sure you'll ask me,
link |
and give me a chance to say that linear algebra
link |
as a subject has just surged in importance.
link |
But also, it was a class that I taught a bunch of times,
link |
so I kind of got it organized and enjoyed doing it.
link |
The videos were just the class.
link |
So they're on OpenCourseWare and on YouTube
link |
and translated, and it's fun.
link |
But there's something about that chalkboard
link |
and the simplicity of the way you explain
link |
the basic concepts in the beginning.
link |
To be honest, when I went to undergrad.
link |
You didn't do linear algebra, probably.
link |
Of course I didn't do linear algebra.
link |
Yeah, yeah, yeah, of course.
link |
But before going through the course at my university,
link |
there was going through OpenCourseWare.
link |
You were my instructor for linear algebra.
link |
And that, I mean, we're using your book.
link |
And I mean, the fact that there is thousands,
link |
hundreds of thousands, millions of people
link |
that watch that video, I think that's really powerful.
link |
So how do you think the idea of putting lectures online,
link |
what really MIT OpenCourseWare has innovated?
link |
That was a wonderful idea.
link |
I think the story that I've heard is the committee
link |
was appointed by the president, President Vest,
link |
at that time, a wonderful guy.
link |
And the idea of the committee was to figure out
link |
how MIT could be like other universities,
link |
market the work we were doing.
link |
And then they didn't see a way.
link |
And after a weekend, and they had an inspiration,
link |
came back to President Vest and said,
link |
what if we just gave it away?
link |
And he decided that was okay, good idea.
link |
You know, that's a crazy idea.
link |
If we think of a university as a thing
link |
that creates a product, isn't knowledge,
link |
the kind of educational knowledge,
link |
isn't the product and giving that away,
link |
are you surprised that it went through?
link |
The result that he did it,
link |
well, knowing a little bit President Vest, it was like him,
link |
I think, and it was really the right idea.
link |
MIT is a kind of, it's known for being high level,
link |
technical things, and this is the best way we can say,
link |
tell, we can show what MIT really is like,
link |
because in my case, those 1806 videos
link |
are just teaching the class.
link |
They were there in 26, 100.
link |
They're kind of fun to look at.
link |
People write to me and say, oh, you've got a sense of humor,
link |
but I don't know where that comes through.
link |
Somehow I'm friendly with the class, I like students.
link |
And then your algebra, the subject,
link |
we gotta give the subject most of the credit.
link |
It really has come forward in importance in these years.
link |
So let's talk about linear algebra a little bit,
link |
because it is such a, it's both a powerful
link |
and a beautiful subfield of mathematics.
link |
So what's your favorite specific topic in linear algebra,
link |
or even math in general to give a lecture on,
link |
to convey, to tell a story, to teach students?
link |
Okay, well, on the teaching side,
link |
so it's not deep mathematics at all,
link |
but I'm kind of proud of the idea of the four subspaces,
link |
the four fundamental subspaces,
link |
which are of course known before,
link |
long before my name for them, but.
link |
Can you go through them?
link |
Can you go through the four subspaces?
link |
So the first one to understand is,
link |
so the matrix is, maybe I should say the matrix is.
link |
Well, so we have like a rectangle of numbers.
link |
So it's got n columns, got a bunch of columns,
link |
and also got an m rows, let's say,
link |
and the relation between,
link |
so of course the columns and the rows,
link |
it's the same numbers.
link |
So there's gotta be connections there,
link |
but they're not simple.
link |
The columns might be longer than the rows,
link |
and they're all different, the numbers are mixed up.
link |
First space to think about is take the columns,
link |
so those are vectors, those are points in n dimensions.
link |
So a physicist would imagine a vector
link |
or might imagine a vector as a arrow in space
link |
or the point it ends at in space.
link |
For me, it's a column of numbers.
link |
You often think of, this is very interesting
link |
in terms of linear algebra, in terms of a vector,
link |
you think a little bit more abstract
link |
than how it's very commonly used, perhaps.
link |
You think this arbitrary multidimensional space.
link |
Right away, I'm in high dimensions.
link |
Yeah, that's right.
link |
In the lecture, I try to,
link |
so if you think of two vectors in 10 dimensions,
link |
I'll do this in class, and I'll readily admit
link |
that I have no good image in my mind
link |
of a vector of an arrow in 10 dimensional space,
link |
You can add one bunch of 10 numbers
link |
to another bunch of 10 numbers,
link |
so you can add a vector to a vector,
link |
and you can multiply a vector by three,
link |
and that's, if you know how to do those,
link |
you've got linear algebra.
link |
10 dimensions, there's this beautiful thing about math,
link |
if we look at string theory and all these theories,
link |
which are really fundamentally derived through math,
link |
but are very difficult to visualize.
link |
How do you think about the things,
link |
like a 10 dimensional vector,
link |
that we can't really visualize?
link |
And yet, math reveals some beauty underlying our world
link |
in that weird thing we can't visualize.
link |
How do you think about that difference?
link |
Well, probably, I'm not a very geometric person,
link |
so I'm probably thinking in three dimensions,
link |
and the beauty of linear algebra is that
link |
it goes on to 10 dimensions with no problem.
link |
I mean, that if you're just seeing what happens
link |
if you add two vectors in 3D,
link |
yeah, then you can add them in 10D.
link |
You're just adding the 10 components.
link |
So, I can't say that I have a picture,
link |
but yet I try to push the class
link |
to think of a flat surface in 10 dimensions.
link |
So a plane in 10 dimensions,
link |
and so that's one of the spaces.
link |
Take all the columns of the matrix,
link |
take all their combinations,
link |
so much of this column, so much of this one,
link |
then if you put all those together,
link |
you get some kind of a flat surface
link |
that I call a vector space, space of vectors.
link |
And my imagination is just seeing
link |
like a piece of paper in 3D, but anyway,
link |
so that's one of the spaces, that's space number one,
link |
the column space of the matrix.
link |
And then there's the row space, which is, as I said,
link |
different, but came from the same numbers.
link |
So we got the column space,
link |
all combinations of the columns,
link |
and then we've got the row space,
link |
all combinations of the rows.
link |
So those words are easy for me to say,
link |
and I can't really draw them on a blackboard,
link |
but I try with my thick chalk.
link |
Everybody likes that railroad chalk, and me too.
link |
I wouldn't use anything else now.
link |
And then the other two spaces are perpendicular to those.
link |
So like if you have a plane in 3D,
link |
just a plane is just a flat surface in 3D,
link |
then perpendicular to that plane would be a line.
link |
So that would be the null space.
link |
So we've got two, we've got a column space, a row space,
link |
and there are two perpendicular spaces.
link |
So those four fit together in a beautiful picture
link |
of a matrix, yeah, yeah.
link |
It's sort of a fundamental, it's not a difficult idea.
link |
It comes pretty early in 1806, and it's basic.
link |
Planes in these multidimensional spaces,
link |
how difficult of an idea is that to come to, do you think?
link |
If you look back in time,
link |
I think mathematically it makes sense,
link |
but I don't know if it's intuitive for us to imagine,
link |
just as we were talking about.
link |
It feels like calculus is easier to intuit.
link |
Well, I have to admit, calculus came earlier,
link |
earlier than linear algebra.
link |
So Newton and Leibniz were the great men
link |
to understand the key ideas of calculus.
link |
But linear algebra to me is like, okay,
link |
it's the starting point,
link |
because it's all about flat things.
link |
Calculus has got, all the complications of calculus
link |
come from the curves, the bending, the curved surfaces.
link |
Linear algebra, the surfaces are all flat.
link |
Nothing bends in linear algebra.
link |
So it should have come first, but it didn't.
link |
And calculus also comes first in high school classes,
link |
in college class, it'll be freshman math,
link |
it'll be calculus, and then I say, enough of it.
link |
Like, okay, get to the good stuff.
link |
Do you think linear algebra should come first?
link |
Well, it really, I'm okay with it not coming first,
link |
but it should, yeah, it should.
link |
Because everything is flat.
link |
Yeah, everything's flat.
link |
Well, of course, for that reason,
link |
calculus sort of sticks to one dimension,
link |
or eventually you do multivariate,
link |
but that basically means two dimensions.
link |
Linear algebra, you take off into 10 dimensions, no problem.
link |
It just feels scary and dangerous
link |
to go beyond two dimensions, that's all.
link |
If everything's flat, you can't go wrong.
link |
So what concept or theorem in linear algebra or in math
link |
you find most beautiful,
link |
that gives you pause that leaves you in awe?
link |
Well, I'll stick with linear algebra here.
link |
I hope the viewer knows that really,
link |
mathematics is amazing, amazing subject
link |
and deep, deep connections between ideas
link |
that didn't look connected, they turned out they were.
link |
But if we stick with linear algebra...
link |
So we have a matrix.
link |
That's like the basic thing, a rectangle of numbers.
link |
And it might be a rectangle of data.
link |
You're probably gonna ask me later about data science,
link |
where often data comes in a matrix.
link |
You have maybe every column corresponds to a drug
link |
and every row corresponds to a patient.
link |
And if the patient reacted favorably to the drug,
link |
then you put up some positive number in there.
link |
Anyway, rectangle of numbers, a matrix is basic.
link |
So the big problem is to understand all those numbers.
link |
You got a big, big set of numbers.
link |
And what are the patterns, what's going on?
link |
And so one of the ways to break down that matrix
link |
into simple pieces is uses something called singular values.
link |
And that's come on as fundamental in the last,
link |
certainly in my lifetime.
link |
Eigenvalues, if you have viewers who've done engineering,
link |
math, or basic linear algebra, eigenvalues were in there.
link |
But those are restricted to square matrices.
link |
And data comes in rectangular matrices.
link |
So you gotta take that next step.
link |
I'm always pushing math faculty, get on, do it, do it.
link |
So those are a way to break, to find the important pieces
link |
of the matrix, which add up to the whole matrix.
link |
So you're breaking a matrix into simple pieces.
link |
And the first piece is the most important part of the data.
link |
The second piece is the second most important part.
link |
And then often, so a data set is a matrix.
link |
And often, so a data scientist will like,
link |
if a data scientist can find those first and second pieces,
link |
stop there, the rest of the data is probably round off,
link |
experimental error maybe.
link |
So you're looking for the important part.
link |
So what do you find beautiful about singular values?
link |
Well, yeah, I didn't give the theorem.
link |
So here's the idea of singular values.
link |
Every matrix, every matrix, rectangular, square, whatever,
link |
can be written as a product
link |
of three very simple special matrices.
link |
So that's the theorem.
link |
Every matrix can be written as a rotation times a stretch,
link |
which is just a diagonal matrix,
link |
otherwise all zeros except on the one diagonal.
link |
And then the third factor is another rotation.
link |
So rotation, stretch, rotation
link |
is the breakup of any matrix.
link |
The structure of that, the ability that you can do that,
link |
what do you find appealing?
link |
What do you find beautiful about it?
link |
Well, geometrically, as I freely admit,
link |
the action of a matrix is not so easy to visualize,
link |
but everybody can visualize a rotation.
link |
Take two dimensional space and just turn it
link |
around the center.
link |
Take three dimensional space.
link |
So a pilot has to know about,
link |
well, what are the three, the yaw is one of them.
link |
I've forgotten all the three turns that a pilot makes.
link |
Up to 10 dimensions, you've got 10 ways to turn,
link |
but you can visualize a rotation.
link |
Take the space and turn it.
link |
And you can visualize a stretch.
link |
So to break a matrix with all those numbers in it
link |
into something you can visualize,
link |
rotate, stretch, rotate is pretty neat.
link |
That's pretty powerful.
link |
On YouTube, just consuming a bunch of videos
link |
and just watching what people connect with
link |
and what they really enjoy and are inspired by,
link |
math seems to come up again and again.
link |
I'm trying to understand why that is.
link |
Perhaps you can help give me clues.
link |
So it's not just the kinds of lectures that you give,
link |
but it's also just other folks like with Numberphile,
link |
there's a channel where they just chat about things
link |
that are extremely complicated, actually.
link |
People nevertheless connect with them.
link |
What do you think that is?
link |
It's wonderful, isn't it?
link |
I mean, I wasn't really aware of it.
link |
We're conditioned to think math is hard,
link |
math is abstract, math is just for a few people,
link |
but it isn't that way.
link |
A lot of people quite like math and they liked it.
link |
I get messages from people saying,
link |
now I'm retired, I'm gonna learn some more math.
link |
I get a lot of those.
link |
It's really encouraging.
link |
And I think what people like is that there's some order,
link |
a lot of order and things are not obvious, but they're true.
link |
So it's really cheering to think that so many people
link |
really wanna learn more about math.
link |
And in terms of truth, again,
link |
sorry to slide into philosophy at times,
link |
but math does reveal pretty strongly what things are true.
link |
I mean, that's the whole point of proving things.
link |
And yet, sort of our real world is messy and complicated.
link |
What do you think about the nature of truth
link |
that math reveals?
link |
Because it is a source of comfort like you've mentioned.
link |
Yeah, that's right.
link |
Well, I have to say, I'm not much of a philosopher.
link |
I just like numbers.
link |
As a kid, this was before you had to go in,
link |
when you had a filly in your teeth,
link |
you had to kind of just take it.
link |
So what I did was think about math,
link |
like take powers of two, two, four, eight, 16,
link |
up until the time the tooth stopped hurting
link |
and the dentist said you're through.
link |
So that was a source of just, source of peace almost.
link |
What is it about math do you think that brings that?
link |
Well, you know where you are.
link |
Yeah, it's symmetry, it's certainty.
link |
The fact that, you know, if you multiply two by itself
link |
10 times, you get 1,024 period.
link |
Everybody's gonna get that.
link |
Do you see math as a powerful tool or as an art form?
link |
That's really one of the neat things.
link |
You can be an artist and like math,
link |
you can be an engineer and use math.
link |
What did you connect with most?
link |
Yeah, I'm somewhere between.
link |
I'm certainly not a artist type, philosopher type person.
link |
Might sound that way this morning, but I'm not.
link |
Yeah, I really enjoy teaching engineers
link |
because they go for an answer.
link |
And yeah, so probably within the MIT math department,
link |
most people enjoy teaching people,
link |
teaching students who get the abstract idea.
link |
I'm okay with, I'm good with engineers
link |
who are looking for a way to find answers.
link |
Actually, that's an interesting question.
link |
Do you think for teaching and in general,
link |
thinking about new concepts,
link |
do you think it's better to plug in the numbers
link |
or to think more abstractly?
link |
So looking at theorems and proving the theorems
link |
or actually building up a basic intuition of the theorem
link |
or the method, the approach,
link |
and then just plugging in numbers and seeing it work.
link |
Yeah, well, certainly many of us like to see examples.
link |
First, we understand,
link |
it might be a pretty abstract sounding example,
link |
like a three dimensional rotation.
link |
How are you gonna understand a rotation in 3D?
link |
And then some of us like to keep going with it
link |
to the point where you got numbers,
link |
where you got 10 angles, 10 axes, 10 angles.
link |
But the best, the great mathematicians probably,
link |
I don't know if they do that,
link |
because for them, an example would be a highly abstract thing
link |
to the rest of it.
link |
Right, but nevertheless, working in the space of examples.
link |
Examples of structure.
link |
Our brains seem to connect with that.
link |
So I'm not sure if you're familiar with him,
link |
but Andrew Yang is a presidential candidate
link |
currently running with math in all capital letters
link |
and his hats as a slogan.
link |
Stands for Make America Think Hard.
link |
Okay, I'll vote for him.
link |
So, and his name rhymes with yours, Yang, Strang.
link |
But he also loves math and he comes from that world
link |
of, but he also, looking at it,
link |
makes me realize that math, science, and engineering
link |
are not really part of our politics, political discourse,
link |
about political government in general.
link |
Why do you think that is?
link |
What are your thoughts on that in general?
link |
Well, certainly somewhere in the system,
link |
we need people who are comfortable with numbers,
link |
comfortable with quantities.
link |
You know, if you say this leads to that,
link |
they see it and it's undeniable.
link |
But isn't that strange to you that we have almost no,
link |
I mean, I'm pretty sure we have no elected officials
link |
in Congress or obviously the president
link |
that either has an engineering degree or a math degree.
link |
Yeah, well, that's too bad.
link |
A few could, a few who could make the connection.
link |
Yeah, it would have to be people who understand
link |
engineering or science and at the same time
link |
can make speeches and lead, yeah.
link |
Yeah, inspire people.
link |
Yeah, inspire, yeah.
link |
You were, speaking of inspiration,
link |
the president of the Society
link |
for Industrial and Applied Mathematics.
link |
It's a major organization in math, applied math.
link |
What do you see as a role of that society,
link |
you know, in our public discourse?
link |
Yeah, so, well, it was fun to be president at the time.
link |
A couple years, a few years.
link |
Two years, around 2000.
link |
I just hope that's president of a pretty small society.
link |
But nevertheless, it was a time when math
link |
was getting some more attention in Washington.
link |
But yeah, I got to give a little 10 minutes
link |
to a committee of the House of Representatives
link |
talking about who I met.
link |
And then, actually, it was fun
link |
because one of the members of the House
link |
had been a student, had been in my class.
link |
What do you think of that?
link |
Yeah, as you say, pretty rare, most members of the House
link |
have had a different training, different background.
link |
But there was one from New Hampshire
link |
who was my friend, really, by being in the class.
link |
Yeah, so those years were good.
link |
Then, of course, other things take over in importance
link |
in Washington, and math just, at this point,
link |
is not so visible.
link |
But for a little moment, it was.
link |
There's some excitement, some concern
link |
about artificial intelligence in Washington now.
link |
Yes, sure. About the future.
link |
Yeah. And I think at the core
link |
Well, it is, yeah.
link |
Maybe it's hidden.
link |
Maybe it's wearing a different hat.
link |
Well, artificial intelligence, and particularly,
link |
can I use the words deep learning?
link |
Deep learning is a particular approach
link |
to understanding data.
link |
Again, you've got a big, whole lot of data
link |
where data is just swamping the computers of the world.
link |
And to understand it, out of all those numbers,
link |
to find what's important in climate, in everything.
link |
And artificial intelligence is two words
link |
for one approach to data.
link |
Deep learning is a specific approach there,
link |
which uses a lot of linear algebra.
link |
I thought, okay, I've gotta learn about this.
link |
So maybe from your perspective,
link |
let me ask the most basic question.
link |
How do you think of a neural network?
link |
What is a neural network?
link |
So can I start with the idea about deep learning?
link |
What does that mean?
link |
What is deep learning?
link |
What is deep learning, yeah.
link |
So we're trying to learn, from all this data,
link |
we're trying to learn what's important.
link |
What's it telling us?
link |
So you've got data, you've got some inputs
link |
for which you know the right outputs.
link |
The question is, can you see the pattern there?
link |
Can you figure out a way for a new input,
link |
which we haven't seen, to understand
link |
what the output will be from that new input?
link |
So we've got a million inputs with their outputs.
link |
So we're trying to create some pattern,
link |
some rule that'll take those inputs,
link |
those million training inputs, which we know about,
link |
to the correct million outputs.
link |
And this idea of a neural net
link |
is part of the structure of our new way to create a rule.
link |
We're looking for a rule that will take
link |
these training inputs to the known outputs.
link |
And then we're gonna use that rule on new inputs
link |
that we don't know the output and see what comes.
link |
Linear algebra is a big part of finding that rule.
link |
That's right, linear algebra is a big part.
link |
People were leaning on matrices, that's good, still do.
link |
Linear is something special.
link |
It's all about straight lines and flat planes.
link |
And data isn't quite like that.
link |
It's more complicated.
link |
So you gotta introduce some complication.
link |
So you have to have some function
link |
that's not a straight line.
link |
And it turned out, nonlinear, nonlinear, not linear.
link |
And it turned out that it was enough to use the function
link |
that's one straight line and then a different one.
link |
Halfway, so piecewise linear.
link |
One piece has one slope,
link |
one piece, the other piece has the second slope.
link |
And so that, getting that nonlinear,
link |
simple nonlinearity in blew the problem open.
link |
That little piece makes it sufficiently complicated
link |
to make things interesting.
link |
Because you're gonna use that piece
link |
over and over a million times.
link |
So it has a fold in the graph, the graph, two pieces.
link |
But when you fold something a million times,
link |
you've got a pretty complicated function
link |
that's pretty realistic.
link |
So that's the thing about neural networks
link |
is they have a lot of these.
link |
A lot of these, that's right.
link |
So why do you think neural networks,
link |
by using sort of formulating an objective function,
link |
very not a plain function of the folds,
link |
lots of folds of the inputs, the outputs,
link |
why do you think they work to be able to find a rule
link |
that we don't know is optimal,
link |
but it just seems to be pretty good in a lot of cases?
link |
What's your intuition?
link |
Is it surprising to you as it is to many people?
link |
Do you have an intuition of why this works at all?
link |
Well, I'm beginning to have a better intuition.
link |
This idea of things that are piecewise linear,
link |
flat pieces but with folds between them.
link |
Like think of a roof of a complicated,
link |
infinitely complicated house or something.
link |
That curve, it almost curved, but every piece is flat.
link |
That's been used by engineers,
link |
that idea has been used by engineers,
link |
is used by engineers, big time.
link |
Something called the finite element method.
link |
If you want to design a bridge,
link |
design a building, design an airplane,
link |
you're using this idea of piecewise flat
link |
as a good, simple, computable approximation.
link |
But you have a sense that there's a lot of expressive power
link |
in this kind of piecewise linear.
link |
Yeah, you used the right word.
link |
If you measure the expressivity,
link |
how complicated a thing can this piecewise flat guys express?
link |
The answer is very complicated, yeah.
link |
What do you think are the limits of such piecewise linear
link |
or just of neural networks?
link |
The expressivity of neural networks.
link |
Well, you would have said a while ago
link |
that they're just computational limits.
link |
It's a problem beyond a certain size.
link |
A supercomputer isn't gonna do it.
link |
But those keep getting more powerful.
link |
So that limit has been moved
link |
to allow more and more complicated surfaces.
link |
So in terms of just mapping from inputs to outputs,
link |
looking at data, what do you think of,
link |
in the context of neural networks in general,
link |
data is just tensor, vectors, matrices, tensors.
link |
How do you think about learning from data?
link |
How much of our world can be expressed in this way?
link |
How useful is this process?
link |
I guess that's another way to ask you,
link |
what are the limits of this approach?
link |
Well, that's a good question, yeah.
link |
So I guess the whole idea of deep learning
link |
is that there's something there to learn.
link |
If the data is totally random,
link |
just produced by random number generators,
link |
then we're not gonna find a useful rule
link |
because there isn't one.
link |
So the extreme of having a rule is like knowing Newton's law.
link |
If you hit a ball, it moves.
link |
So that's where you had laws of physics.
link |
Newton and Einstein and other great, great people
link |
have found those laws and laws of the distribution
link |
of oil in an underground thing.
link |
I mean, so engineers, petroleum engineers understand
link |
how oil will sit in an underground basin.
link |
So there were rules.
link |
Now, the new idea of artificial intelligence is
link |
learn the rules instead of figuring out the rules
link |
with help from Newton or Einstein.
link |
The computer is looking for the rules.
link |
So that's another step.
link |
But if there are no rules at all
link |
that the computer could find,
link |
if it's totally random data, well, you've got nothing.
link |
You've got no science to discover.
link |
It's an automated search for the underlying rules.
link |
Yeah, search for the rules.
link |
And there will be a lot of random parts.
link |
A lot of, I mean, I'm not knocking random
link |
because that's there.
link |
There's a lot of randomness built in,
link |
but there's gotta be some basic.
link |
It's almost always signal, right?
link |
There's gotta be some signal, yeah.
link |
If it's all noise, then you're not gonna get anywhere.
link |
Well, this world around us does seem to be,
link |
does seem to always have a signal of some kind.
link |
Yeah, yeah, that's right.
link |
So what excites you more?
link |
We just talked about a little bit of application.
link |
What excites you more, theory
link |
or the application of mathematics?
link |
Well, for myself, I'm probably a theory person.
link |
I'm not, I'm speaking here pretty freely about applications,
link |
but I'm not the person who really,
link |
I'm not a physicist or a chemist or a neuroscientist.
link |
So for myself, I like the structure
link |
and the flat subspaces
link |
and the relation of matrices, columns to rows.
link |
That's my part in the spectrum.
link |
So really, science is a big spectrum of people
link |
from asking practical questions
link |
and answering them using some math,
link |
then some math guys like myself who are in the middle of it
link |
and then the geniuses of math and physics and chemistry
link |
who are finding fundamental rules
link |
and then doing the really understanding nature.
link |
That's incredible.
link |
At its lowest, simplest level,
link |
maybe just a quick in broad strokes from your perspective,
link |
where does linear algebra sit as a subfield of mathematics?
link |
What are the various subfields that you think about
link |
in relation to linear algebra?
link |
So the big fields of math are algebra as a whole
link |
and problems like calculus and differential equations.
link |
So that's a second, quite different field.
link |
Then maybe geometry deserves to be thought of
link |
as a different field to understand the geometry
link |
of high dimensional surfaces.
link |
So I think, am I allowed to say this here?
link |
I think this is where personal view comes in.
link |
I think math, we're thinking about undergraduate math,
link |
what millions of students study.
link |
I think we overdo the calculus at the cost of the algebra,
link |
at the cost of linear.
link |
So you have this talk titled Calculus Versus Linear Algebra.
link |
That's right, that's right.
link |
And you say that linear algebra wins.
link |
So can you dig into that a little bit?
link |
Why does linear algebra win?
link |
Right, well, okay, the viewer is gonna think
link |
this guy is biased.
link |
Not true, I'm just telling the truth as it is.
link |
Yeah, so I feel linear algebra is just a nice part of math
link |
that people can get the idea of.
link |
They can understand something that's a little bit abstract
link |
because once you get to 10 or 100 dimensions
link |
and very, very, very useful,
link |
that's what's happened in my lifetime
link |
is the importance of data,
link |
which does come in matrix form.
link |
So it's really set up for algebra.
link |
It's not set up for differential equation.
link |
And let me fairly add probability,
link |
the ideas of probability and statistics
link |
have become very, very important, have also jumped forward.
link |
So, and that's different from linear algebra,
link |
So now we really have three major areas to me,
link |
calculus, linear algebra, matrices,
link |
and probability statistics.
link |
And they all deserve an important place.
link |
And calculus has traditionally had a lion's share
link |
A disproportionate share.
link |
It is, thank you, disproportionate, that's a good word.
link |
Of the love and attention from the excited young minds.
link |
I know it's hard to pick favorites,
link |
but what is your favorite matrix?
link |
What's my favorite matrix?
link |
Okay, so my favorite matrix is square, I admit it.
link |
It's a square bunch of numbers
link |
and it has twos running down the main diagonal.
link |
And on the next diagonal,
link |
so think of top left to bottom right,
link |
twos down the middle of the matrix
link |
and minus ones just above those twos
link |
and minus ones just below those twos
link |
and otherwise all zeros.
link |
So mostly zeros, just three nonzero diagonals coming down.
link |
What is interesting about it?
link |
Well, all the different ways it comes up.
link |
You see it in engineering,
link |
you see it as analogous in calculus to second derivative.
link |
So calculus learns about taking the derivative,
link |
the figuring out how much, how fast something's changing.
link |
But second derivative, now that's also important.
link |
That's how fast the change is changing,
link |
how fast the graph is bending, how fast it's curving.
link |
And Einstein showed that that's fundamental
link |
to understand space.
link |
So second derivatives should have a bigger place in calculus.
link |
Second, my matrices,
link |
which are like the linear algebra version
link |
of second derivatives are neat in linear algebra.
link |
Yeah, just everything comes out right with those guys.
link |
What did you learn about the process of learning
link |
by having taught so many students math over the years?
link |
Ooh, that is hard.
link |
I'll have to admit here that I'm not really a good teacher
link |
because I don't get into the exam part.
link |
The exam is the part of my life that I don't like
link |
and grading them and giving the students A or B or whatever.
link |
I do it because I'm supposed to do it,
link |
but I tell the class at the beginning,
link |
I don't know if they believe me.
link |
Probably they don't.
link |
I tell the class, I'm here to teach you.
link |
I'm here to teach you math and not to grade you.
link |
But they're thinking, okay, this guy is gonna,
link |
when is he gonna give me an A minus?
link |
Is he gonna give me a B plus?
link |
What have you learned about the process of learning?
link |
Yeah, well, maybe to give you a legitimate answer
link |
about learning, I should have paid more attention
link |
to the assessment, the evaluation part at the end.
link |
But I like the teaching part at the start.
link |
That's the sexy part.
link |
To tell somebody for the first time about a matrix, wow.
link |
Is there, are there moments,
link |
so you are teaching a concept,
link |
are there moments of learning that you just see
link |
in the student's eyes?
link |
You don't need to look at the grades.
link |
But you see in their eyes that you hook them,
link |
that you connect with them in a way where,
link |
you know what, they fall in love
link |
with this beautiful world of math.
link |
They see that it's got some beauty there.
link |
Or conversely, that they give up at that point
link |
The dark could say that math, I'm just not good at math.
link |
I don't wanna walk away.
link |
Maybe because of the approach in the past,
link |
they were discouraged, but don't be discouraged.
link |
It's too good to miss.
link |
Yeah, well, if I'm teaching a big class,
link |
do I know when, I think maybe I do.
link |
Sort of, I mentioned at the very start,
link |
the four fundamental subspaces
link |
and the structure of the fundamental theorem
link |
of linear algebra.
link |
The fundamental theorem of linear algebra.
link |
That is the relation of those four subspaces,
link |
those four spaces.
link |
Yeah, so I think that, I feel that the class gets it.
link |
What advice do you have to a student
link |
just starting their journey in mathematics today?
link |
How do they get started?
link |
Oh, yeah, that's hard.
link |
Well, I hope you have a teacher, professor,
link |
who is still enjoying what he's doing,
link |
what he's teaching.
link |
They're still looking for new ways to teach
link |
and to understand math.
link |
Cause that's the pleasure,
link |
the moment when you see, oh yeah, that works.
link |
So it's less about the material you study,
link |
it's more about the source of the teacher
link |
being full of passion.
link |
Yeah, more about the fun.
link |
Yeah, the moment of getting it.
link |
But in terms of topics, linear algebra?
link |
Well, that's my topic,
link |
but oh, there's beautiful things in geometry to understand.
link |
What's wonderful is that in the end,
link |
there's a pattern, there are rules
link |
that are followed in biology as there are in every field.
link |
You describe the life of a mathematician
link |
as 100% wonderful.
link |
Except for the grade stuff.
link |
Except for grades.
link |
Yeah, when you look back at your life,
link |
what memories bring you the most joy and pride?
link |
Well, that's a good question.
link |
I certainly feel good when I,
link |
maybe I'm giving a class in 1806,
link |
that's MIT's linear algebra course that I started.
link |
So sort of, there's a good feeling that,
link |
okay, I started this course,
link |
a lot of students take it, quite a few like it.
link |
Yeah, so I'm sort of happy
link |
when I feel I'm helping make a connection
link |
between ideas and students,
link |
between theory and the reader.
link |
Yeah, it's, I get a lot of very nice messages
link |
from people who've watched the videos and it's inspiring.
link |
I just, I'll maybe take this chance to say thank you.
link |
Well, there's millions of students
link |
who you've taught and I am grateful to be one of them.
link |
So Gilbert, thank you so much, it's been an honor.
link |
Thank you for talking today.
link |
It was a pleasure, thanks.
link |
Thank you for listening to this conversation
link |
with Gilbert Strang.
link |
And thank you to our presenting sponsor, Cash App.
link |
Download it, use code LexPodcast,
link |
you'll get $10 and $10 will go to FIRST,
link |
a STEM education nonprofit
link |
that inspires hundreds of thousands of young minds
link |
to learn and to dream of engineering our future.
link |
If you enjoy this podcast, subscribe on YouTube.
link |
We have five stars in Apple Podcast,
link |
support on Patreon or connect with me on Twitter.
link |
Finally, some closing words of advice
link |
from the great Richard Feynman.
link |
Study hard what interests you the most
link |
in the most undisciplined, irreverent
link |
and original manner possible.
link |
Thank you for listening and hope to see you next time.