back to indexGilbert Strang: Linear Algebra, Teaching, and MIT OpenCourseWare | Lex Fridman Podcast #52
link |
The following is a conversation with Gilbert Strang.
link |
He's a professor of mathematics in MIT
link |
and perhaps one of the most famous
link |
and impactful teachers of math in the world.
link |
His MIT OpenCourseWare lectures on linear algebra
link |
have been viewed millions of times.
link |
As an undergraduate student,
link |
I was one of those millions of students.
link |
There's something inspiring about the way he teaches.
link |
There's a once calm, simple, yet full of passion
link |
for the elegance inherent to mathematics.
link |
I remember doing the exercises in his book,
link |
Introduction to Linear Algebra,
link |
and slowly realizing that the world of matrices,
link |
of vector spaces, of determinants and eigenvalues,
link |
of geometric transformations and matrix decompositions
link |
reveal a set of powerful tools
link |
in the toolbox of artificial intelligence.
link |
From signals to images,
link |
from numerical optimization to robotics,
link |
computer vision, deep learning, computer graphics,
link |
and everywhere outside AI,
link |
including, of course, a quantum mechanical study
link |
This is the Artificial Intelligence Podcast.
link |
If you enjoy it, subscribe on YouTube,
link |
give it five stars on Apple Podcasts,
link |
support on Patreon, or simply connect with me on Twitter.
link |
Alex Friedman spelled F R I D M A N.
link |
This podcast is supported by Ziprecruiter.
link |
Hiring great people is hard,
link |
and to me is the most important element
link |
of a successful mission driven team.
link |
I've been fortunate to be a part of
link |
and to lead several great engineering teams.
link |
The hiring I've done in the past
link |
was mostly through tools that we built ourselves,
link |
but reinventing the wheel was painful.
link |
Ziprecruiter is a tool that's already available for you.
link |
It seeks to make hiring simple, fast, and smart.
link |
For example, codable cofounder Gretchen Hebner
link |
used Ziprecruiter to find a new game artist
link |
to join her education tech company.
link |
By using Ziprecruiter's screening questions
link |
to filter candidates,
link |
Gretchen found it easier to focus on the best candidates
link |
and finally hiring the perfect person for the role
link |
in less than two weeks from start to finish.
link |
Ziprecruiter, the smartest way to hire.
link |
See why Ziprecruiter is effective for businesses
link |
of all sizes by signing up as I did for free
link |
at Ziprecruiter.com slash Lex Pod.
link |
That's Ziprecruiter.com slash Lex Pod.
link |
This show is presented by Cash App,
link |
the number one finance app in the App Store.
link |
I personally use Cash App to send money to friends,
link |
but you can also use it to buy, sell, and deposit Bitcoin.
link |
Most Bitcoin exchanges take days
link |
for a bank transfer to become investable.
link |
Through Cash App, it takes seconds.
link |
Cash App also has a new investing feature.
link |
You can buy fractions of a stock,
link |
which to me is a really interesting concept.
link |
So you can buy $1 worth, no matter what the stock price is.
link |
Brokerage services are provided by Cash App Investing,
link |
a subsidiary of Square and member SIPC.
link |
I'm excited to be working with Cash App
link |
to support one of my favorite organizations
link |
that many of you may know and have benefited from,
link |
called FIRST, best known for their FIRST robotics
link |
and Lego competitions.
link |
They educate and inspire hundreds of thousands of students
link |
in over 110 countries and have a perfect rating
link |
on charity navigator, which means the donated money
link |
is used to maximum effectiveness.
link |
When you get Cash App from the App Store or Google Play
link |
and use code LEX Podcast, you'll get $10
link |
and Cash App will also donate $10 to FIRST,
link |
which again, is an organization
link |
that I've personally seen inspire girls and boys
link |
to dream of engineering a better world.
link |
And now, here's my conversation with Gilbert Strang.
link |
How does it feel to be one of the modern day rock stars
link |
I don't feel like a rock star.
link |
That's kind of crazy for old math person.
link |
But it's true that the videos in linear algebra
link |
that I made way back in 2000, I think,
link |
have been watched a lot.
link |
And well, partly the importance of linear algebra,
link |
which I'm sure you'll ask me and give me a chance to say
link |
that linear algebra as a subject is just surged
link |
But also, it was a class that I taught a bunch of times.
link |
So I kind of got it organized and enjoyed doing it.
link |
It was just, the videos were just the class.
link |
So they're on open courseware and on YouTube
link |
and translated, and it's fun.
link |
But there's something about that chalkboard
link |
and the simplicity of the way you explain the basic concepts
link |
in the beginning, to be honest, when I went to undergrad.
link |
You didn't do linear algebra, probably.
link |
Of course I did linear algebra.
link |
Yeah, yeah, yeah, yeah, of course.
link |
But before going through the course at my university,
link |
I was going through open courseware.
link |
You were my instructor for linear algebra.
link |
And that, I mean, we were using your book.
link |
And I mean, the fact that there is thousands,
link |
you know, hundreds of thousands, millions of people
link |
that watch that video, I think that's really powerful.
link |
So how do you think the idea of putting lectures online,
link |
what really MIT Open Courseware has innovated?
link |
That was a wonderful idea.
link |
You know, I think the story that I've heard
link |
is the committee was appointed by the president,
link |
President Vest at that time, a wonderful guy.
link |
And the idea of the committee was to figure out
link |
how MIT could be like other universities,
link |
market the work we were doing.
link |
And then they didn't see a way.
link |
And after a weekend and they had an inspiration
link |
and came back to the president Vest and said,
link |
what if we just gave it away?
link |
And he decided that was okay, good idea.
link |
You know, that's a crazy idea.
link |
That's, if we think of a university as a thing
link |
that creates a product,
link |
isn't knowledge, the kind of educational knowledge
link |
isn't the product and giving that away.
link |
Are you surprised that you went through?
link |
The result that he did it.
link |
Well, knowing a little bit President Vest
link |
was like him, I think.
link |
And it was really the right idea, you know.
link |
MIT is a kind of, it's known for being high level,
link |
And this is the best way we can say,
link |
tell, we can show what MIT really is like.
link |
Because in my case, those 1806 videos
link |
are just teaching the class.
link |
They were there in 26, 100.
link |
They're kind of fun to look at.
link |
People write to me and say,
link |
oh, you've got a sense of humor,
link |
but I don't know where that comes through.
link |
It's somehow friendly with the class.
link |
And linear algebra, the subject,
link |
we gotta give the subject most of the credit.
link |
It really has come forward in importance in these years.
link |
So let's talk about linear algebra a little bit.
link |
Cause it is such a, it's both a powerful and a beautiful
link |
a subfield of mathematics.
link |
So what's your favorite specific topic in linear algebra
link |
or even math in general to give a lecture on,
link |
to convey, to tell a story, to teach students?
link |
Well, on the teaching side,
link |
so it's not deep mathematics at all,
link |
but I'm kind of proud of the idea of the four subspaces,
link |
the four fundamental subspaces,
link |
which are of course known before,
link |
long before my name for them, but...
link |
Can you go through them?
link |
Can you go through the four subspaces?
link |
So the first one to understand is,
link |
so the matrix, maybe I should say the matrix.
link |
What is the matrix?
link |
Well, so we have like a rectangle of numbers.
link |
So it's got n columns, got a bunch of columns
link |
and also got an m rows, let's say.
link |
And the relation between,
link |
so of course the columns and the rows,
link |
it's the same numbers.
link |
So there's gotta be connections there,
link |
but they're not simple.
link |
The columns might be longer than the rows
link |
and they're all different.
link |
The numbers are mixed up.
link |
First space to think about is, take the columns.
link |
So those are vectors, those are points in dimensions.
link |
So a fist test would imagine a vector
link |
or might imagine a vector as a arrow in space
link |
or the point it ends at in space.
link |
For me, it's a column of numbers.
link |
You often think of, this is very interesting
link |
in terms of linear algebra, in terms of a vector.
link |
You think a little bit more abstract
link |
than how it's very commonly used perhaps.
link |
You think this arbitrary multidimensional space.
link |
I'm right away, I'm in high dimensions and in the...
link |
Yeah, that's right, in the lecture, I try to...
link |
So if you think of two vectors in 10 dimensions,
link |
I'll do this in class and I'll readily admit
link |
that I have no good image in my mind
link |
of a vector of an arrow in 10 dimensional space,
link |
but whatever, you can add one bunch of 10 numbers
link |
to another bunch of 10 numbers.
link |
So you can add a vector to a vector
link |
and you can multiply a vector by three.
link |
And that's, if you know how to do those,
link |
you've got linear algebra.
link |
You know, 10 dimensions, there's this beautiful thing
link |
about math, if you look at string theory
link |
and all these theories,
link |
which are really fundamentally derived through math,
link |
but are very difficult to visualize.
link |
How do you think about the things
link |
like a 10 dimensional vector
link |
that we can't really visualize?
link |
And yet math reveals some beauty
link |
underlying our world in that weird thing we can't visualize.
link |
How do you think about that difference?
link |
Well, probably I'm not a very geometric person,
link |
so I'm probably thinking in three dimensions
link |
and the beauty of linear algebra is that
link |
it goes on to 10 dimensions with no problem.
link |
I mean, if you're just seeing what happens
link |
if you add two vectors in 3D,
link |
you then you can add them in 10D.
link |
You're just adding the 10 components.
link |
So I can't say that I have a picture,
link |
but yet I try to push the class
link |
to think of a flat surface in 10 dimensions.
link |
So a plane in 10 dimensions.
link |
And so that's one of the spaces.
link |
Take all the columns of the matrix,
link |
take all their combinations,
link |
so much of this column, so much of this one.
link |
Then if you put all those together,
link |
you get some kind of a flat surface
link |
that I call a vector space, space of vectors.
link |
And my imagination is just seeing
link |
like a piece of paper in 3D.
link |
But anyway, so that's one of the spaces,
link |
that's space number one,
link |
the column space of the matrix.
link |
And then there's the row space,
link |
which is, as I said, different,
link |
but came from the same numbers.
link |
So we got the column space,
link |
all combinations of the columns.
link |
And then we got the row space,
link |
all combinations of the rows.
link |
So those words are easy for me to say,
link |
and I can't really draw them on a blackboard,
link |
but I try with my thick chalk.
link |
Everybody likes that railroad chalk.
link |
And me too, I wouldn't use anything else now.
link |
And then the other two spaces are perpendicular to those.
link |
So like if you have a plane in 3D,
link |
just a plane is just a flat surface in 3D,
link |
then perpendicular to that plane would be a line.
link |
So that would be the null space.
link |
So we've got two, we've got a column space,
link |
a row space, and there are two perpendicular spaces.
link |
So those four fit together
link |
in a beautiful picture of a matrix.
link |
Yeah, yeah, it's sort of fundamental.
link |
It's not a difficult idea.
link |
It comes pretty early in 1806, and it's basic.
link |
So planes in these multi dimensional spaces,
link |
how difficult of an idea is that to come to?
link |
Do you think if you look back in time,
link |
I think mathematically it makes sense,
link |
but I don't know if it's intuitive for us to imagine,
link |
just as what we're talking about.
link |
Feels like calculus is easier to intuit.
link |
Well, calculus, I have to admit calculus came earlier,
link |
earlier than linear algebra.
link |
So Newton and Leibniz were the great men
link |
to understand the key ideas of calculus.
link |
But linear algebra to me is like,
link |
okay, it's the starting point,
link |
because it's all about flat things.
link |
Calculus has got all the complications of calculus
link |
come from the curves, the bending, the curved surfaces.
link |
Linear algebra, the surfaces are all flat.
link |
Nothing bends in linear algebra.
link |
So it should have come first, but it didn't.
link |
And calculus also comes first in high school classes,
link |
and in college class, it'll be freshman math,
link |
it'll be calculus.
link |
And then I say, enough of it.
link |
Like, okay, get to the good stuff.
link |
Do you think linear algebra should come first?
link |
Well, it really, I'm okay with it not coming first,
link |
but it should, yeah, it should.
link |
Because everything's flat.
link |
Yeah, everything's flat.
link |
Well, of course, for that reason,
link |
calculus sort of sticks to one dimension,
link |
or eventually you do multivariate,
link |
but that basically means two dimensions.
link |
Linear algebra, you take off into 10 dimensions, no problem.
link |
It just feels scary and dangerous
link |
to go beyond two dimensions, that's all.
link |
If everything's flat, you can't go wrong.
link |
So what concept or theorem in linear algebra or in math
link |
you find most beautiful?
link |
That gives you pause that leaves you in awe?
link |
Well, I'll stick with linear algebra here.
link |
I hope the viewer knows that really mathematics
link |
is an amazing, amazing subject
link |
and deep connections between ideas
link |
that didn't look connected.
link |
Some, they turned out they were.
link |
But if we stick with linear algebra,
link |
so we have a matrix, that's like the basic thing,
link |
a rectangle of numbers.
link |
And it might be a rectangle of data,
link |
you're probably gonna ask me later about data science,
link |
where often data comes in a matrix,
link |
you have maybe every column corresponds
link |
to a drug and every row corresponds to a patient.
link |
And if the patient reacted favorably to the drug,
link |
then you put up some positive number in there.
link |
Anyway, rectangle of numbers, matrix is basic.
link |
So the big problem is to understand all those numbers.
link |
You got a big set of numbers.
link |
And what are the patterns, what's going on?
link |
And so one of the ways to break down that
link |
matrix into simple pieces is,
link |
uses something called singular values.
link |
And that's come on as fundamental in the last
link |
and certainly in my lifetime.
link |
Eigen values, if you have viewers who've done engineering
link |
math or basic linear algebra,
link |
Eigen values were in there.
link |
But those are restricted to square matrices
link |
and data comes in rectangular matrices.
link |
So you gotta take that next step.
link |
I'm always pushing math faculty, get on, do it, do it,
link |
do it, singular values.
link |
So those are a way to find the important pieces
link |
of the matrix, which add up to the whole matrix.
link |
So you're breaking a matrix into simple pieces.
link |
And the first piece is the most important part of the data,
link |
the second piece is the second most important part.
link |
And then often, so a data scientist will like,
link |
if a data scientist can find those first and second pieces,
link |
stop there, the rest of the data is probably round off,
link |
experimental error maybe.
link |
So you're looking for the important part.
link |
So what do you find beautiful about singular values?
link |
Well, yeah, I didn't give the theorem.
link |
So here's the idea of singular values.
link |
Every matrix, every matrix, rectangular, square,
link |
whatever, can be written as a product
link |
of three very simple special matrices.
link |
So that's the theorem.
link |
Every matrix can be written as a rotation
link |
times a stretch, which is just a matrix,
link |
a diagonal matrix, otherwise all zeros
link |
except on the one diagonal.
link |
And then the third factor is another rotation.
link |
So rotation, stretch, rotation is the breakup
link |
The structure, the ability that you can do that,
link |
what do you find appealing?
link |
What do you find beautiful about it?
link |
Well, geometrically, as I freely admit,
link |
the action of a matrix is not so easy to visualize,
link |
but everybody can visualize a rotation.
link |
Take two dimensional space and just turn it
link |
around the center.
link |
Take three dimensional space.
link |
So a pilot has to know about, well,
link |
what are the three, yaw is one of them?
link |
I've forgotten all the three turns that a pilot makes.
link |
Up to 10 dimensions, you've got 10 ways to turn,
link |
but you can visualize a rotation.
link |
Take the space and turn it.
link |
And you can visualize a stretch.
link |
So to break a matrix with all those numbers in it
link |
into something you can visualize, rotate, stretch, rotate.
link |
That's pretty powerful.
link |
On YouTube, just consuming a bunch of videos
link |
and just watching what people connect with
link |
and what they really enjoy and are inspired by,
link |
math seems to come up again and again.
link |
I'm trying to understand why that is.
link |
Perhaps you can help give me clues.
link |
So it's not just the kinds of lectures that you give,
link |
but it's also just other folks
link |
like with Numberphile, there's a channel
link |
where they just chat about things
link |
that are extremely complicated, actually.
link |
People nevertheless connect with them.
link |
What do you think that is?
link |
It's wonderful, isn't it?
link |
I mean, I wasn't really aware of it.
link |
We're conditioned to think math is hard, math is abstract,
link |
math is just for a few people,
link |
but it isn't that way.
link |
A lot of people quite like math and I get messages
link |
from people saying, you know, now I'm retired,
link |
I'm gonna learn some more math.
link |
I get a lot of those, it's really encouraging.
link |
And I think what people like is that there's some order,
link |
a lot of order and things are not obvious,
link |
So it's really cheering to think that so many people
link |
really want to learn more about math.
link |
In terms of truth, again, I'm sorry to slide
link |
into philosophy at times, but math does reveal
link |
pretty strongly what things are true.
link |
I mean, that's the whole point of proving things.
link |
And yet sort of our real world is messy and complicated.
link |
What do you think about the nature of truth
link |
that math reveals?
link |
Because it is a source of comfort, like you've mentioned.
link |
Yeah, that's right.
link |
Well, I have to say, I'm not much of a philosopher.
link |
I just like numbers, you know, as a kid,
link |
this was before you had to go in
link |
when you had a filling in your teeth,
link |
you had to kind of just take it.
link |
So what I did was think about math,
link |
like take powers of two, two, four, eight, 16,
link |
up until the time the tooth stopped hurting
link |
and the dentist said you're through.
link |
Or counting, yeah, so.
link |
So that was a source of just, a source of peace, almost.
link |
What is it about math do you think that brings that?
link |
I don't know where you are.
link |
Yeah, symmetry, it's certainty.
link |
The fact that, you know, if you multiply two
link |
by itself 10 times, you get 1,024 period.
link |
Everybody's gonna get that.
link |
Do you see math as a powerful tool or as an art form?
link |
So it's both, that's really one of the neat things.
link |
You can be an artist and like math,
link |
you can be an engineer and use math.
link |
What did you connect with most?
link |
Yeah, I'm somewhere between.
link |
I'm certainly not a artist type, philosopher type person.
link |
Might sound that way this morning, but I'm not.
link |
Yeah, I really enjoy teaching engineers
link |
because they go for an answer.
link |
And yeah, so probably within the MIT Math Department,
link |
most people enjoy teaching students
link |
who get the abstract idea.
link |
I'm okay with, I'm good with engineers
link |
who are looking for a way to find answers.
link |
Actually, that's an interesting question.
link |
Do you think for teaching and in general,
link |
thinking about new concepts,
link |
do you think it's better to plug in the numbers
link |
or to think more abstractly?
link |
So looking at theorems and proving the theorems
link |
or actually building up a basic intuition of the theorem
link |
or the methodology approach
link |
and then just plugging in numbers and seeing it work?
link |
Yeah, well, certainly many of us like to see examples.
link |
First, we understand,
link |
it might be a pretty abstract sounding example
link |
like a three dimensional rotation.
link |
How are you gonna understand a rotation in 3D?
link |
And then some of us like to keep going with it
link |
to the point where you got numbers,
link |
where you got 10 angles, 10 axes, 10 angles.
link |
But the best, the great mathematicians is probably,
link |
I don't know if they do that
link |
because for them, an example would be
link |
a highly abstract thing to the rest of us.
link |
Right, but nevertheless working in the space of examples.
link |
It's examples of structure.
link |
Our brains seem to connect with that.
link |
So I'm not sure if you're familiar with them
link |
but Andrew Yang is a presidential candidate
link |
currently running with math
link |
in all capital letters and his hats as a slogan.
link |
Stands for make America think hard.
link |
Okay, I'll vote for him.
link |
And his name rhymes with yours, Yang Strang.
link |
But he also loves math and he comes from that world
link |
but he also looking at it makes me realize
link |
that math, science, and engineering
link |
are not really part of our politics,
link |
political discourse about political life,
link |
government in general.
link |
Why do you think that is?
link |
What are your thoughts on that in general?
link |
Well, certainly somewhere in the system
link |
we need people who are comfortable with numbers,
link |
comfortable with quantities,
link |
you know, if you say this leads to that,
link |
they see it and it's undeniable.
link |
But isn't that strange to you that we have almost no...
link |
I mean, I'm pretty sure we have no elected officials
link |
in Congress or obviously the president
link |
that either has an engineering degree or math.
link |
Yeah, well, that's too bad.
link |
A few who could make the connection.
link |
Yeah, it would have to be people who are...
link |
who understand engineering or science
link |
and at the same time can make speeches and lead.
link |
Yeah, inspire people.
link |
You were, speaking of inspiration,
link |
the president of the Society for Industrial and Applied Mathematics.
link |
It's a major organization in math and applied math.
link |
What do you see as a role of that society,
link |
you know, in our public discourse?
link |
So, well, it was fun to be president at the time.
link |
Two years, around 2000.
link |
There's hope that's present of a pretty small society.
link |
But nevertheless, it was a time when math was getting some...
link |
more attention in Washington.
link |
But, yeah, I got to give a little 10 minutes
link |
to a committee of the House of Representatives
link |
talking about who I met.
link |
And then, actually, it was fun,
link |
because one of the members of the House
link |
had been a student, had been in my class.
link |
What do you think of that?
link |
Yeah, as you say, a pretty rare.
link |
Most members of the House have had a different training,
link |
different background,
link |
but there was one from New Hampshire
link |
who was my friend, really, by being in the class.
link |
So, those years were good.
link |
Then, of course, other things take over
link |
and importance in Washington.
link |
And math, just at this point, is not so visible.
link |
But for a little moment, it was.
link |
There's some excitement, some concern
link |
about artificial intelligence in Washington now.
link |
And I think at the core of that is math.
link |
Well, it is, yeah.
link |
Maybe it's hidden.
link |
Maybe it's wearing a different hat.
link |
Well, artificial intelligence,
link |
and particularly, can I use the words, deep learning,
link |
if the deep learning is a particular approach
link |
to understanding data.
link |
Again, you've got a big whole lot of data.
link |
Data is just swamping the computers of the world
link |
and to understand it,
link |
to out of all those numbers to find what's important
link |
in climate and everything.
link |
And artificial intelligence is two words
link |
for one approach to data.
link |
Deep learning is a specific approach there,
link |
which uses a lot of linear algebra.
link |
So, I got into it.
link |
I thought, okay, I've got to learn about this.
link |
So, maybe from your perspective,
link |
let me ask the most basic question.
link |
How do you think of a neural network?
link |
What is a neural network?
link |
So, can I start with the idea about deep learning?
link |
What does that mean?
link |
What is deep learning?
link |
What is deep learning?
link |
So, we're trying to learn from all this data,
link |
we're trying to learn what's important,
link |
what's it telling us.
link |
So, you've got data.
link |
You've got some inputs for which you know the right outputs.
link |
The question is, can you see the pattern there?
link |
Can you figure out a way for a new input,
link |
which we haven't seen, to get the,
link |
to understand what the output will be from that new input.
link |
So, we've got a million inputs with their outputs.
link |
So, we're trying to create some pattern,
link |
some rule that will take those inputs,
link |
those million training inputs,
link |
which we know about, to the correct million outputs.
link |
And this idea of a neural net is part of the structure
link |
of our new way to create a rule.
link |
We're looking for a rule that will take these training inputs
link |
to the known outputs.
link |
And then we're going to use that rule on new inputs
link |
that we don't know the output and see what comes.
link |
Linear algebra is a big part of defining that rule.
link |
That's right. Linear algebra is a big part.
link |
People were leaning on matrices, that's good, still do.
link |
Linear is something special.
link |
It's all about straight lines and flat planes.
link |
And data isn't quite like that, you know.
link |
It's more complicated.
link |
So, you've got to introduce some complication.
link |
You have to have some function that's not a straight line.
link |
And it turned out nonlinear, nonlinear, not linear.
link |
And it turned out that it was enough to use the function
link |
that's one straight line and then a different one.
link |
Halfway, so piecewise linear.
link |
One piece has one slope, one piece,
link |
the other piece has a second slope.
link |
And so, getting that nonlinear,
link |
simple nonlinearity in blew the problem open.
link |
That little piece makes it sufficiently complicated
link |
to make things interesting.
link |
Exactly, because you're going to use that piece
link |
over and over a million times.
link |
So, it has a fold in the graph.
link |
The graph is two pieces.
link |
But when you fold something a million times,
link |
you've got a pretty complicated function
link |
that's pretty realistic.
link |
So, that's the thing about neural networks is
link |
they have a lot of these.
link |
So, why do you think neural networks
link |
by using a, so formulating an objective function,
link |
very not a plain function of the,
link |
lots of folds of the inputs, the outputs,
link |
why do you think they work to be able to find a rule
link |
that we don't know is optimal,
link |
but it just seems to be pretty good in a lot of cases.
link |
What's your intuition?
link |
Is it surprising to you as it is to many people?
link |
Do you have an intuition of why this works at all?
link |
Well, I'm beginning to have a better intuition.
link |
This idea of things that are piecewise linear,
link |
flat pieces, but with folds between them.
link |
Like, think of a roof of an infinitely complicated house
link |
or something, that curve, it almost curved,
link |
but every piece is flat.
link |
That's been used by engineers.
link |
That idea has been used by engineers, is used by engineers.
link |
Big time, something called the finite element method.
link |
If you want to design a bridge,
link |
design a building, design an airplane,
link |
you're using this idea of piecewise flat
link |
as a good, simple, computable approximation.
link |
But you have a sense that there's a lot of expressive power
link |
in this kind of piecewise linear.
link |
You use the right word.
link |
If you measure the expressivity,
link |
how complicated a thing can this piecewise flat guy's
link |
express, the answer is very complicated.
link |
What do you think are the limits of such piecewise linear
link |
or just neural networks, the expressivity of neural networks?
link |
Well, you would have said a while ago
link |
that they're just computational limits.
link |
It's a problem beyond a certain size.
link |
A supercomputer isn't going to do it.
link |
But those keep getting more powerful.
link |
So that limit has been moved to allow
link |
more and more complicated surfaces.
link |
So in terms of just mapping from inputs to outputs,
link |
looking at data, what do you think of,
link |
in the context in neural networks in general,
link |
data is just tensors, vectors, matrices, tensors.
link |
How do you think about learning from data?
link |
How much of our world can be expressed in this way?
link |
How useful is this process?
link |
I guess that's another way to ask,
link |
what are the limits of this approach?
link |
Well, that's a good question, yeah.
link |
So I guess the whole idea of deep learning
link |
is that there's something there to learn.
link |
If the data is totally random,
link |
just produced by random number generators,
link |
then we're not going to find a useful rule
link |
because there isn't one.
link |
So the extreme of having a rule
link |
is like knowing Newton's law,
link |
if you hit a ball and it moves.
link |
So that's where you had laws of physics.
link |
Newton and Einstein and other great people
link |
have found those laws
link |
and laws of the distribution of oil
link |
in an underground thing.
link |
So engineers, petroleum engineers,
link |
understand how oil will sit in an underground basin.
link |
So there were rules.
link |
Now the new idea of artificial intelligence is
link |
to find the rules.
link |
Instead of figuring out the rules
link |
with help from Newton or Einstein,
link |
the computer is looking for the rules.
link |
So that's another step.
link |
But if there are no rules at all
link |
that the computer could find,
link |
if it's totally random data,
link |
well, you've got nothing.
link |
You've got no science to discover.
link |
It's automated search for the underlying rules.
link |
Yeah, search for the rules, yeah, exactly.
link |
And there will be a lot of random parts.
link |
I mean, I'm not knocking random
link |
because that's there.
link |
There's a lot of randomness built in,
link |
but there's got to be some basic structure.
link |
There's got to be some signal.
link |
If it's all noise, then you're not going to get anywhere.
link |
Well, this world around us
link |
does seem to always have a signal of some kind
link |
So what excites you more?
link |
We just talked about a little bit of application.
link |
What excites you more, theory
link |
or the application of mathematics?
link |
I'm probably a theory person.
link |
I'm speaking here pretty freely about applications,
link |
but I'm not the person who really...
link |
I'm not a physicist or a chemist or a neuroscientist.
link |
So for myself, I like the structure
link |
and the flat subspaces
link |
and the relation of matrices, columns to rows.
link |
That's my part in the spectrum.
link |
So really, science is a big spectrum of people
link |
from asking practical questions and answering them
link |
then some math guys like myself who are in the middle of it,
link |
and then the geniuses of math and physics and chemistry
link |
who are finding fundamental rules
link |
and doing really understanding nature.
link |
That's incredible.
link |
At its lowest, simplest level,
link |
maybe just a quick and broad strokes from your perspective.
link |
Where does linear algebra sit as a subfield of mathematics?
link |
What are the various subfields that you think about
link |
in relation to linear algebra?
link |
So the big fields of math are algebra as a whole
link |
and problems like calculus and differential equations.
link |
So that's a second quite different field
link |
than maybe geometry.
link |
It deserves to be thought of as a different field
link |
to understand the geometry of high dimensional surfaces.
link |
Am I allowed to say this here?
link |
This is where personal view comes in.
link |
I think we're thinking about undergraduate math,
link |
what millions of students study.
link |
I think we overdo the calculus at the cost of the algebra,
link |
at the cost of linear.
link |
See if this dog titled calculus versus linear algebra.
link |
And you say that linear algebra wins.
link |
Can you dig into that a little bit?
link |
Why does linear algebra win?
link |
The viewer is going to think this guy is biased.
link |
I'm just telling the truth as it is.
link |
So I feel linear algebra is just a nice part of math
link |
that people can get the idea of.
link |
They understand something that's a little bit abstract,
link |
because once you get to 10 or 100 dimensions,
link |
and very, very, very useful.
link |
That's what's happened in my lifetime,
link |
is the importance of data,
link |
which does come in matrix form.
link |
So it's really set up for algebra.
link |
It's not set up for differential equation.
link |
And let me fairly add probability.
link |
This is a probability.
link |
And statistics have become very, very important.
link |
I've also jumped forward.
link |
And that's different from linear algebra.
link |
So now we really have three major areas to me.
link |
Calculus, linear algebra, matrices,
link |
and probability statistics.
link |
And they all deserve an important place.
link |
And calculus has traditionally had a lion's share of the time.
link |
A disproportionate share.
link |
Thank you. Disproportionate.
link |
That's a good word.
link |
Of the love and attention from the excited young minds.
link |
I know it's hard to pick favorites,
link |
but what is your favorite matrix?
link |
My favorite matrix.
link |
So my favorite matrix is square.
link |
It's a square bunch of numbers.
link |
And it has twos running down the main diagonal.
link |
And on the next diagonal,
link |
so think of top left to bottom right,
link |
twos down the middle of the matrix.
link |
And minus ones just above those twos
link |
and minus ones just below those twos.
link |
And otherwise all zeros. So mostly zeros.
link |
Just three nonzero diagonals coming down.
link |
What is interesting about it?
link |
Well, all the different ways it comes up.
link |
You see it in engineering.
link |
You see it as analogous in calculus to second derivative.
link |
So calculus learns about taking the derivative,
link |
figuring out how fast something's changing.
link |
But second derivative.
link |
Now that's also important.
link |
That's how fast the change is changing.
link |
How fast the graph is bending.
link |
How fast it's curving.
link |
And Einstein showed that that's fundamental to understand space.
link |
So second derivatives should have a bigger place in calculus.
link |
Second, my matrices which are like the linear algebra
link |
version of second derivatives are neat in linear algebra.
link |
Just everything comes out right with those guys.
link |
What did you learn about the process of learning
link |
by having taught so many students math over the years?
link |
Oh, that is hard. I'll have to admit here that I'm not really
link |
a good teacher because I don't get into the exam part.
link |
The exam is the part of my life that I don't like.
link |
And grading them and giving the students A or B or whatever.
link |
I do it because I'm supposed to do it.
link |
But I tell the class at the beginning.
link |
I don't know if they believe me. Probably they don't.
link |
I tell the class, I'm here to teach you.
link |
I'm here to teach you math and not to grade you.
link |
But they're thinking, OK, this guy, is he going to give me
link |
Is he going to give me a B plus?
link |
What did you learn about the process of learning?
link |
Well, maybe to give you a legitimate answer about learning,
link |
I should have paid more attention to the assessment,
link |
the evaluation part at the end.
link |
But I like the teaching part at the start.
link |
That's the sexy part, to tell somebody for the first time
link |
about a matrix. Wow.
link |
Are there moments, so you are teaching a concept,
link |
are there moments of learning that you just see
link |
in the students eyes?
link |
You don't need to look at the grades.
link |
You see in their eyes that you hook them,
link |
that you connect with them in a way where they fall in love
link |
with this beautiful world of math.
link |
They see that it's got some beauty there.
link |
Or conversely, that they give up at that point.
link |
It's the opposite.
link |
The darker say that math, I'm just not good at math.
link |
I don't want to walk away.
link |
Maybe because of the approach in the past,
link |
they were discouraged, but don't be discouraged.
link |
It's too good to miss.
link |
Well, if I'm teaching a big class, do I know when,
link |
I think maybe I do.
link |
Sort of, I mentioned at the very start,
link |
the four fundamental subspaces and the structure
link |
of the fundamental theorem of linear algebra.
link |
The fundamental theorem of linear algebra.
link |
That is the relation of those four subspaces.
link |
Those four spaces.
link |
So I think that I feel that the class gets it.
link |
What advice do you have to a student just starting
link |
their journey in mathematics today?
link |
How do they get started?
link |
Yeah, that's hard.
link |
Well, I hope you have a teacher,
link |
a professor who is still enjoying what he's doing,
link |
what he's teaching.
link |
He's still looking for new ways to teach
link |
and to understand math.
link |
Because that's the pleasure to the moment when you see,
link |
oh yeah, that works.
link |
So it says about the material you study.
link |
It's more about the source of the teacher
link |
being full of passion.
link |
Yeah, more about the fun.
link |
The moment of getting it.
link |
But in terms of topics, linear algebra?
link |
Well, that's my topic.
link |
But oh, there's beautiful things in geometry to understand.
link |
What's wonderful is that in the end, there's a pattern there.
link |
There are rules that are followed in biology
link |
as there are in every field.
link |
You describe the life of a mathematician as 100% wonderful,
link |
except for the grade stuff.
link |
Except for grades.
link |
Yeah, when you look back at your life,
link |
what memories bring you the most joy and pride?
link |
Well, that's a good question.
link |
I certainly feel good when I maybe I'm giving a class in 1806.
link |
That's MIT's linear algebra course that I started.
link |
So there's a good feeling that, OK, I started this course.
link |
A lot of students take it.
link |
Quite a few like it.
link |
Yeah, so I'm sort of happy when I feel
link |
I'm helping make a connection between ideas and students,
link |
between theory and the reader.
link |
I get a lot of very nice messages from people who've watched the videos
link |
and it's inspiring.
link |
I'll maybe take this chance to say thank you.
link |
Well, there's millions of students who you've taught
link |
and I am grateful to be one of them.
link |
So Gilbert, thank you so much.
link |
It's been an honor.
link |
Thank you for talking today.
link |
It was a pleasure.
link |
Thank you for listening to this conversation with Gilbert Strang.
link |
And thank you to our presenting sponsor, Cash App.
link |
Use code LEX Podcast.
link |
You'll get $10 and $10 will go to FIRST,
link |
a STEM education nonprofit that inspires hundreds of thousands
link |
of young minds to learn and to dream of engineering our future.
link |
If you enjoy this podcast, subscribe on YouTube.
link |
We've got five stars in Apple Podcasts.
link |
Support on Patreon.
link |
I'll connect with me on Twitter.
link |
Finally, some closing words of advice from the great Richard Feynman.
link |
Study hard what interests you the most in the most undisciplined,
link |
irreverent, and original manner possible.
link |
Thank you for listening and hope to see you next time.