back to index## Gilbert Strang: Linear Algebra, Teaching, and MIT OpenCourseWare | Lex Fridman Podcast #52

link |

The following is a conversation with Gilbert Strang.

link |

He's a professor of mathematics at MIT

link |

and perhaps one of the most famous

link |

and impactful teachers of math in the world.

link |

His MIT OpenCourseWare lectures on linear algebra

link |

have been viewed millions of times.

link |

As an undergraduate student,

link |

I was one of those millions of students.

link |

There's something inspiring about the way he teaches.

link |

There's at once calm, simple, and yet full of passion

link |

for the elegance inherent to mathematics.

link |

I remember doing the exercise in his book,

link |

Introduction to Linear Algebra,

link |

and slowly realizing that the world of matrices,

link |

of vector spaces, of determinants and eigenvalues,

link |

of geometric transformations and matrix decompositions

link |

reveal a set of powerful tools

link |

in the toolbox of artificial intelligence.

link |

From signals to images,

link |

from numerical optimization to robotics,

link |

computer vision, deep learning, computer graphics,

link |

and everywhere outside AI,

link |

including, of course, a quantum mechanical study

link |

This is the Artificial Intelligence Podcast.

link |

If you enjoy it, subscribe on YouTube,

link |

give it five stars on Apple Podcast,

link |

support on Patreon,

link |

or simply connect with me on Twitter

link |

at Lex Friedman, spelled F R I D M A N.

link |

This podcast is supported by ZipRecruiter.

link |

Hiring great people is hard,

link |

and to me is the most important element

link |

of a successful mission driven team.

link |

I've been fortunate to be a part of

link |

and to lead several great engineering teams.

link |

The hiring I've done in the past

link |

was mostly through tools that we built ourselves,

link |

but reinventing the wheel was painful.

link |

ZipRecruiter is a tool that's already available for you.

link |

It seeks to make hiring simple, fast, and smart.

link |

For example, Codable cofounder Gretchen Huebner

link |

used ZipRecruiter to find a new game artist

link |

to join her education tech company.

link |

By using ZipRecruiter's screening questions

link |

to filter candidates,

link |

Gretchen found it easier to focus on the best candidates

link |

and finally hiring the perfect person for the role

link |

in less than two weeks from start to finish.

link |

ZipRecruiter, the smartest way to hire.

link |

See why ZipRecruiter is effective

link |

for businesses of all sizes by signing up,

link |

as I did, for free at ziprecruiter.com slash lexpod.

link |

That's ziprecruiter.com slash lexpod.

link |

This show is presented by Cash App,

link |

the number one finance app in the App Store.

link |

I personally use Cash App to send money to friends,

link |

but you can also use it to buy, sell, and deposit Bitcoin.

link |

Most Bitcoin exchanges take days

link |

for a bank transfer to become investable.

link |

Through Cash App, it takes seconds.

link |

Cash App also has a new investing feature.

link |

You can buy fractions of a stock,

link |

which to me is a really interesting concept.

link |

So you can buy $1 worth no matter what the stock price is.

link |

Brokerage services are provided by Cash App Investing,

link |

a subsidiary of Square, and member SIPC.

link |

I'm excited to be working with Cash App

link |

to support one of my favorite organizations

link |

that many of you may know and have benefited from,

link |

called First, best known for their first robotics

link |

and Lego competitions.

link |

They educate and inspire hundreds of thousands of students

link |

in over 110 countries,

link |

and have a perfect rating on Charity Navigator,

link |

which means the donated money is used

link |

to maximum effectiveness.

link |

When you get Cash App from the App Store or Google Play,

link |

and use code LexPodcast,

link |

you get $10, and Cash App will also donate $10 to First,

link |

which again, is an organization

link |

that I've personally seen inspire girls and boys

link |

to dream of engineering a better world.

link |

And now, here's my conversation with Gilbert Strang.

link |

How does it feel to be one of the modern day rock stars

link |

I don't feel like a rock star.

link |

That's kind of crazy for an old math person.

link |

But it's true that the videos in linear algebra

link |

that I made way back in 2000, I think,

link |

have been watched a lot.

link |

And well, partly the importance of linear algebra,

link |

which I'm sure you'll ask me,

link |

and give me a chance to say that linear algebra

link |

as a subject has just surged in importance.

link |

But also, it was a class that I taught a bunch of times,

link |

so I kind of got it organized and enjoyed doing it.

link |

The videos were just the class.

link |

So they're on OpenCourseWare and on YouTube

link |

and translated, and it's fun.

link |

But there's something about that chalkboard

link |

and the simplicity of the way you explain

link |

the basic concepts in the beginning.

link |

To be honest, when I went to undergrad.

link |

You didn't do linear algebra, probably.

link |

Of course I didn't do linear algebra.

link |

Yeah, yeah, yeah, of course.

link |

But before going through the course at my university,

link |

there was going through OpenCourseWare.

link |

You were my instructor for linear algebra.

link |

And that, I mean, we're using your book.

link |

And I mean, the fact that there is thousands,

link |

hundreds of thousands, millions of people

link |

that watch that video, I think that's really powerful.

link |

So how do you think the idea of putting lectures online,

link |

what really MIT OpenCourseWare has innovated?

link |

That was a wonderful idea.

link |

I think the story that I've heard is the committee

link |

was appointed by the president, President Vest,

link |

at that time, a wonderful guy.

link |

And the idea of the committee was to figure out

link |

how MIT could be like other universities,

link |

market the work we were doing.

link |

And then they didn't see a way.

link |

And after a weekend, and they had an inspiration,

link |

came back to President Vest and said,

link |

what if we just gave it away?

link |

And he decided that was okay, good idea.

link |

You know, that's a crazy idea.

link |

If we think of a university as a thing

link |

that creates a product, isn't knowledge,

link |

the kind of educational knowledge,

link |

isn't the product and giving that away,

link |

are you surprised that it went through?

link |

The result that he did it,

link |

well, knowing a little bit President Vest, it was like him,

link |

I think, and it was really the right idea.

link |

MIT is a kind of, it's known for being high level,

link |

technical things, and this is the best way we can say,

link |

tell, we can show what MIT really is like,

link |

because in my case, those 1806 videos

link |

are just teaching the class.

link |

They were there in 26, 100.

link |

They're kind of fun to look at.

link |

People write to me and say, oh, you've got a sense of humor,

link |

but I don't know where that comes through.

link |

Somehow I'm friendly with the class, I like students.

link |

And then your algebra, the subject,

link |

we gotta give the subject most of the credit.

link |

It really has come forward in importance in these years.

link |

So let's talk about linear algebra a little bit,

link |

because it is such a, it's both a powerful

link |

and a beautiful subfield of mathematics.

link |

So what's your favorite specific topic in linear algebra,

link |

or even math in general to give a lecture on,

link |

to convey, to tell a story, to teach students?

link |

Okay, well, on the teaching side,

link |

so it's not deep mathematics at all,

link |

but I'm kind of proud of the idea of the four subspaces,

link |

the four fundamental subspaces,

link |

which are of course known before,

link |

long before my name for them, but.

link |

Can you go through them?

link |

Can you go through the four subspaces?

link |

So the first one to understand is,

link |

so the matrix is, maybe I should say the matrix is.

link |

Well, so we have like a rectangle of numbers.

link |

So it's got n columns, got a bunch of columns,

link |

and also got an m rows, let's say,

link |

and the relation between,

link |

so of course the columns and the rows,

link |

it's the same numbers.

link |

So there's gotta be connections there,

link |

but they're not simple.

link |

The columns might be longer than the rows,

link |

and they're all different, the numbers are mixed up.

link |

First space to think about is take the columns,

link |

so those are vectors, those are points in n dimensions.

link |

So a physicist would imagine a vector

link |

or might imagine a vector as a arrow in space

link |

or the point it ends at in space.

link |

For me, it's a column of numbers.

link |

You often think of, this is very interesting

link |

in terms of linear algebra, in terms of a vector,

link |

you think a little bit more abstract

link |

than how it's very commonly used, perhaps.

link |

You think this arbitrary multidimensional space.

link |

Right away, I'm in high dimensions.

link |

Yeah, that's right.

link |

In the lecture, I try to,

link |

so if you think of two vectors in 10 dimensions,

link |

I'll do this in class, and I'll readily admit

link |

that I have no good image in my mind

link |

of a vector of an arrow in 10 dimensional space,

link |

You can add one bunch of 10 numbers

link |

to another bunch of 10 numbers,

link |

so you can add a vector to a vector,

link |

and you can multiply a vector by three,

link |

and that's, if you know how to do those,

link |

you've got linear algebra.

link |

10 dimensions, there's this beautiful thing about math,

link |

if we look at string theory and all these theories,

link |

which are really fundamentally derived through math,

link |

but are very difficult to visualize.

link |

How do you think about the things,

link |

like a 10 dimensional vector,

link |

that we can't really visualize?

link |

And yet, math reveals some beauty underlying our world

link |

in that weird thing we can't visualize.

link |

How do you think about that difference?

link |

Well, probably, I'm not a very geometric person,

link |

so I'm probably thinking in three dimensions,

link |

and the beauty of linear algebra is that

link |

it goes on to 10 dimensions with no problem.

link |

I mean, that if you're just seeing what happens

link |

if you add two vectors in 3D,

link |

yeah, then you can add them in 10D.

link |

You're just adding the 10 components.

link |

So, I can't say that I have a picture,

link |

but yet I try to push the class

link |

to think of a flat surface in 10 dimensions.

link |

So a plane in 10 dimensions,

link |

and so that's one of the spaces.

link |

Take all the columns of the matrix,

link |

take all their combinations,

link |

so much of this column, so much of this one,

link |

then if you put all those together,

link |

you get some kind of a flat surface

link |

that I call a vector space, space of vectors.

link |

And my imagination is just seeing

link |

like a piece of paper in 3D, but anyway,

link |

so that's one of the spaces, that's space number one,

link |

the column space of the matrix.

link |

And then there's the row space, which is, as I said,

link |

different, but came from the same numbers.

link |

So we got the column space,

link |

all combinations of the columns,

link |

and then we've got the row space,

link |

all combinations of the rows.

link |

So those words are easy for me to say,

link |

and I can't really draw them on a blackboard,

link |

but I try with my thick chalk.

link |

Everybody likes that railroad chalk, and me too.

link |

I wouldn't use anything else now.

link |

And then the other two spaces are perpendicular to those.

link |

So like if you have a plane in 3D,

link |

just a plane is just a flat surface in 3D,

link |

then perpendicular to that plane would be a line.

link |

So that would be the null space.

link |

So we've got two, we've got a column space, a row space,

link |

and there are two perpendicular spaces.

link |

So those four fit together in a beautiful picture

link |

of a matrix, yeah, yeah.

link |

It's sort of a fundamental, it's not a difficult idea.

link |

It comes pretty early in 1806, and it's basic.

link |

Planes in these multidimensional spaces,

link |

how difficult of an idea is that to come to, do you think?

link |

If you look back in time,

link |

I think mathematically it makes sense,

link |

but I don't know if it's intuitive for us to imagine,

link |

just as we were talking about.

link |

It feels like calculus is easier to intuit.

link |

Well, I have to admit, calculus came earlier,

link |

earlier than linear algebra.

link |

So Newton and Leibniz were the great men

link |

to understand the key ideas of calculus.

link |

But linear algebra to me is like, okay,

link |

it's the starting point,

link |

because it's all about flat things.

link |

Calculus has got, all the complications of calculus

link |

come from the curves, the bending, the curved surfaces.

link |

Linear algebra, the surfaces are all flat.

link |

Nothing bends in linear algebra.

link |

So it should have come first, but it didn't.

link |

And calculus also comes first in high school classes,

link |

in college class, it'll be freshman math,

link |

it'll be calculus, and then I say, enough of it.

link |

Like, okay, get to the good stuff.

link |

Do you think linear algebra should come first?

link |

Well, it really, I'm okay with it not coming first,

link |

but it should, yeah, it should.

link |

Because everything is flat.

link |

Yeah, everything's flat.

link |

Well, of course, for that reason,

link |

calculus sort of sticks to one dimension,

link |

or eventually you do multivariate,

link |

but that basically means two dimensions.

link |

Linear algebra, you take off into 10 dimensions, no problem.

link |

It just feels scary and dangerous

link |

to go beyond two dimensions, that's all.

link |

If everything's flat, you can't go wrong.

link |

So what concept or theorem in linear algebra or in math

link |

you find most beautiful,

link |

that gives you pause that leaves you in awe?

link |

Well, I'll stick with linear algebra here.

link |

I hope the viewer knows that really,

link |

mathematics is amazing, amazing subject

link |

and deep, deep connections between ideas

link |

that didn't look connected, they turned out they were.

link |

But if we stick with linear algebra...

link |

So we have a matrix.

link |

That's like the basic thing, a rectangle of numbers.

link |

And it might be a rectangle of data.

link |

You're probably gonna ask me later about data science,

link |

where often data comes in a matrix.

link |

You have maybe every column corresponds to a drug

link |

and every row corresponds to a patient.

link |

And if the patient reacted favorably to the drug,

link |

then you put up some positive number in there.

link |

Anyway, rectangle of numbers, a matrix is basic.

link |

So the big problem is to understand all those numbers.

link |

You got a big, big set of numbers.

link |

And what are the patterns, what's going on?

link |

And so one of the ways to break down that matrix

link |

into simple pieces is uses something called singular values.

link |

And that's come on as fundamental in the last,

link |

certainly in my lifetime.

link |

Eigenvalues, if you have viewers who've done engineering,

link |

math, or basic linear algebra, eigenvalues were in there.

link |

But those are restricted to square matrices.

link |

And data comes in rectangular matrices.

link |

So you gotta take that next step.

link |

I'm always pushing math faculty, get on, do it, do it.

link |

So those are a way to break, to find the important pieces

link |

of the matrix, which add up to the whole matrix.

link |

So you're breaking a matrix into simple pieces.

link |

And the first piece is the most important part of the data.

link |

The second piece is the second most important part.

link |

And then often, so a data set is a matrix.

link |

And often, so a data scientist will like,

link |

if a data scientist can find those first and second pieces,

link |

stop there, the rest of the data is probably round off,

link |

experimental error maybe.

link |

So you're looking for the important part.

link |

So what do you find beautiful about singular values?

link |

Well, yeah, I didn't give the theorem.

link |

So here's the idea of singular values.

link |

Every matrix, every matrix, rectangular, square, whatever,

link |

can be written as a product

link |

of three very simple special matrices.

link |

So that's the theorem.

link |

Every matrix can be written as a rotation times a stretch,

link |

which is just a diagonal matrix,

link |

otherwise all zeros except on the one diagonal.

link |

And then the third factor is another rotation.

link |

So rotation, stretch, rotation

link |

is the breakup of any matrix.

link |

The structure of that, the ability that you can do that,

link |

what do you find appealing?

link |

What do you find beautiful about it?

link |

Well, geometrically, as I freely admit,

link |

the action of a matrix is not so easy to visualize,

link |

but everybody can visualize a rotation.

link |

Take two dimensional space and just turn it

link |

around the center.

link |

Take three dimensional space.

link |

So a pilot has to know about,

link |

well, what are the three, the yaw is one of them.

link |

I've forgotten all the three turns that a pilot makes.

link |

Up to 10 dimensions, you've got 10 ways to turn,

link |

but you can visualize a rotation.

link |

Take the space and turn it.

link |

And you can visualize a stretch.

link |

So to break a matrix with all those numbers in it

link |

into something you can visualize,

link |

rotate, stretch, rotate is pretty neat.

link |

That's pretty powerful.

link |

On YouTube, just consuming a bunch of videos

link |

and just watching what people connect with

link |

and what they really enjoy and are inspired by,

link |

math seems to come up again and again.

link |

I'm trying to understand why that is.

link |

Perhaps you can help give me clues.

link |

So it's not just the kinds of lectures that you give,

link |

but it's also just other folks like with Numberphile,

link |

there's a channel where they just chat about things

link |

that are extremely complicated, actually.

link |

People nevertheless connect with them.

link |

What do you think that is?

link |

It's wonderful, isn't it?

link |

I mean, I wasn't really aware of it.

link |

We're conditioned to think math is hard,

link |

math is abstract, math is just for a few people,

link |

but it isn't that way.

link |

A lot of people quite like math and they liked it.

link |

I get messages from people saying,

link |

now I'm retired, I'm gonna learn some more math.

link |

I get a lot of those.

link |

It's really encouraging.

link |

And I think what people like is that there's some order,

link |

a lot of order and things are not obvious, but they're true.

link |

So it's really cheering to think that so many people

link |

really wanna learn more about math.

link |

And in terms of truth, again,

link |

sorry to slide into philosophy at times,

link |

but math does reveal pretty strongly what things are true.

link |

I mean, that's the whole point of proving things.

link |

And yet, sort of our real world is messy and complicated.

link |

What do you think about the nature of truth

link |

that math reveals?

link |

Because it is a source of comfort like you've mentioned.

link |

Yeah, that's right.

link |

Well, I have to say, I'm not much of a philosopher.

link |

I just like numbers.

link |

As a kid, this was before you had to go in,

link |

when you had a filly in your teeth,

link |

you had to kind of just take it.

link |

So what I did was think about math,

link |

like take powers of two, two, four, eight, 16,

link |

up until the time the tooth stopped hurting

link |

and the dentist said you're through.

link |

So that was a source of just, source of peace almost.

link |

What is it about math do you think that brings that?

link |

Well, you know where you are.

link |

Yeah, it's symmetry, it's certainty.

link |

The fact that, you know, if you multiply two by itself

link |

10 times, you get 1,024 period.

link |

Everybody's gonna get that.

link |

Do you see math as a powerful tool or as an art form?

link |

That's really one of the neat things.

link |

You can be an artist and like math,

link |

you can be an engineer and use math.

link |

What did you connect with most?

link |

Yeah, I'm somewhere between.

link |

I'm certainly not a artist type, philosopher type person.

link |

Might sound that way this morning, but I'm not.

link |

Yeah, I really enjoy teaching engineers

link |

because they go for an answer.

link |

And yeah, so probably within the MIT math department,

link |

most people enjoy teaching people,

link |

teaching students who get the abstract idea.

link |

I'm okay with, I'm good with engineers

link |

who are looking for a way to find answers.

link |

Actually, that's an interesting question.

link |

Do you think for teaching and in general,

link |

thinking about new concepts,

link |

do you think it's better to plug in the numbers

link |

or to think more abstractly?

link |

So looking at theorems and proving the theorems

link |

or actually building up a basic intuition of the theorem

link |

or the method, the approach,

link |

and then just plugging in numbers and seeing it work.

link |

Yeah, well, certainly many of us like to see examples.

link |

First, we understand,

link |

it might be a pretty abstract sounding example,

link |

like a three dimensional rotation.

link |

How are you gonna understand a rotation in 3D?

link |

And then some of us like to keep going with it

link |

to the point where you got numbers,

link |

where you got 10 angles, 10 axes, 10 angles.

link |

But the best, the great mathematicians probably,

link |

I don't know if they do that,

link |

because for them, an example would be a highly abstract thing

link |

to the rest of it.

link |

Right, but nevertheless, working in the space of examples.

link |

Examples of structure.

link |

Our brains seem to connect with that.

link |

So I'm not sure if you're familiar with him,

link |

but Andrew Yang is a presidential candidate

link |

currently running with math in all capital letters

link |

and his hats as a slogan.

link |

Stands for Make America Think Hard.

link |

Okay, I'll vote for him.

link |

So, and his name rhymes with yours, Yang, Strang.

link |

But he also loves math and he comes from that world

link |

of, but he also, looking at it,

link |

makes me realize that math, science, and engineering

link |

are not really part of our politics, political discourse,

link |

about political government in general.

link |

Why do you think that is?

link |

What are your thoughts on that in general?

link |

Well, certainly somewhere in the system,

link |

we need people who are comfortable with numbers,

link |

comfortable with quantities.

link |

You know, if you say this leads to that,

link |

they see it and it's undeniable.

link |

But isn't that strange to you that we have almost no,

link |

I mean, I'm pretty sure we have no elected officials

link |

in Congress or obviously the president

link |

that either has an engineering degree or a math degree.

link |

Yeah, well, that's too bad.

link |

A few could, a few who could make the connection.

link |

Yeah, it would have to be people who understand

link |

engineering or science and at the same time

link |

can make speeches and lead, yeah.

link |

Yeah, inspire people.

link |

Yeah, inspire, yeah.

link |

You were, speaking of inspiration,

link |

the president of the Society

link |

for Industrial and Applied Mathematics.

link |

It's a major organization in math, applied math.

link |

What do you see as a role of that society,

link |

you know, in our public discourse?

link |

Yeah, so, well, it was fun to be president at the time.

link |

A couple years, a few years.

link |

Two years, around 2000.

link |

I just hope that's president of a pretty small society.

link |

But nevertheless, it was a time when math

link |

was getting some more attention in Washington.

link |

But yeah, I got to give a little 10 minutes

link |

to a committee of the House of Representatives

link |

talking about who I met.

link |

And then, actually, it was fun

link |

because one of the members of the House

link |

had been a student, had been in my class.

link |

What do you think of that?

link |

Yeah, as you say, pretty rare, most members of the House

link |

have had a different training, different background.

link |

But there was one from New Hampshire

link |

who was my friend, really, by being in the class.

link |

Yeah, so those years were good.

link |

Then, of course, other things take over in importance

link |

in Washington, and math just, at this point,

link |

is not so visible.

link |

But for a little moment, it was.

link |

There's some excitement, some concern

link |

about artificial intelligence in Washington now.

link |

Yes, sure. About the future.

link |

Yeah. And I think at the core

link |

Well, it is, yeah.

link |

Maybe it's hidden.

link |

Maybe it's wearing a different hat.

link |

Well, artificial intelligence, and particularly,

link |

can I use the words deep learning?

link |

Deep learning is a particular approach

link |

to understanding data.

link |

Again, you've got a big, whole lot of data

link |

where data is just swamping the computers of the world.

link |

And to understand it, out of all those numbers,

link |

to find what's important in climate, in everything.

link |

And artificial intelligence is two words

link |

for one approach to data.

link |

Deep learning is a specific approach there,

link |

which uses a lot of linear algebra.

link |

I thought, okay, I've gotta learn about this.

link |

So maybe from your perspective,

link |

let me ask the most basic question.

link |

How do you think of a neural network?

link |

What is a neural network?

link |

So can I start with the idea about deep learning?

link |

What does that mean?

link |

What is deep learning?

link |

What is deep learning, yeah.

link |

So we're trying to learn, from all this data,

link |

we're trying to learn what's important.

link |

What's it telling us?

link |

So you've got data, you've got some inputs

link |

for which you know the right outputs.

link |

The question is, can you see the pattern there?

link |

Can you figure out a way for a new input,

link |

which we haven't seen, to understand

link |

what the output will be from that new input?

link |

So we've got a million inputs with their outputs.

link |

So we're trying to create some pattern,

link |

some rule that'll take those inputs,

link |

those million training inputs, which we know about,

link |

to the correct million outputs.

link |

And this idea of a neural net

link |

is part of the structure of our new way to create a rule.

link |

We're looking for a rule that will take

link |

these training inputs to the known outputs.

link |

And then we're gonna use that rule on new inputs

link |

that we don't know the output and see what comes.

link |

Linear algebra is a big part of finding that rule.

link |

That's right, linear algebra is a big part.

link |

People were leaning on matrices, that's good, still do.

link |

Linear is something special.

link |

It's all about straight lines and flat planes.

link |

And data isn't quite like that.

link |

It's more complicated.

link |

So you gotta introduce some complication.

link |

So you have to have some function

link |

that's not a straight line.

link |

And it turned out, nonlinear, nonlinear, not linear.

link |

And it turned out that it was enough to use the function

link |

that's one straight line and then a different one.

link |

Halfway, so piecewise linear.

link |

One piece has one slope,

link |

one piece, the other piece has the second slope.

link |

And so that, getting that nonlinear,

link |

simple nonlinearity in blew the problem open.

link |

That little piece makes it sufficiently complicated

link |

to make things interesting.

link |

Because you're gonna use that piece

link |

over and over a million times.

link |

So it has a fold in the graph, the graph, two pieces.

link |

But when you fold something a million times,

link |

you've got a pretty complicated function

link |

that's pretty realistic.

link |

So that's the thing about neural networks

link |

is they have a lot of these.

link |

A lot of these, that's right.

link |

So why do you think neural networks,

link |

by using sort of formulating an objective function,

link |

very not a plain function of the folds,

link |

lots of folds of the inputs, the outputs,

link |

why do you think they work to be able to find a rule

link |

that we don't know is optimal,

link |

but it just seems to be pretty good in a lot of cases?

link |

What's your intuition?

link |

Is it surprising to you as it is to many people?

link |

Do you have an intuition of why this works at all?

link |

Well, I'm beginning to have a better intuition.

link |

This idea of things that are piecewise linear,

link |

flat pieces but with folds between them.

link |

Like think of a roof of a complicated,

link |

infinitely complicated house or something.

link |

That curve, it almost curved, but every piece is flat.

link |

That's been used by engineers,

link |

that idea has been used by engineers,

link |

is used by engineers, big time.

link |

Something called the finite element method.

link |

If you want to design a bridge,

link |

design a building, design an airplane,

link |

you're using this idea of piecewise flat

link |

as a good, simple, computable approximation.

link |

But you have a sense that there's a lot of expressive power

link |

in this kind of piecewise linear.

link |

Yeah, you used the right word.

link |

If you measure the expressivity,

link |

how complicated a thing can this piecewise flat guys express?

link |

The answer is very complicated, yeah.

link |

What do you think are the limits of such piecewise linear

link |

or just of neural networks?

link |

The expressivity of neural networks.

link |

Well, you would have said a while ago

link |

that they're just computational limits.

link |

It's a problem beyond a certain size.

link |

A supercomputer isn't gonna do it.

link |

But those keep getting more powerful.

link |

So that limit has been moved

link |

to allow more and more complicated surfaces.

link |

So in terms of just mapping from inputs to outputs,

link |

looking at data, what do you think of,

link |

in the context of neural networks in general,

link |

data is just tensor, vectors, matrices, tensors.

link |

How do you think about learning from data?

link |

How much of our world can be expressed in this way?

link |

How useful is this process?

link |

I guess that's another way to ask you,

link |

what are the limits of this approach?

link |

Well, that's a good question, yeah.

link |

So I guess the whole idea of deep learning

link |

is that there's something there to learn.

link |

If the data is totally random,

link |

just produced by random number generators,

link |

then we're not gonna find a useful rule

link |

because there isn't one.

link |

So the extreme of having a rule is like knowing Newton's law.

link |

If you hit a ball, it moves.

link |

So that's where you had laws of physics.

link |

Newton and Einstein and other great, great people

link |

have found those laws and laws of the distribution

link |

of oil in an underground thing.

link |

I mean, so engineers, petroleum engineers understand

link |

how oil will sit in an underground basin.

link |

So there were rules.

link |

Now, the new idea of artificial intelligence is

link |

learn the rules instead of figuring out the rules

link |

with help from Newton or Einstein.

link |

The computer is looking for the rules.

link |

So that's another step.

link |

But if there are no rules at all

link |

that the computer could find,

link |

if it's totally random data, well, you've got nothing.

link |

You've got no science to discover.

link |

It's an automated search for the underlying rules.

link |

Yeah, search for the rules.

link |

And there will be a lot of random parts.

link |

A lot of, I mean, I'm not knocking random

link |

because that's there.

link |

There's a lot of randomness built in,

link |

but there's gotta be some basic.

link |

It's almost always signal, right?

link |

There's gotta be some signal, yeah.

link |

If it's all noise, then you're not gonna get anywhere.

link |

Well, this world around us does seem to be,

link |

does seem to always have a signal of some kind.

link |

Yeah, yeah, that's right.

link |

So what excites you more?

link |

We just talked about a little bit of application.

link |

What excites you more, theory

link |

or the application of mathematics?

link |

Well, for myself, I'm probably a theory person.

link |

I'm not, I'm speaking here pretty freely about applications,

link |

but I'm not the person who really,

link |

I'm not a physicist or a chemist or a neuroscientist.

link |

So for myself, I like the structure

link |

and the flat subspaces

link |

and the relation of matrices, columns to rows.

link |

That's my part in the spectrum.

link |

So really, science is a big spectrum of people

link |

from asking practical questions

link |

and answering them using some math,

link |

then some math guys like myself who are in the middle of it

link |

and then the geniuses of math and physics and chemistry

link |

who are finding fundamental rules

link |

and then doing the really understanding nature.

link |

That's incredible.

link |

At its lowest, simplest level,

link |

maybe just a quick in broad strokes from your perspective,

link |

where does linear algebra sit as a subfield of mathematics?

link |

What are the various subfields that you think about

link |

in relation to linear algebra?

link |

So the big fields of math are algebra as a whole

link |

and problems like calculus and differential equations.

link |

So that's a second, quite different field.

link |

Then maybe geometry deserves to be thought of

link |

as a different field to understand the geometry

link |

of high dimensional surfaces.

link |

So I think, am I allowed to say this here?

link |

I think this is where personal view comes in.

link |

I think math, we're thinking about undergraduate math,

link |

what millions of students study.

link |

I think we overdo the calculus at the cost of the algebra,

link |

at the cost of linear.

link |

So you have this talk titled Calculus Versus Linear Algebra.

link |

That's right, that's right.

link |

And you say that linear algebra wins.

link |

So can you dig into that a little bit?

link |

Why does linear algebra win?

link |

Right, well, okay, the viewer is gonna think

link |

this guy is biased.

link |

Not true, I'm just telling the truth as it is.

link |

Yeah, so I feel linear algebra is just a nice part of math

link |

that people can get the idea of.

link |

They can understand something that's a little bit abstract

link |

because once you get to 10 or 100 dimensions

link |

and very, very, very useful,

link |

that's what's happened in my lifetime

link |

is the importance of data,

link |

which does come in matrix form.

link |

So it's really set up for algebra.

link |

It's not set up for differential equation.

link |

And let me fairly add probability,

link |

the ideas of probability and statistics

link |

have become very, very important, have also jumped forward.

link |

So, and that's different from linear algebra,

link |

So now we really have three major areas to me,

link |

calculus, linear algebra, matrices,

link |

and probability statistics.

link |

And they all deserve an important place.

link |

And calculus has traditionally had a lion's share

link |

A disproportionate share.

link |

It is, thank you, disproportionate, that's a good word.

link |

Of the love and attention from the excited young minds.

link |

I know it's hard to pick favorites,

link |

but what is your favorite matrix?

link |

What's my favorite matrix?

link |

Okay, so my favorite matrix is square, I admit it.

link |

It's a square bunch of numbers

link |

and it has twos running down the main diagonal.

link |

And on the next diagonal,

link |

so think of top left to bottom right,

link |

twos down the middle of the matrix

link |

and minus ones just above those twos

link |

and minus ones just below those twos

link |

and otherwise all zeros.

link |

So mostly zeros, just three nonzero diagonals coming down.

link |

What is interesting about it?

link |

Well, all the different ways it comes up.

link |

You see it in engineering,

link |

you see it as analogous in calculus to second derivative.

link |

So calculus learns about taking the derivative,

link |

the figuring out how much, how fast something's changing.

link |

But second derivative, now that's also important.

link |

That's how fast the change is changing,

link |

how fast the graph is bending, how fast it's curving.

link |

And Einstein showed that that's fundamental

link |

to understand space.

link |

So second derivatives should have a bigger place in calculus.

link |

Second, my matrices,

link |

which are like the linear algebra version

link |

of second derivatives are neat in linear algebra.

link |

Yeah, just everything comes out right with those guys.

link |

What did you learn about the process of learning

link |

by having taught so many students math over the years?

link |

Ooh, that is hard.

link |

I'll have to admit here that I'm not really a good teacher

link |

because I don't get into the exam part.

link |

The exam is the part of my life that I don't like

link |

and grading them and giving the students A or B or whatever.

link |

I do it because I'm supposed to do it,

link |

but I tell the class at the beginning,

link |

I don't know if they believe me.

link |

Probably they don't.

link |

I tell the class, I'm here to teach you.

link |

I'm here to teach you math and not to grade you.

link |

But they're thinking, okay, this guy is gonna,

link |

when is he gonna give me an A minus?

link |

Is he gonna give me a B plus?

link |

What have you learned about the process of learning?

link |

Yeah, well, maybe to give you a legitimate answer

link |

about learning, I should have paid more attention

link |

to the assessment, the evaluation part at the end.

link |

But I like the teaching part at the start.

link |

That's the sexy part.

link |

To tell somebody for the first time about a matrix, wow.

link |

Is there, are there moments,

link |

so you are teaching a concept,

link |

are there moments of learning that you just see

link |

in the student's eyes?

link |

You don't need to look at the grades.

link |

But you see in their eyes that you hook them,

link |

that you connect with them in a way where,

link |

you know what, they fall in love

link |

with this beautiful world of math.

link |

They see that it's got some beauty there.

link |

Or conversely, that they give up at that point

link |

The dark could say that math, I'm just not good at math.

link |

I don't wanna walk away.

link |

Maybe because of the approach in the past,

link |

they were discouraged, but don't be discouraged.

link |

It's too good to miss.

link |

Yeah, well, if I'm teaching a big class,

link |

do I know when, I think maybe I do.

link |

Sort of, I mentioned at the very start,

link |

the four fundamental subspaces

link |

and the structure of the fundamental theorem

link |

of linear algebra.

link |

The fundamental theorem of linear algebra.

link |

That is the relation of those four subspaces,

link |

those four spaces.

link |

Yeah, so I think that, I feel that the class gets it.

link |

What advice do you have to a student

link |

just starting their journey in mathematics today?

link |

How do they get started?

link |

Oh, yeah, that's hard.

link |

Well, I hope you have a teacher, professor,

link |

who is still enjoying what he's doing,

link |

what he's teaching.

link |

They're still looking for new ways to teach

link |

and to understand math.

link |

Cause that's the pleasure,

link |

the moment when you see, oh yeah, that works.

link |

So it's less about the material you study,

link |

it's more about the source of the teacher

link |

being full of passion.

link |

Yeah, more about the fun.

link |

Yeah, the moment of getting it.

link |

But in terms of topics, linear algebra?

link |

Well, that's my topic,

link |

but oh, there's beautiful things in geometry to understand.

link |

What's wonderful is that in the end,

link |

there's a pattern, there are rules

link |

that are followed in biology as there are in every field.

link |

You describe the life of a mathematician

link |

as 100% wonderful.

link |

Except for the grade stuff.

link |

Except for grades.

link |

Yeah, when you look back at your life,

link |

what memories bring you the most joy and pride?

link |

Well, that's a good question.

link |

I certainly feel good when I,

link |

maybe I'm giving a class in 1806,

link |

that's MIT's linear algebra course that I started.

link |

So sort of, there's a good feeling that,

link |

okay, I started this course,

link |

a lot of students take it, quite a few like it.

link |

Yeah, so I'm sort of happy

link |

when I feel I'm helping make a connection

link |

between ideas and students,

link |

between theory and the reader.

link |

Yeah, it's, I get a lot of very nice messages

link |

from people who've watched the videos and it's inspiring.

link |

I just, I'll maybe take this chance to say thank you.

link |

Well, there's millions of students

link |

who you've taught and I am grateful to be one of them.

link |

So Gilbert, thank you so much, it's been an honor.

link |

Thank you for talking today.

link |

It was a pleasure, thanks.

link |

Thank you for listening to this conversation

link |

with Gilbert Strang.

link |

And thank you to our presenting sponsor, Cash App.

link |

Download it, use code LexPodcast,

link |

you'll get $10 and $10 will go to FIRST,

link |

a STEM education nonprofit

link |

that inspires hundreds of thousands of young minds

link |

to learn and to dream of engineering our future.

link |

If you enjoy this podcast, subscribe on YouTube.

link |

We have five stars in Apple Podcast,

link |

support on Patreon or connect with me on Twitter.

link |

Finally, some closing words of advice

link |

from the great Richard Feynman.

link |

Study hard what interests you the most

link |

in the most undisciplined, irreverent

link |

and original manner possible.

link |

Thank you for listening and hope to see you next time.