back to indexAndrew Ng: Deep Learning, Education, and Real-World AI | Lex Fridman Podcast #73
link |
The following is a conversation with Andrew Ng,
link |
one of the most impactful educators, researchers, innovators, and leaders
link |
in artificial intelligence and technology space in general.
link |
He cofounded Coursera and Google Brain,
link |
launched Deep Learning AI, Landing AI, and the AI Fund,
link |
and was the chief scientist at Baidu.
link |
As a Stanford professor and with Coursera and Deep Learning AI,
link |
he has helped educate and inspire millions of students, including me.
link |
This is the Artificial Intelligence Podcast.
link |
If you enjoy it, subscribe on YouTube, give it five stars on Apple Podcast,
link |
support it on Patreon, or simply connect with me on Twitter
link |
at Lex Friedman, spelled F R I D M A N.
link |
As usual, I'll do one or two minutes of ads now
link |
and never any ads in the middle that can break the flow of the conversation.
link |
I hope that works for you and doesn't hurt the listening experience.
link |
This show is presented by Cash App, the number one finance app in the App Store.
link |
When you get it, use code LEXPODCAST.
link |
Cash App lets you send money to friends, buy Bitcoin,
link |
and invest in the stock market with as little as $1.
link |
Broker services are provided by Cash App Investing,
link |
a subsidiary of Square, a member SIPC.
link |
Since Cash App allows you to buy Bitcoin,
link |
let me mention that cryptocurrency in the context of the history of money is fascinating.
link |
I recommend Ascent of Money as a great book on this history.
link |
Debits and credits on ledgers started over 30,000 years ago.
link |
The US dollar was created over 200 years ago,
link |
and Bitcoin, the first decentralized cryptocurrency, released just over 10 years ago.
link |
So given that history, cryptocurrency is still very much in its early days of development,
link |
but it's still aiming to and just might redefine the nature of money.
link |
So again, if you get Cash App from the App Store or Google Play
link |
and use the code LEXPODCAST, you'll get $10,
link |
and Cash App will also donate $10 to FIRST,
link |
one of my favorite organizations that is helping to advance robotics and STEM education
link |
for young people around the world.
link |
And now, here's my conversation with Andrew Ng.
link |
The courses you taught on machine learning at Stanford
link |
and later on Coursera that you cofounded have educated and inspired millions of people.
link |
So let me ask you, what people or ideas inspired you
link |
to get into computer science and machine learning when you were young?
link |
When did you first fall in love with the field, is another way to put it.
link |
Growing up in Hong Kong and Singapore, I started learning to code when I was five or six years old.
link |
At that time, I was learning the basic programming language,
link |
and they would take these books and they'll tell you,
link |
type this program into your computer, so type that program to my computer.
link |
And as a result of all that typing, I would get to play these very simple shoot them up games
link |
that I had implemented on my little computer.
link |
So I thought it was fascinating as a young kid that I could write this code.
link |
I was really just copying code from a book into my computer
link |
to then play these cool little video games.
link |
Another moment for me was when I was a teenager and my father,
link |
who's a doctor, was reading about expert systems and about neural networks.
link |
So he got me to read some of these books, and I thought it was really cool.
link |
You could write a computer that started to exhibit intelligence.
link |
Then I remember doing an internship while I was in high school, this was in Singapore,
link |
where I remember doing a lot of photocopying and as an office assistant.
link |
And the highlight of my job was when I got to use the shredder.
link |
So the teenager me, remote thinking, boy, this is a lot of photocopying.
link |
If only we could write software, build a robot, something to automate this,
link |
maybe I could do something else.
link |
So I think a lot of my work since then has centered on the theme of automation.
link |
Even the way I think about machine learning today,
link |
we're very good at writing learning algorithms that can automate things that people can do.
link |
Or even launching the first MOOCs, Mass Open Online Courses, that later led to Coursera.
link |
I was trying to automate what could be automatable in how I was teaching on campus.
link |
Process of education, trying to automate parts of that to make it more,
link |
sort of to have more impact from a single teacher, a single educator.
link |
Yeah, I felt, you know, teaching at Stanford,
link |
teaching machine learning to about 400 students a year at the time.
link |
And I found myself filming the exact same video every year,
link |
telling the same jokes in the same room.
link |
And I thought, why am I doing this?
link |
Why don't we just take last year's video?
link |
And then I can spend my time building a deeper relationship with students.
link |
So that process of thinking through how to do that,
link |
that led to the first MOOCs that we launched.
link |
And then you have more time to write new jokes.
link |
Are there favorite memories from your early days at Stanford,
link |
teaching thousands of people in person and then millions of people online?
link |
You know, teaching online, what not many people know was that a lot of those videos
link |
were shot between the hours of 10 p.m. and 3 a.m.
link |
A lot of times, we were launching the first MOOCs at Stanford.
link |
We had already announced the course, about 100,000 people signed up.
link |
We just started to write the code and we had not yet actually filmed the videos.
link |
So a lot of pressure, 100,000 people waiting for us to produce the content.
link |
So many Fridays, Saturdays, I would go out, have dinner with my friends,
link |
and then I would think, OK, do you want to go home now?
link |
Or do you want to go to the office to film videos?
link |
And the thought of being able to help 100,000 people potentially learn machine learning,
link |
fortunately, that made me think, OK, I want to go to my office,
link |
go to my tiny little recording studio.
link |
I would adjust my Logitech webcam, adjust my Wacom tablet,
link |
make sure my lapel mic was on,
link |
and then I would start recording often until 2 a.m. or 3 a.m.
link |
I think unfortunately, that doesn't show that it was recorded that late at night,
link |
but it was really inspiring the thought that we could create content
link |
to help so many people learn about machine learning.
link |
How did that feel?
link |
The fact that you're probably somewhat alone,
link |
maybe a couple of friends recording with a Logitech webcam
link |
and kind of going home alone at 1 or 2 a.m. at night
link |
and knowing that that's going to reach sort of thousands of people,
link |
eventually millions of people, what's that feeling like?
link |
I mean, is there a feeling of just satisfaction of pushing through?
link |
I think it's humbling.
link |
And I wasn't thinking about what I was feeling.
link |
I think one thing that I'm proud to say we got right from the early days
link |
was I told my whole team back then that the number one priority
link |
is to do what's best for learners, do what's best for students.
link |
And so when I went to the recording studio,
link |
the only thing on my mind was what can I say?
link |
How can I design my slides?
link |
What I need to draw right to make these concepts as clear as possible for learners?
link |
I think I've seen sometimes instructors is tempting to,
link |
hey, let's talk about my work.
link |
Maybe if I teach you about my research,
link |
someone will cite my papers a couple more times.
link |
And I think one of the things we got right,
link |
launching the first few MOOCs and later building Coursera,
link |
was putting in place that bedrock principle of
link |
let's just do what's best for learners and forget about everything else.
link |
And I think that that is a guiding principle
link |
turned out to be really important to the rise of the MOOC movement.
link |
And the kind of learner you imagined in your mind
link |
is as broad as possible, as global as possible.
link |
So really try to reach as many people
link |
interested in machine learning and AI as possible.
link |
I really want to help anyone that had an interest in machine learning
link |
to break into the field.
link |
And I think sometimes I've actually had people ask me,
link |
hey, why are you spending so much time explaining gradient descent?
link |
And my answer was, if I look at what I think the learner needs
link |
and what benefit from, I felt that having that
link |
a good understanding of the foundations coming back to the basics
link |
would put them in a better stead to then build on a long term career.
link |
So try to consistently make decisions on that principle.
link |
So one of the things you actually revealed to the narrow AI community
link |
at the time and to the world is that the amount of people
link |
who are actually interested in AI is much larger than we imagined.
link |
By you teaching the class and how popular it became,
link |
it showed that, wow, this isn't just a small community
link |
of sort of people who go to NeurIPS and it's much bigger.
link |
It's developers, it's people from all over the world.
link |
I mean, I'm Russian, so everybody in Russia is really interested.
link |
There's a huge number of programmers who are interested in machine learning,
link |
India, China, South America, everywhere.
link |
There's just millions of people who are interested in machine learning.
link |
So how big do you get a sense that the number of people
link |
is that are interested from your perspective?
link |
I think the number has grown over time.
link |
I think it's one of those things that maybe it feels like it came out of nowhere,
link |
but it's an insight that building it, it took years.
link |
It's one of those overnight successes that took years to get there.
link |
My first foray into this type of online education
link |
was when we were filming my Stanford class
link |
and sticking the videos on YouTube and some other things.
link |
We had uploaded the horrors and so on,
link |
but it's basically the one hour, 15 minute video that we put on YouTube.
link |
And then we had four or five other versions of websites that I had built,
link |
most of which you would never have heard of
link |
because they reached small audiences,
link |
but that allowed me to iterate,
link |
allowed my team and me to iterate,
link |
to learn what are the ideas that work and what doesn't.
link |
For example, one of the features I was really excited about
link |
and really proud of was build this website
link |
where multiple people could be logged into the website at the same time.
link |
So today, if you go to a website,
link |
if you are logged in and then I want to log in,
link |
you need to log out because it's the same browser, the same computer.
link |
But I thought, well, what if two people say you and me
link |
were watching a video together in front of a computer?
link |
What if a website could have you type your name and password,
link |
have me type my name and password,
link |
and then now the computer knows both of us are watching together
link |
and it gives both of us credit for anything we do as a group.
link |
Influencers feature rolled it out in a high school in San Francisco.
link |
We had about 20 something users.
link |
Where's the teacher there?
link |
Sacred Heart Cathedral Prep, the teacher is great.
link |
I mean, guess what?
link |
Zero people use this feature.
link |
It turns out people studying online,
link |
they want to watch the videos by themselves.
link |
So you can play back, pause at your own speed rather than in groups.
link |
So that was one example of a tiny lesson learned out of many
link |
that allowed us to hone into the set of features.
link |
It sounds like a brilliant feature.
link |
So I guess the lesson to take from that is
link |
there's something that looks amazing on paper and then nobody uses it.
link |
It doesn't actually have the impact that you think it might have.
link |
And so, yeah, I saw that you really went through a lot of different features
link |
and a lot of ideas to arrive at Coursera,
link |
the final kind of powerful thing that showed the world
link |
that MOOCs can educate millions.
link |
And I think with the whole machine learning movement as well,
link |
I think it didn't come out of nowhere.
link |
Instead, what happened was as more people learn about machine learning,
link |
they will tell their friends and their friends will see
link |
how it's applicable to their work.
link |
And then the community kept on growing.
link |
And I think we're still growing.
link |
I don't know in the future what percentage of all developers
link |
will be AI developers.
link |
I could easily see it being north of 50%, right?
link |
Because so many AI developers broadly construed,
link |
not just people doing the machine learning modeling,
link |
but the people building infrastructure, data pipelines,
link |
all the software surrounding the core machine learning model
link |
maybe is even bigger.
link |
I feel like today almost every software engineer
link |
has some understanding of the cloud.
link |
Not all, but maybe this is my microcontroller developer
link |
that doesn't need to deal with the cloud.
link |
But I feel like the vast majority of software engineers today
link |
are sort of having an appreciation of the cloud.
link |
I think in the future, maybe we'll approach nearly 100% of all developers
link |
being in some way an AI developer
link |
or at least having an appreciation of machine learning.
link |
And my hope is that there's this kind of effect
link |
that there's people who are not really interested in being a programmer
link |
or being into software engineering, like biologists, chemists,
link |
and physicists, even mechanical engineers,
link |
all these disciplines that are now more and more sitting on large data sets.
link |
And here they didn't think they're interested in programming
link |
until they have this data set and they realize
link |
there's this set of machine learning tools
link |
that allow you to use the data set.
link |
So they actually become, they learn to program
link |
and they become new programmers.
link |
So like the, not just because you've mentioned
link |
a larger percentage of developers become machine learning people.
link |
So it seems like more and more the kinds of people
link |
who are becoming developers is also growing significantly.
link |
Yeah, I think once upon a time,
link |
only a small part of humanity was literate, could read and write.
link |
And maybe you thought, maybe not everyone needs to learn to read and write.
link |
You just go listen to a few monks read to you and maybe that was enough.
link |
Or maybe you just need a few handful of authors to write the bestsellers
link |
and no one else needs to write.
link |
But what we found was that by giving as many people,
link |
in some countries, almost everyone, basic literacy,
link |
it dramatically enhanced human to human communications.
link |
And we can now write for an audience of one,
link |
such as if I send you an email or you send me an email.
link |
I think in computing, we're still in that phase
link |
where so few people know how to code
link |
that the coders mostly have to code for relatively large audiences.
link |
But if everyone, or most people became developers at some level,
link |
similar to how most people in developed economies are somewhat literate,
link |
I would love to see the owners of a mom and pop store
link |
be able to write a little bit of code to customize the TV display
link |
for their special this week.
link |
And I think it will enhance human to computer communications,
link |
which is becoming more and more important today as well.
link |
So you think it's possible that machine learning
link |
becomes kind of similar to literacy,
link |
where like you said, the owners of a mom and pop shop,
link |
is basically everybody in all walks of life
link |
would have some degree of programming capability?
link |
I could see society getting there.
link |
There's one other interesting thing.
link |
If I go talk to the mom and pop store,
link |
if I talk to a lot of people in their daily professions,
link |
I previously didn't have a good story for why they should learn to code.
link |
We could give them some reasons.
link |
But what I found with the rise of machine learning and data science is that
link |
I think the number of people with a concrete use for data science
link |
in their daily lives, in their jobs,
link |
may be even larger than the number of people
link |
who have concrete use for software engineering.
link |
For example, if you run a small mom and pop store,
link |
I think if you can analyze the data about your sales, your customers,
link |
I think there's actually real value there,
link |
maybe even more than traditional software engineering.
link |
So I find that for a lot of my friends in various professions,
link |
be it recruiters or accountants or people that work in the factories,
link |
which I deal with more and more these days,
link |
I feel if they were data scientists at some level,
link |
they could immediately use that in their work.
link |
So I think that data science and machine learning
link |
may be an even easier entree into the developer world
link |
for a lot of people than the software engineering.
link |
That's interesting.
link |
And I agree with that, but that's beautifully put.
link |
But we live in a world where most courses and talks have slides,
link |
PowerPoint, keynote,
link |
and yet you famously often still use a marker and a whiteboard.
link |
The simplicity of that is compelling,
link |
and for me at least, fun to watch.
link |
So let me ask, why do you like using a marker and whiteboard,
link |
even on the biggest of stages?
link |
I think it depends on the concepts you want to explain.
link |
For mathematical concepts,
link |
it's nice to build up the equation one piece at a time,
link |
and the whiteboard marker or the pen and stylus
link |
is a very easy way to build up the equation,
link |
to build up a complex concept one piece at a time
link |
while you're talking about it,
link |
and sometimes that enhances understandability.
link |
The downside of writing is that it's slow,
link |
and so if you want a long sentence, it's very hard to write that.
link |
So I think there are pros and cons,
link |
and sometimes I use slides,
link |
and sometimes I use a whiteboard or a stylus.
link |
The slowness of a whiteboard is also its upside,
link |
because it forces you to reduce everything to the basics.
link |
Some of your talks involve the whiteboard.
link |
I mean, you go very slowly,
link |
and you really focus on the most simple principles,
link |
and that's a beautiful,
link |
that enforces a kind of a minimalism of ideas
link |
that I think is surprising at least for me is great for education.
link |
Like a great talk, I think, is not one that has a lot of content.
link |
A great talk is one that just clearly says a few simple ideas,
link |
and I think the whiteboard somehow enforces that.
link |
Peter Abbeel, who's now one of the top roboticists
link |
and reinforcement learning experts in the world,
link |
was your first PhD student.
link |
So I bring him up just because I kind of imagine
link |
this must have been an interesting time in your life,
link |
and do you have any favorite memories of working with Peter,
link |
since you were your first student in those uncertain times,
link |
especially before deep learning really sort of blew up?
link |
Any favorite memories from those times?
link |
Yeah, I was really fortunate to have had Peter Abbeel
link |
as my first PhD student,
link |
and I think even my long term professional success
link |
builds on early foundations or early work
link |
that Peter was so critical to.
link |
So I was really grateful to him for working with me.
link |
What not a lot of people know is just how hard research was,
link |
Peter's PhD thesis was using reinforcement learning
link |
to fly helicopters.
link |
And so, even today, the website heli.stanford.edu,
link |
heli.stanford.edu is still up.
link |
You can watch videos of us using reinforcement learning
link |
to make a helicopter fly upside down,
link |
fly loose roses, so it's cool.
link |
It's one of the most incredible robotics videos ever,
link |
so people should watch it.
link |
Oh yeah, thank you.
link |
That's from like 2008 or seven or six, like that range.
link |
Yeah, something like that.
link |
Yeah, so it was over 10 years old.
link |
That was really inspiring to a lot of people, yeah.
link |
What not many people see is how hard it was.
link |
So Peter and Adam Coase and Morgan Quigley and I
link |
were working on various versions of the helicopter,
link |
and a lot of things did not work.
link |
For example, it turns out one of the hardest problems we had
link |
was when the helicopter's flying around upside down,
link |
doing stunts, how do you figure out the position?
link |
How do you localize the helicopter?
link |
So we wanted to try all sorts of things.
link |
Having one GPS unit doesn't work
link |
because you're flying upside down,
link |
the GPS unit's facing down, so you can't see the satellites.
link |
So we experimented trying to have two GPS units,
link |
one facing up, one facing down.
link |
So if you flip over, that didn't work
link |
because the downward facing one couldn't synchronize
link |
if you're flipping quickly.
link |
Morgan Quigley was exploring this crazy,
link |
complicated configuration of specialized hardware
link |
to interpret GPS signals.
link |
Looking at the FPG is completely insane.
link |
Spent about a year working on that, didn't work.
link |
So I remember Peter, great guy, him and me,
link |
sitting down in my office looking at some of the latest things
link |
we had tried that didn't work and saying,
link |
done it, what now?
link |
Because we tried so many things and it just didn't work.
link |
In the end, what we did, and Adam Coles was crucial to this,
link |
was put cameras on the ground and use cameras on the ground
link |
to localize the helicopter.
link |
And that solved the localization problem
link |
so that we could then focus on the reinforcement learning
link |
and inverse reinforcement learning techniques
link |
so it didn't actually make the helicopter fly.
link |
And I'm reminded, when I was doing this work at Stanford,
link |
around that time, there was a lot of reinforcement learning
link |
theoretical papers, but not a lot of practical applications.
link |
So the autonomous helicopter work for flying helicopters
link |
was one of the few practical applications
link |
of reinforcement learning at the time,
link |
which caused it to become pretty well known.
link |
I feel like we might have almost come full circle with today.
link |
There's so much buzz, so much hype, so much excitement
link |
about reinforcement learning.
link |
But again, we're hunting for more applications
link |
of all of these great ideas that David Kuhnke has come up with.
link |
What was the drive sort of in the face of the fact
link |
that most people are doing theoretical work?
link |
What motivates you in the uncertainty and the challenges
link |
to get the helicopter sort of to do the applied work,
link |
to get the actual system to work?
link |
Yeah, in the face of fear, uncertainty, sort of the setbacks
link |
that you mentioned for localization.
link |
I like stuff that works.
link |
In the physical world.
link |
So like, it's back to the shredder.
link |
You know, I like theory, but when I work on theory myself,
link |
and this is personal taste,
link |
I'm not saying anyone else should do what I do.
link |
But when I work on theory, I personally enjoy it more
link |
if I feel that the work I do will influence people,
link |
have positive impact, or help someone.
link |
I remember when many years ago,
link |
I was speaking with a mathematics professor,
link |
and it kind of just said, hey, why do you do what you do?
link |
It kind of just said, hey, why do you do what you do?
link |
And then he said, he had stars in his eyes when he answered.
link |
And this mathematician, not from Stanford,
link |
different university, he said, I do what I do
link |
because it helps me to discover truth and beauty
link |
He had stars in his eyes when he said that.
link |
And I thought, that's great.
link |
I don't want to do that.
link |
I think it's great that someone does that,
link |
fully support the people that do it,
link |
a lot of respect for people that do that.
link |
But I am more motivated when I can see a line
link |
to how the work that my teams and I are doing helps people.
link |
The world needs all sorts of people.
link |
I'm just one type.
link |
I don't think everyone should do things
link |
the same way as I do.
link |
But when I delve into either theory or practice,
link |
if I personally have conviction that here's a pathway
link |
to help people, I find that more satisfying
link |
to have that conviction.
link |
You were a proponent of deep learning
link |
before it gained widespread acceptance.
link |
What did you see in this field that gave you confidence?
link |
What was your thinking process like in that first decade
link |
of the, I don't know what that's called, 2000s, the aughts?
link |
Yeah, I can tell you the thing we got wrong
link |
and the thing we got right.
link |
The thing we really got wrong was the importance of,
link |
the early importance of unsupervised learning.
link |
So early days of Google Brain,
link |
we put a lot of effort into unsupervised learning
link |
rather than supervised learning.
link |
And there was this argument,
link |
I think it was around 2005 after NeurIPS,
link |
at that time called NIPS, but now NeurIPS had ended.
link |
And Jeff Hinton and I were sitting in the cafeteria
link |
outside the conference.
link |
We had lunch, we were just chatting.
link |
And Jeff pulled up this napkin.
link |
He started sketching this argument on a napkin.
link |
It was very compelling, as I'll repeat it.
link |
Human brain has about a hundred trillion.
link |
So there's 10 to the 14 synaptic connections.
link |
You will live for about 10 to the nine seconds.
link |
You actually live for two by 10 to the nine,
link |
maybe three by 10 to the nine seconds.
link |
So just let's say 10 to the nine.
link |
So if each synaptic connection,
link |
each weight in your brain's neural network
link |
has just a one bit parameter,
link |
that's 10 to the 14 bits you need to learn
link |
in up to 10 to the nine seconds.
link |
10 to the nine seconds of your life.
link |
So via this simple argument,
link |
which is a lot of problems, it's very simplified.
link |
That's 10 to the five bits per second
link |
you need to learn in your life.
link |
And I have a one year old daughter.
link |
I am not pointing out 10 to five bits per second
link |
And I think I'm a very loving parent,
link |
but I'm just not gonna do that.
link |
So from this very crude, definitely problematic argument,
link |
there's just no way that most of what we know
link |
is through supervised learning.
link |
But where you get so many bits of information
link |
is from sucking in images, audio,
link |
those experiences in the world.
link |
And so that argument,
link |
and there are a lot of known forces argument
link |
you should go into,
link |
really convinced me that there's a lot of power
link |
to unsupervised learning.
link |
So that was the part that we actually maybe got wrong.
link |
I still think unsupervised learning is really important,
link |
but in the early days, 10, 15 years ago,
link |
a lot of us thought that was the path forward.
link |
Oh, so you're saying that that perhaps
link |
was the wrong intuition for the time.
link |
For the time, that was the part we got wrong.
link |
The part we got right was the importance of scale.
link |
So Adam Coates, another wonderful person,
link |
fortunate to have worked with him,
link |
he was in my group at Stanford at the time
link |
and Adam had run these experiments at Stanford
link |
showing that the bigger we train a learning algorithm,
link |
the better its performance.
link |
And it was based on that.
link |
There was a graph that Adam generated
link |
where the X axis, Y axis lines going up into the right.
link |
So the bigger you make this thing,
link |
the better its performance accuracy is the vertical axis.
link |
So it's really based on that chart that Adam generated
link |
that he gave me the conviction
link |
that you could scale these models way bigger
link |
than what we could on a few CPUs,
link |
which is where we had at Stanford
link |
that we could get even better results.
link |
And it was really based on that one figure
link |
that Adam generated
link |
that gave me the conviction to go with Sebastian Thrun
link |
to pitch starting a project at Google,
link |
which became the Google Brain project.
link |
The Brain, you go find a Google Brain.
link |
And there the intuition was scale
link |
will bring performance for the system.
link |
So we should chase a larger and larger scale.
link |
And I think people don't realize how groundbreaking of it.
link |
It's simple, but it's a groundbreaking idea
link |
that bigger data sets will result in better performance.
link |
It was controversial at the time.
link |
Some of my well meaning friends,
link |
senior people in the machine learning community,
link |
I won't name, but some of whom we know,
link |
my well meaning friends came
link |
and were trying to give me friendly,
link |
I was like, hey, Andrew, why are you doing this?
link |
It's in the near natural architecture.
link |
Look at these architectures of building.
link |
You just want to go for scale?
link |
Like this is a bad career move.
link |
So my well meaning friends,
link |
some of them were trying to talk me out of it.
link |
But I find that if you want to make a breakthrough,
link |
you sometimes have to have conviction
link |
and do something before it's popular,
link |
since that lets you have a bigger impact.
link |
Let me ask you just a small tangent on that topic.
link |
I find myself arguing with people saying that greater scale,
link |
especially in the context of active learning,
link |
so very carefully selecting the data set,
link |
but growing the scale of the data set
link |
is going to lead to even further breakthroughs
link |
And there's currently pushback at that idea
link |
that larger data sets are no longer,
link |
so you want to increase the efficiency of learning.
link |
You want to make better learning mechanisms.
link |
And I personally believe that bigger data sets will still,
link |
with the same learning methods we have now,
link |
will result in better performance.
link |
What's your intuition at this time
link |
on this dual side?
link |
Do we need to come up with better architectures for learning
link |
or can we just get bigger, better data sets
link |
that will improve performance?
link |
I think both are important and it's also problem dependent.
link |
So for a few data sets,
link |
we may be approaching a Bayes error rate
link |
or approaching or surpassing human level performance
link |
and then there's that theoretical ceiling
link |
that we will never surpass,
link |
so Bayes error rate.
link |
But then I think there are plenty of problems
link |
where we're still quite far
link |
from either human level performance
link |
or from Bayes error rate
link |
and bigger data sets with neural networks
link |
without further algorithmic innovation
link |
will be sufficient to take us further.
link |
But on the flip side,
link |
if we look at the recent breakthroughs
link |
using transforming networks or language models,
link |
it was a combination of novel architecture
link |
but also scale had a lot to do with it.
link |
If we look at what happened with GP2 and BERTZ,
link |
I think scale was a large part of the story.
link |
Yeah, that's not often talked about
link |
is the scale of the data set it was trained on
link |
and the quality of the data set
link |
because there's some,
link |
so it was like reddit threads that had,
link |
they were operated highly.
link |
So there's already some weak supervision
link |
on a very large data set
link |
that people don't often talk about, right?
link |
I find that today we have maturing processes
link |
things like Git, right?
link |
It took us a long time to evolve the good processes.
link |
I remember when my friends and I
link |
were emailing each other C++ files in email,
link |
was it CVS or version Git?
link |
Maybe something else in the future.
link |
We're very mature in terms of tools for managing data
link |
and think about the clean data
link |
and how to solve down very hot, messy data problems.
link |
I think there's a lot of innovation there
link |
I love the idea that you were versioning through email.
link |
I'll give you one example.
link |
When we work with manufacturing companies,
link |
it's not at all uncommon
link |
for there to be multiple labels
link |
that disagree with each other, right?
link |
And so we would do the work in visual inspection.
link |
We will take, say, a plastic part
link |
and show it to one inspector
link |
and the inspector, sometimes very opinionated,
link |
they'll go, clearly, that's a defect.
link |
This scratch, unacceptable.
link |
Gotta reject this part.
link |
Take the same part to different inspector,
link |
different, very opinionated.
link |
Clearly, the scratch is small.
link |
Don't throw it away.
link |
You're gonna make us, you know.
link |
And then sometimes you take the same plastic part,
link |
show it to the same inspector
link |
in the afternoon, I suppose, in the morning,
link |
and very opinionated go, in the morning,
link |
they say, clearly, it's okay.
link |
In the afternoon, equally confident.
link |
Clearly, this is a defect.
link |
And so what is an AI team supposed to do
link |
if sometimes even one person doesn't agree
link |
with himself or herself in the span of a day?
link |
So I think these are the types of very practical,
link |
very messy data problems that my teams wrestle with.
link |
In the case of large consumer internet companies
link |
where you have a billion users,
link |
you have a lot of data.
link |
You don't worry about it.
link |
Just take the average.
link |
But in a case of other industry settings,
link |
we don't have big data.
link |
If just a small data, very small data sets,
link |
maybe around 100 defective parts
link |
or 100 examples of a defect.
link |
If you have only 100 examples,
link |
these little labeling errors,
link |
if 10 of your 100 labels are wrong,
link |
that actually is 10% of your data set has a big impact.
link |
So how do you clean this up?
link |
What are you supposed to do?
link |
This is an example of the types of things
link |
that my teams, this is a landing AI example,
link |
are wrestling with to deal with small data,
link |
which comes up all the time
link |
once you're outside consumer internet.
link |
Yeah, that's fascinating.
link |
So then you invest more effort and time
link |
in thinking about the actual labeling process.
link |
What are the labels?
link |
What are the how are disagreements resolved
link |
and all those kinds of like pragmatic real world problems.
link |
That's a fascinating space.
link |
Yeah, I find that actually when I'm teaching at Stanford,
link |
I increasingly encourage students at Stanford
link |
to try to find their own project
link |
for the end of term project,
link |
rather than just downloading someone else's
link |
nicely clean data set.
link |
It's actually much harder if you need to go
link |
and define your own problem and find your own data set,
link |
rather than you go to one of the several good websites,
link |
very good websites with clean scoped data sets
link |
that you could just work on.
link |
You're now running three efforts,
link |
the AI Fund, Landing AI, and deeplearning.ai.
link |
As you've said, the AI Fund is involved
link |
in creating new companies from scratch.
link |
Landing AI is involved in helping
link |
already established companies do AI
link |
and deeplearning.ai is for education of everyone else
link |
or of individuals interested in getting into the field
link |
and excelling in it.
link |
So let's perhaps talk about each of these areas.
link |
First, deeplearning.ai.
link |
How, the basic question,
link |
how does a person interested in deep learning
link |
get started in the field?
link |
Deep learning.ai is working to create courses
link |
to help people break into AI.
link |
So my machine learning course that I taught through Stanford
link |
is one of the most popular courses on Coursera.
link |
To this day, it's probably one of the courses,
link |
sort of, if I asked somebody,
link |
how did you get into machine learning
link |
or how did you fall in love with machine learning
link |
or would get you interested,
link |
it always goes back to Andrew Ng at some point.
link |
I see, yeah, I'm sure.
link |
You've influenced, the amount of people
link |
you've influenced is ridiculous.
link |
So for that, I'm sure I speak for a lot of people
link |
say big thank you.
link |
No, yeah, thank you.
link |
I was once reading a news article,
link |
I think it was tech review
link |
and I'm gonna mess up the statistic,
link |
but I remember reading an article that said
link |
something like one third of all programmers are self taught.
link |
I may have the number one third,
link |
around me was two thirds,
link |
but when I read that article,
link |
I thought this doesn't make sense.
link |
Everyone is self taught.
link |
So, cause you teach yourself.
link |
I don't teach people.
link |
Yeah, so how does one get started in deep learning
link |
and where does deeplearning.ai fit into that?
link |
So the deep learning specialization offered by deeplearning.ai
link |
is I think it was Coursera's top specialization.
link |
It might still be.
link |
So it's a very popular way for people
link |
to take that specialization
link |
to learn about everything from neural networks
link |
to how to tune in your network
link |
to what is a ConvNet to what is a RNN
link |
or a sequence model or what is an attention model.
link |
And so the deep learning specialization
link |
steps everyone through those algorithms
link |
so you deeply understand it
link |
and can implement it and use it for whatever application.
link |
From the very beginning.
link |
So what would you say are the prerequisites
link |
for somebody to take the deep learning specialization
link |
in terms of maybe math or programming background?
link |
Yeah, need to understand basic programming
link |
since there are programming exercises in Python
link |
and the math prereq is quite basic.
link |
So no calculus is needed.
link |
If you know calculus is great, you get better intuitions
link |
but deliberately try to teach that specialization
link |
without requiring calculus.
link |
So I think high school math would be sufficient.
link |
If you know how to multiply two matrices,
link |
I think that's great.
link |
So a little basic linear algebra is great.
link |
Basic linear algebra,
link |
even very, very basic linear algebra in some programming.
link |
I think that people that have done the machine learning course
link |
will find a deep learning specialization a bit easier
link |
but it's also possible to jump
link |
into the deep learning specialization directly
link |
but it will be a little bit harder
link |
since we tend to go over faster concepts
link |
like how does gradient descent work
link |
and what is the objective function
link |
which is covered more slowly in the machine learning course.
link |
Could you briefly mention some of the key concepts
link |
in deep learning that students should learn
link |
that you envision them learning in the first few months
link |
in the first year or so?
link |
So if you take the deep learning specialization,
link |
you learn the foundations of what is a neural network.
link |
How do you build up a neural network
link |
from a single logistic unit to a stack of layers
link |
to different activation functions.
link |
You learn how to train the neural networks.
link |
One thing I'm very proud of in that specialization
link |
is we go through a lot of practical knowhow
link |
of how to actually make these things work.
link |
So what are the differences between different optimization algorithms?
link |
What do you do if the algorithm overfits
link |
or how do you tell if the algorithm is overfitting?
link |
When do you collect more data?
link |
When should you not bother to collect more data?
link |
I find that even today, unfortunately,
link |
there are engineers that will spend six months
link |
trying to pursue a particular direction
link |
such as collect more data
link |
because we heard more data is valuable
link |
but sometimes you could run some tests
link |
and could have figured out six months earlier
link |
that for this particular problem, collecting more data isn't going to cut it.
link |
So just don't spend six months collecting more data.
link |
Spend your time modifying the architecture or trying something else.
link |
So go through a lot of the practical knowhow
link |
so that when someone, when you take the deep learning specialization,
link |
you have those skills to be very efficient
link |
in how you build these networks.
link |
So dive right in to play with the network, to train it,
link |
to do the inference on a particular data set,
link |
to build intuition about it without building it up too big
link |
to where you spend, like you said, six months
link |
learning, building up your big project
link |
without building any intuition of a small aspect of the data
link |
that could already tell you everything you need to know about that data.
link |
Yes, and also the systematic frameworks of thinking
link |
for how to go about building practical machine learning.
link |
Maybe to make an analogy, when we learn to code,
link |
we have to learn the syntax of some programming language, right?
link |
Be it Python or C++ or Octave or whatever.
link |
But the equally important or maybe even more important part of coding
link |
is to understand how to string together these lines of code
link |
into coherent things.
link |
So when should you put something in a function column?
link |
When should you not?
link |
How do you think about abstraction?
link |
So those frameworks are what makes a programmer efficient
link |
even more than understanding the syntax.
link |
I remember when I was an undergrad at Carnegie Mellon,
link |
one of my friends would debug their code
link |
by first trying to compile it, and then it was C++ code.
link |
And then every line in the syntax error,
link |
they want to get rid of the syntax errors as quickly as possible.
link |
So how do you do that?
link |
Well, they would delete every single line of code with a syntax error.
link |
So really efficient for getting rid of syntax errors
link |
for horrible debugging errors.
link |
So I think we learn how to debug.
link |
And I think in machine learning,
link |
the way you debug a machine learning program
link |
is very different than the way you do binary search or whatever,
link |
or use a debugger, trace through the code
link |
in traditional software engineering.
link |
So it's an evolving discipline,
link |
but I find that the people that are really good
link |
at debugging machine learning algorithms
link |
are easily 10x, maybe 100x faster at getting something to work.
link |
And the basic process of debugging is,
link |
so the bug in this case,
link |
why isn't this thing learning, improving,
link |
sort of going into the questions of overfitting
link |
and all those kinds of things?
link |
That's the logical space that the debugging is happening in
link |
with neural networks.
link |
Yeah, often the question is, why doesn't it work yet?
link |
Or can I expect it to eventually work?
link |
And what are the things I could try?
link |
Change the architecture, more data, more regularization,
link |
different optimization algorithm,
link |
different types of data.
link |
So to answer those questions systematically,
link |
so that you don't spend six months hitting down the blind alley
link |
before someone comes and says,
link |
why did you spend six months doing this?
link |
What concepts in deep learning
link |
do you think students struggle the most with?
link |
Or sort of is the biggest challenge for them
link |
was to get over that hill.
link |
It hooks them and it inspires them and they really get it.
link |
Similar to learning mathematics,
link |
I think one of the challenges of deep learning
link |
is that there are a lot of concepts
link |
that build on top of each other.
link |
If you ask me what's hard about mathematics,
link |
I have a hard time pinpointing one thing.
link |
Is it addition, subtraction?
link |
Is it multiplication?
link |
There's just a lot of stuff.
link |
I think one of the challenges of learning math
link |
and of learning certain technical fields
link |
is that there are a lot of concepts
link |
and if you miss a concept,
link |
then you're kind of missing the prerequisite
link |
for something that comes later.
link |
So in the deep learning specialization,
link |
try to break down the concepts
link |
to maximize the odds of each component being understandable.
link |
So when you move on to the more advanced thing,
link |
we learn confidence,
link |
hopefully you have enough intuitions
link |
from the earlier sections
link |
to then understand why we structure confidence
link |
and then eventually why we built RNNs and LSTMs
link |
or attention models in a certain way
link |
building on top of the earlier concepts.
link |
Actually, I'm curious,
link |
you do a lot of teaching as well.
link |
Do you have a favorite,
link |
this is the hard concept moment in your teaching?
link |
Well, I don't think anyone's ever turned the interview on me.
link |
I'm glad you get first.
link |
I think that's a really good question.
link |
Yeah, it's really hard to capture the moment
link |
when they struggle.
link |
I think you put it really eloquently.
link |
I do think there's moments
link |
that are like aha moments
link |
that really inspire people.
link |
I think for some reason,
link |
reinforcement learning,
link |
especially deep reinforcement learning
link |
is a really great way
link |
to really inspire people
link |
and get what the use of neural networks can do.
link |
Even though neural networks
link |
really are just a part of the deep RL framework,
link |
but it's a really nice way
link |
to paint the entirety of the picture
link |
of a neural network
link |
being able to learn from scratch,
link |
knowing nothing and explore the world
link |
and pick up lessons.
link |
I find that a lot of the aha moments
link |
happen when you use deep RL
link |
to teach people about neural networks,
link |
which is counterintuitive.
link |
I find like a lot of the inspired sort of fire
link |
in people's passion,
link |
it comes from the RL world.
link |
Do you find reinforcement learning
link |
to be a useful part
link |
of the teaching process or no?
link |
I still teach reinforcement learning
link |
in one of my Stanford classes
link |
and my PhD thesis was on reinforcement learning.
link |
So I clearly loved a few.
link |
I find that if I'm trying to teach
link |
students the most useful techniques
link |
for them to use today,
link |
I end up shrinking the amount of time
link |
I talk about reinforcement learning.
link |
It's not what's working today.
link |
Now, our world changes so fast.
link |
Maybe this will be totally different
link |
in a couple of years.
link |
But I think we need a couple more things
link |
for reinforcement learning to get there.
link |
One of my teams is looking
link |
to reinforcement learning
link |
for some robotic control tasks.
link |
So I see the applications,
link |
but if you look at it as a percentage
link |
of all of the impact
link |
of the types of things we do,
link |
it's at least today outside of
link |
playing video games, right?
link |
In a few of the games, the scope.
link |
Actually, at NeurIPS,
link |
a bunch of us were standing around
link |
saying, hey, what's your best example
link |
of an actual deploy reinforcement
link |
learning application?
link |
senior machine learning researchers, right?
link |
And again, there are some emerging ones,
link |
but there are not that many great examples.
link |
I think you're absolutely right.
link |
The sad thing is there hasn't been
link |
a big impactful real world application
link |
of reinforcement learning.
link |
I think its biggest impact to me
link |
has been in the toy domain,
link |
in the game domain,
link |
in the small example.
link |
That's what I mean for educational purpose.
link |
It seems to be a fun thing to explore
link |
in your networks with.
link |
But I think from your perspective,
link |
and I think that might be
link |
the best perspective is
link |
if you're trying to educate
link |
with a simple example
link |
in order to illustrate
link |
how this can actually be grown
link |
to scale and have a real world impact,
link |
then perhaps focusing on the fundamentals
link |
of supervised learning
link |
in the context of a simple data set,
link |
even like an MNIST data set
link |
is the right path to take.
link |
The amount of fun I've seen people
link |
have with reinforcement learning
link |
but not in the applied impact
link |
in the real world setting.
link |
So it's a trade off,
link |
how much impact you want to have
link |
versus how much fun you want to have.
link |
Yeah, that's really cool.
link |
And I feel like the world
link |
actually needs all sorts.
link |
Even within machine learning,
link |
I feel like deep learning
link |
shouldn't just use deep learning.
link |
I find that my teams
link |
use a portfolio of tools.
link |
And maybe that's not the exciting thing
link |
to say, but some days
link |
we use a neural net,
link |
some days we use a PCA.
link |
Actually, the other day,
link |
I was sitting down with my team
link |
looking at PCA residuals,
link |
trying to figure out what's going on
link |
to manufacturing problem.
link |
And some days we use
link |
a probabilistic graphical model,
link |
some days we use a knowledge draft,
link |
which is one of the things
link |
that has tremendous industry impact.
link |
But the amount of chatter
link |
about knowledge drafts in academia
link |
is really thin compared
link |
to the actual real world impact.
link |
So I think reinforcement learning
link |
should be in that portfolio.
link |
And then it's about balancing
link |
how much we teach all of these things.
link |
And the world should have
link |
It'd be sad if everyone
link |
just learned one narrow thing.
link |
Yeah, the diverse skill
link |
help you discover the right tool
link |
What is the most beautiful,
link |
surprising or inspiring idea
link |
in deep learning to you?
link |
Something that captivated
link |
Is it the scale that could be,
link |
the performance that could be
link |
achieved with scale?
link |
Or is there other ideas?
link |
I think that if my only job
link |
was being an academic researcher,
link |
if an unlimited budget
link |
and didn't have to worry
link |
about short term impact
link |
and only focus on long term impact,
link |
I'd probably spend all my time
link |
doing research on unsupervised learning.
link |
I still think unsupervised learning
link |
is a beautiful idea.
link |
At both this past NeurIPS and ICML,
link |
I was attending workshops
link |
or listening to various talks
link |
about self supervised learning,
link |
which is one vertical segment
link |
maybe of unsupervised learning
link |
that I'm excited about.
link |
Maybe just to summarize the idea,
link |
I guess you know the idea
link |
about describing fleet.
link |
So here's the example
link |
of self supervised learning.
link |
Let's say we grab a lot
link |
of unlabeled images off the internet.
link |
So with infinite amounts
link |
of this type of data,
link |
I'm going to take each image
link |
and rotate it by a random
link |
multiple of 90 degrees.
link |
And then I'm going to train
link |
a supervised neural network
link |
to predict what was
link |
the original orientation.
link |
So it has to be rotated 90 degrees,
link |
180 degrees, 270 degrees,
link |
So you can generate
link |
an infinite amounts of labeled data
link |
because you rotated the image
link |
so you know what's the
link |
ground truth label.
link |
And so various researchers
link |
have found that by taking
link |
unlabeled data and making
link |
up labeled data sets
link |
and training a large neural network
link |
you can then take the hidden
link |
layer representation and transfer
link |
it to a different task
link |
Learning word embeddings
link |
where we take a sentence,
link |
predict the missing word,
link |
which is how we learn.
link |
One of the ways we learn
link |
is another example.
link |
And I think there's now
link |
this portfolio of techniques
link |
for generating these made up tasks.
link |
Another one called jigsaw
link |
would be if you take an image,
link |
cut it up into a three by three grid,
link |
three by three puzzle piece,
link |
jump up the nine pieces
link |
and have a neural network predict
link |
which of the nine factorial
link |
possible permutations
link |
Peter B has been doing
link |
some work on this too,
link |
Facebook, Google Brain,
link |
Aaron van der Oort
link |
has great work on the CPC objective.
link |
So many teams are doing exciting work
link |
and I think this is a way
link |
to generate infinite label data
link |
and I find this a very exciting
link |
piece of unsupervised learning.
link |
So long term you think
link |
that's going to unlock
link |
in machine learning systems
link |
is this kind of unsupervised learning.
link |
I don't think there's
link |
a whole enchilada,
link |
I think it's just a piece of it
link |
and I think this one piece
link |
self supervised learning
link |
is starting to get traction.
link |
to it being useful.
link |
Well, word embedding
link |
I think we're getting
link |
to just having a significant
link |
maybe in computer vision and video
link |
but I think this concept
link |
and I think there'll be
link |
other concepts around it.
link |
You know, other unsupervised
link |
learning things that I worked on
link |
I've been excited about.
link |
I was really excited
link |
about sparse coding
link |
slow feature analysis.
link |
I think all of these are ideas
link |
that various of us
link |
about a decade ago
link |
before we all got distracted
link |
by how well supervised
link |
learning was doing.
link |
So we would return
link |
we would return to the fundamentals
link |
of representation learning
link |
that really started
link |
this movement of deep learning.
link |
I think there's a lot more work
link |
that one could explore around
link |
this theme of ideas
link |
to come up with better algorithms.
link |
So if we could return
link |
to maybe talk quickly
link |
about the specifics
link |
of deep learning.ai
link |
the deep learning specialization
link |
perhaps how long does it take
link |
to complete the course
link |
The official length
link |
of the deep learning specialization
link |
is I think 16 weeks
link |
so about four months
link |
but it's go at your own pace.
link |
So if you subscribe
link |
to the deep learning specialization
link |
there are people that finished it
link |
in less than a month
link |
by working more intensely
link |
and studying more intensely
link |
so it really depends on
link |
on the individual.
link |
the deep learning specialization
link |
we wanted to make it
link |
and very affordable.
link |
Coursera and deep learning.ai
link |
that's really important to me
link |
is that if there's someone
link |
for whom paying anything
link |
is a financial hardship
link |
then just apply for financial aid
link |
and get it for free.
link |
If you were to recommend
link |
a daily schedule for people
link |
in learning whether it's
link |
through the deep learning.ai
link |
specialization or just learning
link |
in the world of deep learning
link |
what would you recommend?
link |
How do they go about day to day
link |
sort of specific advice
link |
about their journey in the world
link |
of deep learning machine learning?
link |
I think getting the habit of learning
link |
is key and that means regularity.
link |
we send out a weekly newsletter
link |
the batch every Wednesday
link |
so people know it's coming Wednesday
link |
you can spend a little bit of time
link |
catching up on the latest news
link |
catching up on the latest news
link |
through the batch on Wednesday
link |
I've picked up a habit of spending
link |
some time every Saturday
link |
and every Sunday reading or studying
link |
and so I don't wake up on the Saturday
link |
and have to make a decision
link |
do I feel like reading
link |
or studying today or not
link |
it's just what I do
link |
and the fact is a habit
link |
So I think if someone can get into that habit
link |
it's like you know
link |
just like we brush our teeth every morning
link |
I don't think about it
link |
if I thought about it
link |
it's a little bit annoying
link |
to have to spend two minutes doing that
link |
but it's a habit that it takes
link |
but this would be so much harder
link |
if we have to make a decision every morning
link |
and actually that's the reason
link |
why I wear the same thing every day as well
link |
it's just one less decision
link |
I just get up and wear my blue shirt
link |
so but I think if you can get that habit
link |
that consistency of studying
link |
then it actually feels easier.
link |
So yeah it's kind of amazing
link |
like I play guitar every day for
link |
I force myself to at least for five minutes
link |
it's just it's a ridiculously short period of time
link |
but because I've gotten into that habit
link |
it's incredible what you can accomplish
link |
in a period of a year or two years
link |
you know exceptionally good
link |
at certain aspects of a thing
link |
by just doing it every day
link |
for a very short period of time
link |
it's kind of a miracle
link |
that that's how it works
link |
it adds up over time.
link |
Yeah and I think this is often
link |
not about the bursts of sustained efforts
link |
and the all nighters
link |
because you could only do that
link |
a limited number of times
link |
it's the sustained effort over a long time
link |
I think you know reading two research papers
link |
is a nice thing to do
link |
but the power is not reading two research papers
link |
it's reading two research papers a week
link |
then you read a hundred papers
link |
and you actually learn a lot
link |
when you read a hundred papers.
link |
So regularity and making learning a habit
link |
do you have general other study tips
link |
for particularly deep learning
link |
that people should
link |
in their process of learning
link |
is there some kind of recommendations
link |
or tips you have as they learn?
link |
One thing I still do
link |
when I'm trying to study something really deeply
link |
is take handwritten notes
link |
I know there are a lot of people
link |
that take the deep learning courses
link |
during a commute or something
link |
where it may be more awkward to take notes
link |
so I know it may not work for everyone
link |
but when I'm taking courses on Coursera
link |
and I still take some every now and then
link |
the most recent one I took
link |
was a course on clinical trials
link |
because I was interested about that
link |
I got out my little Moleskine notebook
link |
and what I was seeing on my desk
link |
was just taking down notes
link |
so what the instructor was saying
link |
and that act we know that
link |
that act of taking notes
link |
preferably handwritten notes
link |
increases retention.
link |
So as you're sort of watching the video
link |
just kind of pausing maybe
link |
and then taking the basic insights down on paper.
link |
Yeah so there have been a few studies
link |
if you search online
link |
you find some of these studies
link |
that taking handwritten notes
link |
because handwriting is slower
link |
as we're saying just now
link |
it causes you to recode the knowledge
link |
in your own words more
link |
and that process of recoding
link |
promotes long term retention
link |
this is as opposed to typing
link |
again typing is better than nothing
link |
or in taking a class
link |
and not taking notes is better
link |
than not taking any class at all
link |
but comparing handwritten notes
link |
you can usually type faster
link |
for a lot of people
link |
you can handwrite notes
link |
and so when people type
link |
they're more likely to just transcribe
link |
verbatim what they heard
link |
and that reduces the amount of recoding
link |
and that actually results
link |
in less long term retention.
link |
I don't know what the psychological effect
link |
there is but so true
link |
there's something fundamentally different
link |
about writing hand handwriting
link |
I wonder what that is
link |
I wonder if it is as simple
link |
as just the time it takes to write it slower
link |
yeah and because you can't write
link |
you have to take whatever they said
link |
and summarize it into fewer words
link |
and that summarization process
link |
requires deeper processing of the meaning
link |
which then results in better retention
link |
that's fascinating
link |
oh and I think because of Coursera
link |
I spent so much time studying pedagogy
link |
this is actually one of my passions
link |
I really love learning
link |
how to more efficiently
link |
you know one of the things I do
link |
both when creating videos
link |
or when we write the batch is
link |
I try to think is one minute spent of us
link |
going to be a more efficient learning experience
link |
than one minute spent anywhere else
link |
and we really try to you know
link |
make it time efficient for the learners
link |
because you know everyone's busy
link |
so when when we're editing
link |
I often tell my teams
link |
every word needs to fight for its life
link |
and if you can delete a word
link |
let's just delete it and not wait
link |
let's not waste the learning time
link |
let's not waste the learning time
link |
oh that's so it's so amazing
link |
that you think that way
link |
because there is millions of people
link |
that are impacted by your teaching
link |
and sort of that one minute spent
link |
has a ripple effect right
link |
through years of time
link |
which is it's just fascinating to think about
link |
how does one make a career
link |
out of an interest in deep learning
link |
do you have advice for people
link |
we just talked about
link |
sort of the beginning early steps
link |
but if you want to make it
link |
an entire life's journey
link |
or at least a journey of a decade or two
link |
how do you how do you do it
link |
so most important thing is to get started
link |
right and and I think in the early parts
link |
of a career coursework
link |
um like the deep learning specialization
link |
or it's a very efficient way
link |
to master this material
link |
so because you know instructors
link |
uh be it me or someone else
link |
or you know Lawrence Maroney
link |
teaches our TensorFlow specialization
link |
or other things we're working on
link |
spend effort to try to make it time efficient
link |
for you to learn a new concept
link |
so coursework is actually a very efficient way
link |
for people to learn concepts
link |
and the beginning parts of breaking
link |
in fact one thing I see at Stanford
link |
some of my PhD students want to jump
link |
in the research right away
link |
and I actually tend to say look
link |
in your first couple years of PhD
link |
and spend time taking courses
link |
because it lays a foundation
link |
it's fine if you're less productive
link |
in your first couple years
link |
you'll be better off in the long term
link |
beyond a certain point
link |
there's materials that doesn't exist in courses
link |
because it's too cutting edge
link |
the course hasn't been created yet
link |
there's some practical experience
link |
that we're not yet that good
link |
as teaching in a course
link |
and I think after exhausting
link |
the efficient coursework
link |
then most people need to go on
link |
to either ideally work on projects
link |
and then maybe also continue their learning
link |
by reading blog posts and research papers
link |
and things like that
link |
doing projects is really important
link |
and again I think it's important
link |
to start small and just do something
link |
today you read about deep learning
link |
feels like oh all these people
link |
doing such exciting things
link |
what if I'm not building a neural network
link |
that changes the world
link |
then what's the point?
link |
Well the point is sometimes building
link |
that tiny neural network
link |
you know be it MNIST or upgrade
link |
to a fashion MNIST to whatever
link |
so doing your own fun hobby project
link |
that's how you gain the skills
link |
to let you do bigger and bigger projects
link |
I find this to be true at the individual level
link |
and also at the organizational level
link |
for a company to become good at machine learning
link |
sometimes the right thing to do
link |
is not to tackle the giant project
link |
is instead to do the small project
link |
that lets the organization learn
link |
and then build out from there
link |
but this is true both for individuals
link |
taking the first step
link |
and then taking small steps is the key
link |
should students pursue a PhD
link |
do you think you can do so much
link |
that's one of the fascinating things
link |
in machine learning
link |
you can have so much impact
link |
without ever getting a PhD
link |
so what are your thoughts
link |
should people go to grad school
link |
should people get a PhD?
link |
I think that there are multiple good options
link |
of which doing a PhD could be one of them
link |
I think that if someone's admitted
link |
to a top PhD program
link |
you know at MIT, Stanford, top schools
link |
I think that's a very good experience
link |
or if someone gets a job
link |
at a top organization
link |
at the top AI team
link |
I think that's also a very good experience
link |
there are some things you still need a PhD to do
link |
if someone's aspiration is to be a professor
link |
you know at the top academic university
link |
you just need a PhD to do that
link |
but if it goes to you know
link |
start a company, build a company
link |
do great technical work
link |
I think a PhD is a good experience
link |
but I would look at the different options
link |
available to someone
link |
you know where are the places
link |
where you can get a job
link |
where are the places to get a PhD program
link |
and kind of weigh the pros and cons of those
link |
So just to linger on that for a little bit longer
link |
what final dreams and goals
link |
do you think people should have
link |
so what options should they explore
link |
so you can work in industry
link |
so for a large company
link |
like Google, Facebook, Baidu
link |
all these large sort of companies
link |
that already have huge teams
link |
of machine learning engineers
link |
you can also do with an industry
link |
sort of more research groups
link |
that kind of like Google Research, Google Brain
link |
then you can also do
link |
like we said a professor in academia
link |
oh you can build your own company
link |
you can do a startup
link |
is there anything that stands out
link |
between those options
link |
or are they all beautiful different journeys
link |
that people should consider
link |
I think the thing that affects your experience more
link |
is less are you in this company
link |
versus that company
link |
or academia versus industry
link |
I think the thing that affects your experience most
link |
is who are the people you're interacting with
link |
so even if you look at some of the large companies
link |
the experience of individuals
link |
in different teams is very different
link |
and what matters most is not the logo above the door
link |
when you walk into the giant building every day
link |
what matters the most is who are the 10 people
link |
who are the 30 people you interact with every day
link |
so I actually tend to advise people
link |
if you get a job from a company
link |
ask who is your manager
link |
who are your peers
link |
who are you actually going to talk to
link |
we're all social creatures
link |
we tend to become more like the people around us
link |
and if you're working with great people
link |
you will learn faster
link |
or if you get admitted
link |
if you get a job at a great company
link |
or a great university
link |
maybe the logo you walk in is great
link |
but you're actually stuck on some team
link |
doing really work that doesn't excite you
link |
and then that's actually a really bad experience
link |
so this is true both for universities
link |
and for large companies
link |
for small companies you can kind of figure out
link |
who you'll be working with quite quickly
link |
and I tend to advise people
link |
if a company refuses to tell you
link |
who you will work with
link |
someone say oh join us
link |
the rotation system will figure it out
link |
I think that that's a worrying answer
link |
because it because it means you may not get sent
link |
to you may not actually get to a team
link |
with great peers and great people to work with
link |
it's actually a really profound advice
link |
that we kind of sometimes sweep
link |
we don't consider too rigorously or carefully
link |
the people around you are really often
link |
especially when you accomplish great things
link |
it seems the great things are accomplished
link |
because of the people around you
link |
so that's a it's not about the the
link |
where whether you learn this thing
link |
or that thing or like you said
link |
the logo that hangs up top
link |
it's the people that's a fascinating
link |
and it's such a hard search process
link |
of finding just like finding the right friends
link |
and somebody to get married with
link |
and that kind of thing
link |
it's a very hard search
link |
it's a people search problem
link |
yeah but I think when someone interviews
link |
you know at a university
link |
or the research lab or the large corporation
link |
it's good to insist on just asking
link |
who are the people
link |
and if you refuse to tell me
link |
I'm gonna think well maybe that's
link |
because you don't have a good answer
link |
it may not be someone I like
link |
and if you don't particularly connect
link |
if something feels off with the people
link |
then don't stick to it
link |
you know that's a really important signal to consider
link |
yeah yeah and actually I actually
link |
in my standard class CS230
link |
as well as an ACM talk
link |
I think I gave like a hour long talk
link |
including on the job search process
link |
and then some of these
link |
so you can find those videos online
link |
awesome and I'll point them
link |
I'll point people to them
link |
so the AI fund helps AI startups
link |
get off the ground
link |
or perhaps you can elaborate
link |
on all the fun things it's involved with
link |
what's your advice
link |
and how does one build a successful AI startup
link |
you know in Silicon Valley
link |
a lot of startup failures
link |
come from building other products
link |
that no one wanted
link |
so when you know cool technology
link |
but who's going to use it
link |
so I think I tend to be very outcome driven
link |
and customer obsessed
link |
ultimately we don't get to vote
link |
if we succeed or fail
link |
it's only the customer
link |
that they're the only one
link |
that gets a thumbs up or thumbs down vote
link |
you know there are various people
link |
that get various votes
link |
but in the long term
link |
that's what really matters
link |
so as you build the startup
link |
you have to constantly ask the question
link |
will the customer give a thumbs up on this
link |
I think startups that are very customer focused
link |
deeply understand the customer
link |
and are oriented to serve the customer
link |
are more likely to succeed
link |
with the provisional
link |
I think all of us should only do things
link |
that we think create social good
link |
and moves the world forward
link |
so I personally don't want to build
link |
addictive digital products
link |
just to sell a lot of ads
link |
or you know there are things
link |
that could be lucrative
link |
but if we can find ways to serve people
link |
in meaningful ways
link |
I think those can be
link |
great things to do
link |
either in the academic setting
link |
or in a corporate setting
link |
or a startup setting
link |
so can you give me the idea
link |
of why you started the AI fund
link |
I remember when I was leading
link |
the AI group at Baidu
link |
two parts of my job
link |
one was to build an AI engine
link |
to support the existing businesses
link |
and that was running
link |
just performed by itself
link |
there was a second part of my job at the time
link |
which was to try to systematically initiate
link |
new lines of businesses
link |
using the company's AI capabilities
link |
so you know the self driving car team
link |
came out of my group
link |
the smart speaker team
link |
similar to what is Amazon Echo Alexa in the US
link |
but we actually announced it
link |
so Baidu wasn't following Amazon
link |
that came out of my group
link |
and I found that to be
link |
actually the most fun part of my job
link |
so what I wanted to do was
link |
to build AI fund as a startup studio
link |
to systematically create new startups
link |
with all the things we can now do with AI
link |
I think the ability to build new teams
link |
to go after this rich space of opportunities
link |
is a very important way
link |
to very important mechanism
link |
to get these projects done
link |
that I think will move the world forward
link |
so I've been fortunate to build a few teams
link |
that had a meaningful positive impact
link |
and I felt that we might be able to do this
link |
in a more systematic repeatable way
link |
so a startup studio is a relatively new concept
link |
there are maybe dozens of startup studios
link |
you know right now
link |
but I feel like all of us
link |
many teams are still trying to figure out
link |
how do you systematically build companies
link |
with a high success rate
link |
so I think even a lot of my you know
link |
venture capital friends are
link |
seem to be more and more building companies
link |
rather than investing in companies
link |
but I find a fascinating thing to do
link |
to figure out the mechanisms
link |
by which we could systematically build
link |
successful teams, successful businesses
link |
in areas that we find meaningful
link |
so a startup studio is something
link |
is a place and a mechanism
link |
for startups to go from zero to success
link |
to try to develop a blueprint
link |
it's actually a place for us
link |
to build startups from scratch
link |
so we often bring in founders
link |
and work with them
link |
or maybe even have existing ideas
link |
that we match founders with
link |
and then this launches
link |
you know hopefully into successful companies
link |
so how close are you to figuring out
link |
a way to automate the process
link |
of starting from scratch
link |
and building a successful AI startup
link |
yeah I think we've been constantly
link |
improving and iterating on our processes
link |
so things like you know
link |
how many customer calls do we need to make
link |
in order to get customer validation
link |
how do we make sure this technology
link |
quite a lot of our businesses
link |
need cutting edge machine learning algorithms
link |
so you know kind of algorithms
link |
have developed in the last one or two years
link |
and even if it works in a research paper
link |
it turns out taking the production
link |
there are a lot of issues
link |
for making these things work in the real life
link |
that are not widely addressed in academia
link |
so how do we validate
link |
that this is actually doable
link |
how do you build a team
link |
get the specialized domain knowledge
link |
be it in education or health care
link |
whatever sector we're focusing on
link |
so I think we've actually getting
link |
we've been getting much better
link |
at giving the entrepreneurs
link |
a high success rate
link |
but I think we're still
link |
I think the whole world is still
link |
in the early phases of figuring this out
link |
but do you think there is some aspects
link |
of that process that are transferable
link |
from one startup to another
link |
to another to another
link |
you know starting from scratch
link |
you know starting a company
link |
to most entrepreneurs
link |
is a really lonely thing
link |
and I've seen so many entrepreneurs
link |
not know how to make certain decisions
link |
like when do you need to
link |
how do you do B2B sales right
link |
if you don't know that
link |
or how do you market this efficiently
link |
other than you know buying ads
link |
which is really expensive
link |
are there more efficient tactics for that
link |
or for a machine learning project
link |
you know basic decisions
link |
can change the course of
link |
whether machine learning product works or not
link |
and so there are so many hundreds of decisions
link |
that entrepreneurs need to make
link |
and making a mistake
link |
and a couple key decisions
link |
can have a huge impact
link |
on the fate of the company
link |
so I think a startup studio
link |
provides a support structure
link |
that makes starting a company
link |
much less of a lonely experience
link |
and also when facing with these key decisions
link |
like trying to hire your first
link |
uh the VP of engineering
link |
what's a good selection criteria
link |
should I hire this person or not
link |
by helping by having a ecosystem
link |
around the entrepreneurs
link |
the founders to help
link |
I think we help them at the key moments
link |
and hopefully significantly
link |
make them more enjoyable
link |
and then higher success rate
link |
so there's somebody to brainstorm with
link |
in these very difficult decision points
link |
and also to help them recognize
link |
what they may not even realize
link |
is a key decision point
link |
that's that's the first
link |
and probably the most important part
link |
yeah actually I can say one other thing
link |
um you know I think
link |
building companies is one thing
link |
but I feel like it's really important
link |
that we build companies
link |
that move the world forward
link |
for example within the AI Fund team
link |
there was once an idea
link |
that if it had succeeded
link |
would have resulted in people
link |
watching a lot more videos
link |
in a certain narrow vertical type of video
link |
the business case was fine
link |
the revenue case was fine
link |
but I looked and just said
link |
I don't want to do this
link |
like you know I don't actually
link |
just want to have a lot more people
link |
watch this type of video
link |
wasn't educational
link |
it's an educational baby
link |
and so and so I I I I code the idea
link |
on the basis that I didn't think
link |
it would actually help people
link |
so um whether building companies
link |
or working enterprises
link |
or doing personal projects
link |
I think um it's up to each of us
link |
to figure out what's the difference
link |
we want to make in the world
link |
you help already established companies
link |
grow their AI and machine learning efforts
link |
how does a large company
link |
integrate machine learning
link |
into their efforts?
link |
AI is a general purpose technology
link |
and I think it will transform every industry
link |
our community has already transformed
link |
the software internet sector
link |
most software internet companies
link |
outside the top right
link |
five or six or three or four
link |
already have reasonable
link |
machine learning capabilities
link |
or or getting there
link |
it's still room for improvement
link |
but when I look outside
link |
the software internet sector
link |
everything from manufacturing
link |
agriculture, healthcare
link |
logistics transportation
link |
there's so many opportunities
link |
that very few people are working on
link |
so I think the next wave of AI
link |
is for us to also transform
link |
all of those other industries
link |
there was a McKinsey study
link |
estimating 13 trillion dollars
link |
of global economic growth
link |
US GDP is 19 trillion dollars
link |
so 13 trillion is a big number
link |
or PwC estimates 16 trillion dollars
link |
so whatever number is is large
link |
but the interesting thing to me
link |
was a lot of that impact
link |
the software internet sector
link |
so we need more teams
link |
to work with these companies
link |
to help them adopt AI
link |
and I think this is one thing
link |
help drive global economic growth
link |
and make humanity more powerful
link |
and like you said the impact is there
link |
so what are the best industries
link |
the biggest industries
link |
perhaps outside the software tech sector
link |
frankly I think it's all of them
link |
some of the ones I'm spending a lot of time on
link |
are manufacturing agriculture
link |
look into healthcare
link |
for example in manufacturing
link |
we do a lot of work in visual inspection
link |
where today there are people standing around
link |
using the eye human eye
link |
to check if you know
link |
this plastic part or the smartphone
link |
or this thing has a scratch
link |
or a dent or something in it
link |
we can use a camera to take a picture
link |
deep learning and other things
link |
to check if it's defective or not
link |
and thus help factories improve yield
link |
and improve quality
link |
and improve throughput
link |
it turns out the practical problems
link |
we run into are very different
link |
than the ones you might read about
link |
in in most research papers
link |
the data sets are really small
link |
so we face small data problems
link |
you know the factories
link |
keep on changing the environment
link |
so it works well on your test set
link |
something changes in the factory
link |
the lights go on or off
link |
recently there was a factory
link |
in which a bird threw through the factory
link |
and pooped on something
link |
and so that changed stuff
link |
and so increasing our algorithm
link |
so all the changes happen in the factory
link |
I find that we run a lot of practical problems
link |
that are not as widely discussed
link |
and it's really fun
link |
kind of being on the cutting edge
link |
solving these problems before
link |
maybe before many people are even aware
link |
that there is a problem there
link |
and that's such a fascinating space
link |
you're absolutely right
link |
but what is the first step
link |
that a company should take
link |
it's just scary leap
link |
into this new world of
link |
going from the human eye
link |
inspecting to digitizing that process
link |
having an algorithm
link |
what's the first step
link |
like what's the early journey
link |
that you recommend
link |
that you see these companies taking
link |
I published a document
link |
called the AI Transformation Playbook
link |
and taught briefly in the AI for Everyone
link |
course on Coursera
link |
about the long term journey
link |
that companies should take
link |
but the first step
link |
is actually to start small
link |
I've seen a lot more companies fail
link |
by starting too big
link |
than by starting too small
link |
you know most people don't realize
link |
and how controversial it was
link |
so when I started Google Brain
link |
it was controversial
link |
you know people thought
link |
deep learning near nest
link |
tried it didn't work
link |
why would you want to do deep learning
link |
so my first internal customer
link |
was the Google speech team
link |
which is not the most lucrative
link |
not the most important
link |
it's not web search or advertising
link |
but by starting small
link |
my team helped the speech team
link |
build a more accurate speech recognition system
link |
and this caused their peers
link |
other teams to start
link |
to have more faith in deep learning
link |
my second internal customer
link |
was the Google Maps team
link |
where we used computer vision
link |
to read house numbers
link |
from basic street view images
link |
to more accurately locate houses
link |
within Google Maps
link |
so improve the quality of geodata
link |
and it was only after those two successes
link |
that I then started
link |
a more serious conversation
link |
with the Google Ads team
link |
and so there's a ripple effect
link |
that you showed that it works
link |
and then it just propagates
link |
through the entire company
link |
that this thing has a lot of value
link |
I think the early small scale projects
link |
it helps the teams gain faith
link |
but also helps the teams learn
link |
what these technologies do
link |
I still remember when our first GPU server
link |
it was a server under some guy's desk
link |
and you know and then that taught us
link |
early important lessons about
link |
how do you have multiple users
link |
share a set of GPUs
link |
which is really not obvious at the time
link |
but those early lessons were important
link |
we learned a lot from that first GPU server
link |
that later helped the teams think through
link |
how to scale it up
link |
to much larger deployments
link |
Are there concrete challenges
link |
that companies face
link |
that you see is important for them to solve?
link |
I think building and deploying
link |
machine learning systems is hard
link |
there's a huge gulf between
link |
something that works
link |
in a jupyter notebook on your laptop
link |
versus something that runs
link |
their production deployment setting
link |
in a factory or agriculture plant or whatever
link |
so I see a lot of people
link |
get something to work on your laptop
link |
and say wow look what I've done
link |
and that's great that's hard
link |
that's a very important first step
link |
but a lot of teams underestimate
link |
the rest of the steps needed
link |
I've heard this exact same conversation
link |
between a lot of machine learning people
link |
and business people
link |
the machine learning person says
link |
look my algorithm does well on the test set
link |
and it's a clean test set at the end of peak
link |
and the machine and the business person says
link |
thank you very much
link |
but your algorithm sucks it doesn't work
link |
and the machine learning person says
link |
no wait I did well on the test set
link |
and I think there is a gulf between
link |
what it takes to do well on the test set
link |
on your hard drive
link |
versus what it takes to work well
link |
in a deployment setting
link |
some common problems
link |
robustness and generalization
link |
you deploy something in the factory
link |
maybe they chop down a tree outside the factory
link |
so the tree no longer covers the window
link |
and the lighting is different
link |
so the test set changes
link |
and in machine learning
link |
and especially in academia
link |
we don't know how to deal with test set distributions
link |
that are dramatically different
link |
than the training set distribution
link |
you know that this research
link |
the stuff like domain annotation
link |
you know there are people working on it
link |
but we're really not good at this
link |
so how do you actually get this to work
link |
because your test set distribution
link |
is going to change
link |
and I think also if you look at the number of lines of code
link |
in the software system
link |
the machine learning model is maybe five percent
link |
relative to the entire software system
link |
so how do you get all that work done
link |
and make it reliable and systematic
link |
so good software engineering work
link |
is fundamental here
link |
to building a successful small machine learning system
link |
yes and the software system
link |
needs to interface with the machine learning system
link |
needs to interface with people's workloads
link |
so machine learning is automation on steroids
link |
if we take one task out of many tasks
link |
that are done in the factory
link |
so the factory does lots of things
link |
one task is vision inspection
link |
if we automate that one task
link |
it can be really valuable
link |
but you may need to redesign a lot of other tasks
link |
around that one task
link |
for example say the machine learning algorithm
link |
says this is defective
link |
what are you supposed to do
link |
do you throw it away
link |
do you get a human to double check
link |
do you want to rework it or fix it
link |
so you need to redesign a lot of tasks
link |
around that thing you've now automated
link |
so planning for the change management
link |
and making sure that the software you write
link |
is consistent with the new workflow
link |
and you take the time to explain to people
link |
what needs to happen
link |
so I think what landing AI has become good at
link |
and then I think we learned by making the steps
link |
and you know painful experiences
link |
well my what would become good at is
link |
working with our partners to think through
link |
all the things beyond just the machine learning model
link |
or running the jupyter notebook
link |
but to build the entire system
link |
manage the change process
link |
and figure out how to deploy this in a way
link |
that has an actual impact
link |
the processes that the large software tech companies
link |
use for deploying don't work
link |
for a lot of other scenarios
link |
for example when I was leading large speech teams
link |
if the speech recognition system goes down
link |
what happens well alarms goes off
link |
and then someone like me would say hey
link |
you 20 engine environment
link |
you 20 engineers please fix this
link |
but if you have a system girl in the factory
link |
there are not 20 machine learning engineers
link |
sitting around you can page your duty
link |
and have them fix it
link |
so how do you deal with the maintenance
link |
or the or the dev ops or the mo ops
link |
or the other aspects of this
link |
so these are concepts that I think landing AI
link |
and a few other teams on the cutting edge
link |
but we don't even have systematic terminology yet
link |
to describe some of the stuff we do
link |
because I think we're inventing it on the fly.
link |
So you mentioned some people are interested
link |
in discovering mathematical beauty
link |
and truth in the universe
link |
and you're interested in having
link |
a big positive impact in the world
link |
so let me ask the two are not inconsistent
link |
no they're all together
link |
I'm only half joking
link |
because you're probably interested a little bit in both
link |
but let me ask a romanticized question
link |
so much of the work
link |
your work and our discussion today
link |
has been on applied AI
link |
maybe you can even call narrow AI
link |
where the goal is to create systems
link |
that automate some specific process
link |
that adds a lot of value to the world
link |
but there's another branch of AI
link |
starting with Alan Turing
link |
that kind of dreams of creating human level
link |
or superhuman level intelligence
link |
is this something you dream of as well
link |
do you think we human beings
link |
will ever build a human level intelligence
link |
or superhuman level intelligence system?
link |
I would love to get to AGI
link |
and I think humanity will
link |
but whether it takes 100 years
link |
I find hard to estimate
link |
some folks have worries
link |
about the different trajectories
link |
that path would take
link |
even existential threats of an AGI system
link |
do you have such concerns
link |
whether in the short term or the long term?
link |
I do worry about the long term fate of humanity
link |
I do wonder as well
link |
I do worry about overpopulation on the planet Mars
link |
I think there will be a day
link |
when maybe someday in the future
link |
Mars will be polluted
link |
there are all these children dying
link |
and someone will look back at this video
link |
and say Andrew how is Andrew so heartless?
link |
He didn't care about all these children
link |
dying on the planet Mars
link |
and I apologize to the future viewer
link |
I do care about the children
link |
but I just don't know how to
link |
productively work on that today
link |
your picture will be in the dictionary
link |
for the people who are ignorant
link |
about the overpopulation on Mars
link |
yes so it's a long term problem
link |
is there something in the short term
link |
we should be thinking about
link |
in terms of aligning the values of our AI systems
link |
with the values of us humans
link |
sort of something that Stuart Russell
link |
and other folks are thinking about
link |
as this system develops more and more
link |
we want to make sure that it represents
link |
the better angels of our nature
link |
the ethics the values of our society
link |
you know if you take self driving cars
link |
the biggest problem with self driving cars
link |
is not that there's some trolley dilemma
link |
and you teach this so you know
link |
how many times when you are driving your car
link |
did you face this moral dilemma
link |
who do I crash into?
link |
so I think self driving cars
link |
will run into that problem roughly as often
link |
as we do when we drive our cars
link |
the biggest problem with self driving cars
link |
is when there's a big white truck across the road
link |
and what you should do is break
link |
and not crash into it
link |
and the self driving car fails
link |
and it crashes into it
link |
so I think we need to solve that problem first
link |
I think the problem with some of these discussions
link |
about AGI you know alignments
link |
the paperclip problem
link |
is that is a huge distraction
link |
from the much harder problems
link |
that we actually need to address today
link |
it's not the hardest problems
link |
we need to address today
link |
it's not the hard problems
link |
we need to address today
link |
I think bias is a huge issue
link |
I worry about wealth and equality
link |
the AI and internet are causing
link |
an acceleration of concentration of power
link |
because we can now centralize data
link |
use AI to process it
link |
and so industry after industry
link |
we've affected every industry
link |
so the internet industry has a lot of
link |
or win and take all dynamics
link |
but we've infected all these other industries
link |
so we're also giving these other industries
link |
most of them to take all flavors
link |
so look at what Uber and Lyft
link |
did to the taxi industry
link |
so we're doing this type of thing
link |
it's a lot and so this
link |
so we're creating tremendous wealth
link |
but how do we make sure that the wealth
link |
I think that and then how do we help
link |
people whose jobs are displaced
link |
you know I think education is part of it
link |
there may be even more
link |
that we need to do than education
link |
I think bias is a serious issue
link |
there are adverse uses of AI
link |
like deepfakes being used
link |
for various and various purposes
link |
so I worry about some teams
link |
maybe accidentally
link |
and I hope not deliberately
link |
making a lot of noise about things
link |
that problems in the distant future
link |
rather than focusing on
link |
some of the much harder problems
link |
yeah the overshadow of the problems
link |
that we have already today
link |
they're exceptionally challenging
link |
like those you said
link |
and even the silly ones
link |
but the ones that have a huge impact
link |
which is the lighting variation
link |
outside of your factory window
link |
that that ultimately is
link |
what makes the difference
link |
between like you said
link |
the Jupiter notebook
link |
and something that actually transforms
link |
an entire industry potentially
link |
and then just to some companies
link |
or a regulator comes to you
link |
and says look your product
link |
is messing things up
link |
fixing it may have a revenue impact
link |
well it's much more fun to talk to them
link |
about how you promise
link |
not to wipe out humanity
link |
and to face the actually really hard problems we face
link |
so your life has been a great journey
link |
from teaching to research
link |
to entrepreneurship
link |
one are there regrets
link |
moments that if you went back
link |
you would do differently
link |
and two are there moments
link |
you're especially proud of
link |
moments that made you truly happy
link |
you know I've made so many mistakes
link |
it feels like every time
link |
I discover something
link |
I go why didn't I think of this
link |
you know five years earlier
link |
or even 10 years earlier
link |
and then sometimes I read a book
link |
and I go I wish I read this book 10 years ago
link |
my life would have been so different
link |
although that happened recently
link |
and then I was thinking
link |
if only I read this book
link |
when we're starting up Coursera
link |
I could have been so much better
link |
but I discovered the book
link |
had not yet been written
link |
we're starting Coursera
link |
so that made me feel better
link |
so that made me feel better
link |
but I find that the process of discovery
link |
we keep on finding out things
link |
that seem so obvious in hindsight
link |
but it always takes us so much longer
link |
than than I wish to to figure it out
link |
so on the second question
link |
are there moments in your life
link |
that if you look back
link |
that you're especially proud of
link |
or you're especially happy
link |
what would be the that filled you with happiness
link |
one does my daughter know of her
link |
because I know how much time I spent with her
link |
I just can't spend enough time with her
link |
congratulations by the way
link |
and then second is helping other people
link |
I think the meaning of life
link |
is helping others achieve
link |
whatever are their dreams
link |
and then also to try to move the world forward
link |
making humanity more powerful as a whole
link |
so the times that I felt most happy
link |
most proud was when I felt
link |
someone else allowed me the good fortune
link |
of helping them a little bit
link |
on the path to their dreams
link |
I think there's no better way to end it
link |
than talking about happiness
link |
and the meaning of life
link |
so Andrew it's a huge honor
link |
me and millions of people
link |
thank you for all the work you've done
link |
thank you for talking today
link |
thank you so much thanks
link |
thanks for listening to this conversation with Andrew Ng
link |
and thank you to our presenting sponsor Cash App
link |
download it use code LEX podcast
link |
you'll get ten dollars
link |
and ten dollars will go to FIRST
link |
an organization that inspires and educates young minds
link |
to become science and technology innovators of tomorrow
link |
if you enjoy this podcast
link |
subscribe on YouTube
link |
give it five stars on Apple podcast
link |
support it on Patreon
link |
or simply connect with me on Twitter
link |
and now let me leave you with some words of wisdom from Andrew Ng
link |
if what you're working on succeeds beyond your wildest dreams
link |
would you have significantly helped other people?
link |
if not then keep searching for something else to work on
link |
otherwise you're not living up to your full potential
link |
thank you for listening and hope to see you next time