back to indexCharles Isbell and Michael Littman: Machine Learning and Education | Lex Fridman Podcast #148
link |
The following is a conversation with Charles Isbell
link |
and Michael Whitman.
link |
Charles is the Dean of the College of Computing
link |
at Georgia Tech and Michael is a computer science professor
link |
at Brown University.
link |
I've spoken with each of them individually on this podcast
link |
and since they are good friends in real life,
link |
we all thought it would be fun
link |
to have a conversation together.
link |
Quick mention of each sponsor,
link |
followed by some thoughts related to the episode.
link |
Thank you to Athletic Greens,
link |
the all in one drink that I start every day with
link |
to cover all my nutritional bases.
link |
Eight Sleep, a mattress that cools itself
link |
and gives me yet another reason to enjoy sleep.
link |
Masterclass, online courses
link |
from some of the most amazing humans in history
link |
and Cash App, the app I use to send money to friends.
link |
Please check out the sponsors in the description
link |
to get a discount and to support this podcast.
link |
As a side note, let me say that having two guests
link |
on the podcast is an experiment
link |
that I've been meaning to do for a while.
link |
In particular, because down the road,
link |
I would like to occasionally be a kind of moderator
link |
for debates between people that may disagree
link |
in some interesting ways.
link |
If you have suggestions for who you would like to see debate
link |
on this podcast, let me know.
link |
As with all experiments of this kind,
link |
it is a learning process.
link |
Both the video and the audio might need improvement.
link |
I realized I think I should probably do
link |
three or more cameras next time
link |
as opposed to just two.
link |
And also try different ways to mount the microphone
link |
for the third person.
link |
Also, after recording this intro,
link |
I'm going to have to go figure out the thumbnail
link |
for the video version of the podcast
link |
since I usually put the guest's head on the thumbnail.
link |
And now there's two heads and two names
link |
to try to fit into the thumbnail.
link |
It's a kind of a bin packing problem
link |
which in theoretical computer science
link |
happens to be an NP hard problem.
link |
Whatever I come up with, if you have better ideas
link |
for the thumbnail, let me know as well.
link |
And in general, I always welcome ideas
link |
how this thing can be improved.
link |
If you enjoy it, subscribe on YouTube,
link |
review it with Five Stars and Apple Podcast,
link |
follow on Spotify, support on Patreon,
link |
or connect with me on Twitter at Lex Friedman.
link |
And now here's my conversation with Charles Isbell
link |
and Michael Littman.
link |
You'll probably disagree about this question,
link |
but what is your biggest, would you say, disagreement
link |
about either something profound and very important
link |
or something completely not important at all?
link |
I don't think we have any disagreements at all.
link |
I'm not sure that's true.
link |
We walked into that one, didn't we?
link |
So one thing that you sometimes mention is that,
link |
and we did this one on air too, as it were,
link |
whether or not machine learning
link |
is computational statistics.
link |
And in particular, and more importantly,
link |
it is not just computational statistics.
link |
So what's missing in the picture?
link |
All the rest of it.
link |
That which is missing.
link |
Oh, yes, well, you can't be wrong now.
link |
Well, it's not just the statistics.
link |
He doesn't even believe this.
link |
We've had this conversation before.
link |
If it were just the statistics,
link |
then we would be happy with where we are.
link |
But it's not just the statistics.
link |
That's why it's computational statistics.
link |
Or if it were just the computational.
link |
I agree that machine learning is not just statistics.
link |
It is not just statistics.
link |
We can agree on that.
link |
Nor is it just computational statistics.
link |
It's computational statistics.
link |
It is computational.
link |
What is the computational and computational statistics?
link |
Does this take us into the realm of computing?
link |
It does, but I think perhaps the way I can get him
link |
to admit that he's wrong is that it's about rules.
link |
It's about symbols.
link |
It's about all these other things.
link |
But statistics is not about rules?
link |
I'm gonna say statistics is about rules.
link |
But it's not just the statistics, right?
link |
It's not just a random variable that you choose
link |
and you have a probability.
link |
I think you have a narrow view of statistics.
link |
Okay, well then what would be the broad view of statistics
link |
that would still allow it to be statistics
link |
and not say history that would make
link |
computational statistics okay?
link |
Well, okay, so I had my first sort of research mentor,
link |
a guy named Tom Landauer,
link |
taught me to do some statistics, right?
link |
And I was annoyed all the time
link |
because the statistics would say
link |
that what I was doing was not statistically significant.
link |
And I was like, but, but, and basically what he said to me
link |
is statistics is how you're gonna keep
link |
from lying to yourself, which I thought was really deep.
link |
It is a way to keep yourself honest in a particular way.
link |
I agree with that.
link |
Yeah, and so you're trying to find rules.
link |
I'm just gonna bring it back to rules.
link |
Wait, wait, wait, could you possibly try to define rules?
link |
Even regular statisticians, noncomputational statisticians,
link |
do spend some of their time evaluating rules, right?
link |
Applying statistics to try to understand
link |
does this rule capture this?
link |
Does this not capture that?
link |
You mean like hypothesis testing kind of thing?
link |
Or like confidence intervals?
link |
I think more like hypothesis.
link |
Like I feel like the word statistic
link |
literally means like a summary,
link |
like a number that summarizes other numbers.
link |
But I think the field of statistics
link |
actually applies that idea to things like rules,
link |
to understand whether or not a rule is valid.
link |
Does software engineering statistics?
link |
Programming languages statistics?
link |
Because I think there's a very,
link |
it's useful to think about a lot of what AI
link |
and machine learning is or certainly should be
link |
as software engineering, as programming languages.
link |
Just to put it in language that you might understand,
link |
the hyperparameters beyond the problem itself.
link |
The hyperparameters is too many syllables
link |
for me to understand.
link |
The hyperparameters.
link |
That goes around it, right?
link |
It's the decisions you choose to make.
link |
It's the metrics you choose to use.
link |
It's the loss function.
link |
You wanna say the practice of machine learning
link |
is different than the practice of statistics.
link |
Like the things you have to worry about
link |
and how you worry about them are different,
link |
therefore they're different.
link |
At a very little, I mean, at the very least.
link |
It's that much is true.
link |
It doesn't mean that statistics,
link |
computational or otherwise aren't important.
link |
I mean, I do a lot of that, for example.
link |
But I think it goes beyond that.
link |
I think that we could think about game theory
link |
in terms of statistics,
link |
but I don't think it's very as useful to do.
link |
I mean, the way I would think about it
link |
or a way I would think about it is this way.
link |
Chemistry is just physics.
link |
But I don't think it's as useful to think about chemistry
link |
as being just physics.
link |
It's useful to think about it as chemistry.
link |
The level of abstraction really matters here.
link |
there are contexts in which it is useful.
link |
I think of it that way, right?
link |
So finding that connection is actually helpful.
link |
And I think that's when I emphasize
link |
the computational statistics thing.
link |
I think I want to befriend statistics and not absorb them.
link |
Here's the A way to think about it
link |
beyond what I just said, right?
link |
So what would you say,
link |
and I want you to think back to a conversation
link |
we had a very long time ago.
link |
What would you say is the difference between,
link |
say, the early 2000s, ICML
link |
and what we used to call NIPS, NeurIPS?
link |
Is there a difference?
link |
A lot of, particularly on the machine learning
link |
that was done there?
link |
ICML was around that long.
link |
So iClear is the new conference, newish.
link |
And ICML was around the 2000.
link |
So ICML predates that.
link |
I think my most cited ICML paper is from 94.
link |
Michael knows this better than me
link |
because, of course, he's significantly older than I am.
link |
But the point is, what is the difference
link |
between ICML and NeurIPS in the late 90s, early 2000s?
link |
I don't know what everyone else's perspective would be,
link |
but I had a particular perspective at that time,
link |
which is I felt like ICML was more
link |
of a computer science place
link |
and that NIPS, NeurIPS was more of an engineering place,
link |
like the kind of math that happened at the two places.
link |
As a computer scientist,
link |
I felt more comfortable with the ICML math.
link |
And the NeurIPS people would say
link |
that that's because I'm dumb.
link |
And that's such an engineering thing to say, so.
link |
I agree with that part of it,
link |
but I do it a little differently.
link |
We actually had a nice conversation
link |
with Tom Dietrich about this in public.
link |
On Twitter just a couple of days ago.
link |
I put it a little differently,
link |
which is that ICML was machine learning done
link |
by a computer scientist.
link |
And NeurIPS was machine learning done
link |
by a computer scientist trying to impress statisticians.
link |
Which was weird because it was the same people,
link |
at least by the time I started paying attention.
link |
But it just felt very, very different.
link |
And I think that that perspective
link |
of whether you're trying to impress the statisticians
link |
or you're trying to impress the programmers
link |
is actually very different and has real impact
link |
on what you choose to worry about
link |
and what kind of outcomes you come to.
link |
So I think it really matters.
link |
I think computational statistics is a means to an end.
link |
It is not an end in some sense.
link |
And I think that really matters here
link |
in the same way that I don't think computer science
link |
is just engineering or just science
link |
or just math or whatever.
link |
Okay, so I'd have to now agree
link |
that now we agree on everything.
link |
The important thing here is that
link |
my opinions may have changed,
link |
but not the fact that I'm right,
link |
I think is what we just came to.
link |
Right, and my opinions may have changed
link |
and not the fact that I'm wrong.
link |
I think I lost myself there too.
link |
But anyway, we're back.
link |
This happens to us sometimes.
link |
How does neural networks change this,
link |
just to even linger on this topic,
link |
change this idea of statistics,
link |
how big of a pie statistics is
link |
within the machine learning thing?
link |
Like, because it sounds like hyperparameters
link |
and also just the role of data.
link |
You know, people are starting to use
link |
this terminology of software 2.0,
link |
which is like the act of programming
link |
as a, like you're a designer
link |
in the hyperparameter space of neural networks,
link |
and you're also the collector and the organizer
link |
and the cleaner of the data,
link |
and that's part of the programming.
link |
So how did, on the NeurIPS versus ICML topic,
link |
what's the role of neural networks
link |
in redefining the size and the role of machine learning?
link |
I can't wait to hear what Michael thinks about this,
link |
but I would add one.
link |
That's true, I will, I'll force myself to.
link |
I think there's one other thing
link |
I would add to your description,
link |
which is the kind of software engineering part
link |
of what does it mean to debug, for example.
link |
But this is a difference between
link |
the kind of computational statistics view
link |
of machine learning and the computational view
link |
of machine learning, which is, I think,
link |
one is worried about the equation, as it were.
link |
And by the way, this is not a value judgment.
link |
I just think it's about perspective.
link |
But the kind of questions you would ask
link |
when you start asking yourself,
link |
well, what does it mean to program
link |
and develop and build the system,
link |
is a very computer sciencey view of the problem.
link |
I mean, if you get on data science Twitter
link |
and econ Twitter, you actually hear this a lot
link |
with the economist and the data scientist
link |
complaining about the machine learning people.
link |
Well, it's just statistics,
link |
and I don't know why they don't see this.
link |
But they're not even asking the same questions.
link |
They're not thinking about it
link |
as a kind of programming problem.
link |
And I think that that really matters,
link |
just asking this question.
link |
I actually think it's a little different
link |
from programming in hyperparameter space
link |
and sort of collecting the data.
link |
But I do think that that immersion really matters.
link |
So I'll give you a quick example
link |
of the way I think about this.
link |
So I teach machine learning.
link |
Michael and I have co taught a machine learning class,
link |
which has now reached, I don't know, 10,000 people at least
link |
over the last several years, or somewhere there's abouts.
link |
And my machine learning assignments are of this form.
link |
So the first one is something like,
link |
implement these five algorithms,
link |
KNN and SVMs and boosting and decision trees
link |
and neural networks, and maybe that's it, I can't remember.
link |
And when I say implement, I mean steal the code.
link |
I am completely uninterested.
link |
You get zero points for getting the thing to work.
link |
I don't want you spending your time worrying about
link |
getting the corner case right of what happens
link |
when you are trying to normalize distances
link |
and the points on the thing.
link |
And so you divide by zero.
link |
I'm not interested in that, right?
link |
However, you're going to run those algorithms
link |
The data sets have to be interesting.
link |
What does it mean to be interesting?
link |
Well, data sets interesting if it reveals differences
link |
between algorithms, which presumably are all the same
link |
because they can represent whatever they can represent.
link |
And two data sets are interesting together
link |
if they show different differences, as it were.
link |
And you have to analyze them.
link |
You have to justify their interestingness
link |
and you have to analyze them in a whole bunch of ways.
link |
But all I care about is the data in your analysis,
link |
not the programming.
link |
And I occasionally end up in these long discussions
link |
with students, well, I don't really,
link |
I copy and paste the things that I've said
link |
the other 15,000 times it's come up,
link |
which is they go, but the only way to learn,
link |
really understand is to code them up,
link |
which is a very programmer,
link |
software engineering view of the world.
link |
If you don't program it, you don't understand it,
link |
which is, by the way, I think is wrong
link |
in a very specific way.
link |
But it is a way that you come to understand
link |
because then you have to wrestle with the algorithm.
link |
But the thing about machine learning
link |
is it's not just sorting numbers
link |
where in some sense the data doesn't matter.
link |
What matters is, well, does the algorithm work
link |
on these abstract things, one less than the other?
link |
In machine learning, the data matters.
link |
It matters more than almost anything.
link |
And not everything, but almost anything.
link |
And so as a result, you have to live with the data
link |
and don't get distracted by the algorithm per se.
link |
And I think that that focus on the data
link |
and what it can tell you
link |
and what question it's actually answering for you
link |
as opposed to the question you thought you were asking
link |
is a key and important thing about machine learning
link |
and is a way that computationalists
link |
as opposed to statisticians bring a particular view
link |
about how to think about the process.
link |
The statisticians, by contrast, bring,
link |
I think I'd be willing to say,
link |
a better view about the kind of formal math that's behind it
link |
and what an actual number ultimately is saying
link |
And those are both important, but they're also different.
link |
I didn't really think of it this way
link |
is to build intuition about the role of data,
link |
the different characteristics of data
link |
by having two data sets that are different
link |
and they reveal the differences in the differences.
link |
That's a really fascinating,
link |
that's a really interesting educational approach.
link |
The students love it, but not right away.
link |
No, they love it at the end.
link |
They love it later.
link |
They love it at the end.
link |
Not at the beginning.
link |
Not even immediately after.
link |
I feel like there's a deep profound lesson
link |
about education there.
link |
That you can't listen to students
link |
about whether what you're doing is the right
link |
or the wrong thing.
link |
Yeah, well, as a wise, Michael Lippmann once said to me
link |
about children, which I think applies to teaching,
link |
is you have to give them what they need
link |
without bending to their will.
link |
And students are like that.
link |
You have to figure out what they need.
link |
Your whole job is to curate and to present
link |
because on their own,
link |
they're not gonna necessarily know where to search.
link |
So you're providing pushes in some direction
link |
and learn space and you have to give them what they need
link |
in a way that keeps them engaged enough
link |
so that they eventually discover what they want
link |
and they get the tools they need to go
link |
and learn other things off of.
link |
Let me put on my Russian hat,
link |
which believes that life is suffering.
link |
I like Russian hats, by the way.
link |
If you have one, I would like this.
link |
Those are ridiculous, yes.
link |
But in a delightful way.
link |
What do you think is the role of,
link |
we talked about balance a little bit.
link |
What do you think is the role of hardship in education?
link |
Like I think the biggest things I've learned,
link |
like what made me fall in love with math, for example,
link |
is by being bad at it until I got good at it.
link |
So like struggling with a problem,
link |
which increased the level of joy I felt
link |
when I finally figured it out.
link |
And it always felt with me, with teachers,
link |
especially modern discussions of education,
link |
how can we make education more fun,
link |
more engaging, more all those things?
link |
Or from my perspective, it's like,
link |
you're maybe missing the point
link |
that education, that life is suffering.
link |
Education is supposed to be hard
link |
and that actually what increases the joy you feel
link |
when you actually learn something.
link |
Is that ridiculous?
link |
Do you like to see your students suffer?
link |
Okay, so this may be a point where we differ.
link |
I'm gonna do go on.
link |
Well, what would your answer be?
link |
I wanna hear you first.
link |
Okay, well, I was gonna not answer the question.
link |
You don't want the students to know you enjoy them suffering?
link |
No, no, no, no, no, no.
link |
I was gonna say that there's,
link |
I think there's a distinction that you can make
link |
in the kind of suffering, right?
link |
So I think you can be in a mode
link |
where you're suffering in a hopeless way
link |
versus you're suffering in a hopeful way, right?
link |
Where you're like, you can see that if you,
link |
that you still have,
link |
you can still imagine getting to the end, right?
link |
And as long as people are in that mindset
link |
where they're struggling,
link |
but it's not a hopeless kind of struggling,
link |
that's productive.
link |
I think that's really helpful.
link |
But it's struggling, like if you break their will,
link |
if you leave them hopeless.
link |
No, that don't, sure, some people are gonna,
link |
whatever, lift themselves up by their bootstraps,
link |
but like mostly you give up
link |
and certainly it takes the joy out of it.
link |
And you're not gonna spend a lot of time
link |
on something that brings you no joy.
link |
So it is a bit of a delicate balance, right?
link |
You have to thwart people in a way
link |
that they still believe that there's a way through.
link |
Right, so that's a, we strongly agree actually.
link |
So I think, well, first off,
link |
struggling and suffering aren't the same thing, right?
link |
Yeah, just being poetic.
link |
Oh, no, no, I actually appreciate the poetry.
link |
And one of the reasons I appreciate it
link |
is that they are often the same thing
link |
and often quite different, right?
link |
So you can struggle without suffering,
link |
you can certainly suffer pretty easily.
link |
You don't necessarily have to struggle to suffer.
link |
So I think that you want people to struggle,
link |
but that hope matters.
link |
You have to, they have to understand
link |
that they're gonna get through it on the other side.
link |
And it's very easy to confuse the two.
link |
I actually think Brown University has a very,
link |
just philosophically has a very different take
link |
on the relationship with their students,
link |
particularly undergrads from say a place like Georgia Tech,
link |
Which university is better?
link |
Well, I have my opinions on that.
link |
I mean, remember, Charles said,
link |
it doesn't matter what the facts are, I'm always right.
link |
The correct answer is that it doesn't matter,
link |
they're different.
link |
But clearly, clearly the answer is different.
link |
He went to a school like the school
link |
where he is as an undergrad.
link |
I went to a school, specifically the same school,
link |
though it was changed a bit in the intervening years.
link |
Brown or Georgia Tech?
link |
No, I was talking about Georgia Tech.
link |
And I went to an undergrad place
link |
that's a lot like the place where I work now.
link |
And so it does seem like we're more familiar
link |
with these models.
link |
So there's a similarity between Brown and Yale?
link |
Yeah, I think they're quite similar, yeah.
link |
Duke has some similarities too,
link |
but it's got a little Southern draw.
link |
You've kind of worked your,
link |
you've sort of worked at universities
link |
that are like the places where you learned.
link |
And the same would be true for me.
link |
Are you uncomfortable venturing outside the box?
link |
Is that what you're saying?
link |
That's not what I'm saying.
link |
Yeah, Charles is definitely.
link |
He only goes to places
link |
that have institute in the name, right?
link |
It has worked out that way.
link |
Well, academic places anyway.
link |
Well, no, I was a visiting scientist at UPenn
link |
or visiting something at UPenn.
link |
Oh, wow, I just understood your joke.
link |
Five minutes later.
link |
I like to set the sort of time bomb.
link |
The institute is in the,
link |
that Charles only goes to places
link |
that have institute in the name.
link |
So I guess Georgia,
link |
I forget that Georgia Tech
link |
is Georgia Institute of Technology.
link |
The number of people who refer to it
link |
as Georgia Tech University is large
link |
and incredibly irritating.
link |
It's one of the few things
link |
that genuinely gets under my skin.
link |
But like schools like Georgia Tech and MIT
link |
have as part of the ethos,
link |
I wanna say there's an abbreviation
link |
that someone taught me,
link |
like IHTFP, something like that.
link |
Like there's an expression
link |
which is basically I hate being here,
link |
which they say so proudly.
link |
And that is definitely not the ethos at Brown.
link |
there's a little more pampering
link |
and empowerment and stuff.
link |
And it's not like we're gonna crush you
link |
and you're gonna love it.
link |
So yeah, I think there's a,
link |
I think the ethoses are different.
link |
That's interesting, yeah.
link |
We had Drown Proofing.
link |
In order to graduate from Georgia Tech,
link |
this is a true thing.
link |
Feel free to look it up.
link |
A lot of schools have this by the way.
link |
No, actually Georgia Tech was barely the first.
link |
I feel like Georgia Tech was the first
link |
in a lot of things.
link |
It was the first in a lot of things.
link |
Had the first master's degree.
link |
First Bumblebee mascot.
link |
First master's in computer science actually.
link |
Right, online master's.
link |
Well that too, but way back in the 60s.
link |
You're the first information
link |
and computer science master's degree in the country.
link |
But the Georgia Tech,
link |
it used to be the case
link |
that in order to graduate from Georgia Tech,
link |
you had to take a Drown Proofing class.
link |
Where effectively, they threw you in the water
link |
If you didn't drown, you got to graduate.
link |
There were certainly versions of it,
link |
but I mean luckily they ended it
link |
just before I had to graduate
link |
because otherwise I would have never graduated.
link |
It wasn't going to happen.
link |
I want to say 84, 83,
link |
somewhere around then they ended it.
link |
But yeah, you used to have to prove
link |
you could tread water for some ridiculous amount of time
link |
or you couldn't graduate.
link |
No, it was more than two minutes.
link |
I bet it was two minutes.
link |
Okay, well we'll look at it.
link |
And it was in a bathtub.
link |
You could just stare.
link |
But it was a real thing.
link |
But that idea that, you know, push you.
link |
Yeah, fully clothed.
link |
I bet it was that and not tied up.
link |
Because who needs to learn how to swim when you're tied?
link |
But who needs to learn to swim
link |
when you're actually falling into the water dressed?
link |
That's a real thing.
link |
I think your facts are getting in the way
link |
with a good story.
link |
All right, so they tie you up.
link |
Sometimes the narrative matters.
link |
But whatever it was, you had to,
link |
it was called drown proofing for a reason.
link |
The point of the story, Michael, is that it's,
link |
well, no, but that's good.
link |
It doesn't bring it back to struggle.
link |
That's a part of what Georgia Tech has always been.
link |
And we struggle with that, by the way,
link |
about what we want to be, particularly as things go.
link |
how much can you be pushed without breaking?
link |
And you come out of the other end stronger, right?
link |
There's a saying we used to have
link |
when I was an undergrad there.
link |
It was just Georgia Tech,
link |
building tomorrow the night before.
link |
And it was just kind of idea that,
link |
give me something impossible to do
link |
and I'll do it in a couple of days
link |
because that's what I just spent
link |
the last four or five or six years with.
link |
That ethos definitely stuck to you.
link |
Having now done a number of projects with you,
link |
you definitely will do it the night before.
link |
That's not entirely true.
link |
There's nothing wrong with waiting until the last minute.
link |
The secret is knowing when the last minute is.
link |
Right, that's brilliantly put.
link |
Yeah, that is a definite Charles statement
link |
that I am trying not to embrace.
link |
And I appreciate that
link |
because you helped move my last minute up.
link |
That's the social construct
link |
the way you converge together
link |
what the definition of last minute is.
link |
We figure that all out together.
link |
In fact, MIT, I'm sure a lot of universities have this,
link |
but MIT has like MIT time
link |
that everyone has always agreed together
link |
that there is such a concept
link |
and everyone just keeps showing up like 10 to 15 to 20,
link |
depending on the department, late to everything.
link |
So there's like a weird drift that happens.
link |
It's kind of fascinating.
link |
Yeah, we're five minutes.
link |
We're five minutes.
link |
In fact, the classes will say,
link |
well, this is no longer true actually,
link |
but it used to be a class that started at eight,
link |
but actually it started at eight oh five,
link |
it ends at nine, actually it ends at eight fifty five.
link |
Everything's five minutes off
link |
and nobody expects anything to start until five minutes
link |
after the half hour, whatever it is.
link |
Well, let's rewind the clock back to the fifties and sixties
link |
when you guys met, how did you,
link |
I'm just kidding, I don't know.
link |
But what, can you tell the story of how you met?
link |
So you've, like the internet and the world
link |
kind of knows you as connected in some ways
link |
in terms of education of teaching the world.
link |
That's like the public facing thing,
link |
but how did you as human beings
link |
and as collaborators meet?
link |
I think there's two stories.
link |
One is how we met and the other is how we
link |
got to know each other.
link |
I'm not gonna say fell in love.
link |
I'm gonna say that we came to understand that we
link |
Had some common something.
link |
Cause on the surface, I think we're different
link |
in a lot of ways, but there's something
link |
Yeah, I mean, now we complete each other's
link |
So I will tell the story of how we met
link |
and I'll let Michael tell the story of how we met.
link |
Okay, so here's how we met.
link |
I was already at that point, it was AT&T labs.
link |
There's a long, interesting story there.
link |
But anyway, I was there and Michael was coming to interview.
link |
He was a professor at Duke at the time,
link |
but decided for reasons that he wanted to be in New Jersey.
link |
And so that would mean Bell Labs slash AT&T labs.
link |
And we were doing the interview.
link |
Interviews are very much like academic interviews.
link |
And so I had to be there.
link |
We all had to meet with him afterwards
link |
and so on, one on one.
link |
But it was obvious to me that he was going to be hired.
link |
Like no matter what, because everyone loved him.
link |
They were just talking about all the great stuff he did.
link |
Oh, he did this great thing.
link |
And you had just won something at AAAI, I think.
link |
Or maybe you got 18 papers in AAAI that year.
link |
I got the best paper award at AAAI
link |
for the crossword stuff.
link |
So that had all happened and everyone was going on
link |
and on and on about it.
link |
Actually, so Tinder was saying incredibly nice things
link |
He can be very grumpy.
link |
That's nice to hear.
link |
He was grumpily saying very nice things.
link |
Oh, that makes sense.
link |
And that does make sense.
link |
So, you know, it was going to come.
link |
So why was I meeting him?
link |
I had something else I had to do.
link |
I can't remember what it was.
link |
It probably involved comedy.
link |
So he remembers meeting me
link |
as inconveniencing his afternoon.
link |
So I eventually came to my office.
link |
I was in the middle of trying to do something.
link |
I can't remember what.
link |
And he came and he sat down.
link |
And for reasons that are purely accidental,
link |
despite what Michael thinks,
link |
my desk at the time was set up
link |
in such a way that it had sort of an L shape.
link |
And the chair on the outside was always lower
link |
than the chair that I was in.
link |
And, you know, the kind of point was to...
link |
The only reason I think that it was on purpose
link |
is because you told me it was on purpose.
link |
I don't remember that.
link |
Anyway, the thing is, is that, you know, it kind of gives...
link |
His guest chair was really low
link |
so that he could look down at everybody.
link |
The idea was just to simply create a nice environment
link |
that you were asking for a mortgage
link |
and I was going to say no.
link |
That was the point.
link |
It was a very simple idea here.
link |
Anyway, so we sat there
link |
and we just talked for a little while.
link |
And I think he got the impression that I didn't like him.
link |
Which wasn't true.
link |
I strongly got that impression.
link |
The talk was really good.
link |
The talk, by the way, was terrible.
link |
And right after the talk,
link |
I said to my host, Michael Kearns,
link |
who ultimately was my boss.
link |
I'm a friend and a huge fan of Michael, yeah.
link |
Yeah, he is a remarkable person.
link |
After my talk, I went into the...
link |
He went into basketball.
link |
Racquetball, he's good at everything.
link |
No, but basketball and racquetball too.
link |
Squash, squash, squash, not racquetball.
link |
Yes, squash, which is not...
link |
And I hope you hear that, Michael.
link |
Oh, Michael Kearns.
link |
As a game, not his skill level,
link |
because I'm pretty sure he's...
link |
All right, there's some competitiveness there,
link |
but the point is that it was like the middle of the day,
link |
I had full day of interviews.
link |
I got met with people,
link |
but then in the middle of the day, I gave a job talk.
link |
And then there was gonna be more interviews,
link |
but I pulled Michael aside and I said,
link |
I think it's in both of our best interest
link |
if I just leave now, because that was so bad
link |
that it'd just be embarrassing
link |
if I have to talk to any more people.
link |
You look bad for having invited me.
link |
It's just, let's just forget this ever happened.
link |
So I don't think the talk went well.
link |
That's one of the most Michael Lipman set of sentences
link |
I think I've ever heard.
link |
He did great, or at least everyone knew he was great,
link |
so maybe it didn't matter.
link |
I was there, I remember the talk,
link |
and I remember him being very much the way
link |
I remember him now, on any given week.
link |
And we met and we talked about stuff.
link |
He thinks I didn't like him, but...
link |
Because he was so grumpy.
link |
Must've been the chair thing.
link |
The chair thing and the low voice, I think.
link |
But like, he obviously...
link |
And that slight skeptical look.
link |
I have no idea what you're talking about.
link |
Well, I probably didn't have any idea
link |
what you were talking about.
link |
Anyway, I liked him.
link |
He asked me questions, I answered questions.
link |
I felt bad about myself.
link |
It was a normal day.
link |
It was a normal day.
link |
And then he left, and that's how you met.
link |
And then I got hired and I was in the group.
link |
Can we take a slight tangent on this topic of,
link |
it sounds like, maybe you could speak
link |
to the bigger picture.
link |
It sounds like you're quite self critical.
link |
I'll be so self critical.
link |
Yeah, that was like a three out of 10 response.
link |
So let's try to work it up to five and six.
link |
Yeah, I remember Marvin Minsky said on a video interview,
link |
something that the key to success in academic research
link |
is to hate everything you do.
link |
For some reason...
link |
I think I followed that because I hate everything he's done.
link |
That's a good line.
link |
That's a six out of 10.
link |
Maybe that's a keeper.
link |
But do you find that resonates with you at all
link |
in how you think about talks and so on?
link |
I would say it differently.
link |
That's such an MIT view of the world though.
link |
So I remember talking about this when, as a student,
link |
you were basically told I will clean it up
link |
for the purpose of the podcast.
link |
Then you go to a conference or something.
link |
You're like, everybody else's work is crap.
link |
Everybody else's work is crap.
link |
And you feel better and better about it, relatively speaking.
link |
And then you sort of keep working on it.
link |
I don't hate my work.
link |
That resonates with me.
link |
Yes, I've never hated my work,
link |
but I have been dissatisfied with it.
link |
And I think being dissatisfied,
link |
being okay with the fact that you've taken a positive step,
link |
the derivative's positive,
link |
maybe even the second derivative's positive,
link |
that's important because that's a part of the hope, right?
link |
But you have to, but I haven't gotten there yet.
link |
If that's not there, that I haven't gotten there yet,
link |
then it's hard to move forward, I think.
link |
So I buy that, which is a little different
link |
from hating everything that you do.
link |
Yeah, I mean, there's things that I've done
link |
that I like better than I like myself.
link |
So it's separating me from the work, essentially.
link |
So I think I am very critical of myself,
link |
but sometimes the work I'm really excited about.
link |
And sometimes I think it's kind of good.
link |
Does that happen right away?
link |
So I found the work that I've liked, that I've done,
link |
most of it, I liked it in retrospect
link |
more when I was far away from it in time.
link |
I have to be fairly excited about it to get done.
link |
No, excited at the time, but then happy with the result.
link |
But years later, or even I might go back,
link |
you know what, that actually turned out to matter.
link |
That turned out to matter.
link |
Or, oh gosh, it turns out I've been thinking about that.
link |
It's actually influenced all the work that I've done since
link |
without realizing it.
link |
Boy, that guy was smart.
link |
Yeah, that guy had a future.
link |
Yeah, I think there's something to it.
link |
I think there's something to the idea
link |
you've got to hate what you do, but it's not quite hate.
link |
It's just being unsatisfied.
link |
And different people motivate themselves differently.
link |
I don't happen to motivate myself with self loathing.
link |
I happen to motivate myself with something else.
link |
So you're able to sit back and be proud of,
link |
in retrospect, of the work you've done.
link |
Well, and it's easier when you can connect it
link |
with other people, because then you can be proud of them.
link |
Proud of the people, yeah.
link |
And then the question is.
link |
You can still safely hate yourself.
link |
Yeah, that's right.
link |
It's win, win, Michael.
link |
Or at least win, lose, which is what you're looking for.
link |
Oh, wow, there's so many brilliant minds in this.
link |
So how did you actually meet me?
link |
So the way I think about it is,
link |
because we didn't do much research together at AT&T,
link |
but then we all got laid off.
link |
By the way, sorry to interrupt,
link |
but that was one of the most magical places
link |
historically speaking.
link |
They did not appreciate what they had.
link |
I feel like there's a profound lesson in there too.
link |
How do we get it, like what was, why was it so magical?
link |
Is it just a coincidence of history?
link |
Or is there something special about?
link |
There were some really good managers
link |
and people who really believed in machine learning
link |
as this is gonna be important.
link |
Let's get the people who are thinking about this
link |
in creative and insightful ways
link |
and put them in one place and stir.
link |
Yeah, but even beyond that, right?
link |
It was Bell Labs at its heyday.
link |
And even when we were there, which I think was past its heyday.
link |
And to be clear, he's gotten to be at Bell Labs.
link |
I never got to be at Bell Labs.
link |
I joined after that.
link |
Yeah, I showed up in 91 as a grad student.
link |
So I was there for a long time, every summer, except for two.
link |
So twice I worked for companies
link |
that had just stopped being Bell Labs.
link |
Bellcore and then AT&T Labs.
link |
So Bell Labs was several locations or for the research
link |
I don't know if Jersey's involved somehow.
link |
They're all in Jersey.
link |
Yeah, they're all over the place.
link |
But they were in a couple of places in Jersey.
link |
Murray Hill was the Bell Labs place.
link |
So you had an office in Murray Hill
link |
at one point in your career.
link |
Yeah, and I played Ultimate Frisbee
link |
on the cricket pitch at Bell Labs at Murray Hill.
link |
And then it became AT&T Labs when it split off
link |
with loose during what we called Trivestiture.
link |
Are you better than Michael Korn's at Ultimate Frisbee?
link |
But I think that one's not boasting.
link |
I think Charles plays a lot of Ultimate
link |
and I don't think Michael does.
link |
Yes, but that wasn't the point.
link |
I'm finally better.
link |
Okay, I have played on a championship winning
link |
Ultimate Frisbee team or whatever,
link |
Ultimate team with Charles.
link |
So I know how good he is.
link |
How good I was anyway, when I was younger.
link |
I know how young he was when he was younger.
link |
So much younger than now.
link |
Michael was a much better basketball player than I was.
link |
Yes, no, not Michael.
link |
Let's be very clear about that.
link |
To be clear, I've not played basketball with you.
link |
So you don't know how terrible I am,
link |
but you have a probably pretty good guess.
link |
And that you're not as good as Michael Kearns.
link |
He's tall and athletic.
link |
And he cared about it.
link |
He's very athletic.
link |
And probably competitive.
link |
I love hanging out with Michael.
link |
Anyway, but we were talking about something else,
link |
although I no longer remember what it was.
link |
What were we talking about?
link |
So this was kind of cool about what was magical about it.
link |
The first thing you have to know
link |
is that Bell Labs was an arm of the government, right?
link |
Because AT&T was an arm of the government.
link |
It was a monopoly.
link |
And every month you paid a little thing on your phone bill,
link |
which turned out was a tax
link |
for all the research that Bell Labs was doing.
link |
And they invented transistors and the laser
link |
and whatever else is that they did.
link |
The Big Bang or whatever, the cosmic background radiation.
link |
Yeah, they did all that stuff.
link |
They had some amazing stuff with directional microphones,
link |
I got to go in this room
link |
where they had all these panels and everything.
link |
And we would talk and one another,
link |
and he'd move some panels around.
link |
And then he would have me step two steps to the left.
link |
And I couldn't hear a thing he was saying
link |
because nothing was bouncing off the walls.
link |
And then he would shut it all down
link |
and you could hear your heartbeat,
link |
which is deeply disturbing to hear your heartbeat.
link |
I mean, you can feel it now.
link |
There's just so much all this sort of noise around.
link |
Anyway, Bell Labs was about pure research.
link |
It was a university, in some sense,
link |
the purest sense of a university, but without students.
link |
So it was all the faculty working with one another
link |
and students would come in to learn.
link |
They would come in for three or four months
link |
during the summer and they would go away.
link |
But it was just this kind of wonderful experience.
link |
I could walk out my door.
link |
In fact, I would often have to walk out my door
link |
and deal with Rich Sutton and Michael Kearns
link |
yelling at each other about whatever it is
link |
they were yelling about the proper way
link |
to prove something or another.
link |
And I could just do that.
link |
And Dave McAllister and Peter Stone
link |
and all of these other people,
link |
including, it's a tender and then eventually Michael.
link |
And it was just a place where you could think thoughts.
link |
And it was okay because so long as once every 25 years or so
link |
somebody invented a transistor, it paid for everything else.
link |
You could afford to take the risk.
link |
And then when that all went away,
link |
it became harder and harder and harder to justify it
link |
as far as the folks who were very far away were concerned.
link |
And there was such a fast turnaround
link |
among mental management on the AT&T side
link |
that you never had a chance to really build a relationship.
link |
At least people like us didn't have a chance
link |
to build a relationship.
link |
So when the diaspora happened, it was amazing, right?
link |
Everybody left and I think everybody ended up
link |
at a great place and made a huge,
link |
continued to do really good work with machine learning.
link |
But it was a wonderful place.
link |
And people will ask me, what's the best job you've ever had?
link |
And as a professor, anyway, the answer that I would give is
link |
well, probably Bell Labs in some very real sense.
link |
And I will never have a job like that again
link |
because Bell Labs doesn't exist anymore.
link |
And Microsoft research is great and Google does good stuff.
link |
And you can pick IBM, you can tell if you want to,
link |
but Bell Labs was magical.
link |
It was around for, it was an important time
link |
and it represents a high watermark
link |
in basic research in the US.
link |
Is there something you could say about the physical proximity
link |
and the chance collisions?
link |
Like we live in this time of the pandemic
link |
where everyone is maybe trying to see the silver lining
link |
and accepting the remote nature of things.
link |
Is there one of the things that people like faculty
link |
that I talk to miss is the procrastination.
link |
Like the chance to make everything is about meetings
link |
that are supposed to be,
link |
there's not a chance to just talk about comic book
link |
or whatever, like go into discussion that's totally pointless.
link |
So it's funny you say this
link |
because that's how we met, met, it was exactly that.
link |
So I'll let Michael say that, but I'll just add one thing
link |
which is just that research is a social process
link |
and it helps to have random social interactions
link |
even if they don't feel social at the time,
link |
that's how you get things done.
link |
One of the great things about the AI Lab when I was there,
link |
I don't quite know what it looks like now
link |
once they moved buildings,
link |
but we had entire walls that were whiteboards
link |
and people would just get up there
link |
and they were just right and people would walk up
link |
and you'd have arguments
link |
and you'd explain things to one another
link |
and you got so much out of the freedom to do that.
link |
You had to be okay with people challenging
link |
every fricking word you said,
link |
which I would sometimes find deeply irritating,
link |
but most of the time it was quite useful.
link |
But the sort of pointlessness and the interaction
link |
was in some sense the point, at least for me.
link |
Yeah, I think offline yesterday I mentioned
link |
Josh Tenenbaum and he's very much, he's a man,
link |
he's such an inspiration in the childlike way
link |
that he pulls you in on any topic.
link |
It doesn't even have to be about machine learning
link |
or the brain, he'll just pull you in
link |
to a closest writable surface,
link |
which is still, you can find whiteboards
link |
at MIT everywhere, and just like basically cancel
link |
all meetings and talk for a couple hours
link |
about some aimless thing and it feels like
link |
the whole world, the time space continuum kind of warps
link |
and that becomes the most important thing.
link |
And then it's just, it's definitely something
link |
worth missing in this world where everything's remote.
link |
There's some magic to the physical presence.
link |
Whenever I wonder myself whether MIT really is
link |
as great as I remember it, I just go talk to Josh.
link |
Yeah, you know, that's funny.
link |
There's a few people in this world that carry
link |
the best of what particular institutions stand for, right?
link |
I mean, I don't, my guess is he's unaware of this.
link |
That the masters are not aware of their mastery.
link |
Yes, but first a tangent, no.
link |
How did you meet me?
link |
So I'm not sure what you were thinking,
link |
but when it started to dawn on me
link |
that maybe we had a longer term bond
link |
was after we all got laid off.
link |
And you had decided at that point
link |
that we were still paid.
link |
We were given an opportunity to like do a job search
link |
and kind of make a transition,
link |
but it was clear that we were done.
link |
And I would go to my office to work
link |
and you would go to my office to keep me from working.
link |
That was my recollection of it.
link |
You had decided that there was no,
link |
really no point in working for the company
link |
because our relationship with the company was done.
link |
Yeah, but remember I felt that way beforehand.
link |
It wasn't about the company.
link |
It was about the set of people there
link |
doing really cool things.
link |
And it always, always been that way.
link |
But we were working on something together.
link |
Oh yeah, yeah, yeah.
link |
So at the very end, we all got laid off,
link |
but then our boss came to, our boss's boss came to us
link |
because our boss was Michael Kearns
link |
and he had jumped ship brilliantly, like perfect timing.
link |
Like things like right before the ship was about to sink,
link |
he was like, gotta go and landed perfectly
link |
because Michael Kearns.
link |
Because Michael Kearns.
link |
And leaving the rest of us to go like, this is fine.
link |
And then it was clear that it wasn't fine
link |
and we were all toast.
link |
So we had this sort of long period of time.
link |
But then our boss figured out, okay, wait,
link |
maybe we can save a couple of these people
link |
if we can have them do something really useful.
link |
And the useful thing was we were gonna make
link |
basically an automated assistant
link |
that could help you with your calendar.
link |
You could like tell it things
link |
and it would respond appropriately.
link |
It would just kind of integrate across
link |
all sorts of your personal information.
link |
And so me and Charles and Peter Stone
link |
were set up as the crack team
link |
to actually solve this problem.
link |
Other people maybe were too theoretical that they thought,
link |
but we could actually get something done.
link |
So we sat down to get something done
link |
and there wasn't time and it wouldn't have saved us anyway.
link |
And so it all kind of went downhill.
link |
But the interesting, I think, coda to that
link |
is that our boss's boss is a guy named Ron Brockman.
link |
And when he left AT&T,
link |
cause we were all laid off,
link |
he went to DARPA, started up a program there
link |
which is the program from which Siri sprung,
link |
which is a digital assistant
link |
that helps you with your calendar
link |
and a bunch of other things.
link |
It really, in some ways got its start
link |
with me and Charles and Peter trying to implement this vision
link |
that Ron Brockman had,
link |
that he ultimately got implemented
link |
through his role at DARPA.
link |
So when I'm trying to feel less bad
link |
about having been laid off
link |
from what is possibly the greatest job of all time,
link |
I think about, well, we kind of helped birth Siri.
link |
And then he did other things too.
link |
But we got to spend a lot of time in his office
link |
and talk about lots of things.
link |
We got to spend a lot of time in my office, yeah.
link |
And so then we went on our merry way.
link |
Everyone went to different places.
link |
Charles landed at Georgia Tech,
link |
which was what he always dreamed he would do.
link |
And so that worked out well.
link |
I came up with a saying at the time,
link |
which is luck favors the Charles.
link |
It's kind of like luck favors the prepared,
link |
but Charles, like he wished something
link |
and then it would basically happen just the way he wanted.
link |
It was inspirational to see things go that way.
link |
Things worked out.
link |
And we stayed in touch.
link |
And then I think it really helped
link |
when you were working on,
link |
I mean, you'd kept me in the loop for things like threads
link |
and the work that you were doing at Georgia Tech.
link |
But then when they were starting
link |
their online master's program,
link |
he knew that I was really excited about MOOCs
link |
and online teaching.
link |
And he's like, I have a plan.
link |
And I'm like, tell me your plan.
link |
He's like, I can't tell you the plan yet.
link |
Cause they were deep in negotiations
link |
between Georgia Tech and Udacity to make this happen.
link |
And they didn't want it to leak.
link |
So Charles would kept teasing me about it,
link |
but wouldn't tell me what was actually going on.
link |
And eventually it was announced and he said,
link |
I would like you to teach the machine learning course
link |
I'm like, that can't possibly work.
link |
But it was a great idea.
link |
And it was super fun.
link |
It was a lot of work to put together,
link |
but it was really great.
link |
Was that the first time you thought about,
link |
first of all, was it the first time
link |
you got seriously into teaching?
link |
I mean, I was a professor.
link |
This was already after you jumped to,
link |
so like there's a little bit of jumping around in time.
link |
Yeah, sorry about that.
link |
There's a pretty big jump in time.
link |
So like the MOOCs thing.
link |
So Charles got to Georgia Tech and he,
link |
I mean, maybe Charles, maybe this is a Charles story.
link |
I got to Georgia Tech in 2002.
link |
He got to Georgia Tech in 2002.
link |
And worked on things like revamping the curriculum,
link |
the undergraduate curriculum,
link |
so that it had some kind of semblance of modular structure
link |
because computer science was at the time
link |
moving from a fairly narrow specific set of topics
link |
to touching a lot of other parts of intellectual life.
link |
And the curriculum was supposed to reflect that.
link |
And so Charles played a big role in kind of redesigning that.
link |
And for my labors, I ended up as associate dean.
link |
Right, he got to become associate dean
link |
of charge of educational stuff.
link |
Yeah, I was under.
link |
This should be a valuable lesson.
link |
If you're good at something,
link |
they will give you responsibility to do more of that thing.
link |
Don't show competence.
link |
Don't show competence if you.
link |
Don't want responsibility.
link |
Here's what they say.
link |
The reward for good work is more work.
link |
The reward for bad work is less work.
link |
Which, I don't know.
link |
Depending on what you're trying to do that week,
link |
one of those is better than the other.
link |
Well, one of the problems with the word work,
link |
sorry to interrupt, is that it seems to be an antonym
link |
in this particular language.
link |
We have the opposite of happiness.
link |
But it seems like they're.
link |
That's one of, you know, we talked about balance.
link |
It's always like work life balance.
link |
It always rubbed me the wrong way as a terminology.
link |
I know it's just words.
link |
Right, the opposite of work is play.
link |
But ideally, work is play.
link |
Oh, I can't tell you how much time I'd spend.
link |
Certainly, when I was at Bell Labs,
link |
except for a few very key moments,
link |
as a professor, I would do this too.
link |
I would just say, I cannot believe
link |
they're paying me to do this.
link |
It's something that I would do for a hobby
link |
if I could anyway.
link |
So that sort of worked out.
link |
Are you sure you want to be saying that
link |
when this is being recorded?
link |
As a dean, that is not true at all.
link |
But I think here with this,
link |
even though a lot of time passed,
link |
Mike and I talked almost every, well, we texted,
link |
almost every day during the period.
link |
Charles, at one point, took me,
link |
the ICML conference, the machine learning conference
link |
I was the chair, the general chair of the conference.
link |
Charles was my publicity chair or something like that,
link |
or fundraising chair.
link |
Yeah, but he decided it'd be really funny
link |
if he didn't actually show up for the conference
link |
in his own home city.
link |
So he didn't, but he did at one point
link |
pick me up at the conference in his Tesla
link |
and drove me to the Atlanta mall
link |
and forced me to buy an iPhone
link |
because he didn't like how it was to text with me
link |
and thought it would be better for him
link |
if I had an iPhone, the text would be somehow smoother.
link |
And it is, and his life is better.
link |
And my life is better.
link |
And so, yeah, but it was, yeah,
link |
Charles forced me to get an iPhone
link |
so that he could text me more efficiently.
link |
I thought that was an interesting moment.
link |
Anyway, so we kept talking the whole time
link |
and then eventually we did the teaching thing
link |
And there's a couple of reasons for that, by the way.
link |
One is I really wanted to do something different.
link |
Like you've got this medium here,
link |
people claim it can change things.
link |
What's a thing that you could do in this medium
link |
that you could not do otherwise besides edit, right?
link |
I mean, what could you do?
link |
And being able to do something with another person
link |
was that kind of thing.
link |
I mean, you can take turns,
link |
but teaching together, having conversations is very hard.
link |
So that was a cool thing.
link |
The second thing, give me an excuse
link |
to do more stuff with him.
link |
Yeah, I always thought, he makes it sound brilliant.
link |
And it is, I guess.
link |
But at the time it really felt like
link |
I've got a lot to do, Charles is saying,
link |
and it would be great if Michael could teach the course
link |
and I could just hang out.
link |
Yeah, just kind of coast on that.
link |
Well, that's what the second class was more like that.
link |
Because the second class was explicitly like that.
link |
But the first class, it was at least half.
link |
Yeah, but I do all the stuff.
link |
So the structure that we came up with.
link |
I think you're once again letting the facts
link |
get in the way of a good story.
link |
I should just let Charles talk to us.
link |
But that's the facts that he saw.
link |
So that was kind of true for 7642.
link |
Yeah, that was sort of true for 7642,
link |
which is the reinforcement learning class,
link |
because that was really his class.
link |
You started with reinforcement learning or machine learning?
link |
Intro machine learning, 7641,
link |
which is supervised learning, unsupervised learning,
link |
and reinforcement learning and decision making,
link |
cram all that in there,
link |
the kind of assignments that we talked about earlier.
link |
And then eventually, about a year later,
link |
we did a follow on 7642,
link |
which is reinforcement learning and decision making.
link |
The first class was based on something
link |
I'd been teaching at that point for well over a decade.
link |
And the second class was based on something
link |
Michael had been teaching.
link |
Actually, I learned quite a bit
link |
teaching that class with him, but he drove most of that.
link |
But the first one I drove most, it was all my material.
link |
Although I had stolen that material originally
link |
from slides I found online from Michael,
link |
who had originally stolen that material
link |
from, I guess, slides he found online,
link |
probably from Andrew Moore,
link |
because the jokes were the same anyway.
link |
At least some of the, at least when I found the slides,
link |
some of the stuff with it.
link |
Yes, every machine learning class taught in the early 2000s
link |
stole from Andrew Moore.
link |
A particular joke or two?
link |
At least the structure.
link |
Now, I did, and he did, actually,
link |
a lot more with reinforcement learning and such,
link |
and game theory and those kinds of things.
link |
But, you know, we all sort of built in.
link |
You mean in the research world?
link |
No, no, no, in that class.
link |
No, I mean in teaching that class.
link |
The coverage was different than what we started.
link |
Most people were just doing supervised learning
link |
and maybe a little bit of clustering and whatnot,
link |
but we took it all the way to machine learning.
link |
A lot of it just comes from Tom Mitchell's book.
link |
Oh, no, yeah, except, well,
link |
half of it comes from Tom Mitchell's book, right?
link |
I mean, the other half doesn't.
link |
This is why it's all readings, right?
link |
Because certain things weren't invented
link |
when Tom wrote that stuff.
link |
Yeah, okay, that's true.
link |
All right, but it was quite good.
link |
But there's a reason for that besides, you know,
link |
just, I wanted to do it.
link |
I wanted to do something new,
link |
and I wanted to do something with him,
link |
which is a realization,
link |
which is despite what you might believe,
link |
he's an introvert and I'm an introvert,
link |
or I'm on the edge of being an introvert anyway.
link |
But both of us, I think, enjoy the energy of the crowd,
link |
There's something about talking to people
link |
and bringing them into whatever we find interesting
link |
that is empowering, energizing, or whatever.
link |
And I found the idea of staring alone at a computer screen
link |
and then talking off of materials
link |
less inspiring than I wanted it to be.
link |
And I had in fact done a MOOC for Udacity on algorithms.
link |
And it was a week in a dark room talking at the screen,
link |
writing on the little pad.
link |
And I didn't know this was happening,
link |
but they had watched,
link |
the crew had watched some of the videos
link |
while, you know, like in the middle of this,
link |
and they're like, something's wrong.
link |
You're sort of shutting down.
link |
And I think a lot of it was I'll make jokes
link |
and no one would laugh.
link |
And I felt like the crowd hated me.
link |
Now, of course, there was no crowd.
link |
So like, it wasn't rational.
link |
But each time I tried it and I got no reaction,
link |
it just was taking the energy out of my performance,
link |
out of my presentation.
link |
Such a fantastic metaphor for grad school.
link |
Anyway, by working together,
link |
we could play off each other and have a good time.
link |
And keep the energy up,
link |
because you can't let your guard down for a moment
link |
with Charles, he'll just overpower you.
link |
I have no idea what you're talking about.
link |
But we would work really well together, I thought,
link |
and we knew each other,
link |
so I knew that we could sort of make it work.
link |
Plus, I was the associate dean,
link |
so they had to do what I told them to do.
link |
We had to make it work.
link |
And so it worked out very well, I thought,
link |
well enough that we.
link |
With great power comes great power.
link |
And we became smooth and curly.
link |
And that's when we did the overfitting thriller video.
link |
Yeah, that's a thing.
link |
So can we just, like, smooth and curly,
link |
where did that come from?
link |
Okay, so it happened.
link |
It was completely spontaneous.
link |
These are nicknames you go by.
link |
Yeah, so it's what the students call us.
link |
So the way that we structured the lectures
link |
is one of us is the lecturer
link |
and one of us is basically the student.
link |
And so he was lecturing on.
link |
The lecturer prepares all the materials,
link |
comes up with the quizzes,
link |
and then the student comes in not knowing anything.
link |
So it was just like being on campus.
link |
And I was doing game theory in particular,
link |
the Prisoner's Dilemma.
link |
Prisoner's Dilemma.
link |
And so he needed to set up a little Prisoner's Dilemma grid.
link |
So he drew it and I could see what he was drawing.
link |
And the Prisoner's Dilemma consists of two players,
link |
So he decided he would make little cartoons
link |
And so there was two criminals, right,
link |
that were deciding whether or not to rat each other out.
link |
One of them he drew as a circle with a smiley face
link |
and a kind of goatee thing, smooth head.
link |
And the other one with all sorts of curly hair.
link |
And he said, this is smooth and curly.
link |
I said, smooth and curly?
link |
He said, no, no, smooth with a V.
link |
It's very important that it have a V.
link |
And then the students really took to that.
link |
Like they found that relatable.
link |
He started singing Smooth Criminal by Michael Jackson.
link |
And those names stuck.
link |
So we now have a video series,
link |
an episode, our kind of first actual episode
link |
should be coming out today,
link |
Smooth and Curly on video,
link |
where the two of us discuss episodes of Westworld.
link |
We watch Westworld and we're like, huh,
link |
what does this say about computer science and AI?
link |
And we've never, we did not watch it.
link |
I mean, no, it's on season three or whatever we have.
link |
As of this recording, it's on season three.
link |
We've watched now two episodes total.
link |
Yeah, I think I watched three.
link |
What do you think about Westworld?
link |
So I can tell you so far,
link |
I'm just guessing what's gonna happen next.
link |
It seems like bad things are gonna happen
link |
with the robots uprising.
link |
So I have not, I have not,
link |
I mean, you know, I vaguely remember a movie existing.
link |
So I assume it's related to that, but.
link |
That was more my time than your time, Charles.
link |
That's right, cause you're much older than I am.
link |
I think the important thing here is that
link |
it's narrative, right?
link |
It's all about telling a story.
link |
That's the whole driving thing.
link |
But the idea that they would give these reveries,
link |
that they would make people,
link |
they would make them.
link |
Let them remember.
link |
Remember the awful things that happened.
link |
The terrible things that happened.
link |
Who could possibly think that was gonna,
link |
I gotta, I mean, I don't know.
link |
I've only seen the first two episodes
link |
or maybe the third one.
link |
I think I've only seen the first one.
link |
You know what it was?
link |
You know what the problem is?
link |
That the robots were actually designed by Hannibal Lecter.
link |
So like, what do you think is gonna happen?
link |
It's clear that things are happening
link |
and characters are being introduced
link |
and we don't yet know anything,
link |
but still I was just struck by how
link |
it's all driven by narrative and story.
link |
And there's all these implied things like programming,
link |
the programming interface is talking to them
link |
about what's going on in their heads,
link |
which is both, I mean, artistically,
link |
it's probably useful to film it that way.
link |
But think about how it would work in real life.
link |
That just seems very great.
link |
But there was, we saw in the second episode,
link |
You could see things.
link |
They were wearing like Kubrick's glasses.
link |
It was quite interesting to just kind of ask this question
link |
I mean, I assume it veers off into Never Never Land
link |
We can't answer that question.
link |
I'm also a fan of a guy named Alex Garland.
link |
He's a director of Ex Machina.
link |
And he is the first,
link |
I wonder if Kubrick was like this actually,
link |
is he like studies,
link |
what would it take to program an AI systems?
link |
Like he's curious enough to go into that direction.
link |
On the Westworld side,
link |
I felt there was more emphasis on the narratives
link |
than like actually asking like computer science questions.
link |
Like, how would you build this?
link |
How would you, and.
link |
How would you debug it?
link |
I still think, to me, that's the key issue.
link |
They were terrible debuggers.
link |
Well, they said specifically,
link |
so we make a change and we put it out in the world
link |
and that's bad because something terrible could happen.
link |
Like if you're putting things out in the world
link |
and you're not sure whether something terrible
link |
is going to happen, your process is probably.
link |
I just feel like there should have been someone
link |
whose sole job it was to walk around and poke his head in
link |
and say, what could possibly go wrong?
link |
Just over and over again.
link |
I would have loved if there was an,
link |
and I did watch a lot more and I'm not giving anything away.
link |
I would have loved it if there was like an episode
link |
where like the new intern is like debugging
link |
a new model or something and like it just keeps failing
link |
and they're like, all right.
link |
And then it's more turns into like a episode
link |
of Silicon Valley or something like that.
link |
Versus like this ominous AI systems
link |
that are constantly like threatening the fabric
link |
of this world that's been created.
link |
Yeah, and you know the other,
link |
this reminds me of something that,
link |
so I agree that that should be very cool,
link |
at least for the small percentage of people
link |
who care about debugging systems.
link |
But the other thing is.
link |
Right, debugging, the series.
link |
Yeah, it falls into, think of the sequels,
link |
fear of the debugger.
link |
It's a nightmare show, it's a horror movie.
link |
I think that's where we lose people, by the way,
link |
early on is the people who either decide,
link |
either figure out debugging or think debugging is terrible.
link |
This is where we lose people in computer science.
link |
This is a part of the struggle versus suffering, right?
link |
You get through it and you kind of get the skills of it,
link |
or you're just like, this is dumb,
link |
and this is a dumb way to do anything.
link |
And I think that's when we lose people.
link |
But, well, I'll leave it at that.
link |
But I think that there's something really, really neat
link |
about framing it that way.
link |
But what I don't like about all of these things,
link |
and I love Tex Machina, by the way,
link |
although the ending was very depressing.
link |
One of the things I have to talk to Alex about,
link |
he says that the thing that nobody noticed he put in
link |
is at the end, spoiler alert,
link |
the robot turns and looks at the camera and smiles, briefly.
link |
And to him, he thought that his definition
link |
of passing the general version of the Turing test,
link |
or the consciousness test, is smiling for no one.
link |
It's like the Chinese room kind of experiment.
link |
It's not always trying to act for others,
link |
but just on your own, being able to have a relationship
link |
with the actual experience and just take it in.
link |
I don't know, he said nobody noticed the magic of it.
link |
I have this vague feeling that I remember the smile,
link |
but now you've just put the memory in my head,
link |
But I do think that that's interesting.
link |
Although, by looking at the camera,
link |
you are smiling for the audience, right?
link |
You're breaking the fourth wall.
link |
It seems, I mean, well, that's a limitation of the medium.
link |
But I like that idea.
link |
But here's the problem I have with all of those movies,
link |
all of them, is that, but I know why it's this way,
link |
and I enjoy those movies, and Westworld,
link |
is it sets up the problem of AI as succeeding
link |
and then having something we cannot control.
link |
But it's not the bad part of AI.
link |
The bad part of AI is the stuff
link |
we're living through now, right?
link |
It's using the data to make decisions that are terrible.
link |
It's not the intelligence that's gonna go out there
link |
and surpass us and take over the world
link |
or lock us into a room to starve to death slowly
link |
over multiple days.
link |
It's instead the tools that we're building
link |
that are allowing us to make the terrible decisions
link |
we would have less efficiently made before, right?
link |
Computers are very good at making us more efficient,
link |
including being more efficient at doing terrible things.
link |
And that's the part of the AI we have to worry about.
link |
It's not the true intelligence that we're gonna build
link |
sometime in the future, probably long after we're around.
link |
But I think that whole framing of it
link |
sort of misses the point, even though it is inspiring.
link |
And I was inspired by those ideas, right?
link |
I got into this in part
link |
because I wanted to build something like that.
link |
Philosophical questions are interesting to me,
link |
but that's not where the terror comes from.
link |
The terror comes from the everyday.
link |
And you can construct situations
link |
in the subtlety of the interaction between AI and the human,
link |
like with social networks,
link |
all the stuff you're doing
link |
with interactive artificial intelligence.
link |
But I feel like Cal 9000 came a little bit closer to that
link |
in 2001 Space Odyssey,
link |
because it felt like a personal assistant.
link |
It felt like closer to the AI systems we have today.
link |
And the real things we might actually encounter,
link |
which is over relying in some fundamental way
link |
on our dumb assistants or on social networks,
link |
like over offloading too much of us
link |
onto things that require internet and power and so on
link |
and thereby becoming powerless as a standalone entity.
link |
And then when that thing starts to misbehave
link |
in some subtle way, it creates a lot of problems.
link |
And those problems are dramatized when you're in space,
link |
because you don't have a way to walk away.
link |
Well, as the man said,
link |
once we started making the decisions for you,
link |
it stopped being your world, right?
link |
That's the matrix, Michael, in case you don't remember.
link |
But on the other hand, I could say no,
link |
because isn't that what we do with people anyway?
link |
You know, just kind of the shared intelligence
link |
that is humanity is relying on other people constantly.
link |
I mean, we hyper specialize, right?
link |
As individuals, we're still generally intelligent.
link |
We make our own decisions in a lot of ways,
link |
but we leave most of this up to other people.
link |
And that's perfectly fine.
link |
And by the way, everyone doesn't necessarily share our goals.
link |
Sometimes they seem to be quite against us.
link |
Sometimes we make decisions that others would see
link |
as against our own interests.
link |
And yet we somehow manage it, manage to survive.
link |
I'm not entirely sure why an AI
link |
would actually make that worse or even different, really.
link |
You mentioned the matrix.
link |
Do you think we're living in a simulation?
link |
It does feel like a thought game
link |
more than a real scientific question.
link |
Well, I'll tell you why I think
link |
it's an interesting thought experiment.
link |
Let's see what you think.
link |
From a computer science perspective,
link |
it's a good experiment of how difficult would it be
link |
to create a sufficiently realistic world
link |
that us humans would enjoy being in.
link |
That's almost like a competition.
link |
If we're living in a simulation,
link |
then I don't believe that we were put in the simulation.
link |
I believe that it's just physics playing out
link |
and we came out of that.
link |
Like, I don't think.
link |
So you think you have to build the universe
link |
and have all the fun in the world?
link |
I think that the universe itself,
link |
we can think of that as a simulation.
link |
And in fact, sometimes I try to think about,
link |
to understand what it's like for a computer
link |
to start to think about the world.
link |
I try to think about the world.
link |
Things like quantum mechanics,
link |
where it doesn't feel very natural to me at all.
link |
And it really strikes me as,
link |
I don't understand this thing that we're living in.
link |
It has, there's weird things happening in it
link |
that don't feel natural to me at all.
link |
Now, if you want to call that as the result of a simulator,
link |
okay, I'm fine with that.
link |
But like, I don't.
link |
There's the bugs in the simulation.
link |
I mean, the interesting thing about the simulation
link |
is that it might have bugs.
link |
I mean, that's the thing that I,
link |
But there would be bugs for the people in the simulation.
link |
That's just reality.
link |
Unless you were aware enough to know that there was a bug.
link |
Back to the matrix.
link |
Yeah, the way you put the question though.
link |
I don't think that we live in a simulation created for us.
link |
Okay, I would say that.
link |
I think that's interesting.
link |
I've actually never thought about it that way.
link |
I mean, the way you asked the question though,
link |
could you create a world that is enough for us humans?
link |
It's an interestingly sort of self referential question
link |
because the beings that created the simulation
link |
probably have not created the simulation
link |
that's realistic for them.
link |
But we're in the simulation and so it's realistic for us.
link |
So we could create a simulation
link |
that is fine for the people in the simulation, as it were.
link |
That would not necessarily be fine for us
link |
as the creators of the simulation.
link |
But, well, you can forget.
link |
I mean, if you play video games in virtual reality,
link |
you can, if some suspension of disbelief or whatever.
link |
It becomes a world.
link |
It becomes a world.
link |
Even like in brief moments,
link |
you forget that another world exists.
link |
I mean, that's what like good stories do.
link |
And the question is, is it possible to pull,
link |
our brains are limited.
link |
Is it possible to pull the brain in
link |
to where we actually stay in that world
link |
longer and longer and longer and longer?
link |
And like, not only that, but we don't wanna leave.
link |
And so, especially this is the key thing
link |
about the developing brain,
link |
is if we journey into that world early on in life, often.
link |
How would you even know, yeah.
link |
Yeah, so I, but like from a video game design perspective,
link |
from a Westworld perspective,
link |
it's, I think it's an important thing
link |
for even computer scientists to think about
link |
because it's clear that video games are getting much better.
link |
And virtual reality,
link |
although it's been ups and downs
link |
just like artificial intelligence,
link |
it feels like virtual reality will be here
link |
in a very impressive form
link |
if we were to fast forward 100 years into the future
link |
in a way that might change society fundamentally.
link |
Like if I were to,
link |
I'm very limited in predicting the future as all of us are,
link |
but if I were to try to predict,
link |
like in which way I'd be surprised
link |
to see the world 100 years from now,
link |
it'd be that, or impressed,
link |
it'd be that we're all no longer living
link |
in this physical world,
link |
that we're all living in a virtual world.
link |
You really need to read Calculating God by Sawyer.
link |
It's a, he'll read it in the night.
link |
It's a very easy read,
link |
but it's, assuming you're that kind of reader,
link |
but it's a good story.
link |
And it's kind of about this,
link |
but not in a way that it appears.
link |
And I really enjoyed the thought experiment.
link |
And I think it's pretty sure it's Robert Sawyer.
link |
But anyway, he's apparently
link |
Canadian's top science fiction writer,
link |
which is why the story mostly takes place in Toronto.
link |
But it's a very good sort of story
link |
that sort of imagines this.
link |
Very different kind of simulation hypothesis sort of thing
link |
from say, The Egg, for example.
link |
You know, I'm talking about the short story.
link |
By the guy who did The Martian.
link |
Who wrote The Martian?
link |
You know what I'm talking about.
link |
The Martian. Matt Damon.
link |
So we had this whole discussion
link |
that Michael doesn't partake in this exercise of reading.
link |
He doesn't seem to like it,
link |
which seems very strange to me,
link |
considering how much he has to read.
link |
I read all the time.
link |
I used to read 10 books every week
link |
when I was in sixth grade or whatever.
link |
I was, a lot of it's science fiction,
link |
a lot of it's history, but I love to read.
link |
But anyway, you should read Calculating God.
link |
I think you'll, it's very easy to read, like I said,
link |
and I think you'll enjoy sort of the ideas that it presents.
link |
Yeah, I think the thought experiment is quite interesting.
link |
One thing I've noticed about people growing up now,
link |
I mean, we talk about social media,
link |
but video games is a much bigger,
link |
bigger and bigger and bigger part of their lives.
link |
And the video games have become much more realistic.
link |
I think it's possible that the three of us are not,
link |
maybe the two of you are not familiar exactly
link |
with the numbers we're talking about here.
link |
The number of people.
link |
It's bigger than movies, right?
link |
I used to do a lot of the computational narrative stuff.
link |
I understand that economists can actually see
link |
the impact of video games on the labor market.
link |
That there's fewer young men of a certain age
link |
participating in like paying jobs than you'd expect.
link |
And that they trace it back to video games.
link |
I mean, the problem with Star Trek
link |
was not warp drive or teleportation.
link |
It was the holodeck.
link |
Like if you have the holodeck, that's it.
link |
That's it, you go in the holodeck, you never come out.
link |
I mean, it just never made, once I saw that,
link |
I thought, okay, well, so this is the end of humanity
link |
as we know it, right?
link |
They've invented the holodeck.
link |
Because that feels like the singularity,
link |
not some AGI or whatever.
link |
It's some possibility to go into another world
link |
that can be artificially made better than this one.
link |
And slowing it down so you live forever.
link |
Or speeding it up so you appear to live forever.
link |
Or making the decision of when to die.
link |
And then most of us will just be old people on the porch
link |
yelling at the kids these days in their virtual reality.
link |
But they won't hear us because they've got headphones on.
link |
So, I mean, rewinding back to Mook's,
link |
is there lessons that you've, speaking to kids these days?
link |
That was a transition.
link |
That was fantastic.
link |
I'll fix it in post.
link |
That's Charles's favorite phrase.
link |
When we were recording all the time,
link |
whenever the editor didn't like something or whatever,
link |
I would say, we'll fix it in post.
link |
He hated that more than anything.
link |
Because it's Charles's way of saying,
link |
I'm not gonna do it again.
link |
You're on your own for this one.
link |
But it always got fixed in post.
link |
So is there something you've learned about,
link |
I mean, it's interesting to talk about Mook's.
link |
Is there something you've learned
link |
about the process of education,
link |
about thinking about the present?
link |
I think there's two lines of conversation to be had here.
link |
There's the future of education in general
link |
that you've learned about.
link |
And more passionately is the education
link |
in the times of COVID.
link |
The second thing in some ways matters more than the first,
link |
for at least in my head,
link |
not just because it's happening now,
link |
but because I think it's reminded us of a lot of things.
link |
Coincidentally, today, there's an article out
link |
by a good friend of mine,
link |
who's also a professor at Georgia Tech,
link |
but more importantly, a writer and editor
link |
at the Atlantic, a guy named Ian Bogost.
link |
And the title is something like,
link |
Americans Will Sacrifice Anything
link |
for the College Experience.
link |
And it's about why we went back to college
link |
and why people wanted us to go back to college.
link |
And it's not greedy presidents
link |
trying to get the last dollar from someone.
link |
It's because they want to go to college.
link |
And what they're paying for is not the classes.
link |
What they're paying for is the college experience.
link |
It's not the education that's being there.
link |
I've believed this for a long time,
link |
that we continually make this mistake of,
link |
people want to go back to college
link |
as being people want to go back to class.
link |
They want to go back to campus.
link |
They want to move away from home.
link |
They want to do all those things that people experience.
link |
It's a rite of passage.
link |
It's an identity, if I can steal some of Ian's words here.
link |
And I think that's right.
link |
And I think what we've learned through COVID
link |
is it has made it,
link |
the disaggregation was not the disaggregation
link |
of the education from the place, the university place,
link |
and that you can get the best anywhere you want to.
link |
Turns out there's lots of reasons
link |
why that is not necessarily true.
link |
The disaggregation is having it shoved in our faces
link |
that the reason to go, again,
link |
that the reason to go to college
link |
is not necessarily to learn.
link |
It's to have the college experience.
link |
And that's very difficult for us to accept,
link |
even though we behave that way,
link |
most of us, when we were undergrads.
link |
A lot of us didn't go to every single class.
link |
We learned and we got it and we look back on it
link |
and we're happy we had the learning experience as well,
link |
obviously, particularly us,
link |
because this is the kind of thing that we do.
link |
And my guess is that's true
link |
of the vast majority of your audience.
link |
But that doesn't mean the,
link |
I'm standing in front of you telling you this,
link |
is the thing that people are excited about.
link |
And that's why they want to be there,
link |
primarily why they want to be there.
link |
So to me, that's what COVID has forced us to deal with,
link |
even though I think we're still all in deep denial about it
link |
and hoping that it'll go back to that.
link |
And I think about 85% of it will.
link |
We'll be able to pretend
link |
that that's really the way it is, again,
link |
and we'll forget the lessons of this.
link |
But technically what'll come out of it,
link |
or technologically what'll come out of it
link |
is a way of providing a more dispersed experience
link |
through online education
link |
and these kinds of remote things that we've learned.
link |
And we'll have to come up with new ways to engage them
link |
in the experience of college,
link |
which includes not just the parties
link |
or the whatever kids do,
link |
but the learning part of it
link |
so that they actually come out four or five
link |
or six years later with having actually learned something.
link |
So I think the world
link |
will be radically different afterwards.
link |
And I think technology will matter for that,
link |
just not in the way that the people
link |
who were building the technology originally
link |
imagined it would be.
link |
And I think this would have been true even without COVID,
link |
but COVID has accelerated that reality.
link |
So it's happening in two or three years or five years,
link |
as opposed to 10 or 15.
link |
That was an amazing answer that I did not understand.
link |
It was passionate and meaningful.
link |
But I don't, no, I just didn't,
link |
no, I'm not trying to criticize it.
link |
I just think, I don't think I'm getting it.
link |
So you mentioned disaggregation.
link |
Well, so the power of technology
link |
that if you go on the West Coast and hang out long enough
link |
is all about we're gonna disaggregate these things together.
link |
The books from the bookstore, that kind of a thing.
link |
And then suddenly Amazon controls the universe, right?
link |
And technology is a disruptor, right?
link |
And people have been predicting that
link |
for higher education for a long time,
link |
but certainly in the age of moves.
link |
So is this the sort of idea like
link |
students can aggregate on a campus someplace
link |
and then take classes over the network anywhere?
link |
Yeah, this is what people thought was gonna happen,
link |
or at least people claimed it was gonna happen, right?
link |
Because my daughter is essentially doing that now.
link |
She's on one campus, but learning in a different campus.
link |
Sure, and COVID makes that possible, right?
link |
COVID makes that legal, all but avoidable, right?
link |
But the idea originally was that,
link |
you and I were gonna create this machine learning class
link |
and it was gonna be great,
link |
and then no one else would,
link |
there'd be the machine learning class everyone takes, right?
link |
That was never gonna happen, but something like that,
link |
you can see happening. But I feel like
link |
you didn't address that.
link |
Why, why, why is it that, why, why?
link |
I don't think that will be the thing that happens.
link |
So the college experience,
link |
maybe I missed what the college experience was.
link |
I thought it was peers, like people hanging around.
link |
A large part of it is peers.
link |
Well, it's peers and independence.
link |
Yeah, but none of that,
link |
you can do classes online for all of that.
link |
No, no, no, no, because we're social people, right?
link |
So you wanna be in the same room.
link |
So when we take the classes,
link |
that also has to be part of an experience.
link |
It's in a context, and the context is the university.
link |
And by the way, it actually matters
link |
that Georgia Tech really is different from Brown.
link |
I see, because then students can choose
link |
the kind of experience they think
link |
is gonna be best for them.
link |
Okay, I think we're giving too much agency to the students
link |
in making an informed decision.
link |
Okay. But the truth,
link |
but yes, they will make choices
link |
and they will have different experiences.
link |
And some of those choices will be made for them.
link |
Some of them will be choices they're making
link |
because they think it's this, that, or the other.
link |
I just don't want to say,
link |
I don't want to give the idea.
link |
It's not homogenous.
link |
Yes, it's certainly not homogenous, right?
link |
I mean, Georgia Tech is different from Brown.
link |
Brown is different from pick your favorite state school
link |
in Iowa, Iowa State, okay?
link |
Which I guess is my favorite state school in Iowa.
link |
But these are all different.
link |
They have different contexts.
link |
And a lot of those contexts are,
link |
they're about history, yes,
link |
but they're also about the location of where you are.
link |
They're about the larger group of people who are around you,
link |
whether you're in Athens, Georgia,
link |
and you're basically the only thing that's there
link |
as a university, you're responsible for all the jobs,
link |
or whether you're at Georgia State University,
link |
which is an urban campus,
link |
where you're surrounded by six million people
link |
in your campus where it ends and begins in the city,
link |
ends and begins, we don't know.
link |
It actually matters whether you're a small campus
link |
or a large campus.
link |
I mean, these things matter.
link |
Why is it that if you go to Georgia Tech,
link |
you're forever proud of that,
link |
and you say that to people at dinners,
link |
like bars and whatever,
link |
and if you get a degree at an online university somewhere,
link |
that's not a thing that comes up at a bar.
link |
Well, it's funny you say that.
link |
So the students who take our online masters
link |
by several measures are more loyal
link |
than the students who come on campus,
link |
certainly for the master's degree.
link |
The reason for that, I think,
link |
and you'd have to ask them,
link |
but based on my conversations with them,
link |
I feel comfortable saying this,
link |
is because this didn't exist before.
link |
I mean, we talk about this online masters
link |
and that it's reaching 11,000 students,
link |
and that's an amazing thing,
link |
and we're admitting everyone we believe who can succeed.
link |
We got a 60% acceptance rate.
link |
It's amazing, right?
link |
It's also a $6,600 degree.
link |
The entire degree costs $6,600 or $7,000,
link |
depending on how long you take.
link |
A dollar degree, as opposed to $46,000
link |
it would cost you to come on campus.
link |
So that feels, and I can do it while I'm working full time,
link |
and I've got a family and a mortgage
link |
and all these other things.
link |
So it's an opportunity to do something you wanted to do,
link |
but you didn't think was possible
link |
without giving up two years of your life,
link |
as well as all the money
link |
and everything else in the life that you had built.
link |
So I think we created something that's had an impact,
link |
but importantly, we gave a set of people opportunities
link |
they otherwise didn't feel they had.
link |
So I think people feel very loyal about that.
link |
And my biggest piece of evidence for that,
link |
besides the surveys,
link |
is that we have somewhere north of 80 students,
link |
might be 100 at this point,
link |
who graduated, but come back in TA for this class,
link |
for basically minimum wage,
link |
even though they're working full time,
link |
because they believe in sort of having that opportunity
link |
and they wanna be a part of something.
link |
Now, will generation three feel this way?
link |
15 years from now, will people have that same sense?
link |
I don't know, but right now they kind of do.
link |
And so it's not the online,
link |
it's a matter of feeling as if you're a part of something.
link |
Right, we're all very tribal, right?
link |
And I think there's something very tribal
link |
about being a part of something like that.
link |
Being on campus makes that easier,
link |
going through a shared experience makes that easier.
link |
It's harder to have that shared experience
link |
if you're alone looking at a computer screen.
link |
We can create ways to make that true.
link |
But is it possible?
link |
The question is, it still is the intuition to me,
link |
and it was at the beginning when I saw something
link |
like the online master's program,
link |
is that this is gonna replace universities.
link |
No, it won't replace universities.
link |
Because it's living
link |
in a different part of the ecosystem, right?
link |
The people who are taking it are already adults,
link |
they've gone through their undergrad experience.
link |
I think their goals have shifted from when they were 17.
link |
They have other things that are going on.
link |
But it does do something really important,
link |
something very social and very important, right?
link |
You know this whole thing about,
link |
don't build the sidewalks, just leave the grass
link |
and the students or the people will walk
link |
and you put the sidewalks where they create paths,
link |
this kind of thing.
link |
That's interesting, yeah.
link |
Their architects apparently believe
link |
that's the right way to do things.
link |
The metaphor here is that we created this environment,
link |
we didn't quite know how to think about the social aspect,
link |
but we didn't have time to solve all,
link |
do all the social engineering, right?
link |
The students did it themselves,
link |
they created these groups, like on Google Plus,
link |
there were like 30 something groups created
link |
in the first year because somebody had used Google Plus.
link |
And they created these groups
link |
and they divided up in ways that made sense.
link |
We live in the same state or we're working
link |
on the same things or we have the same background
link |
or whatever and they created these social things.
link |
We sent them T shirts and they wear,
link |
we have all these great pictures of students
link |
putting on their T shirts as they travel around the world.
link |
I climbed this mountain top, I'm putting this T shirt on,
link |
I'm a part of this, they were a part of them.
link |
They created the social environment
link |
on top of the social network and the social media
link |
that existed to create this sense of belonging
link |
and being a part of something.
link |
They found a way to do it, right?
link |
And I think they had other,
link |
it scratched an itch that they had,
link |
but they had scratched some of that itch
link |
that might've required they'd be physically
link |
in the same place long before, right?
link |
So I think, yes, it's possible
link |
and it's more than possible, it's necessary.
link |
But I don't think it's going to replace the university
link |
The university as we know it will change.
link |
But there's just a lot of power
link |
in the kind of rite of passage
link |
kind of going off to yourself.
link |
Now, maybe there'll be some other rite of passage
link |
That'll drive us somewhere else, it's possible.
link |
So the university is such a fascinating mess of things.
link |
So just even the faculty position is a fascinating mess.
link |
Like it doesn't make any sense.
link |
It's stabilized itself,
link |
but like why are the world class researchers
link |
spending a huge amount of time or their time teaching
link |
Like you're doing like three jobs.
link |
And I mean, it turns out it's maybe an accident of history
link |
or human evolution, I don't know.
link |
It seems like the people who are really good at teaching
link |
are often really good at research.
link |
There seems to be a parallel there,
link |
but like it doesn't make any sense
link |
that you should be doing that.
link |
At the same time, it also doesn't seem to make sense
link |
that your place where you party
link |
is the same place where you go to learn calculus
link |
But it's a safe space.
link |
Safe space for everything.
link |
Yeah, relatively speaking, it's a safe space.
link |
Now, by the way, I feel the need very strongly
link |
to point out that we are living
link |
in a very particular weird bubble, right?
link |
Most people don't go to college.
link |
And by the way, the ones who do go to college,
link |
they're not 18 years old, right?
link |
They're like 25 or something.
link |
I forget the numbers.
link |
The places where we've been, where we are,
link |
they look like whatever we think
link |
the traditional movie version of universities are.
link |
But for most people, it's not that way at all.
link |
By the way, most people who drop out of college,
link |
it's entirely for financial reasons, right?
link |
So we were talking about a particular experience.
link |
And so for that set of people,
link |
which is very small, but larger than it was a decade
link |
or two or three or four, certainly, ago,
link |
I don't think that will change.
link |
My concern, which I think is kind of implicit
link |
in some of these questions,
link |
is that somehow we will divide the world up further
link |
into the people who get to have this experience
link |
and get to have the network
link |
and they sort of benefit from it,
link |
and everyone else, while increasingly requiring
link |
that they have more and more credentials
link |
in order to get a job as a barista, right?
link |
You gotta have a master's degree
link |
in order to work at Starbucks.
link |
I mean, we're gonna force people to do these things,
link |
but they're not gonna get to have that experience,
link |
and there'll be a small group of people who do
link |
who will continue to, you know, positive feedback,
link |
look, et cetera, et cetera, et cetera.
link |
I worry a lot about that, which is why, for me,
link |
and by the way, here's an answer
link |
to your question about faculty,
link |
which is why, to me, that you have to focus
link |
on access and the mission.
link |
I think the reason, whether it's good, bad, or strange,
link |
I mean, I agree, it's strange,
link |
but I think it's useful to have the faculty member,
link |
particularly at large R1 universities
link |
where we've all had experiences,
link |
that you tie what they get to do
link |
and with the fundamental mission of the university
link |
and let the mission drive.
link |
What I hear when I talk to faculty is,
link |
they love their PhD students
link |
because they're reproducing, basically, right?
link |
And it lets them do their research and multiply.
link |
But they understand that the mission is the undergrads,
link |
and so they will do it without complaint, mostly,
link |
because it's a part of the mission and why they're here,
link |
and they have experiences with it themselves,
link |
and it was important to get them
link |
where they were going.
link |
The people who tend to get squeezed in that, by the way,
link |
are the master's students, right,
link |
who are neither the PhDs who are like us
link |
nor the undergrads we have already bought into the idea
link |
that we have to teach, though.
link |
That's increasingly changing.
link |
Anyway, I think tying that mission in really matters,
link |
and it gives you a way to unify people
link |
around making it an actual higher calling.
link |
Education feels like more of a higher calling to me
link |
than even research,
link |
because education, you cannot treat it as a hobby
link |
if you're going to do it well.
link |
But that's the pushback on this whole system
link |
is that education should be a full time job, right?
link |
And it's almost like research is a distraction from that.
link |
Yes, although I think most of our colleagues,
link |
many of our colleagues would say that research is the job
link |
and education is the distraction.
link |
Right, but that's the beautiful dance.
link |
It seems to be that tension in itself seems to work,
link |
seems to bring out the best in the faculty.
link |
But I will point out two things.
link |
One thing I'm going to point out,
link |
and the other thing I want Michael to point out,
link |
because I think Michael is much closer
link |
to sort of the ideal professor in some sense than I am.
link |
Well, he is a dean.
link |
You're the platonic sense of a professor.
link |
I don't know what he meant by that,
link |
but he is a dean, so he has a different experience.
link |
I'm giving him time to think of the profound thing
link |
he's going to say.
link |
But let me point this out,
link |
which is that we have lecturers
link |
in the College of Computing where I am.
link |
There's 10 or 12 of them, depending on how you count,
link |
as opposed to the 90 or so tenure track faculty.
link |
Those 10 lecturers who only teach,
link |
well, they don't only teach, they also do service.
link |
Some of them do research as well, but primarily they teach.
link |
They teach 50%, over 50% of our credit hours,
link |
and we teach everybody, right?
link |
So they're doing not just,
link |
they're doing more than eight times the work
link |
of the tenure track faculty,
link |
just more closer to nine or 10.
link |
And that's including our grad courses, right?
link |
So they're doing this, they're teaching more,
link |
they're touching more than anyone,
link |
and they're beloved for it.
link |
I mean, so we recently had a survey.
link |
Everyone does these alumni surveys.
link |
You hire someone from the outside to do whatever,
link |
and I was really struck by something.
link |
You saw all these really cool numbers.
link |
I'm not going to talk about it
link |
because it's all internal, confidential stuff.
link |
But one thing I will talk about
link |
is there was a single question we asked our alum,
link |
and these are people who graduated,
link |
born in the 30s and 40s,
link |
all the way up to people who graduated last week, right?
link |
Well, last semester.
link |
And it was the question,
link |
name a single person who had a strong positive impact on you,
link |
something like that.
link |
I think it was special impact?
link |
Yeah, special impact on you.
link |
And then, so they got all the answers from people,
link |
and they created a word cloud.
link |
It was clearly a word cloud created by people
link |
who don't do word clouds for a living
link |
because they had one person whose name appeared
link |
like nine different times,
link |
like Philip, Phil, Dr. Phil, you know, but whatever.
link |
But they got all this.
link |
And I looked at it, and I noticed something really cool.
link |
The five people from the College of Computing,
link |
I recognized, were in that cloud.
link |
And four of them were lecturers,
link |
the people who teach.
link |
Two of them, relatively modern,
link |
both were chairs of our division of computing instruction.
link |
One just, one retired, one is going to retire soon.
link |
And the other two were lecturers,
link |
I remembered, from the 1980s.
link |
Two of those four actually have.
link |
By the way, the fifth person was Charles.
link |
That's not important.
link |
The thing is, I don't tell people that.
link |
But the two of those people
link |
our teaching awards are named after.
link |
Thank you, Michael.
link |
Two of those our teaching awards are named after, right?
link |
So when you ask students, alumni,
link |
people who are now 60, 70 years old even,
link |
you know, who touched them?
link |
They say the Dean of Students.
link |
They say the big teachers who taught
link |
the big introductory classes that got me into it.
link |
There's a guy named Richard Park who's on there,
link |
who's, you know, who's known as a great teacher.
link |
The Phil Adler guy who,
link |
I probably just said his last name wrong,
link |
but I know the first name's Phil
link |
because he kept showing up over and over again.
link |
Adler is what it said.
link |
But different people spelled it differently.
link |
So he appeared multiple times.
link |
So he was a, clearly,
link |
he was a professor in the business school.
link |
But when you read about him,
link |
I went to read about him because I was curious who he was.
link |
You know, it's all about his teaching
link |
and the students that he touched, right?
link |
So whatever it is that we're doing
link |
and we think we're doing that's important
link |
or why we think the universities function,
link |
the people who go through it,
link |
they remember the people who were kind to them,
link |
the people who taught them something,
link |
and they do remember it.
link |
They remember it later.
link |
I think that's important.
link |
That's why the mission matters.
link |
Not to completely lose track of the fundamental problem
link |
of how do we replace the party aspect of universities
link |
before we go to the what makes the platonic professor.
link |
Do you think, like, what in your sense is the role of MOOCs
link |
in this whole picture during COVID?
link |
Like, should we desperately be clamoring
link |
to get back on campus?
link |
Or is this a stable place to be for a little while?
link |
I know that the online teaching experience
link |
and learning experience has been really rough.
link |
I think that people find it to be a struggle
link |
in a way that's not a happy, positive struggle,
link |
that when you got through it,
link |
you just feel like glad that it's over
link |
as opposed to I've achieved something.
link |
So, you know, I worry about that.
link |
But, you know, I worry about just even before this happened,
link |
I worry about lecture teaching,
link |
how well is that actually really working
link |
as far as a way to do education,
link |
as a way to inspire people.
link |
I mean, all the data that I'm aware of seems to indicate,
link |
and this kind of fits, I think, with Charles's story,
link |
is that people respond to connection, right?
link |
They actually feel, if they feel connected
link |
to the person teaching the class,
link |
they're more likely to go along with it.
link |
They're more able to retain information.
link |
They're more motivated to be involved
link |
in the class in some way.
link |
And that really matters.
link |
You mean to the human themselves.
link |
Okay, can't you do that actually
link |
perhaps more effectively online?
link |
Like you mentioned, science communication.
link |
So I literally, I think, learned linear algebra
link |
from Gilbert Strang by watching MIT OpenCourseWare
link |
when I was in track.
link |
Like, and he was a personality,
link |
he was a bit like a tiny...
link |
In this tiny little world of math,
link |
he's a bit of a rockstar, right?
link |
So you kind of look up to that person.
link |
Can't that replace the in person education?
link |
I will point out something, I can't share the numbers,
link |
but we have surveyed our students,
link |
and even though they have feelings
link |
about what I would interpret as connection,
link |
I like that word, in the different modes of classrooms,
link |
there's no difference between how well
link |
they think they're learning.
link |
For them, the thing that makes them unhappy
link |
is the situation they're in.
link |
And I think the lack of connection,
link |
it's not whether they're learning anything.
link |
They seem to think they're learning something anyway, right?
link |
In fact, they seem to think
link |
they're learning it equally well,
link |
presumably because the faculty are putting in,
link |
or the instructors, more generally speaking,
link |
are putting in the energy and effort
link |
to try to make certain that what they've curated
link |
can be expressed to them in a useful way.
link |
But the connection is missing.
link |
And so there's huge differences in what they prefer.
link |
And as far as I can tell,
link |
what they prefer is more connection, not less.
link |
That connection just doesn't have to be physically
link |
I mean, look, I used to teach 348 students
link |
in my machine learning class on campus.
link |
That was the biggest classroom on campus.
link |
They're sitting in theater seats.
link |
I'm literally on a stage looking down on them
link |
and talking to them, right?
link |
There's no, I mean, we're not sitting down,
link |
having a one on one conversation,
link |
reading each other's body language,
link |
trying to communicate and going,
link |
we're not doing any of that.
link |
So if you're past the third row,
link |
it might as well be online anyway
link |
is the kind of thing that people have said.
link |
Daphne has actually said some version of this
link |
that online starts on the third row or something like that.
link |
And I think that's not, yeah, I like it.
link |
I think it captures something important.
link |
But people still came, by the way.
link |
Even the people who had access to our material
link |
would still come to class.
link |
I mean, there's a certain element
link |
about looking to the person next to you.
link |
It's just like their presence there, their boredom.
link |
And like when the parts are boring
link |
and their excitement when the parts are exciting,
link |
like in sharing in that,
link |
like unspoken kind of, yeah, communication.
link |
In part, the connection is with the other people
link |
Yeah, watching the circus on TV alone is not really.
link |
Ever been to a movie theater
link |
and been the only one there at a comedy?
link |
It's not as funny as when you're in a room
link |
full of people all laughing.
link |
Well, you need, maybe you need just another person.
link |
It's like, as opposed to many.
link |
Maybe there's some kind of.
link |
Well, there's different kinds of connection, right?
link |
And there's different kinds of comedy.
link |
Well, in the sense that.
link |
As we're learning today.
link |
I wasn't sure if that was gonna land.
link |
But just the idea that different jokes,
link |
I've now done a little bit of standup.
link |
And so different jokes work in different size crowds too.
link |
Where sometimes if it's a big enough crowd,
link |
then even a really subtle joke can take root someplace
link |
and then that cues other people.
link |
there's a whole statistics of.
link |
I did this terrible thing to my brother.
link |
So when I was really young,
link |
I decided that my brother was only laughing
link |
as it comes when I laughed.
link |
Like he was taking cues from me.
link |
So I like purposely didn't laugh
link |
just to see if I was right.
link |
And did you laugh at non funny things?
link |
You really wanna do both sides.
link |
And at the end of it, I told him what I did.
link |
He was very upset about this.
link |
And from that day on.
link |
He lost his sense of humor.
link |
But from that day on, he laughed on his own.
link |
He stopped taking cues from me.
link |
So I wanna say that it was a good thing that I did.
link |
You saved that man's life.
link |
Yes, but it was mostly mean.
link |
But it's true though.
link |
That people, I think you're right.
link |
But okay, so where does that get us?
link |
That gets us the idea that,
link |
I mean, certainly movie theaters are a thing, right?
link |
Where people like to be watching together,
link |
even though the people on the screen
link |
aren't really co present with the people in the audience.
link |
The audience is co present with themselves.
link |
By the way, and that point,
link |
it's an open question that's being raised by this,
link |
whether movies will no longer be a thing
link |
because Netflix's audience is growing.
link |
So that's, it's a very parallel question for education.
link |
Will movie theaters still be a thing in 2021?
link |
No, but I think the argument is
link |
that there is a feeling of being in the crowd
link |
that isn't replicated by being at home watching it
link |
and that there's value in that.
link |
And then I think just.
link |
It scales better online.
link |
But I feel like we're having a conversation
link |
about whether concerts will still exist
link |
after the invention of the record or the CD
link |
or wherever it is, right?
link |
You're right, concerts are dead.
link |
Well, okay, I think the joke is only funny
link |
if you say it before now.
link |
Right, yeah, that's true.
link |
Like three years ago.
link |
It's like, well, no, obviously concerts are still a big thing.
link |
I'll wait to publish this until we have a vaccine.
link |
No, you know, we'll fix it in post.
link |
But I think the important thing is.
link |
Fix the virus post.
link |
Concerts changed, right?
link |
First of all, movie theaters weren't this way, right?
link |
In like the 60s and 70s, they weren't like this.
link |
Like blockbusters were basically what?
link |
Well, Jaws and Star Wars created blockbusters, right?
link |
Before then, there weren't.
link |
Like the whole shared summer experience
link |
didn't exist in our lifetimes, right?
link |
Certainly you were well into adulthood
link |
by the time this was true, right?
link |
So it's just a very different.
link |
It's very different.
link |
So what we've been experiencing in the last 10 years
link |
is not like the majority of human history,
link |
but more importantly, concerts, right?
link |
Concerts mean something different.
link |
Most people don't go to concerts anymore.
link |
Like there's an age where you care about it.
link |
You sort of stop doing it,
link |
but you keep listening to music or whatever
link |
and da, da, da, da, da, da, da.
link |
So I think that's a painful way of saying that
link |
It was not the same thing as it going away.
link |
Replace is too strong of a word, but it will change.
link |
Actually, like to push back, I wonder,
link |
because I think you're probably just throwing
link |
that your intuition now.
link |
And it's possible that concerts,
link |
more people go to concerts now,
link |
but obviously much more people listen to,
link |
well, that's dumb, than before there was records.
link |
It's possible to argue that if you look at the data,
link |
that it just expanded the pie of what music listening means.
link |
So it's possible that universities grow in the parallel
link |
or the theaters grow,
link |
but also more people get to watch movies,
link |
more people get to be educated.
link |
Yeah, I hope that is true.
link |
Yeah, and to the extent that we can grow the pie
link |
and have education be not just something you do
link |
for four years when you're done with your other education,
link |
but it be a more lifelong thing,
link |
that would have tremendous benefits,
link |
especially as the economy and the world change rapidly.
link |
People need opportunities to stay abreast of these changes.
link |
And so, I don't know,
link |
that's all part of the ecosystem.
link |
It's all to the good.
link |
I mean, I'm not gonna have an argument
link |
about whether we lost fidelity
link |
when we went from Laserdisc to DVDs
link |
or record players to CDs.
link |
I mean, I'm willing to grant that that is true,
link |
but convenience matters and the ability to do something
link |
that you couldn't do otherwise
link |
because that convenience matters.
link |
And you can tell me I'm only getting 90% of the experience,
link |
but I'm getting the experience.
link |
I wasn't getting it before or it wasn't lasting as long
link |
or it wasn't as easy.
link |
I mean, this just seems straightforward to me.
link |
It's gonna, it's going to change.
link |
It is for the good that more people get access
link |
and it is our job to do two separate things.
link |
One, to educate them and make access available.
link |
That's our mission.
link |
But also for very simple selfish reasons,
link |
we need to figure out how to do it better
link |
so that we individually stay in business.
link |
We can do both of those things at the same time.
link |
They are not in, they may be intention,
link |
but they are not mutually exclusive.
link |
So you've educated some scary number of people.
link |
So you've seen a lot of people succeed,
link |
find their path through life.
link |
Is there a device that you can give to a young person today
link |
about computer science education,
link |
about education in general, about life,
link |
about whatever the journey that one takes in there,
link |
maybe in their teens, in their early 20s,
link |
sort of in those underground years
link |
as you try to go through the essential process of partying
link |
and not going to classes
link |
and yet somehow trying to get a degree?
link |
If you get to the point where you're far enough up
link |
in the hierarchy of needs that you can actually
link |
make decisions like this,
link |
then find the thing that you're passionate about
link |
And sometimes it's the thing that drives your life
link |
and sometimes it's secondary.
link |
And you'll do other things because you've got to eat, right?
link |
You've got a family, you've got to feed,
link |
you've got people you have to help or whatever.
link |
And I understand that and it's not easy for everyone,
link |
but always take a moment or two
link |
to pursue the things that you love,
link |
the things that bring passion and happiness to your life.
link |
And if you don't, I know that sounds corny,
link |
but I genuinely believe it.
link |
And if you don't have such a thing,
link |
then you're lying to yourself.
link |
You have such a thing.
link |
You just have to find it.
link |
And it's okay if it takes you a long time to get there.
link |
Rodney Dangerfield became a comedian in his 50s, I think.
link |
Certainly wasn't his 20s.
link |
And lots of people failed for a very long time
link |
before getting to where they were going.
link |
I try to have hope and it wasn't obvious.
link |
I mean, you and I talked about the experience that I had
link |
a long time ago with a particular police officer.
link |
Was it my first one and was it my last one?
link |
But in my view, I wasn't supposed to be here after that
link |
So it's all gravy.
link |
So you might as well go ahead and grab life as you can
link |
That's sort of how I see it.
link |
While recognizing, again, the delusion matters, right?
link |
Allow yourself to be deluded.
link |
Allow yourself to believe that it's all gonna work out.
link |
Just don't be so deluded that you miss the obvious.
link |
And you're gonna be fine.
link |
It's gonna be there.
link |
It's gonna be there.
link |
It's gonna work out.
link |
What do you think?
link |
I like to say choose your parents wisely
link |
because that has a big impact on your life.
link |
Yeah, I mean, there's a whole lot of things
link |
that you don't get to pick.
link |
And whether you get to have one kind of life
link |
or a different kind of life can depend a lot
link |
on things out of your control.
link |
But I really do believe in the passion, excitement thing.
link |
My, I was talking to my mom on the phone the other day
link |
and essentially what came out is that computer science
link |
is really popular right now.
link |
And I get to be a professor teaching something
link |
that's very attractive to people.
link |
And she was like trying to give me some appreciation
link |
for how foresightful I was for choosing this line of work
link |
as if somehow I knew that this is what was gonna happen
link |
in 2020, but that's not how it went for me at all.
link |
Like I studied computer science
link |
because I was just interested.
link |
It was just so interesting to me.
link |
I didn't think it would be particularly lucrative.
link |
And I've done everything I've can to keep it
link |
as unlucrative as possible.
link |
Some of my friends and colleagues have not done that.
link |
And I pride myself on my ability to remain unrich.
link |
But I do believe that, like I'm glad.
link |
I mean, I'm glad that it worked out for me.
link |
It could have been like, oh, what I was really fascinated by
link |
is this particular kind of engraving
link |
that nobody cares about.
link |
But so I got lucky and the thing that I cared about
link |
happened to be a thing that other people
link |
eventually cared about.
link |
But I don't think I would have had a fun time
link |
choosing anything else.
link |
Like this was the thing that kept me interested and engaged.
link |
Well, one thing that people tell me,
link |
especially around the early undergraduate,
link |
and the internet is part of the problem here,
link |
is they say they're passionate about so many things.
link |
How do I choose a thing?
link |
Which is a harder thing for me to know what to do with.
link |
I mean, don't you know which, I mean, you know, look.
link |
A long time ago, I walked down a hallway
link |
and I took a left turn.
link |
I could have taken a right turn.
link |
And my world could be better or it could be worse.
link |
I have no way of knowing.
link |
Is there anything about this particular hallway
link |
that's relevant or you're just in general choices?
link |
Yeah, you were on the left.
link |
It sounds like you regret not taking the right turn.
link |
Oh no, not at all.
link |
You brought it up.
link |
Well, because there was a turn there.
link |
On the left was Michael Newman's office, right?
link |
I mean, these sorts of things happen, right?
link |
But here's the thing.
link |
On the right, by the way, there was just a blank wall.
link |
It wasn't a huge choice.
link |
It would have really hurt.
link |
No, but it's true, right?
link |
You know, I think about Ron Brockman, right?
link |
I went, I took a trip I wasn't supposed to take
link |
and I ended up talking to Ron about this
link |
and I ended up going down this entire path
link |
that allowed me to, I think, get tenure.
link |
But by the way, I decided to say yes to something
link |
that didn't make any sense
link |
and I went down this educational path.
link |
But it would have been, you know, who knows, right?
link |
Maybe if I hadn't done that,
link |
I would be a billionaire right now.
link |
My life could be so much better.
link |
My life could also be so much worse.
link |
You know, you just gotta feel that sometimes
link |
you have decisions you're gonna make.
link |
You cannot know what's gonna do.
link |
You should think about it, right?
link |
Some things are clearly smarter than other things.
link |
You gotta play the odds a little bit.
link |
But in the end, if you've got multiple choices,
link |
there are lots of things you think you might love.
link |
Go with the thing that you actually love,
link |
the thing that jumps out at you
link |
and sort of pursue it for a little while.
link |
The worst thing that'll happen is you took a left turn
link |
instead of a right turn and you ended up merely happy.
link |
So, so accepting, so taking the step
link |
and just accepting, accepting that,
link |
that don't like question, question the choice.
link |
Life is long and there's time to actually pursue.
link |
Every once in a while, you have to put on a leather suit
link |
and make a thriller video.
link |
Every once in a while.
link |
If I ever get the chance again, I'm doing it.
link |
I was told that you actually dance,
link |
but that part was edited out.
link |
There was a thing where we did do the zombie thing.
link |
We did do the zombie thing.
link |
That wasn't edited out.
link |
It just wasn't put into the final thing.
link |
There was a reason for that too, right?
link |
Like I wasn't wearing something right.
link |
There was a reason for that.
link |
I can't remember what it was.
link |
Is that what it was?
link |
Anyway, the right thing happened.
link |
You took the left turn and ended up being the right thing.
link |
So a lot of people ask me that are a little bit
link |
tangential to the programming and the computing world
link |
and they're interested to learn programming,
link |
like all kinds of disciplines that are outside
link |
of the particular discipline of computer science.
link |
What advice do you have for people
link |
that want to learn how to program
link |
or want to either taste this little skill set
link |
or discipline or try to see if it can be used somehow
link |
in their own life?
link |
What stage of life are they in?
link |
One of the magic things about the internet
link |
of the people that write me is I don't know.
link |
Because my answer's different for, my daughter
link |
is taking AP computer science right now.
link |
She's amazing and doing amazing things
link |
and my son's beginning to get interested
link |
and I'll be really curious where he takes it.
link |
I think his mind actually works very well
link |
for this sort of thing and she's doing great.
link |
But one of the things I have to tell her all the time,
link |
she points, well, I want to make a rhythm game.
link |
So I want to go for two weeks and then build a rhythm game.
link |
Show me how to build a rhythm game.
link |
Start small, learn the building blocks
link |
and how to take the time.
link |
Have patience, eventually you'll build a rhythm game.
link |
I was in grad school when I suddenly woke up one day
link |
over the Royal East and I thought, wait a minute,
link |
I'm a computer scientist.
link |
I should be able to write Pac Man in an afternoon.
link |
And I did, not with great graphics.
link |
It was actually a very cool game.
link |
I had to figure out how the ghost moved and everything
link |
and I did it in an afternoon in Pascal
link |
on an old Apple 2GS.
link |
But if I had started out trying to build Pac Man,
link |
I think it probably would have ended very poorly for me.
link |
Luckily back then, there weren't
link |
these magical devices we call phones
link |
and software everywhere to give me this illusion
link |
that I could create something by myself
link |
from the basics inside of a weekend like that.
link |
I mean, that was a culmination of years and years and years
link |
right before I decided, oh, I should be able to write this
link |
So my advice if you're early on is you've got the internet.
link |
There are lots of people there to give you the information.
link |
Find someone who cares about this.
link |
Remember, they've been doing it for a very long time.
link |
Take it slow, learn the little pieces, get excited about it
link |
and then keep the big project you want to build in mind.
link |
You'll get there soon enough.
link |
Because as a wise man once said, life is long.
link |
Sometimes it doesn't seem that long, but it is long
link |
and you'll have enough time to build it all out.
link |
All the information is out there, but start small.
link |
Generate Fibonacci numbers.
link |
That's not exciting, but it'll get you the language.
link |
Well, there's only one programming language, it's Lisp.
link |
But if you have to pick a programming language,
link |
I guess in today's day, what would I do?
link |
Python is basically Lisp, but with better syntax.
link |
Yeah, with C syntax, how about that?
link |
So you're gonna argue that C syntax
link |
is better than anything?
link |
Anyway, also I'm gonna answer Python despite what he said.
link |
Tell your story about somebody's dissertation
link |
that had a Lisp program in it.
link |
This is Dave's, Dave's dissertation was like,
link |
Dave McAllister, who was a professor at MIT for a while
link |
and then he came to Bell Labs and now he's at
link |
Technology Technical Institute of Chicago.
link |
Such an interesting guy.
link |
Anyway, his thesis, it was a theorem prover
link |
and he decided to have as an appendix his actual code,
link |
which of course was all written in Lisp
link |
because of course it was.
link |
And like the last 20 pages are just right parenthesis.
link |
It's just wonderful.
link |
That's programming right there.
link |
Pages upon pages of right parenthesis.
link |
Anyway, Lisp is the only real language,
link |
but I understand that that's not necessarily
link |
the place where you start.
link |
Python is just fine.
link |
If you're, you know, of a certain age,
link |
if you're really young and trying to figure it out,
link |
graphical languages that let you kind of see
link |
how the thing works and that's fine too.
link |
It almost doesn't matter.
link |
But there are people who spend a lot of time
link |
thinking about how to build languages that get people in.
link |
The question is, are you trying to get in
link |
and figure out what it is?
link |
Or do you already know what you want?
link |
And that's why I asked you what stage of life people are in
link |
because if you're different stages of life,
link |
you would attack it differently.
link |
The answer to that question of which language
link |
keeps changing, I mean, there's some value
link |
to exploring, a lot of people write to me about Julia.
link |
There's these like more modern languages
link |
that keep being invented, Rust and Kotlin.
link |
There's stuff that, for people who love
link |
functional languages like Lisp,
link |
that apparently there's echoes of that,
link |
but much better in the modern languages.
link |
And it's worthwhile to,
link |
especially when you're learning languages,
link |
it feels like it's okay to try one
link |
that's not like the popular one.
link |
Oh yeah, but you want something simple.
link |
And I think you get that way of thinking
link |
almost no matter what language.
link |
And if you push far enough,
link |
like it can be assembly language,
link |
but you need to push pretty far
link |
before you start to hit the really deep concepts
link |
that you would get sooner in other languages.
link |
But like, I don't know, computation is kind of computation,
link |
is kind of Turing equivalent, is kind of computation.
link |
And so it matters how you express things,
link |
but you have to build out that mental structure
link |
And I don't think it's super matters which language.
link |
I mean, it matters a little,
link |
because some things are just
link |
at the wrong level of abstraction.
link |
I think assembly is at the wrong level of abstraction
link |
for someone coming in new.
link |
I think that if you start.
link |
For someone coming in new.
link |
Yes, for frameworks, big frameworks are quite a bit.
link |
You know, you've got to get to the point
link |
where I want to learn a new language,
link |
means I just pick up a reference book
link |
and I think of a project and I go through it in a weekend.
link |
Right, you got to get there.
link |
You're right though, the languages that are designed
link |
for that are, it almost doesn't matter.
link |
Pick the ones that people have built tutorials
link |
and infrastructure around to help you get kind of,
link |
kind of ease into it.
link |
Because it's hard.
link |
I mean, I did this little experiment once.
link |
I was teaching intro to CS in the summer as a favor.
link |
I was teaching intro to CS as a favor.
link |
And it was very funny because I'd go in every single time
link |
and I would think to myself,
link |
how am I possibly going to fill up an hour and a half
link |
talking about for loops, right?
link |
And there wasn't enough time.
link |
Took me a while to realize this, right?
link |
There are only three things, right?
link |
There's reading from a variable,
link |
writing to a variable and conditional branching.
link |
Everything else is syntactic sugar, right?
link |
The syntactic sugar matters, but that's it.
link |
And when I say that's it, I don't mean it's simple.
link |
I mean, it's hard.
link |
Like conditional branching, loops, variable.
link |
Those are really hard concepts.
link |
So you shouldn't be discouraged by this.
link |
Here's a simple experiment.
link |
I'm gonna ask you a question now.
link |
I'm gonna mess this up.
link |
That's one of the trickiest things to get for programmers,
link |
that there's a memory and the variables are pointing
link |
to a particular thing in memory,
link |
and sometimes the languages hide that from you
link |
and they bring it closer
link |
to the way you think mathematics works.
link |
Right, so in fact, Mark Guzdal,
link |
who worries about these sorts of things,
link |
or used to worry about these sorts of things anyway,
link |
had this kind of belief that actually,
link |
people when they see these statements,
link |
X equals something, Y equals something, Y equals X,
link |
that you have now made a mathematical statement
link |
that Y and X are the same.
link |
Which you can if you just put like an anchor in front of it.
link |
Yes, but people, that's not what you're doing, right?
link |
I thought, and I kind of asked the question,
link |
and I think I had some evidence for this,
link |
it's hardly a study,
link |
is that most of the people who didn't know the answer,
link |
weren't sure about the answer, they had used spreadsheets.
link |
And so it's, you know,
link |
it's by reference, or by name really, right?
link |
And so depending upon what you think they are,
link |
you get completely different answers.
link |
The fact that I could go, or one could go,
link |
two thirds of the way through a semester,
link |
and people still hadn't figured out in their heads,
link |
when you say Y equals X, what that meant,
link |
tells you it's actually hard.
link |
Because all those answers are possible,
link |
and in fact, when you said,
link |
oh, if you just put an ampersand in front of it,
link |
I mean, that doesn't make any sense for an intro class,
link |
and of course a lot of languages
link |
don't even give you the ability
link |
to think about it in terms of ampersand.
link |
Do we want to have a 45 minute discussion
link |
about the difference between equal EQ and equal in Lisp?
link |
But you know, you could do that.
link |
This is actually really hard stuff.
link |
So you shouldn't be, it's not too hard, we all do it,
link |
but you shouldn't be discouraged.
link |
It's why you should start small,
link |
so that you can figure out these things,
link |
so you have the right model in your head,
link |
so that when you write the language,
link |
you can execute it, and build the machine
link |
that you want to build, right?
link |
Yeah, the funny thing about programming,
link |
and those very basic things,
link |
is the very basics are not often made explicit,
link |
which is actually what drives everybody away
link |
from basically any discipline,
link |
but programming is just another one.
link |
Like even a simpler version of the equal sign
link |
that I kind of forget, is in mathematics,
link |
equals is not assignment.
link |
Like, I think basically every single programming language
link |
with just a few handful of exceptions,
link |
equals is assignment.
link |
And you have some other operator for equality.
link |
And even that, like everyone kind of knows it,
link |
once you started doing it,
link |
but like you need to say that explicitly,
link |
or you just realize it, like yourself.
link |
Otherwise you might be stuck for,
link |
you said like half a semester,
link |
you could be stuck for quite a long time.
link |
And I think also part of the programming
link |
is being okay in that state of confusion for a while.
link |
It's to the debugging point.
link |
It's like, I just wrote two lines of code,
link |
why doesn't this work?
link |
And staring at that for like hours,
link |
and trying to figure out.
link |
And then every once in a while,
link |
you just have to restart your computer
link |
and everything works again.
link |
And then you just kind of stare into the void
link |
with the tear slowly rolling down your eye.
link |
By the way, the fact that they didn't get this
link |
actually had no impact on,
link |
I mean, they were still able to do their assignments.
link |
Because it turns out their misunderstanding
link |
wasn't being revealed to them
link |
by the problem sets we were giving them.
link |
It's pretty profound actually, yeah.
link |
I wrote a program a long time ago,
link |
actually for my master's thesis,
link |
and in C++ I think, or C, I guess it was C.
link |
And it was all memory management and terrible.
link |
And it wouldn't work for a while.
link |
And it was some kind of,
link |
it was clear to me that it was overriding memory.
link |
And I just couldn't, I was like,
link |
look, I got to pay for this time for this.
link |
So I basically declared a variable
link |
at the front in the main that was like 400K,
link |
just an array, and it worked.
link |
Because wherever I was scribbling over memory,
link |
it would scribble into that space and it didn't matter.
link |
And so I never figured out what the bug was.
link |
But I did create something to sort of deal with it.
link |
To work around it.
link |
And it, you know, that's crazy, that's crazy.
link |
It was okay, because that's what I wanted.
link |
But I knew enough about memory managed to go,
link |
you know, management to go, you know,
link |
I'm just going to create an empty array here
link |
and hope that that deals with this scribbling memory problem.
link |
That takes a long time to figure out.
link |
And by the way, the language you first learned
link |
probably just garbage collection anyway,
link |
so you're not even going to come up across,
link |
you're not going to come across that problem.
link |
So we talked about the Minsky idea
link |
of hating everything you do and hating yourself.
link |
So let's end on a question
link |
that's going to make both of you very uncomfortable.
link |
Which is, what is your, Charles,
link |
what's your favorite thing that you're grateful for
link |
And Michael, what is your favorite thing
link |
that you're grateful for about Charles?
link |
Well, that answer is actually quite easy.
link |
He stole the easy answer.
link |
Yeah, I can tell you what I hate about Charles,
link |
he steals my good answers.
link |
The thing I like most about Charles,
link |
he sees the world in a similar enough,
link |
but different way that I,
link |
it's sort of like having another life.
link |
It's sort of like I get to experience things
link |
that I wouldn't otherwise get to experience
link |
because I would not naturally gravitate to them that way.
link |
And so he just, he just shows me a whole other world.
link |
Yeah, the inner product is not zero for sure.
link |
It's not quite one, 0.7 maybe.
link |
Just enough that you can learn.
link |
Just enough that you can learn.
link |
That's the definition of friendship.
link |
The inner product is 0.7.
link |
That's the answer to life really.
link |
Charles sometimes believes in me
link |
when I have not believed in me.
link |
He also sometimes works as an outward confidence
link |
that he has so much, so much confidence and self,
link |
I don't know, comfortableness.
link |
Okay, let's go with that.
link |
That I feel better a little bit.
link |
If he thinks I'm okay,
link |
then maybe I'm not as bad as I think I am.
link |
At the end of the day, luck favors the Charles.
link |
It's a huge honor to talk with you.
link |
Thank you so much for taking this time,
link |
wasting your time with me.
link |
It was an awesome conversation.
link |
You guys are an inspiration to a huge number of people
link |
and to me, so really enjoyed this.
link |
Thanks for talking to me.
link |
I enjoyed it as well.
link |
Thank you so much.
link |
And by the way, if luck favors the Charles,
link |
then it's certainly the case
link |
that I've been very lucky to know you.
link |
I'm gonna edit that part out.
link |
Thanks for listening to this conversation
link |
with Charles Isbell and Michael Littman.
link |
And thank you to our sponsors,
link |
Athletic Greens, Super Nutritional Drink,
link |
Eight Sleep, Self Cooling Mattress,
link |
Masterclass Online Courses
link |
from some of the most amazing humans in history,
link |
and Cash App, the app I use to send money to friends.
link |
Please check out the sponsors in the description
link |
to get a discount and to support this podcast.
link |
If you enjoy this thing, subscribe on YouTube,
link |
review it with Five Stars Napa Podcast,
link |
follow on Spotify, support it on Patreon,
link |
or connect with me on Twitter at Lex Friedman.
link |
And now, let me leave you with some words from Desmond Tutu.
link |
Don't raise your voice, improve your argument.
link |
Thank you for listening and hope to see you next time.